Author Topic: [FF8] world map and objects (world.fs)  (Read 69258 times)

Halfer

  • *
  • Posts: 142
    • View Profile
Re: [FF8] world map and objects (world.fs)
« Reply #25 on: 2015-05-28 08:30:37 »
Great breakthrough! Since you're both working on this, is it possible to use the hi-res worldmap textures and the UV coordinates from the PSX version?


This is an interesting question. I still haven't looked on the textures for much but what I've been taking to note is that it should be possible to give faces a bigger resolution texture, so hi-res textures may be possible in the future. As for PSX version, there isn't any reason for PC to be any different when speaking of textures since they are similar. However it seems like the UV calculation for road isn't correct thus making the white lines disappear. It can also be that PSX version uses some polished version of PC world map, one way to confirm this is to get the wmx.obj from PSX files and import it to PC version assuming that the file hierarchy is same.
« Last Edit: 2015-05-28 08:32:29 by Halfer »

Maki

  • 0xBAADF00D
  • *
  • Posts: 621
  • 0xCCCCCCCC
    • View Profile
Re: [FF8] world map and objects (world.fs)
« Reply #26 on: 2015-05-28 10:10:21 »
I could run a BinDiff on those two files, but I only have a PSN version of FF8, so I would have to "download" one.

Halfer

  • *
  • Posts: 142
    • View Profile
Re: [FF8] world map and objects (world.fs)
« Reply #27 on: 2015-05-28 10:21:57 »
Well I have PSX version on my parents house, but I can't get it for a while :P. But I hardly doubt that the files differ from each other, it may be instruction related problem with PSX GPU and computer GPU. But if that's not the case then the PC is indeed an older version of world map which is pretty hard to believe because it was released later than PSX.
« Last Edit: 2015-05-28 10:32:27 by Halfer »

Maki

  • 0xBAADF00D
  • *
  • Posts: 621
  • 0xCCCCCCCC
    • View Profile
Re: [FF8] world map and objects (world.fs)
« Reply #28 on: 2015-05-28 17:26:52 »
I found it little hard to browse FF8DISC1.img, as almost every extractor for PSOne files is dead. I had to manually extract it. My HexEditor just exploded, so had to work with buffer blockcopy. Thank you Visual Studio. <3

wmx.obj:
FFVIII Steam 2013: A594059C0D6D16F5DBEB8A82C4BE696FEB8A0E3D
FF8DISC1.IMG 0x108800 (size: 0x1D5B000): A594059C0D6D16F5DBEB8A82C4BE696FEB8A0E3D
FF8DISC1.IMG 0x19C8800 (size: 0x1D5B000): 500A79C34243AE19A49078BF36AB156C361F1043
FF8DISC1.IMG 0x388B800 (size: 0x333000): 950BCB3B5C8A3FE7F236C248044EC44653781329

This is a bit confusing. Offset 108800 is the same file that is present in Steam release. I'll run a BinDiff on 0x19C8800 and Steam one.

Update: There is a massive difference in 0x108800 and 0x19C8800 file. See the example:
 
I'll replace Steam wmx.obj with 0x19C8800 dump.

Well. It works. xD But I'm stuck inside some mountain and I can't move. Wait, I'll force ragnarok.

What the...

« Last Edit: 2015-05-28 18:21:26 by MaKiPL »

Halfer

  • *
  • Posts: 142
    • View Profile
Re: [FF8] world map and objects (world.fs)
« Reply #29 on: 2015-05-28 18:26:27 »
haha :D, seems like there is 2 wmx.obj in the psx image file? That's odd.

paul

  • *
  • Posts: 179
    • View Profile
Re: [FF8] world map and objects (world.fs)
« Reply #30 on: 2015-05-29 21:56:15 »
Probably done like FF7 where this is grid map and optimized for cd loading map?

Halfer

  • *
  • Posts: 142
    • View Profile
Re: [FF8] world map and objects (world.fs)
« Reply #31 on: 2015-05-29 23:01:33 »
If by grid map you mean the placement of sections and how they are loaded to match the map, it isn't like FF7. In FF7 they are separate files and in FF8 there is only wmx.obj container which contains the sections and other files which contain textures. However there may be the placements also but I think they are coded in .exe for optimization.

Edit: Well, to be honest, I don't even know that yet. wmx.obj is 30MB big and I'm 99% sure that it contains only the geometry and texture calls. Other files mostly contains textures, but there're still lot to inspect. If these files contains the geometry placements, then it's packed very well compared to what FF7 respective data takes.

Edit 2: As I suspected, I found water geometry and texture calls from wmsetus.obj, but it's only size of a block when comparing to wmx.obj. It does contain much more data than the models itself, probably metadata mostly. I'll take closer look tomorrow for wmsetus.obj since I can't manage to get any information from wmx.obj for now because the texture calls don't make sense.
« Last Edit: 2015-05-29 23:36:35 by Halfer »

Halfer

  • *
  • Posts: 142
    • View Profile
Re: [FF8] world map and objects (world.fs)
« Reply #32 on: 2015-05-30 21:42:02 »
Okay, little analysis I did today. Here's what I documented:

Tested by nullifying these offsets from the beginning of wmsetus.obj and little inspection from the offset also.
wmsetus.obj:

0xB38 contains texture calls for rails and streets. There are 13 different pieces which have own texture. Each call is 4 bytes long and is somehow connected to their respective texture.

0xB70 contains data for the transition from world to fields. If offset indicator from the beginning of file is nullified you can't enter towns, except ragnarok cockpit (probably garden also).

0x18D0 has something to do with visibility of world models, especially ragnarok.

0x1CE0 has something to do with lunatic Pandora on tear’s point, nullifying offset indicator disables whole entry.

0x21B8 contains world map dialogues, probably all the texts that are shown in world map.

0x1AB04 contains color and lighting options for world map.


« Last Edit: 2015-05-30 22:25:59 by Halfer »

Mcindus

  • *
  • Posts: 929
  • Artist, Modder, Musician.
    • View Profile
    • Lunatic Pandora
Re: [FF8] world map and objects (world.fs)
« Reply #33 on: 2015-05-30 22:37:58 »
Okay, little analysis I did today. Here's what I documented:

Tested by nullifying these offsets from the beginning of wmsetus.obj and little inspection from the offset also.
wmsetus.obj:

0xB38 contains texture calls for rails and streets. There are 13 different pieces which have own texture. Each call is 4 bytes long and is somehow connected to their respective texture.

0xB70 contains data for the transition from world to fields. If offset indicator from the beginning of file is nullified you can't enter towns, except ragnarok cockpit (probably garden also).

0x18D0 has something to do with visibility of world models, especially ragnarok.

0x1CE0 has something to do with lunatic Pandora on tear’s point, nullifying offset indicator disables whole entry.

0x21B8 contains world map dialogues, probably all the texts that are shown in world map.

0x1AB04 contains color and lighting options for world map.




I learned a lot by modding the textures using Tonberry for the world map... here are a few notes:

1:  UV's are incorrect on the PC worldmap - textures are not 'seamless' and strange artifacts happen over the textures where the walkmesh is either 'doubled' over itself, causing strange wrapping issues, or some kind of glitch is present.

2:  The Ocean, Roads, and a couple other 'animation' effects that exist in the world map all 'rotate' to the same cycle of sprites.  If you change a 'road' texture from intercepting the textures at ram with hash-codes, unless you use a code that 'overrides' the entire animation, the road blinks between animation sprites.  As for the rails/streets Halfer was talking about - the textures for them are all on one texture page.

3:  When you are using the English version and are on the ground, the world textures are hi-res -- if you're flying around the world in the Ragnarok, textures 'pop' back down to lower resolution, so this behavior might help you guys figure out what's going on.

Thanks for all your hard work!

Halfer

  • *
  • Posts: 142
    • View Profile
Re: [FF8] world map and objects (world.fs)
« Reply #34 on: 2015-05-30 22:46:04 »
I learned a lot by modding the textures using Tonberry for the world map... here are a few notes:

1:  UV's are incorrect on the PC worldmap - textures are not 'seamless' and strange artifacts happen over the textures where the walkmesh is either 'doubled' over itself, causing strange wrapping issues, or some kind of glitch is present.

2:  The Ocean, Roads, and a couple other 'animation' effects that exist in the world map all 'rotate' to the same cycle of sprites.  If you change a 'road' texture from intercepting the textures at ram with hash-codes, unless you use a code that 'overrides' the entire animation, the road blinks between animation sprites.  As for the rails/streets Halfer was talking about - the textures for them are all on one texture page.

3:  When you are using the English version and are on the ground, the world textures are hi-res -- if you're flying around the world in the Ragnarok, textures 'pop' back down to lower resolution, so this behavior might help you guys figure out what's going on.

Thanks for all your hard work!

Really? That's useful, thanks!
Couple questions though. You are saying that rails/streets are on one texture page. Do you know this for sure because tim viewer separates them as own images. Though if they are on same page then the hex values would make sense also.

And the hi-res textures. I don't seem to see any kind of difference in textures while on ragnarok or ground. Is there a setting which operates this behavior in ff8config?

Also, 0x15CC contains placement to world from field. Here's ending replicated in world map :P


« Last Edit: 2015-05-30 22:51:11 by Halfer »

Mcindus

  • *
  • Posts: 929
  • Artist, Modder, Musician.
    • View Profile
    • Lunatic Pandora
Re: [FF8] world map and objects (world.fs)
« Reply #35 on: 2015-05-31 02:03:00 »
Really? That's useful, thanks!
Couple questions though. You are saying that rails/streets are on one texture page. Do you know this for sure because tim viewer separates them as own images. Though if they are on same page then the hex values would make sense also.

And the hi-res textures. I don't seem to see any kind of difference in textures while on ragnarok or ground. Is there a setting which operates this behavior in ff8config?


It's hard to tell because the images are so pixelated anyway, but have you used tonberry?  If you want to run a test, you can install Tonberry (1.6 or later), then install my 'horizonpack' mod and disable the low res png's by removing wm_texpg_low1, _low2, and _low3 from the 'wm' textures folder.  Then hop on ragnarok in Balamb and then dismount.  You will see how the textures are being swapped back and forth.

When I get a debug dump from Tonberry, I get a single bitmap dump with all 16 road/textures in one image - and Tonberry -normally- doesn't dump single textures, but the entire page the texture came from (sometimes this is estimated, since all blocks come into Tonberry as 256px images, even though many texture pages are 128x256, 128x128, 256x128, etc.  These textures fit in a 256x128 bitmap, I believe.

Maki

  • 0xBAADF00D
  • *
  • Posts: 621
  • 0xCCCCCCCC
    • View Profile
Re: [FF8] world map and objects (world.fs)
« Reply #36 on: 2015-05-31 13:55:37 »
This is World map with wmsetus.obj with null'ied all textures. 0x1cbd8-0x86cf8 (First file which textures are in center of file, not at the ending)


NOTE: wmsetxx.obj also is responsible for forest walking particle. I have some theory about that ragnarok texture change. Give me a sec, gonna test it.

As I thoughŁ


The textures in wmsetus(sp/it...).obj of world map are for Ragnarok. When you ride ragnarok, the game loads 1/2 sized textures from wmsetus.obj. When you land, the game loads texl.obj file.

TOWNS textures are gone too!
« Last Edit: 2015-05-31 14:03:45 by MaKiPL »

Halfer

  • *
  • Posts: 142
    • View Profile
Re: [FF8] world map and objects (world.fs)
« Reply #37 on: 2015-05-31 14:36:02 »
I see.... I can't replicate this for some reason. I nullified the textures from the file like this but I still see all the textures??:

Notice that the first texture is the water textures!





Maki

  • 0xBAADF00D
  • *
  • Posts: 621
  • 0xCCCCCCCC
    • View Profile
Re: [FF8] world map and objects (world.fs)
« Reply #38 on: 2015-05-31 15:13:14 »
I see... Then steam re-release is different than the original one. My first texture is not a water and is at 117720. Hm... I have an idea. Let's see what happens if we mix files between releases.

Here: https://mega.co.nz/#!qRQVSAqC!Pkw2dyeOxnt235spPwdM34D2bhNRaMLybIGPvIfj-QA

BTW> I'm having fun with rail.obj :D

Aaaaand... Done. TRAAAAAINS!
http://wiki.qhimm.com/view/FF8/WorldMap_rail
« Last Edit: 2015-05-31 16:52:04 by MaKiPL »

Halfer

  • *
  • Posts: 142
    • View Profile
Re: [FF8] world map and objects (world.fs)
« Reply #39 on: 2015-05-31 15:32:33 »
Oh, I mean that is how timviewer shows my wmsetus.obj when I nullified the header of tim textures. I nullified 6 or 7 textures from the beginning and 117720 is the first texture offset for me also. I just tried to demonstrate that for me the textures didn't change for some reason while and while not in ragnarok. I wonder if i have some settings differently because I use dxwnd for windowed screen

Halfer

  • *
  • Posts: 142
    • View Profile
Re: [FF8] world map and objects (world.fs)
« Reply #40 on: 2015-05-31 15:38:37 »
Oh I think it is because I have software rendering on from ff8config. I'm using laptop with gt 840m and HD 4400 and for some reason with both graphic cards I can't use direct3d acceleration because it does this:





Any solutions for this?

Halfer

  • *
  • Posts: 142
    • View Profile
Re: [FF8] world map and objects (world.fs)
« Reply #41 on: 2015-05-31 17:04:54 »
So, I tried to analyze a bit more wmx.obj and this is what I got.

Near the end are the sections that are placed to world map according to game progression:


And also there is the whole area which is showed in place of esthar.

I also found that the crater done by Lunar Cry is there but for some reason the area is split in two and the parts are next to each other like this:


You can see that they are horizontally next to each other while in game they should be vertically on top of each other.

This led me to think that most likely there isn't a simple algorithm to read this file in the engine, but somehow it's described which sections corresponds which area. So it's either somewhere in wmset files or described in engine which would be unfortunate for us. I'll try to search more info for the placements.

EDIT: Can someone confirm if the wmset.obj contains japanese texts? There seems to be text differences between these files.
« Last Edit: 2015-06-04 19:06:09 by Halfer »

Mcindus

  • *
  • Posts: 929
  • Artist, Modder, Musician.
    • View Profile
    • Lunatic Pandora
Re: [FF8] world map and objects (world.fs)
« Reply #42 on: 2015-06-03 04:16:06 »
Will there be any way to change the UV of the textures, since they aren't seamless in the PC version?  It looks like the UV is about 15% too large.

Maki

  • 0xBAADF00D
  • *
  • Posts: 621
  • 0xCCCCCCCC
    • View Profile
Re: [FF8] world map and objects (world.fs)
« Reply #43 on: 2015-06-03 05:37:56 »
Sure, but this can take some time, Halfer is making amazing progress, and after we know how texturing works I could convert data to UV mask only, fix it up and re-parse to FF8.

Edit: I tried to trick game to think I am ragnarok and can pass mountains, sea etc. but no luck. I'll try with coding sea and mountains as grass, thanks to Halfer I know it's possible to do, and where to search.

While coding ultimate hack found this:

Probably GPU related, but it's fun. :D
« Last Edit: 2015-06-03 14:25:07 by MaKiPL »

Halfer

  • *
  • Posts: 142
    • View Profile
Re: [FF8] world map and objects (world.fs)
« Reply #44 on: 2015-06-04 13:24:32 »
Good news guys. I finally managed to crack the UV placements and no wonder it was so hard because I didn't realize they are picked from psx emulated framebuffer. Well I'm gonna do a little more research and write here at evening how it is read from the file. Get ready! :D
« Last Edit: 2015-06-04 13:27:44 by Halfer »

Halfer

  • *
  • Posts: 142
    • View Profile
Re: [FF8] world map and objects (world.fs)
« Reply #45 on: 2015-06-04 17:59:30 »
UPDATE!

wmsetusXX.obj contains world map textures. The object textures used for grounds are size of 128x128 pixels. Here's an example:



Here is 1 block which includes the first 4 textures from wmsetusXX.obj. There are 16 polygons and/or 32 triangles in one block. Each polygon has been made to show 4 32x32 texture area which are the small square areas in the image above.

As I described geometry before I wrote UV coordinates also while I hadn't researched it much yet. Turns out I was correct, but here's better struct with explanation after:

Code: [Select]
struct
{

        u8    face indices [6];         // Describes which indices are connected to each other. See first post, there is an example of face indices for blocks used in sea.
        u8    U vertex 1;
        u8    V vertex 1:
        u8    U vertex 2;
        u8    V vertex 2;
        u8    U vertex 3;
        u8    V vertex 3;
        u4    Texture page [4-bits];    // This changes the texture page according to value (0-F)
        u4    CLUT_ID [4-bits];          // The current palettes CLUT info. This needs to point for the texture you are using (first row 0-3, second row 4-7, and so on...).
        u8    Material type;             // Defines if the geometry is mountain, forest, sea or so on.... (00 seems to be forest)
        u8    Unknown;        // I'm not quite sure what this operates, but it seems to change texture pages too. Currently got only the sea textures while changing the value.
        u8    Unknown;       // Also unknown, however the last 4-bits seems to operate collision somehow

}Triangle

Now for the UV's. Each UV must be set according to triangle's face indices. UV values are from 00 to FF. They are set to indicate a rectangular area where the texture is present in current texture page. For example, if you have 128x128 texture and you want to fill a block with 16 polygon, you have to give each U and V coordinate a value in 2D axis where (0,0) indicates top left corner and (FF,FF) bottom left corner. The textures used in wmsetusXX.obj are 128x128 and it contains 16 32x32 textures so if you want to give a specific texture, you set each face indices UV's to the corresponding area by scaling UV's according to texture location on the page.

Here's an example how I did the image above (and great demonstration of my paint skills  :evil:):



Blue dots are face indices. In this example the top left dots values are (U = 0x00, V = 0x00).
Middle dots values are (U = 0x7F, V = 0x7F).
Bottom right dots values are (U = 0xFF, V = 0xFF).

This indicates that the UV area contains the whole area. While there is now 4 polygons used for showing one texture, you can set the area inside 1 polygon also. It's also possible to enlarge it, but it needs more "waypoints" (dots) to determine in between the desired area and making it manually is frustrating.

So, this means that in theory it is possible to use large textures in world, but since the memory space is limited it may be difficult. I'll look on this later.


If you have questions please ask. I'm pretty tired and noticing my own mistakes and the "shortness" of this post makes me think I may have forgotten to explain something so ask if there's something you don't understand or needs a little explanation.
« Last Edit: 2015-07-07 16:53:20 by Halfer »

Kaldarasha

  • *
  • Posts: 2449
  • Prince of Model Editing
    • View Profile
Re: [FF8] world map and objects (world.fs)
« Reply #46 on: 2015-06-04 18:26:13 »
But the tile system has also information for low res textures. How does the game handle these? Are they also stored in wmsetusXX.obj?

Halfer

  • *
  • Posts: 142
    • View Profile
Re: [FF8] world map and objects (world.fs)
« Reply #47 on: 2015-06-04 18:35:17 »
Actually these are the low-res textures. The higher resolution textures are stored in texl.obj and as far as I have been informed they are used when not in ragnarok (or any other vehicle?). I'm using Aali's drivers to run the game in direct3d mode, but for some reason my game isn't using these higher resolution textures like it doesn't if running with software rendering on.

Oh and here's a picture with all the texture pages showcased on the 2 top rows:
« Last Edit: 2015-06-04 18:40:14 by Halfer »

Maki

  • 0xBAADF00D
  • *
  • Posts: 621
  • 0xCCCCCCCC
    • View Profile
Re: [FF8] world map and objects (world.fs)
« Reply #48 on: 2015-06-04 18:50:25 »
Halfer, excelent work. Will try this on my own.
Oh, by the way- try to completely delete texl.obj file and see what would happen.

Kaldarasha

  • *
  • Posts: 2449
  • Prince of Model Editing
    • View Profile
Re: [FF8] world map and objects (world.fs)
« Reply #49 on: 2015-06-04 18:52:15 »
The Steam version shows the high res textures but if you open and close the menu the game falls back to the low res ones.

Halfer, excelent work. Will try this on my own.
Oh, by the way- try to completely delete texl.obj file and see what would happen.

Maybe re-name texl.obj to wmsetusXX.obj do would work to force the game to use the high res textures.  ???
« Last Edit: 2015-06-04 18:56:05 by Kaldarasha »