[PSX/PC] World map conversion tool - wmx2obj (2015-11-17)

[Blue]

Deprecated. Use this instead.

wmx2obj

wmx2obj is a reverse engineering conversion tool for Final Fantasy 8. It converts the world map file of Final Fantasy 8 into the known Wavefront .obj format — thus making it possible to import the world map geometry into Blender, for instance.



To extract the world map model file wmx.obj from world.fs, use Deling.

Although the original world map file's format is listed as .obj, it's not the commonly known Wavefront .obj format. Hence, this program was created.

Download

wmx2obj on GitHub
Java Archive (JAR)

Features

• Converts the original Final Fantasy 8 world map file wmx.obj into the known Wavefront .obj format.
• You can either convert the whole map (i.e. segments 1 to 835) or select your own segment range of choice for conversion.

Possible future upgrades

• Pushing back a Wavefront .obj file to the format supported by Final Fantasy 8.
• Texture support, as the program currently only supports conversion of the map geometry.
• A preview screen

I, myself, won't unfortunately be working on this program anymore for now. Those interested in upgrading the program, feel free to do so by gathering the source code from GitHub.

[Halfer]

Awesome job!




I'll start improving the program by adding a support for texture exporting with the geometry. After that I'll make an importer. In theory the importer could work without texture support but it would be pretty pointless since you would only be able to modify current vertex and face positions.

[kaspar01]

Amazing!

I love you guys

Can't wait to see it textured!
I already prepared all the textures whith the correct palette since 2 years ago for this moment XD

[Blue]

Just a quick note to those looking to upgrade the software: the source code is commented to clear confusion over its functionality. However, if you are having trouble trying to understand some parts of the code, just drop a message.

[DanTsukasa]

Oh man this is AWESOME! I've been waiting for exports for FF8 and FF9, I'm trying to create my own world map and looking at how these were done is really helpful.

Great work.

[Blue]

Thanks, everyone!

Halfer is currently in the progress of adding texture support to wmx2obj. Check out this thread for smaller updates before a full release.

[DanTsukasa]

Is tere any way to fix one edge of the map where it seems to just be a bunch of pieces stuck together that otherwise make an island or 2?

[kaspar01]

Is tere any way to fix one edge of the map where it seems to just be a bunch of pieces stuck together that otherwise make an island or 2?

As I was sayng in the other topic those "pieces stuck togheter" are extra interchangeable parts of the map (for example the balamb garden area before and after the school become a vehicle)

You can just delete them if you just want to see the full map only (or export only the segments you want instead of all the 835 ones).

[Blue]

As I was sayng in the other topic those "pieces stuck togheter" are extra interchangeable parts of the map (for example the balamb garden area before and after the school become a vehicle)

You can just delete them if you just want to see the full map only (or export only the segments you want instead of all the 835 ones).

This is correct. Only convert segments 1 to 768 to exclude the interchangeable parts.

[Halfer]

wmx2obj version 2!

This program exports the world map as a wavefront .obj and material tempalte library .mtl file.

https://www.dropbox.com/s/dg6su56s5x3dlv5/wmx2obj.jar?dl=0

What's new compared to prior version:

Texture support added with correct material and vertex groups. UV's have been corrected to minimize the UV errors present in the PC version for some reason. Please report any UV calculation errors or improvements if you find the current settings to be a bit off.

Instructions:

• Import wmx.obj
• Choose the segments/sections you want to export. (1-768 are sections for the correct map, 768-835 are placeholders which are placed according to story progress in game.
• Export .obj and .mtl to same directory

Good to know before use:

• When exporting different segments and importing them to 3d program like blender, they will appear in the place where it would be on a world map instead of origin in 3d space!
• It may take some time to export the file due to large file size, same goes for importing the exported file to 3d editing program.
• You have to supply the textures for materials yourself by extracting them from wmsetXX.obj files or texl.obj or by some other way. The problem with extracting these files are that they use CLUT info which isn't automatically applied when exporting them. Timviewer is a good program for extracting these files as a bmp file but it extracts only 1 CLUT info at a time.

Thanks to kaspar01 for his corrected textures that he did. I was able to test my calculations and if the textures worked with these and this is how it looks:


Rails, roads and water have their own texture pages. Sadly the rails and roads are not in one texture page so I didn't spend time to correct this manually. However, you can create your own textures by looking up the UV maps from blender for example.

I purposely didn't put the source code for now, but if you want it I'll publish it!
 
Please report any bugs and wishes for improving the program. I'll start to make the importer once I have set my mind on how to wisely develop it. Also the fact that textures are hard to extract from the files is pretty stupid so maybe I'll look on to it someday.

[kaspar01]

Awesome!

I'll test it (as soon as I get home) and try to manually fix rails and road texture.. and probably I'll try to re-create animated material for water

About new feature you could add.. maybe a gui for what I think could be useful..

The idea should be a simple switcher for the interchangeable zones that automatically replace the interchangeable mesh during the export..

A bunch of choice that let you chose which version you want to export (for example : Balamb garden "on ground" or "crater" )

We should list all the matches between interchangeable pieces (I can help whith that I guess)

I think this function could be useful for future modding

[Update]
For water texture it's been easy but... rails and road is not..

I'm looking at their UV and it makes me think they should be 256x256 texture like the others..



I found some little pieces that seems to match whith the UV groups but I have to guess exactly where to place each..

[Update 2]

This is time consuming but my theory looks to be correct..

pieces of rails appear to be scattered in wmsetus file.. I'm manually rebuilding the texture for it :

[Halfer]

Yeah, I should have probably mentioned that the game engine treats the rail and road textures as an one texture the same way as others. Timviewer shows them sepereated in wmsetXX.obj file but they are loaded to frame buffer in complete texture (128x128 I think). I found a program which can tell the VRAM locations here: http://www.romhacking.net/utilities/799/

[kaspar01]

Yeah, I should have probably mentioned that the game engine treats the rail and road textures as an one texture the same way as others. Timviewer shows them sepereated in wmsetXX.obj file but they are loaded to frame buffer in complete texture (128x128 I think). I found a program which can tell the VRAM locations here: http://www.romhacking.net/utilities/799/

Thanks! Never seen that tool before but I'm gonna try it for sure

Anyway the size for rail and road is most likely 256x256 even if only a little portion of it is actually used (according to my UV tests..I placed those piece from wmsetXX and placed in a 256x256 texture and they fit correctly..just need time to understand which piece goes where.. so I guess I'm right about this..)

Another thing that should be said is that rails material does not striclty applies on wolrd map rails only but seems that is used for some "road" pieces (usually for city entrance like the green one you can see in my pic)..and for FH bridge.

[Halfer]

Well the UV's don't depend on the resolution so you can put higher or lower resolution images in blender or 3dsmax. However I'm not sure what happens to UV's when you put for example 1024x1024 texture, it may be messed up a bit, but then it's just a matter of new calculation and that's it. Speaking of it I can make an improvement so that when you export the world map you can choose which resolution textures you want to use so it doesn't mess up the UV's.

The problem with the UV placement is that they are presented in hex values 0-255, so I have to manually do the calculations because for example 0x7f is 127 in decimals and 127/255 = 0.498... while it's supposed to be 0.5 on UV grid.

[kaspar01]

Well the UV's don't depend on the resolution so you can put higher or lower resolution images in blender or 3dsmax. However I'm not sure what happens to UV's when you put for example 1024x1024 texture, it may be messed up a bit, but then it's just a matter of new calculation and that's it. Speaking of it I can make an improvement so that when you export the world map you can choose which resolution textures you want to use so it doesn't mess up the UV's.

The problem with the UV placement is that they are presented in hex values 0-255, so I have to manually do the calculations because for example 0x7f is 127 in decimals and 127/255 = 0.498... while it's supposed to be 0.5 on UV grid.

The reasong why I think it's 256 is that if I export the UV map in that dimension to see where UV group are and then I pick the pieces from wmsetXX files and place in a map which is exactly that big they fit whih no need of scaling.
Moreover every other texture for the map is that big so..


I'm tryng to configure an emulator to rip out those 2 texture but whith bad result till now..


[Update]

Fun fact: most likely road and rail use same textures in same position..or apparently the things which are in the same position in the uv actually have the same texture so..basically they could probably use the same "merged" texture.. i guess..



[Update 2]

Done!



I'm not 100% sure about 2 or 3 texture pieces that could be switched but it looks almost the same so.. I guess it doesn't matter


I confirm that road and rail can use the same texture.

fun fact 2: there is a little piece of texture which is apparently not used (not from road or rail at least..)

[Halfer]

wmsetXX textures are 128x128 each. texl.obj contains only 256x256 big textures. I think that with the current calculations both works with the geometry, don't know about larger textures.

Fun fact: most likely road and rail use same textures in same position..or apparently the things which are in the same position in the uv actually have the same texture so..basically they could probably use the same "merged" texture.. i guess..

They may be on the same page with each other, but I don't know how the engine treats it since the texture call from wmx.obj indicates that the page is different. There may be some sort of memory handler in the engine that works with the PSX VRAM to ease the job for GPU. One reason why I believe that is when memory hacking the camera in the game to value of 300 (1024 is normal camera) which zooms out the camera from the player makes the game crash when flying over esthar and FH bridge. It makes the bridge textures disappear before crash. It could also mean that the memory buffer can't keep up with the textures, but it may be too soon to make this kind of conclusions.

I confirm that road and rail can use the same texture.

fun fact 2: there is a little piece of texture which is apparently not used (not from road or rail at least..)

Nice job! Looking great

[halkun]

Can I get a close-up of the train tracks? I'm interested in making a metro map of those (Once an for all)

[kaspar01]

Can I get a close-up of the train tracks? I'm interested in making a metro map of those (Once an for all)

it's really low poly and low res but here it is : http://s2.postimg.org/jqe3yenjc/close_up.jpg

[Halfer]

it's really low poly and low res but here it is : http://s2.postimg.org/jqe3yenjc/close_up.jpg

Seems like the road now has the white lines like in the PSX version. Show me if you see any UV misplacement's so I can try to fix them .

[kaspar01]

Seems like the road now has the white lines like in the PSX version. Show me if you see any UV misplacement's so I can try to fix them .

Of course

I found this not far from deling city (north from missile base).



This is the only "reallly bad" thing i found.

Some other are just "not perfect seamless" but I don't know if it's a problem from the original game.

Another weird thing is that many face have probably wrong normals.

[Halfer]

Can you post a larger scale image of the above, can't identify the beach. Never mind, found it, I've problems with reading 

The textures not being seamless is probably my calculation error since it's not perfect yet because of the hex values, I'll try to search an answer for it.

About normals. I documented in reverse engineer forums that wmx.obj contains some sort of shadowing values. Well I now know that they are normals and probably used for Gourard shading which PSX GPU can utilize (look it up if interested). For the next version I'll include normals too so the shading is not a problem after it. Though before I put normals in use I want to improve to current algorithm for texture vertexes because right now it's taking too much memory as you can see from the file size. After optimizing the algorithm I would say the file size will drop about 1/4 or 1/3

I had to check the seam thing and it appears in FF8 also:


Sorry for the first picture but I hope you can see it, my FF8 uses 128x128 textures since it's the older release. However these kind of problems occurs all over the map because of the developers. Also the odd stretches like in the kaspar01's picture are misplaced UV's by developers. There's probably nothing that can be done automatically to these unless all the UV's are re-generated.

However it looks better in game because of the renderer which makes it look more seamless, although the problem exists there even if it's harder to notice.

[halkun]

it's really low poly and low res but here it is : http://s2.postimg.org/jqe3yenjc/close_up.jpg

No, I mean the routes, not the actual track. I want to make a metro map like the London tube underground map



to something like below

[kaspar01]

About normals. I documented in reverse engineer forums that wmx.obj contains some sort of shadowing values. Well I now know that they are normals and probably used for Gourard shading which PSX GPU can utilize (look it up if interested). For the next version I'll include normals too so the shading is not a problem after it. Though before I put normals in use I want to improve to current algorithm for texture vertexes because right now it's taking too much memory as you can see from the file size. After optimizing the algorithm I would say the file size will drop about 1/4 or 1/3

I had to check the seam thing and it appears in FF8 also

Sorry for the first picture but I hope you can see it, my FF8 uses 128x128 textures since it's the older release. However these kind of problems occurs all over the map because of the developers. Also the odd stretches like in the kaspar01's picture are misplaced UV's by developers. There's probably nothing that can be done automatically to these unless all the UV's are re-generated.

However it looks better in game because of the renderer which makes it look more seamless, although the problem exists there even if it's harder to notice.

Yep it looks better since it render pixels as they are I guess..

Anyway for the idea I gave you about the switcher I wrote a list that you could use for the switch funcion:

Code: [Select]

Interchangeables zones:

Eshtar

374 to 381  <==>  769 to 776
406 to 413  <==>  777 to 784
438 to 445  <==>  785 to 792
470 to 477  <==>  793 to 800
502 to 509  <==>  801 to 808
534 to 541  <==>  809 to 816
566 to 573  <==>  817 to 824

Trabia Garden

150 to 151  <==>  825 to 826


Galbadia Garden

268  <==>  827


Balamb Garden

275 to 276  <==>  828 to 829


Missile Base

328  <==>  830

Trabia Crater

215 to 216  <==>  831 to 832
247 to 248  <==>  833 to 834


Desert Prison

362  <==>  835


The list is not yet confirmed (I'll test it tomorrow..it's 3 A.M. here XD) Edit2 : Checked the list and it looks to be all correct


Edit:

@halkun this should be what you asked for: http://s16.postimg.org/m6kmcyl1h/Full_Map_HD.jpg

[Zervox]

Code: [Select]

public void exportFile(File file) throws IOException {
    PrintWriter printWriter = new PrintWriter(file);
    printWriter.println("# Converted to Wavefront .obj with wmx2obj");
    printWriter.println("# wmx2obj \u00A9 2015 Aleksanteri Hirvonen");
    printWriter.println();
for (Segment segment : segments) {
printWriter.println("g " + segment.getID());
writeVertices(segment,printWriter);
writeFaces(segment,printWriter);
}
    printWriter.close();
}

private void writeVertices(Segment segment,PrintWriter printWriter) {
    // take the coordinate values of each vertex instance
    // and write to file
    printWriter.println("# List of geometric vertices");
       
    for (Block block : segment.getBlocks()) {
        for (Vertex vertex : block.getVertices()) {
            printWriter.println(
                    "v "
                    + vertex.getCoordinate(Vertex.X) + " "
                    + -vertex.getCoordinate(Vertex.Y) + " "
                    + -vertex.getCoordinate(Vertex.Z)
            );
        }
    }
    printWriter.println();
}

private void writeFaces(Segment segment,PrintWriter printWriter) {
    // take the vertex index values of each face instance
    // and write to file
    printWriter.println("# Polygonal face elements");
    for (Block block : segment.getBlocks()) {
        for (Face face : block.getFaces()) {
            printWriter.print("f ");
            for (Integer vertexIndex : face.getVertexIndices()) {
                printWriter.print(vertexIndex + " ");
            }
            printWriter.println();
        }
    }
}
I added this code for exporting  each segment as a separate wavefront part hope it would please someone.
not sure about javascript and how I am passing the segments as I converted my code from C++

[kaspar01]

I added this code for exporting  each segment as a separate wavefront part hope it would please someone.
not sure about javascript and how I am passing the segments as I converted my code from C++

This is really good indeed.
Since they're separate chunk it makes perfectly sense  (and will be handy for editing/modding)

Not sure it can be done but.. naming the chunk whith the segment number could be useful too (it's been quite frustrating calculate the number for interchangeable parts)