Qhimm.com Forums
Miscellaneous Forums => Scripting and Reverse Engineering => Topic started by: Tonberry on 2006-04-22 21:29:24
-
Hi everyone!
I've been playing with the "ff9.img" file for a while and found some of its structures. Now I'm trying to make some utilities to simplify the task of understanding them. I only have access to the Spanish version of FF9 so I was wondering if someone would be kind enough to verify some information on the English version for me.
I'm only interested in verifying the order of the directories inside of "ff9.img" and the number of files for each one. As I couldn't find information on the Net or the forums about this, I'm assuming it's not known to everyone so here it goes...
"ff9.img" is divided in sectors of 0x800 bytes. What I need is information on the first sector of the first disk, that goes from 0x00000000 to 0x000007ff. It's similar to a root directory and its structure is something like this (in the Spanish version):
000: (4 bytes) Signature "FF9 ".
004: (4 bytes) Don't know yet. (Value=0x6)
008: (4 bytes) Number of entries in directory. (Value=0xf)
00C: (4 bytes) Don't know yet. (Value=0x1)
Next follow each directoy entry up to a number equal to the value at offset 0x008.
(4 bytes) - Type. (02=directory; 03=array of files; 04=end of volume) (I think.)
(4 bytes) - Number of files in directory. (Except for type 04.)
(4 bytes) - Pointer to sector with more information about files in directory. (Different meaning for type 04.)
(4 bytes) - Pointer to first sector of first file. (Except for type 04.)
The rest is padded with the string "FF9 ".
Next is the list of values I need you to check. I added the value in the Spanish version, just in case someone is interested.
008: (0xF)
014: (0x15)
024: (0x23)
034: (0x13)
044: (0xF)
054: (0xde)
064: (0x151)
074: (0x97)
084: (0x65)
094: (0x55)
0a4: (0x37)
0b4: (0x2a)
0c4: (0x1ab)
0d4: (0x2)
0e4: (0x1ff)
One more thing. May I ask what's been found out about this file? I only saw posts by Zande and Cyberman but most of the work is on FF7 and the new engine. I've just started and my list is more or less:
- Directory structure and file extraction. (Done, with details pending.)
- General structure for some container files. (DBxx structures only.)
- Models for weapons, enemies and characters. (Bones and animation pending.) Other models too, but I haven't spent much time to recognize them. (Maybe some scenery.)
- Geometry for World Map. (Learning OpenGL to add textures. Plus, some details pending.)
Thank you very much for any information you can give me.
-
Umm well the first sector of the disk are you refering to the file FF9.img or are you refering to the entire disk? IE mode 1 Form 1 0:0:0 or LBA 0 on the disk?
If it's the file that's probably good (LOL).
The reason the file is in 0x800 byte chunks is that is the playstation sector size, which makes it easier to seek data in the program. Likely to be read in Mode 2 form 1 this might be the reason I've been having some trouble dealing with the data analysis (hmmm) because the header may shift it 8 bytes if it's a Mode 2 sector. Windows does not deal with the larger sector size of Mode 2 form 2 at all (shrug).
In any case a quick look at the structure indicates that the directory information is fairly simple it just has a list of sector count and start values.
What is the difference to you between an array of files and a directory? 04 may indicate that there isn't anything more in directory entries.
Your 02 and 03 designations puzzle me. I'll have to look at what it means using the actual data.
I'll write a quickie bit of code to use this information to see if it works right. I can also detect TIM and model files already, and possibly find the field information files as well with this. PRIOR to this what I did was search through the image file and make references. However with exact pointers this will change things a lot in terms of knowing where data is and decoding it.
Cyb
-
Cyberman, thanks for your answer.
I messed it a bit when I said "the first sector of the first disk". It's the first sector of the "ff9.img" file in the first disk as I'm not using the other disks at the moment. (First sector of the file.) Everywhere I mention sector I mean sector in the file, not the disk.
Type 02 entries have elements like this:
(4 bytes) - I don't know what they are, but include incrementing numbers and some attributes. (Maybe identifiers.)
(4 bytes) - Pointer to first sector of file.
This means Type 02 entries are like directories, with entries pointing directly to the files.
Type 03 entries are a bit different. I've got only one. Its elements are like this:
(2 bytes) - Number of sectors to add to base sector or 0xffff for empty entries, with "base sector" being the sector pointer that appears in the Type 03 entry.
I see Type 03 entries as an array of pointers to files inside another file. Perhaps a better term would be subdirectory, but then, the structure is not like the others. You pick the names as I'm not very good at that.
I think all files in this entry are the effects you see (hear?) in the battles as there are a lot of images of little stars and thunder.
The Type 04 entry effectively indicates there are no more entries. It points to the same sector pointed by the first entry.
I checked "ff9.img" for sectors not accessible using this structures and there are none, so the whole file is covered.
Most of the files I just described are containers to other files so you'll find most TIMs, models, etc. inside those containers. You can also be sure that no file inside a container has a pointer ouside the container unless it coincides with the end of the container itself. It may have an indirect way to reference something outside the container (like an index) but it doesn't point to it directly.
Each Type 02 entry groups the information in logical modules: Initialization and disk management, cards game, field, battle, enemies, weapons, characters, music, world map and some other things. (I think the order is right but I'm missing some as I've not yet gotten to the point of documenting anything.) The order of these things and the number of their files in the English version is what I wanted to check when I made my first post.
I wrote some code to extract the files, to look at what I know about the models, to extract TIMs and to look for other graphics not in TIM format, but it's too messy at the moment. If there's someone brave enough I can give it away as I'm more interested in studying the structures at the moment. It's written in C# and I was thinking in rewritting it in C when everything was more clear. Anyway, I don't like to code as much as I like decoding.
I also wrote some code in C, using OpenGL to render the geometry of the World Map and it does quite well. It still doesn't render the textures, but I'm working on it. Again, if there's interest I'll post the code, but not the binary as it's sort of a hack.
WARNING Cyberman: Spoiler ahead, or not. ;)
Once you separate the files I described you'll find most of them are containers to other files. Some of these have what I've named "DBxx structures". (Feel free to rename them.)
You'll recognize the DBxx structures because in an hex editor they start like this:
DB xx 00 00-yy yy yy tt-...
Where "DB" is the byte value 0xdb and "xx" tells the number of parts that describe the complete DBxx structure. Next come a list of pointers to each part, relative to offset 4 (where "yy" starts). No part overlaps another one in the same level (there are some nested DBxx.)
The pointer to each part has the structure "yy yy yy tt", where "yyyyyy" is the pointer relative to offset 4 and "tt" seems to be the type for the part. Some of the values of "tt" I've found are:
tt=02 --> 3D model (weapon, enemies, etc.)
tt=03 --> Skins. (enemies)
tt=04 --> Skins. (weapons)
tt=08 --> AKAO?
tt=12 --> AKAO?
tt=14 --> End of AKAO?
tt=19 -->
tt=1B --> Sub-structure container.
With this information it's easy to further subdivide some files.
You can find DBxx by recursively going deeper in the structure or try the quickly and dirtly way: Search for "DB xx 00 00 yy yy yy" (without "tt") and force yyyyyy=xx*4.
If "tt"=0x1b, you'll have more nested structures. And keep in mind that depending on the structure you'll have to change the base offset for references, some are relative to a common offset and some are relative to themselves.
If there's someone interested in a little more info just let me know and I'll order my mess a bit. Anyone want to split the file 50-50? ;)
Thank you again Cyberman and I expect we can compare the number of files for each entry.
-
AKAO is related to sound. It's the name of Square's sound guy and has the honor of even having a fieldscript command named after him.
-
AKAO is related to sound. It's the name of Square's sound guy and has the honor of even having a fieldscript command named after him.
Wasn't the sound guy Nobuo Uematsu?
-
Thank you, halkun. I thought that was the case so I decided to leave it for later as I'm used to looking into graphics and code, not sound for the Playstation. I'll read the document I found in the forums about the sound chip.
chaoscontrol: He's refering to the sound programmer, not the music guy (composer.)
-
Well I have the root directory partially perusable I just haven't automatically identified files as yet.
Seems the first file in the first directory is just the number 21 the second file appears to be a debugging program because it sports text formating information about disk changing and frames per second and triangle drawing (hmmm).
That's about all for tonight :D
Cyb
Here is what is currently viewable
(http://www.geocities.com/cyberman_phillips/Images/PSXVIEW_FF9_001.JPG)
-
Wow, I wasn't expecting a full fledged application, but that looks great. :) Look... I don't want trouble with the fans of the ff7 engine. ;)
So, we have different number of files in DIR01, but the other directories have the same number of files.
The flags for the files in DIR00 are the same I have, but there are some differences in the starting sectors, so there must be differences in the sizes. Those must be files where localization was needed, but we shouldn't have differences in most graphics, sound, etc.
Comparing the data for DIR00 I get:
File Yours Mine SectorDiff SizeDiff
F<00>000 17 17 +3
F<00>001 18 21 3 0
F<00>002 91 94 3 +2
F<00>003 280 285 5 0
F<00>004 373 378 5 0
F<00>005 399 404 5 0
F<00>006 408 413 5 0
F<00>007 437 442 5 -1
F<00>008 465 469 4 +3
F<00>009 495 502 7 +1
F<00>010 502 510 8 0
F<00>011 516 524 8 +2
F<00>012 542 552 10 0
F<00>013 549 559 10 +1
F<00>014 578 589 11 0
F<00>015 592 603 11 0
F<00>016 606 617 11 +2
F<00>017 635 648 13 0
F<00>018 636 649 13 0
F<00>019 654 667 13 0
F<00>020 671 684 13 ???
Some of the files I have are bigger than yours. I think a sentence in Spanish has a little bit more characters than a sentence in English, but I couldn't find a reference to confirm this. And I don't see why an image, even translated, would have a different size.
Are you saying that F<00>000 has only a byte with value 0x15? Mine is a little bit strange: It has some 'JFIF' tags and a few "Photoshop" strings. JPEGs in the psx? I couldn't decompress them, but the headers are similar to a JPEG.
I have the same kind of information as you do in F<00>001. The strings are in English so they weren't meant for translation and as you say, they show some debugging information.
Most of the files in DIR00 start with an identifier, followed by a table of values 0x80wwyyzz. Can these be memory addresses from some sort of relocation table?
I think I'll take a closer look at the files in DIR00. There are some images not in TIM format in some of the files that may help classify them and to find some pointers.
----------------------
I looked into DIR00 for graphics and there is certainly a pattern. All the images I found are noticeable in greyscale, but most of them have a palette before the image and the greyscale value is the index. I'm still trying to find height and width attributes.
It should be easy to figure out the purpose of some of the files.
F<00>007:
25 low contrast images. Chocograph targets. (width=42)
F<00>010:
Symbols for the cards game. (width=128)
Rose of the Winds. (width=84)
F<00>011:
Paper-like key item background. (width=88)
F<00>013:
Portraits of 10 characters. (width=32)
Animation. Passing pages of a book. (Accessing memory card.) (frames=4; total_width=176)
F<00>014:
Portraits of 13 characters. (width=32)
F<00>015:
Portraits of 13 characters. (width=32)
F<00>016:
Portraits of 13 characters. (width=32)
Right-Up/Right-Down arrows and 4 other symbols. (width=48)
F<00>018:
"Wrong disk!!" image. (width=194)
F<00>20:
"Level up!" image. (width=112)
-
Oh, phun!
I haven't worked with the FF9 stuff for quite some, I gathered quite a bit of info around the data though!
Let see... I could post up some docs I made, but they're a mess. :D
(I'll see if I can rewrite em into something usefull :P).
Here's a short rundown of what's in some of the directories (you found out this already?):
0000 - Status/Menu/Battle/... -Text and random crap. ;P
0001 - Misc Images (Logos, Fonts, World 'mini' Map images, etc).
0002 - Dialog Text
0003 - Images (My docs doesn't say what kinda :P).
0004 - Object/Supporting Characters Data
0005 - Monster Data (Part I, stats, names, etc).
0006 - Location Data (Dungeon, Cities, etc).
0007 - Monster Data (Part II, 3d models)
0008 - Weapon Data
0009 - Samplebanks and Sequencer Data (ie. ze music ;)).
0010 - Character Data
0011 - Sound effects
0012 - World Map Data (It's a guess, but I'm fairly sure).
0013 - Summon models/animation data, seems to differ quite a bit from the other DIRs.
0014 - Whatever...
That's about it I think, might give you and idea. ;P
tt=02 --> 3D model (weapon, enemies, etc.)
tt=03 --> Skins. (enemies)
tt=04 --> Skins. (weapons)
tt=08 --> AKAO?
tt=12 --> AKAO?
tt=14 --> End of AKAO?
tt=19 -->
tt=1B --> Sub-structure container.
Here's the chunk ID's I've found (there seems to be 0x1f, or 31, different chunk types):
0x02= 3d Model
0x03= Animations (Guess, hope it's a good guess :P).
0x04= TIM (Images/Textures)
0x06= Text (Single Tile Encoded with commands)
0x07= Sequencer data (Mr. Akao's annoying format :P).
0x08= Dunno, FM/Noise/Reverb settings? (as odd as all of Akao's formats)
0x09= Audio, 4bit (Sony) ADPCM with Akaos header.
0x10= Monster stats (HP, weakness etc), might contain other things aswell
0x1b= Pointer chunk (Doesn't contain any actuall data, just pointers to other chunks).
I thought I had more, but I guess I never wrote em down or I couldn't find them, have to dig thru all files.
I got how sequncer, audio, text encoding and (most) commands, 3d models (mostly) documented in either a shitty txt or c++ file (or both).
I'll come back when I've check all the old data laying around on millions of places of my discs. See if I can throw together some, more or less, proper docs. :P
Think I posted this one sometime ago. :P (http://zine.mine.nu/beast.jpg)
-
Thank you Zande, and my appologies... I said in my first post that I read some posts by Cyberman and Zidane, but it was Zande. Sorry. I could just as well have said Zorro. :D
I have some idea as to what is inside each directory, but not all the entries. Your list is more complete than mine. This is the first psx game I look into so I'm not too familiar with the structures. After stripping off the TIMs I jumped directly to the weapon models that I thought would be simple and then to the directories.
Entry 0012, as you say is the World Map geometry. It's a little strange as there is some redundancy and missing parts. I'll post more information when I get home.
----------------- I'm home :)
Zande, what I didn't mention earlier is that I am interested in any information I can get. I don't mind no mess as it can't be worst than mine. :D
I don't remember seeing that model before, but it looks great. With bones and all. (Mine are all "contractured". No bones.) I think I'll skip the models then. Did you get to add the textures?
This is what I have at the moment about the World Map. Its geometry is defined in DIR12. There are 2 files and each one of them has more or less the same information. I'll describe the first one, as the second one seems to be transposed. Both of them are divided into sectors (0x800 bytes).
The World is defined as a grid of 24 columns and 20 rows in the first file. (The number of rows may differ a little because of rows full of water. Should check the second file.) Each cell of the grid points to a block and each block is divided into 4 meshes (2 meshes * 2 meshes).
The general structure is:
- Grid definition.
- Definition of blocks and texture groups.
The first two sectors have the grid definition. They don't define the dimensions for the grid and the data for the first cell can be found at offset 0. At the moment, I'm not sure if each cell is defined by 4 or 8 bytes. To render a correct version of the map I assumed they are 8 bytes long:
(4 bytes) Reference to block, version 1.
(4 bytes) Reference to block, version 2.
Each version may represent the state of a cell at a given moment: open cave/closed cave, standing construction/destroyed construction, etc. (Or perhaps some animation.) The few I checked had the open/closed cave.
The way I rendered a version of the World Map that makes sense is by rendering all the version 2 blocks. If version 1 blocks are used instead, the World is almost empty.
Each reference to a block is:
(2 bytes) Pointer to block. (Number of sector where block definition starts.)
(1 byte) Attribute1.
(1 byte) Attribute2.
The blocks are ordered in the file starting with Chokobo's Paradise in the top-left corner, then the rest of the cells of the first row, second row, etc.; and so are the cells of the grid.
Note: A cell of the grid may have more than one block associated, but only one is shown at a given time when rendering the World.
If Attribute1==0, then the cell is water without consideration for the block pointed to. If Attribute1!=0, then the pointer to the block is used. I haven't figured out any other meaning for Attribute1 nor Attribute2.
After the grid definition, come the blocks and some TIM textures in between a variable number of blocks. As an example:
Block-Block-Textures-Block-Block-Block-Block-Textures-Block-...
Each block has exactly 4 meshes arranged like this:
+-------------+
|mesh0 | mesh1|
|------+------|
|mesh2 | mesh3|
+-------------+
The definition for a block is (all pointers in bytes, not sectors, relative to offset 0 of block):
(1 byte) 0x04
(1 byte) Unknown.
(1 byte) Unknown.
(1 byte) Unknown.
(4 bytes) Pointer to start of next block or texture group.
(4 bytes) Pointer to start of first mesh.
(4 bytes) Pointer to start of second mesh.
(4 bytes) Pointer to start of third mesh.
(4 bytes) Pointer to start of fourth mesh.
Start of first mesh:
(1 byte) Number of vertices.
(1 byte) Unknown.
(2 bytes) Number of triangles.
(4 bytes) Pointer to vertex data.
(4 bytes) Pointer to triangle data.
(20 bytes) Unknown.
(4 bytes) Unknown.
Start of vertex data for first mesh:
(2 bytes) Coordinate x. (Admits negative numbers.)
(2 bytes) Coordinate y. (Admits negative numbers.)
(2 bytes) Coordinate z. (Admits negative numbers.)
(2 bytes) Unknown.
... (Repeat for each vertex.)
Start of triangle data for first mesh:
(1 byte) Index of first vertex.
(1 byte) Index of second vertex.
(1 byte) Index of third vertex.
(1 byte) Unknown.
(1 byte) Coordinate u in texture for the first vertex.
(1 byte) Coordinate v in texture for the first vertex.
(2 bytes) Unknown.
(1 byte) Coordinate u in texture for the second vertex.
(1 byte) Coordinate v in texture for the second vertex.
(2 bytes) Unknown.
(1 byte) Coordinate u in texture for the third vertex.
(1 byte) Coordinate v in texture for the third vertex.
(2 bytes) Unknown.
... (Repeat for each triangle.)
Start of second mesh:
... (Idem first mesh.)
Start of third mesh:
... (Idem first mesh.)
Start of fourth mesh:
... (Idem first mesh.)
Zero padded up to end of sector.
Each mesh has a rectangular proyection on the base, but they aren't limited to concave/convex surfaces.
I haven't figured out the association between triangles and textures yet. I should install epsxe to watch the vram.
Texture groups have this structure:
(4 bytes) Number of textures.
(4 bytes) Pointer to first texture.
... Ponters to more textures.
First texture. (TIM)
... More textures. (TIM)
I estimate the in-game field of vision should be of 5*5 square blocks, with the protagonist in the center. Perhaps a little wider in higher places. That's the approximate distance where the fog makes it impossible to see the mountains.
I'm having problems with the rendering of locations like cities, etc. as there is a hole instead. Perhaps the model for the cities is somewhere else or I'm making some mistake.
I also found some kind of correlation I still haven't figured out: The texture groups seem to be near the cities or places of interest.
I hope you don't mind there is some especulation in what I describe. I'll try to get one of those services to upload some images soon.
-
Thank you Zande, and my appologies... I said in my first post that I read some posts by Cyberman and Zidane, but it was Zande. Sorry. I could just as well have said Zorro.
Oh, I didn't even noticed, just skimmed thru the posts @ work. No problem though. ;)
Did you get to add the textures?
Yea, I got the textures + mapping to work with the models.
Not that particular model though, as you might see it's the summon Fenrir, the summon data is stored in the directory 13 which is really odd (like 0xff as padding instead of 0x00??).
The world map information is interesting, don't have time to read it thru deeply at the moment, I'll do it later.
A little note though, my docs says there's both triangle and quadrange polygons. The world map might aswell have, or I might be wrong. Other 3d models have both triangle and quadrangle polygons (and there's actually 3 different types of each).
-
A little note though, my docs says there's both triangle and quadrange polygons. The world map might aswell have, or I might be wrong. Other 3d models have both triangle and quadrangle polygons (and there's actually 3 different types of each).
I found triangles and quadrangles in the models for weapons and enemies, but I don't understand when you say there are 3 different types. Do you mean they are arranged in different ways or are you refering to some other attribute?
Anyway, what you said about quadrangles made me look a little better into the World Map and I had a bug in the code that made holes appear where cities should be. I was taking the number of triangles as a 1 byte value instead of 2 bytes as documented above. I was missing 256 triangles in complex meshes.
I think I also found the relation between the textures and the cities, but that's for another moment.
-
I found triangles and quadrangles in the models for weapons and enemies, but I don't understand when you say there are 3 different types. Do you mean they are arranged in different ways or are you refering to some other attribute?
Yeah, the data is slightly different. Nevermind though, the "normal" ones are most common, the other are not used too much.
-
These are more likely to be script information than just dialog :)
This means they likely have window and various other symbol information embeded into them. Fortunately I've no problem reading the dialog (because of my cute app).
I'll have to explore the DB XX XX etc thing more.
I think you should think of most data as 32 bit aligned also. In any case I'll see what I can find out based on that. Sheesh you folks are already twiddling with complete 3d data (sigh). I'm getting old :D
Cyb
-
Most of the files in DIR00 start with an identifier, followed by a table of values 0x80wwyyzz. Can these be memory addresses from some sort of relocation table?
The playstation uses memory locations 0x80XXXXXX internally as 'system'/ user memory. These are likely load locations for data or program code.
Also it is very unlikely they used anything other than TIM's for images. As they had to use the PsyQ library. They could use JPEG images possibly however there was no reason to do this. From my sccanning of the data originally that was on the disk all image data is stored as TIM's including the field background images. So as I said.. it's not likely image data at all.
In addition to all that, you need to consider the playstation is a 32 bit machine. This is why you have so many uknowns in your world map. This is because ALL the data is likely 32 bit aligned or double word aligned. Furthermore the playstation information tends to be little endian so for example you mesh data begins with 0x04 ZZ ZZ ZZ well it's likely it's just a double word of 0x04 instead.
Geometry data tends to be double word aligned because of the playstations GTE and GPU (32 bit DMA)
The Vertex data is X Y Z U (U being unused) probably. You will notice they pad the UV coordinates with a word sized chunk of data.
Cyb
-
Also it is very unlikely they used anything other than TIM's for images. As they had to use the PsyQ library. They could use JPEG images possibly however there was no reason to do this. From my sccanning of the data originally that was on the disk all image data is stored as TIM's including the field background images. So as I said.. it's not likely image data at all.
Well, it must be that I have the wrong definition of TIM. As I said this is the first time I look into a psx game. The idea I got from reading some documents in the Web is that they have a header starting with 0x00000010, followed by the color format (2, 3, 8 or 9), ..., cluts, ..., color information (RGB or index to clut depending on color format). The first routines I made read this data and converted them to BMP. Am I wrong up to this point?
Then I wrote some other routines to convert the bytes in a generic file to grayscale pixels, allowing to resize in the X dimension. With this I found some other images I wasn't able to find with the first routines; I looked in an hex editor and they didn't have the header described above, so I called them non-TIM. Perhaps they are TIMs without a header?
If that's the case, I'm sorry I didn't research more. Can you recomend a doc?
The first image I found with the header is the one with the letters, numbers, kanji, etc. The first images I found without the header were the ones I described in a previous post.
About the 32-bit alignment, I know what you mean, and I think you don't understand the meaning of the byte/unknown thing because I never told you about it.
I know the psx is a 32-bit machine so the first thing I would do is try to align the data to 32-bit boundaries. I'm not trying to do a byte by byte description of the structures.
The "0x04 ZZ ZZ ZZ" is not 0x00000004 as I described it above. It is in *most* blocks of the World Map, but not in some of them; especially the ones that are immediately before a image group. (Bytes at offset 1 and 2 seem to have independant meaning.)
What I'm sure about is that byte 0x04 is allways present at offset 0, that offset 1 and 2 are mostly 0, that they don't seem to depend one on the other, and that I don't know what they mean.
Another example is the "number of vertices/number of triangles" word. I'm almost sure you'd be tempted to take the byte I marked as unknown and make half a word with the "number of vertices". I did that at first, but when I kept on looking, I found some blocks that where generating vertices' coordinates with information that was clearly triangle data.
The bug I had with the triangles was because I didn't look at the messy docs I had when programming that bit. I did it on memory and thought the triangles where 1 byte also. (I even padded to the next word.)
The 20 bytes I marked as unknown, I did because of lazyness and because I didn't see a clear separation between them. The next 4 bytes I marked as unknown, I did because they were clearly appart from the previous 20 bytes.
Sorry again I didn't explain everything, but althogh I'm a newbie in the psx "deconstruction", I'm from the zxSpectrum "deconstruction" ages and I've been doing it for 13 years in the PC and some other minor platforms.
(Yes, I know it's not right to talk about oneself, especially without proof, but at least that could give you an idea of what to expect.)
I'll try to add all the information and little quirks I find in the structures, but as I said, I'm a little messy with the docs and I take some things for granted.
Some of the quirks I didn't mention are:
- The easy way to find the end of the last file in a directory. (But I'd say you already figured that out.)
- That there is a block in the World Map that is all 0x0. It doesn't even have the 0x04 at offset 0. (I simply skip it.)
- That you can have image groups with 4 images, so you'll have to watch out not to interpret them as block definitions.
About the World Map, I didn't have much time these days and I haven't even made the first texture appear on the screen, but I did some "hand-mapping" (don't even expect 1FPS.) Image groups seem to be loaded into memory in a rectangle of 128*256 pixels where the images in the group are arranged. They seem to be loaded dynamically when rendering a city.
I think I'll shut up about the World Map until I'm able to show a textured city. :D
-
Well, it must be that I have the wrong definition of TIM. As I said this is the first time I look into a psx game. The idea I got from reading some documents in the Web is that they have a header starting with 0x00000010, followed by the color format (2, 3, 8 or 9), ..., cluts, ..., color information (RGB or index to clut depending on color format). The first routines I made read this data and converted them to BMP. Am I wrong up to this point?
:D 2 3 8 9 are not the correct format info.
0 1 2 3 (4/8/16/24) bits respectively with the high order bit of the nybble indicating if it has a palette or not. 4/8 bit TIMs do not require a palette. Just that they be loaded in the correct location of memory. Decoding on a TIM for paletted data is application specific. IE the palette is optional.
Then I wrote some other routines to convert the bytes in a generic file to grayscale pixels, allowing to resize in the X dimension. With this I found some other images I wasn't able to find with the first routines; I looked in an hex editor and they didn't have the header described above, so I called them non-TIM. Perhaps they are TIMs without a header?
This may be the case as this is what Square used in FF7 (files labeled MIM on the FF7 disk are multi image Tims). All background data however does have TIM's for there image. of course I can accept easily if I'm wrong on this as well LOL!
If that's the case, I'm sorry I didn't research more. Can you recomend a doc?
I think the MIM format is discused in the qhimm wiki (http://wiki.qhimm.com/FF7/Field_Module#PSX_MIM_Format) data for FF7. I'm going to slowly put FF9 data in there and hopefully FF8 data as well.
The first image I found with the header is the one with the letters, numbers, kanji, etc. The first images I found without the header were the ones I described in a previous post.
headerless isn't surprising really. Though do you recall what the images were?
About the 32-bit alignment, I know what you mean, and I think you don't understand the meaning of the byte/unknown thing because I never told you about it.
Doh :D Ok that would do it.
I know the psx is a 32-bit machine so the first thing I would do is try to align the data to 32-bit boundaries. I'm not trying to do a byte by byte description of the structures.
The "0x04 ZZ ZZ ZZ" is not 0x00000004 as I described it above. It is in *most* blocks of the World Map, but not in some of them; especially the ones that are immediately before a image group. (Bytes at offset 1 and 2 seem to have independant meaning.)
What I'm sure about is that byte 0x04 is allways present at offset 0, that offset 1 and 2 are mostly 0, that they don't seem to depend one on the other, and that I don't know what they mean.
Another example is the "number of vertices/number of triangles" word. I'm almost sure you'd be tempted to take the byte I marked as unknown and make half a word with the "number of vertices". I did that at first, but when I kept on looking, I found some blocks that where generating vertices' coordinates with information that was clearly triangle data.
The bug I had with the triangles was because I didn't look at the messy docs I had when programming that bit. I did it on memory and thought the triangles where 1 byte also. (I even padded to the next word.)
The 20 bytes I marked as unknown, I did because of lazyness and because I didn't see a clear separation between them. The next 4 bytes I marked as unknown, I did because they were clearly appart from the previous 20 bytes.
Sorry again I didn't explain everything, but althogh I'm a newbie in the psx "deconstruction", I'm from the zxSpectrum "deconstruction" ages and I've been doing it for 13 years in the PC and some other minor platforms.
(Yes, I know it's not right to talk about oneself, especially without proof, but at least that could give you an idea of what to expect.)
I'll try to add all the information and little quirks I find in the structures, but as I said, I'm a little messy with the docs and I take some things for granted.
Some of the quirks I didn't mention are:
- The easy way to find the end of the last file in a directory. (But I'd say you already figured that out.)
- That there is a block in the World Map that is all 0x0. It doesn't even have the 0x04 at offset 0. (I simply skip it.)
- That you can have image groups with 4 images, so you'll have to watch out not to interpret them as block definitions.
About the World Map, I didn't have much time these days and I haven't even made the first texture appear on the screen, but I did some "hand-mapping" (don't even expect 1FPS.) Image groups seem to be loaded into memory in a rectangle of 128*256 pixels where the images in the group are arranged. They seem to be loaded dynamically when rendering a city.
I think I'll shut up about the World Map until I'm able to show a textured city. :D
Hey the more you give the better it is for me. I think there are 2 world maps for a reason. One is a 'walking' scale and the other is a flying scale. The world is shrunk if you use an airship visually. I'll also help beat the information into a nice readable format (check the wiki some).
Side note, FF9 is probably one of the more visually detailed FF's. I think in the world map area's you'll notice they include anything flying in the air and chocobo's possibly as well. I've noticed that there is texture data for little airships and chocobo's in the IMG file.
Cyb
-
Thank you Cyberman. I'll take a look at the MIM format.
headerless isn't surprising really. Though do you recall what the images were?
I'll try to classify all the images I found and associate them to files and directories. It will take me some time, but if you are interested in the ones I described as non-TIM, you can take a look at DIR00. I described all the images I found in that directory. You can find that info in the eighth post of this thread, but I'll make a little recount: (I used your filename nomenclature and all widths are in pixels.)
They seem to be all images that appear in the menus/mini-games options and they are logically grouped in each file, so I assume they give information on what to expect to find in each file.
F<00>007:
25 low contrast images. Chocograph targets. (width=42)
I meant, the little images that tell you where to find goodies in the world with the chocobos. I said "low contrast" because I was looking to them in grayscale and they have a palette with few colors, so in grayscale they look almost black. (I added an inverse look up "feature" to the application as it's easier to see them with a white background.) I remember I added the corresponding palettes some time ago and the colors where right.
Just look into file 007 of DIR00.
In file 008 you can find some symbols and backgrounds used in the card game, so I'd expect the rest of the file has instructions or something related to the card game.
F<00>011:
Paper-like key item background. (width=88 )
This file has only 1 image (that I was able to find) with the background where characters are displayed when you read a letter you got from someone, the ticket Vivi has when you start the game, and any other key item. (Maybe the name in English isn't "key item"?)
F<00>013:
Portraits of 10 characters. (width=32)
Animation. Passing pages of a book. (Accessing memory card.) (frames=4; total_width=176)
When I say portraits, I mean the head/face of the characters that apper in the menus. (By character in this context I mean Zidane, Vivi, etc., not letters and numbers.)
If you notice, when you are about to save and you access a memory card there is a book that passes some pages as if you were reading them. That's what I meant about the animation.
As the file seems to be related to file loading/saving, I'd say that the portraits are of the characters you have the possibility to save games with.
I don't remember if the number is limited to 10 characters when saving.
F<00>014:
Portraits of 13 characters. (width=32)
F<00>015:
Portraits of 13 characters. (width=32)
I only found images for characters' heads/faces so I don't know what they are for, but I'm pretty sure they are part of the menus as they look exactly like the 10 above.
F<00>016:
Portraits of 13 characters. (width=32)
Right-Up/Right-Down arrows and 4 other symbols. (width=48 )
The portraits are the same as above, and I didn't recognize the arrows at first, but all are symbols shown when you buy/sell stuff. The arrows tell if a particular character would be at an advantage/disadvantage if it equips a given armor/weapon/item.
So, file 016 must be related to the buy/sell stuff.
F<00>018:
"Wrong disk!!" image. (width=194 )
This one has an image that tells you to put the right disk in the drive. So it must be related to disk management.
F<00>20:
"Level up!" image. (width=112)
This one is the little text that appears over the characters that leveled up after a battle.
If you found any of those images in DIR00 (or much less likely in other directory) I'd say it's a mistake I made because I didn't know the TIM format as I should.
About the cards' monsters, world textures, weapon textures, enemies textures, field backgound/character textures, palettes and some other I have them as TIM with the header, just as you mention. I'll document it.
I have as non-TIM the images of the last directory, where Zande said there was summon data and I mentioned there where little stars and thunder (lightning really - I messed up there. Although there are AKAO bits there.)
Hey the more you give the better it is for me.
Well, I didn't say anything because I took it for granted, so that's my fault. I don't usually explain every bit of info I don't know unless I have a feeling of what it is and that's not the case with the fields in the structure above.
I'd say the more we give the better for everyone. :) The reason I started working on ff9.img is because I couldn't find tools to look at the models, scenery, etc. There was little info, also. Not that I would do anything with the data, just to look what's in there. I'll probably drop it as soon as I lose interest, but at least I think there is some movement forward. :)
Anyway, I'd like to ask if you have any objective for this data. Not that I mind, just curious.
Some other thing I forgot to mention is that the padding for the u,v texture coordinates isn't always 0x0000. I won't know what they mean until I have some textures in the world.
I think there are 2 world maps for a reason. One is a 'walking' scale and the other is a flying scale. The world is shrunk if you use an airship visually. I'll also help beat the information into a nice readable format (check the wiki some).
That's possible. I think the world in the second file is transposed to the first file (but maybe it's rotated 90 degrees.) That would make it unnecesarilly complicated to associate data from one file to the other. But who knows, it seems to be an optimization as you say.
It's been a long time since I played the game and I lost all the savegames so I don't remember quite well but wasn't there a strange movement when you entered the airship? (I'm slowly starting to play again.)
That would be great if you entered information in the wiki as I looked there a few weeks ago and didn't find anything on ff9.
Side note, FF9 is probably one of the more visually detailed FF's. I think in the world map area's you'll notice they include anything flying in the air and chocobo's possibly as well. I've noticed that there is texture data for little airships and chocobo's in the IMG file.
I remember those images. There are some that include a small Zidane skin (for the World Map), along with chocobo's and moogle's (the one that lets you save the game.) The little airships are the ones you see when you exit the ice cave, near Dali. (I just got there.)
About the textures, I think the World Map files only include textures for the buildings and some details, but I found lots of terrain/building TIMs into file 021 of DIR01 or somewhere near. I don't know if they are used for the World Map, for the battles, or maybe both, but that would explain the lack of textures in the World Map files. (Most common textures accessible to the World, not so common textures accessible to the continents and less common defined near cities, etc.)
I even saw scaled textures so the World rendering is surelly optimized to account for the distance to the camera, but then, this is pretty common.
And I said I wasn't going to mention the WM. :)
-
Looks like my FF9 perusing code is going to get complicated by the DB NN 00 00 thing.
So here is what I currently going to do. Files that begin with DB and have more than one section I am going to add as a subdirectory in my tree view. This will make things look stranger but hey..
What does make sense is that the files begining with DB are likely to be loaded into memory as is. If none are > 1 meg in size that is. It makes sense to have the data with type and offset information. It's likely a load format.
It means I also need to add cute folder Icons and such for the tree view section.
Why all this whining? Well I can't identify any of the file types because few are a single object in the group of data. So I'm going to have to think things through. So far I have a mess, I don't like messes (surprised?)
I have to get up early.. so I'll twiddle tomorrow. For model construction is there a a set of vertexs and bones with seperate animation controls?
Cyb
PS I noticed FF9's program size is almost 2 megs which is the entire PS1 user memory. I suspect this means it reads chunks of it'self from the disk on ocassion.
-
I'm going to brush off my theory hat. This has helped with poor souls stuck in the past.
The program flow for any Final fantasy game up to and including 9 uses the same module system. These all use the same codebase, or design pholosophy with some tweaks. 10 had a massive rewrite of the field module, a new "cinamatic" module added, and the worldmap was reduced, and battle was coupled more tightly to the modules.
Anyway.
Blindly guessing, not having a FF9 disk in front to me, you have to therorize that FF9 executes much like it's Final Fantasy brothers and sisters of the past. Start at the top level and work your way down.
We know that Square like st seperate it's system and give each section to deveopment teams. So you break out battle, menu, field, minigame, and worldmap.
Careful with minigame though, a lot of minigames are written in field script, which are a part of field. In FF7. the CPR game, Chocobo breeding game, and some puzzles are examples of this. (Also of note, the "wandering in the snowstorm/plant posts to keep yourself stright" game is actually a worldmap minigame...)
The icons save to the memory card are "almost tims" and very tiny. They are in the menu system.
Keep in mind. The Menu module is probably the most core system to the game, second to the kernel and above field. It's the global variable manager, and is responsable for save/loading of data. It itself is broken into sub modules that are staticly pasted into memory. (This has been the same Modulus Operondi for 7 and 6)
Fieldscript, save for it's 3d functions, is the same as FF6, and was only slightly tweaked for FF8. The commands, from what very liittle poking I have found, are the same for FF8, PE, and most likely FF9 as well. (Is there a debug room for 9, or did square finally remove that from production releases?) 8 and 7's "Startmap" ,the inital debug room, is set up exactly the same (0,0,0 camera and 0,0,0 character position)
The upshot....
Somewhere in that blob of data is a "kernel.bin" that has inital savemap info plus a "start game" overwrite. (The inital savemap should drop you into the debug room, while the overwrite starts the inital fieldscript for the game)
Look for zgip headers, they start with 0x1F8B080000000000, actually, the "magic number" is 0x1F8B, but exsize that out if you see it, uncompress it, and se what's inside.
Many times the polygon headers are actually GPU command headers, cross refrence with my PSX tech docm looking in the GPU section. In FF8 the ploygons were defined by "blank" gpu commands. (Missing rotation data, which would be supplied by the GTE during the creation of the OT for DME to the GPU)
Oops, Let me retranslate that....
When a PSX renders 3d stuff to the screen it has to go though the GPU. This can be done by directly sending a packet via hardware registers, which is slow, or though Direct Memory Access ->(DME). If you do a DME transfer, a real keen way is to line your GPU packets up in a linked list, sorted by depth. These sorted linked lists are called "ordering tables" ->(OT) and are sorted by an algo by depth on the Z access. An efficent way to get a model into the GPU as fast as possable is to have a model pre-made of GPU packets in a psudo-linked list with the GTE rotation data missing. During the GPU "kick" each packet has it's GTE data put in, sorted by Z depth, and then the whole Linked list is fired off to the GPU for rendering via DME.
(wow, it almost seems like I know what I'm talking about)
So here's some tips.
1. Look for PsyQ headers (TIM, SEQ, VAG)
2. Look for gzip headers
3. Look for GPU packets.
4. The system was broken to descrete parts during deveopment, they are "self contained" for the most part. You will rarely see data of one module bleeding into another. Find the boundries. Think in terms of what programmer was responsable for what part. Assume the only talked to each other at tame-wasting meetings every so often, and only to give update on how thier module is going.
I don't think I answered your questions. But That ws really fun to write.
-
I was able to add some textures to the WM, not like I wanted to but they don't look too bad. Especially considering I like the CAD/technological look. :D
I posted 3 images to the trash bin provided by mirex (Thank you!), so they'll probably disapear.
In the first one you'll see the landscape near Esto Gaza and Shimmering Island.
http://bin.mypage.sk/FILES/FF9_WM1.GIF
In the second one you'll see the Lifa Tree.
http://bin.mypage.sk/FILES/FF9_WM2.GIF
In the third one, some texture mapping.
http://bin.mypage.sk/FILES/FF9_WM4.JPG
You'll notice it doesn't look right, but look closer at Dali, the Observatory and that landing circle in between and you'll see the mapping is right.
Now, to work on the proper textures, as there are lot of problems.
----------------------------------
Cyberman, besides what halkun sugested (looking for known headers, with which I can't help much and looking for modules which is what I've been trying to do), I'd sugest you take your time to classify things. I'll tell you the way I started with all this in the hope that it may be useful; maybe you did the same and think it's obvious, but I don't know. :)
1 - Look for files and directories. Didn't work.
2 - Look for TIMs. Found a lot, grouped logically.
3 - Create 1 file for each block of unknown data; that is, the parts that weren't TIMs.
4 - Look for images without header. It's time consuming, but helpful.
5 - Take a look at 4 or 5 files with unknown data that are near a group of TIMs or headerless images and compare. (This was when I noticed the DB NN 00 00s and some of them aligned to 0x800 bytes.)
6 - Take the group of files that seem easier/shorter than the others and study them deeper. (I decided to go with the info near TIMs of weapons, as they wouldn't have animation.)
7 - Look deeper into the DB NN 00 00 stuff for weapons. I found lots of unknowns, but also a lot of similar data.
8 - See if similar data is near pointers in header. This, with the presumption that there weren't overlaping sections was extremelly useful to further subdivide the DB NN 00 00. There I found vertices, triangles, texture mapping and some blocks with the AKAO string. I was able to make sense of the geometry, but not from the AKAO thing.
9 - Look in Internet for AKAO. Not many results on data format, but many about the sound programmer for FF games. :) Leave for later.
10 - Once with the model for weapons "complete", look near enemies' TIMs. I recognized a lot of the structures as they were the same as the weapons'. (At this moment I found a lot of DB NN 00 00s.)
11 - Dump the first 64/128/whatever number of bytes of the ff9.img starting with DB NN 00 00. I grouped them by the NN and I finally understood them.
12 - Take the previous unordered dump and look for containers.
13 - I found the biggest container for every weapon and enemy, and noticed every container was bound to 0x800.
14 - Look for files and directories with this extra info. Eureka!
The rest is pretty obvious: Separate the files found and do some checking.
The thing is I started trying to reduce things from the whole, but ended up making some "holes" and building from that.
I also started very orderly, but I had to rewrite so many times the code when I discovered something new that I decided it was no use. I was spending more time writing code than looking for things. :(
For model construction is there a a set of vertexs and bones with seperate animation controls?
I didn't get to find animation data.
I'll start with the classification of images and their respective files and dirs tomorrow.
-
There has been a long calm. :lol:
In any case I've been quietly (noisely is not possible) working on a better UI and processing the data files.
However I've discovered a few things about DIR00 and it's associated files
F<00>000 contains only 1 DWORD in little endian format (00000015) or 21 in decimal First word..
The first DWORD I believe identifies the file (hmm).
F<00>001 begins with 00000005
F<00>002 begins with 00000006 and is followed by 'Disc Cover Open\0'
F<00>003 begins with 00000007 it also has a number of printf format strings in it such as
"Illigal Animation Data" <--- actual data hehehe
"w_movementChange : %d -> %d\xA\0"
"MAP JUMP\xA\0"
"w_frameMapDestructor:\xA\0"
"w_frameSystemDestructor:\xA\0"
"*[ Request from script:Weather:%d ]\xA\0"
"*[ Request from script:Navi Active:%d]\xA\0"
etc.
F<00>004 begins with 00000008 it has a number of words that run together
It contains a load table of locations within the PS1 memory and a series of phrases
"ENEMY CARD SELECT\xA\0"
My guess is this is the card game :)
F<00>005 begins with 00000009 it also has a long load table of PS1 memory locations.
F<00>006 begins with 0000000A has a short load table and has a few printf format strings in it
"%2d Bankroll\0"
"%ld Wager\0"
"Stand\0"
"Hit\0"
"Double\0"
"Split\0"
F<00>007 begins with 0000000E
F<00>008 begins with 0000000F
F<00>009 begins with 00000011
F<00>010 begins with 00000010
F<00>011 begins with 00000012
F<00>012 begins with 00000013
F<00>013 begins with 00000014
It has words such as
"File00\0"
"/DISC\0"
in the begining
F<00>014 begins with 00000015
has a PS1 memory load table at the begining
(5) locations
F<00>015 begins with 00000016
F<00>016 begins with 00000017
F<00>017 begins with 0000000F
that's all that is contained in it
F<00>018 begins with 0000000D
and looks to have a TIM file in it.
I'm getting false positives reguarding the DB files (mutter) so I may need to work on that a bit more :)
Current screen caps.
Directory Information (woo?) including those pesky 0xDB encoded files ;)
http://www.geocities.com/cyberman_phillips/Images/DIR04_1.JPG
Hex Dump of first data section of first file on screen.
http://www.geocities.com/cyberman_phillips/Images/DIR04_2.JPG
Note: These are linked because the system killed the images so there was no point in putting them in with \[IMG\] \[\IMG] usage. All Images appear to be restricted to 640 pixels across.
Cyb
-
Hi, again! It's nice to see the forums are back.
I haven't done much lately and won't be able to do much for the next couple of weeks as it's crunch time at work. :( I'll post a little more info then.
Anyway, I'll comment a little bit on your data and our differences.
F<00>000: Has some JPEG tags.
F<00>017: Begins with 00000014. The rest is 00000000.
F<00>020: Begins with 0000000C.
The other numbers match.
Were you able to solve the false positives with the DBs? Where are you having problems?
What I was able to figure out a little after the forums went down:
- Some of the fields I marked as unknown in the WM struct. (I still have to add textures to the ground.)
- Some other WM related data in DIR03. (Model for little Zidane, Chocobo, etc; textures for the ground and text/dialog for the WM.)
- Partial in-game text encoding.
- Some parts of the files in DIR13 that could be used to further subdivide them. (Not complete.)
DIR02 has only text (and some tables to index the text). Each file contains the dialog to a section of the game:
- F<02>000: Intro, theater play (the one about the canary).
- F<02>001: Intro, Vivi in town and Steiner looking for the princess.
- F<02>002: Escape from the forest.
- F<02>003: Ice Cavern and Dali.
- F<02>004: Lindblum.
... etc.
There's a lot of encoded text in other directories, including DIR00.
In the DB structures, text blocks (or files) have "type" 0x06.
I looked at the wiki info on FF9, and I have some comments, as some info is not what I have:
- In the "Root Directory Header", the "Signature" field ends in 0x00, but in the FF9.img I have it is 0x20. (The padding for the Root Dir is the same.)
- In "Sub Directory Information --> File Size information" it says:
The last file is a bit different you need the sector of the next sub directory file list and subtract from that the first sector of the file you wish the size of.
Maybe I'm reading it wrong, but the way to get the size for the last file is quite similar to the previous files, as the last element in the structure has 0xFFFFFFFF in the "Flags" field and the "First Sector" field points to the first sector after the current sub-dir (which is the same as the first sector of the next sub-dir.) I forgot to mention that in my previous posts.
Some other thing I'd change (feel free to ignore this) to the wiki is the description for "Directory Types" 0x03; I wouldn't use the word "Heirarchical", but perhaps "Indexed" or something like that, as there are some blank entries (0xFFFF) that make me think there were effects/summon data left out, but the place was kept to access every file by an index that wasn't easily changed in the application. (I simply don't see the hierarchy.)
-
:D
1) JPEG files are unlikely to be in a PS1 game.
2) my computer crashed so I'm waiting to restore my files (sigh). To be precise the Hard Disk expired mostly. :)
3) I'm trying to not scream a lot. From frustration with crashing software. I think my MB is partly to blame. Life goes on!
4) DIR02 is likely script data. This means FF9's engines is a bit different in how it organizes it's data compared to FF7.
Since FF9 is a linear progression game it means that it's likely the script and text for each part of the game progression is located in DIR02 in the order they appear. That also means that you should find the tutorial things that appear during the game in there as well.
If you find there is more data than text then it's script if not the script is hidden elsewhere. It would be good if you could discover this though :D
I suppose I need to restore my code ASAP :P
Hopefully I can cool my dead HD brain a bit to do so :)
Cyb
-
Sorry to hear about your computer crashing. :( (That reminds me to backup.)
1) JPEG files are unlikely to be in a PS1 game.
Yes, now I think so too, but the tags are there and I wasn't able to decompress any image. Perhaps they were added while translating the game. Anyway I don't mind too much as your file is almost empty and I couldn't find similar tags anywhere else.
4) DIR02 is likely script data. This means FF9's engines is a bit different in how it organizes it's data compared to FF7.
Since FF9 is a linear progression game it means that it's likely the script and text for each part of the game progression is located in DIR02 in the order they appear. That also means that you should find the tutorial things that appear during the game in there as well.
If you find there is more data than text then it's script if not the script is hidden elsewhere. It would be good if you could discover this though Cheesy
You are right, the text follows the script order and there are tutorials, but no, there's no script. Well, I think there is not enough room for a script language there.
The text strings include some control characters (<cr>/<lf> like) and argument placeholders (similar to %s from printf.) There are a lot of characters I haven't looked up, but they appear sparcelly and seem to be punctuation marks or special graphics.
Before the text there is a table with pointers to every string in the file. Each entry has some attributes that as usual I don't know what they mean, and they could reference a script somewhere else; but it seem unlikely as there are a lot of repeated values.
So, the script must be somewhere else. I was thinking the Field directory (DIR04), the World Map 2 (DIR03), DIR01 or DIR00.
The text/dialog should be referenced in the script by a numeric index and assuming the state machine used follows the script, a lot of those consecutive indexes should appear in short distances. That could help automate some search, but I'd prefer to fill-in some easier blanks first.
Rereading this comment I just remembered Zande had mostly all of this figured out long ago. :) I'll have to check his comments again.
-
You know, you should really start nameing these directories. It will make things a little more standard and easier to read.
I mean it can't be any different than FF7's
INIT
ENEMY
WM
MINI
-
You know, you should really start nameing these directories. It will make things a little more standard and easier to read.
I mean it can't be any different than FF7's
INIT
ENEMY
WM
MINI
It's a thought save for the fact the directories are offsets from a table.. and the files are offsets from a table. So there are no names readily available. Also the index value cannot be directly associated to a name unless one memorizes what the index value of a name is. My memory is crap of late, so I know I would constantly have to use a little sheet irreguardless of the names.
When we finish hashing out the information contained in each directory I suppose it would be more functional to do it that way. As yet we are acertaining what files are where and such. It would be nice if the scripting were the similar to FF7 and FF8's format.
Cyb
-
I agree with both of you. :)
Naming dirs and files can help people understand where things are located and such, but as Cyberman pointed out it is too early to do that. I'd suggest to simply create tables mapping the numbers to a short description so people interested can use them and don't get lost. But I'll keep on using numbers until things are a little more clear. (Anyway I wouldn't mind explaining something without the numbers if necessary.)
I think the table Zande provided for the directories is a good start. Now that I can read some of the encoded text, I'm doing a similar table for files in DIR00 in my free time (not much).
For those interested in reading the encoded text I found, I'll post this table:
0 1 2 3 4 5 6 7 8 9 A B C D E F
+---------------------------------------------------
0| 0 1 2 3 4 5 6 7 8 9 + - = * % SPC
1| A B C D E F G H I J K L M N O P
2| Q R S T U V W X Y Z ( ! ? " : .
3| a b c d e f g h i j k l m n o p
4| q r s t u v w x y z ) , / º ~ &
5|
6| á Ã
7| ó ú
8|
9|
A|
B| ¿
C|
D|
E|
F| \n
SPC=SPACE (0x20)
\n=New line
It's not complete, as I've added only the bytes I verified, but it should be enough to read some text; especially the dialog, some debug messages, and the text from the menus.
The row number is the most significant nibble in a byte and the column number is the less significant nibble. You'll have to convert that byte to the character inside the cell in those coordinates.
It's not too difficult to get the position for other characters as they can be found in some of the TIM images of fonts.
I already know how to get the image for each individual character from the font image, but I haven't looked too much into it so I'll leave that for when I have more time.
If anyone feels like completing the table I won't mind. :D
A note: there are probably similar tables for other fonts.
Besides that, I have a description of what I know is inside of F<00>013. It's interesting as I was able to recognize most elements from the first half of the file, but I think I won't be able to know what's inside the other half just by looking at it.
File F<00>013 seems to be the "load/save game" menu. Just to give some bad news to Cyberman ;D, I wasn't able to find a header nicelly describing the components, plus I think it's the same thing for all DIR00 files. My only hope is that those elements are pointed to from a script.
I won't give the offsets of my version, as they'll surelly be different for the English version; or any other language for that matter.
(4 bytes) File identifier? (=0x00000014)
(4 bytes) Unknown (=0x00003030)
(8 bytes) String "/DISCO" padded with 0x00. (Translation: "DISK")
(4 bytes) Unknown (0x00)
(2 bytes) Unknown (0x7e)
(2 bytes) Unknown (0xe0)
(4 bytes) Number of pointers to strings that follow. (=0x3a)
Pointer to string:
(2 bytes) Number of bytes (relative to offset of first pointer) till the first char of the string.
(2 bytes) Unknown. (Attributes of string?)
... (Repeat for every pointer.)
Encoded string. (Variable length.)
... (Repeat for every string.)
(256 bytes) CLUT for Zidane. (entries=128, 16bpp, size=256 bytes)
(1472 bytes) Picture of Zidane's face. (width=32, height=46, size=1472 bytes)
(256 bytes) CLUT for Vivi. (entries=128, 16bpp, size=256 bytes)
(1472 bytes) Picture of Vivi's face. (width=32, height=46, size=1472 bytes)
(256 bytes) CLUT for Garnet. (entries=128, 16bpp, size=256 bytes)
(1472 bytes) Picture of Garnet's face. (width=32, height=46, size=1472 bytes)
(256 bytes) CLUT for ***. (entries=128, 16bpp, size=256 bytes)
(1472 bytes) Picture of ***'s face. (width=32, height=46, size=1472 bytes)
(256 bytes) CLUT for Steiner. (entries=128, 16bpp, size=256 bytes)
(1472 bytes) Picture of Steiner's face. (width=32, height=46, size=1472 bytes)
(256 bytes) CLUT for Quina. (entires=128, 16bpp, size=256 bytes)
(1472 bytes) Picture of Quina's face. (width=32, height=46, size=1472 bytes)
(256 bytes) CLUT for Eiko. (entries=128, 16bpp, size=256 bytes)
(1472 bytes) Picture of Eiko's face. (width=32, height=46, size=1472 bytes)
(256 bytes) CLUT for Freya. (entries=128, 16bpp, size=256 bytes)
(1472 bytes) Picture of Freya's face. (width=32, heigth=46, size=1472 bytes)
(256 bytes) CLUT for Amarant. (entires=128, 16bpp, size=256 bytes)
(1472 bytes) Picture of Amarant's face. (width=32, height=46, size=1472 bytes)
(256 bytes) CLUT for Marcus. (entires=128, 16bpp, size=256 bytes)
(1472 bytes) Picture of Marcus' face. (entries=128, 16bpp, size=1472 bytes)
(512 bytes) CLUT for "passing pages of a book". (entries=256, 16bpp, size=512 bytes)
(5808 bytes) Picture of "passign pages of a book". (width=176, height=33, size=5808 bytes)
... (Rest is unknown at the moment and is barelly half the file.)
The *** above means I don't remember the name of the protagonist, but if I'm not wrong it's Dagger. I really don't remember if Garnet changed names and the picture of her face changed in the menus as well.
The "passing pages of a book" is, as I mentioned before, the animation of the book that appears when the game is opening a memory card.
After that image I don't know what's in there. That may be a good place to look for some script or compiled code but I don't know machine code nor assembly for the psx and neither do I know ff7 nor ff8 scripting language. So, if anyone could look a little bit into it, please... :)
It's quite homogeneous and if it was a 8086 architecture I'd go for code, but I might as well be completelly wrong.
-
It's not bad news that's good news. I suspected DIR00 of being odd. I believe this to be splash screens and menu things for the game now. (MENU? :D)
In summary things are potentially better because of this.
Hmmm I believe there is a font that looks similiar to your matrix coding for text.. I'll have to nose around for it.
The game has a script somewhere, but where is the question, finding the script data is likely going to be troublesome I suspect.
It may be in DIR02 but not with the text... only time will tell.
Erstwhile I am looking at cooling my hard disk and trying to get the data off of it (or at least Hoping too).
Cyb
-
It's not bad news that's good news. I suspected DIR00 of being odd. I believe this to be splash screens and menu things for the game now. (MENU?)
I was just kidding about the bad news, but I'd prefer headers in the files. I agree with the MENU designation but there can be some exception to it.
It may be in DIR02 but not with the text... only time will tell.
All the files in DIR02 have this structure:
(4 bytes) DB structure with one sub-element. (Always DB 01 00 00)
(3 bytes) Pointer to first (and only) sub-element in DB struct. (Always 0x04)
(1 byte) Type of sub-element. (Always 0x06. Text array?)
First sub-element:
(4 bytes) Unknown. (Always 06 01 00 00)
(4 bytes) Unknown. It's incremented by 1 or more units with each file.
(4 bytes) Unknown. (Always 08 00 00 00)
(4 bytes) Pointer to first byte after last string. (Pointer to slack space.)
(4 bytes) Number of string pointers.
Pointer to first string:
(2 bytes) Number of bytes (relative to offset of first pointer) till the first char of the string.
(2 bytes) Unknown. (Attributes of string?)
... (Repeat for every pointer.)
First encoded string. (Variable length, ends in 0xFF.)
... (Repeat for every string.)
(0x00 padded to complete last sector.)
As you can see they are quite simple. Every file is "text oriented" (meaning they define arrays of strings and not much more.) As I said before, the only relation to a script can be defined in the attributes or by some special character inside the strings, but there aren't many examples where that's possible; and there is no chance to find a complex script there.
---------------------------------------------
Mmmmmm.... Just a correction, as I messed up a bit with the description of F<00>013 (save/load menu). I said I couldn't recognize the second half of the file and thought it could be code, but from a comment by halkun I realized I was missing something. I opened a memory card image and found the first 256 unknown bytes into it. (So much for code. :oops:)
-
I think I found assembler code in F<00>013 (load/save game menu), and I mean "valid" assembler code; but it would be great if someone with experience on the psx could tell me if I'm making any mistakes.
I made some quick routines to disassemble the portions of F<00>013 that I couldn't classify and I found something like this at the very begining:
* ADDIU $sp, $sp, 65504
...
JR $ra
ADDIU $sp, $sp, 32
* ADDIU $sp, $sp, 65496
...
JR $ra
ADDIU $sp, $sp, 40
* ADDIU $sp, $sp, 65512
...
JR $ra
ADDIU $sp, $sp, 24
* ADDIU $sp, $sp, 65488
...
JR $ra
ADDIU $sp, $sp, 48
* ADDIU $sp, $sp, 65464
...
JR $ra
ADDIU $sp, $sp, 72
* ADDIU $sp, $sp, 65472
...
JR $ra
ADDIU $sp, $sp, 64
* ADDIU $sp, $sp, 65448
...
etc.
Now, if I'm not mistaken, those should be typical entry/exit points for functions, as the instructions marked with an asterisk reserve certain amount of memory on the stack and the one after the "JR $ra" frees the same amount of memory.
I tried with other files from DIR00 and got similar results.
-
I was bored and decided to play with Tile Molester, I found the Change Disc Images & Game over screen...
*Note: I used Zidane_2's ff9_extractor and db_extract programs to tear apart my ff9.img...
I redownload img and db extractors to:
http://zidane_games.webhost.ru/FF9_TOOLS.rar
And upload new version of the "battle scene viewer":
http://zidane_games.webhost.ru/New_viewer.rar
In new version i fix old bug with textures ))
Archive contain old and new versions and 2 files for example.
Dir 01 file 2 (Standard 16bpp TIM File):
Game Over Screen
Height: 224px
Width: 320px
Dir 01 files 7-10:
Change Disc 1-4
...after a little cut and paste...
Height: 224px
Width: 320px
...One thing I noticed is that the right half is missing 19 (8x8pixel) tiles from the bottom right of all of them...
(http://www.geocities.com/scmark15/gfx/dir01file07old.bmp)
*Edit .... Searched through the original ff9.img file (too big to open with tile molester) using my hex editor and
cut out about 1400 bytes and pasted it at the end of the 7th file in the 01 directory saved it as a new file and
tried to view it again and the missing tiles are there now. I have no idea why it got cut off, maybe a problem with
Zande_2's ff9_extractor, I don't know C++ so I can't tell ^_^ (my original ripped 7th file is 143, 360 bytes).
(http://www.geocities.com/scmark15/gfx/dir01file07new.bmp)
Best viewed settings for change disc images...
Codec: 16bpp ABGR (1555) *Corrected*
Mode: 2-Dimensional
Zoom: 100%
Width: 20 (8x8 pixel) tiles
Height: ?????? (don't feel like counting)
The image will be cut in half with the right half of the image below the left half.
I know a little hex but I really gotta learn Visual C++ so I write my own custom TIM/file viewers.
Any recommendations on books/commented source (I like the cut, delete, paste method of learning also) that would help, I'd greatly appreciate it!
Thanks,
scmark15
-
I'm afraid not much source code exists for TIM file reading.
I have header information for the file type however.
I believe the TIM images on FF9's change disk screens are 320x224.
Since most of the TIM images that are directly readable are 16bit (this includes background images). They are very easy to decode, save the back ground images although 16 bit TIM's include palettes. I suspect they did this to make things easier programming wise.
#define _PSX_STRUCT_H_
//typedef long INT32;
//typedef unsigned long UINT32;
typedef short int INT16;
typedef unsigned short UINT16;
typedef char INT8;
typedef unsigned char UINT8;
typedef struct
{
UINT32 Magic;
UINT32 Type;
} base_tim_hdr;
typedef struct
{
UINT32 WCount;
UINT16 IMGX, IMGY; // origin of image
UINT16 Pitch; // in 16 bit words per horizontal line
UINT16 Height; // number of lines;
} base_tim_img;
typedef struct
{
base_tim_hdr INFO;
UINT32 Bytes;
UINT16 PALX, PALY;
UINT16 ZZ01;
UINT16 CLUT_CNT;
} tim_4bpp;
typedef struct
{
base_tim_hdr INFO;
UINT32 Bytes;
UINT16 PALX, PALY;
UINT16 ZZ01;
UINT16 CLUT_CNT;
} tim_8bpp;
typedef struct
{
base_tim_hdr INFO;
base_tim_img IMAGEDAT;
} tim_16bpp;
typedef struct
{
base_tim_hdr INFO;
base_tim_img IMAGEDAT;
} tim_24bpp;
#define TIM_4BPP 8
#define TIM_8BPP 9
#define TIM_16BPP 2
#define TIM_24BPP 3
#define TIM_MAGIC 0x10
// Pallette/Color Look Up Tables for TIM files
typedef UINT16 CLUT_4bpp[16];
typedef UINT16 CLUT_8bpp[256];The code I have simply will not work for whatever you are using it for (It is VCL specific).
The best method is to convert the BGR16 (note it IS 16 bit data not 15 bit don't forget this EVER as that could really mess things up if you do).
The data in all tim's other than 24bpp is 16 bit WORD sized ALWAYS and more importantly it is NOT BGR15 data EVER. I repeat it WILL be a big mistake if you assume that. Why? Bit 16 is the ALPHA bit or mask bit. IF it is set be VERY wary of the data that's associated with it. It can indicate any of 4 modes of transparency and this IS used in FF9 background data. FF9's background data is 16 bit TIM information, however they use a custom method to decode the data in the TIM (IE it's internally not 16bit data :D).
My suggestion is start with the data you have the header information I've included should be enough to get you started and look at the wiki (http://wiki.qhimm.com/PSX/TIM_format)for further details on TIM images.
-
Thanks Cyb!
I was just messing around with Tile Molester and found them accidently, they weren't found
with any TIM viewer/scanner (Yuri 0.99e, Tim Viewer, PSXMC, etc...) that I had.
I realized that they were actually 16bpp after when I viewed them the second time after reading
qhimms wiki on TIM files with the extra 1400 (approx) bytes pasted at the end (to see if the
missing tiles would appear). I just found them by accident with the 15bpp BGR (555) codec setting
first, they display the same (or there is an un-noticable difference). In Tile Molester they are not
colored correctly at 16bpp BGRA (5551) or any other display codec but at 16bpp ABGR (1555)....
I'm not including the 15bpp (555) setting ^_^.
Just thought that others may wish to view them ^_^.
---------------------------------------------------------------------------------------------------------------------------------------
I have Visual Studio (6 & 2008 .Net) as well as Borlands C++ & Delphi (My old mans a
programer/systems analyst). I have very little programming experience (I know HTML, PHP and a
little hex) but want to learn C++, I've played with MS Visual C# .Net and Visual Basic as well as
Borlands Delphi but have no idea where to start with C++, like should I start with Borland C++ or
MS Visual C++, what is the difference. Any recommendations on language and/or books for
beginners would be greatly appreciated.
!!Thanks again!!,
scmark15
---------------------------------------------------------------------------------------------------------------------------------------
PS Cyb,
Thanks for the code! I'll save it for later.
-
A few Things VCL (BC++) makes things very easy. However MS created C# to countermand this concept (IE that coding for there SDK is a pain in the arse). It's now 6 of this half dozen of the other. It may or may not be easier. the important thing is to use what you have examples of that you can make work. BCB was what worked for me. (It really is Rapid Application Development) If your father (petrarch), can help you with it that's even better.
I used VCL because I have little patients to build wheels over and over again (VC++). C# is a bit easier ... a bit. It does have a large learning curve. Also it is not C++. My opinion on it aside, it is what it is. Don't be worried I guess I'm getting old and inflexible :D
Anyhow try one thing and stick with it, don't use Visual Basic.. too many limitations, it's for business oriented applications not something that would require some pretty precarious methods of getting at data. C++ or C# is a better bet.
Cyb
-
Thanks Cyb! I picked up some books that my father recommended, he also recommended some webpages that show examples and source. Unfortunently though he's grown accustomed to working with Microsoft Products (the company he works for has deals with microsoft, one of the CEO works for MS also) but he told me to start with Borland C++ and then give Visual C++ a try a little later down the road, and he also recommended a while back to learn C# (that's why I dabbled with it).
Anyways...Time to crack open some books
Thanks again Cyb!
Scmark15
-
Some info:
Dir 0:
File "1" - Field Module
File "2" - Battle Module
File "3" - WORLDMAP Module
File "4" - TetraMaster module
File "5" - ???? module.
File "6" - ???? module.
File "9" - ???? module.
File "14" - ???? module.