Author Topic: FF7 Crisis Core - File Format and Data Investigation  (Read 109195 times)

koral

  • Guest
--------------------------------------------------------------------------------------
[time saver]

All the findings and knowledge is being wiki-fied here: http://wiki.qhimm.com/FF7:CC

Tools to view/extract Crises-Core data and Models:


Everything else in this first post is old and probably will become obsolete.
However I have kept it mostly intact because of nostalgic value, as it was my first post here at Quimms!  :lol:

HUGE thanks to everyone here on the Quimm forums for your support!
It's all for you guys, enjoy!  :-D

[/time saver]
--------------------------------------------------------------------------------------




Not much seems to be discussed about FF7:CC regarding its files and data, so I thought I'd look into things myself and maybe help to get the ball rolling  :-D

Thanks to G for his FF7CC extractor, I quickly browsed through the 8000 odd files it churned out and noticed some really interesting things; I'm not sure how the extractor works, but same file-types (having same or similar header data or possibly function) are grouped together via their filenames.

(EDIT: Get G's-Extractor from here: http://forums.qhimm.com/index.php?topic=7053.0 also uploaded here if anyone can't get it from there: http://www.mediafire.com/?nwm24zynjmk )

Taking G's extractor filenames as reference, here's what I've discovered [these are the files output by his extractor, eg "00001.raw"]:

(NOTE: these are my initial thoughts ONLY, I may be wrong on several accounts, which is why I'm posting here amongst all the elite FF r-engineers, quimm and mirex deserve special mention at this point  8-) )

(NOTE2: When I refer to "Header:" I don't mean actual byte values, just some common ASCII text contained within the first 16 or so bytes of the files)

(NOTE3: I didn't look through ALL the files, just randomly poking around and narrowing the boundries; I may have missed some clusters or mistyped some filename values)


00000 - 00005   possibly misc system files, nothing too important
00006 - 00062 Header: "PSMF001" - PSP format Movies / Video files
00063 - 01349 Header: "RIFF" - Audio files
01350 - 01417 Header: "SSCF" - more Audio files
01418 - 01442 Menu and DMW related stuff (I think)
 Mix of two types of files:
    Header: [seemingly random] - files contain bunch of indexed Long's or something, no idea
    Header: "IMG" - guessing they're image / Texture files
01443 - 01445  standard PNG images
01446              Now this is interesting - seems to contain a list of Locations and Mission Titles!
01447              Contains a long list of character names, probably used when talking to NPC's etc
01448 - 01455  No discernable header (pretty much random). Maybe mesh data?
01456              Header: "MBD" - Indices or Counts of something
01457 - 02011  Story progression and Event data
  Mix of two types of files:
    Header: "GT"  - Chapter End Images
    Header: "ATEL" - LOTS of these, contain various Textures, dialogue scripts, etc
02012 - 02457 Models: Characters, Monsters, NPCs and possibly also Locations
  Mix of three types of files:-
    Header: [seemingly random]
    Header: "!" - contain single Textures for each Model. Maybe mesh-data too, but haven't found any.
    Header: "&&"
02458 - 03352  Tiny files, no common header, no ascii info visible. No idea.
03353 - 08538  unknown, no common header, fairly small files <500 kb. Ditto
08539 - 08622  Monster / Boss Data!
   Header: "MOT_" - later also contain "SSCF" data later. Interesting...
   maybe "file-names", Event Scripts, Boss Battle scripts, Monster behaviour or something else. Its all there, but no idea what.
08623 - 08627  Header: "zack_" - later also contain "ATEL" data
08628 - 08690  Materia (Magic) effect files!
  Header: "MAGIC_W0"   
08671 - 08706  unknown, tiny files
08707 - 08725  Header: "PSMF0015" - more Videos


So there you have it, my first two cents for the community here.

I'm mainly interested in the Character Model data myself, having already written my own viewers for FF7 and FF8 field and battle models (now looking to fry bigger fish :-P ) but that doesn't mean I'll ignore everything else possibly important to understand the game better (Maps, Events, etc).

Anyone care to help?
 :-D






EDIT:
Here'll I'll try keep all discovered File-Formats listed in an organised way, with a description of what they are and how to extract their contents:

==========================================
FF7 CC File Format Information Library
==========================================
LAST UPDATED: 15 MAR 2009


See: http://wiki.qhimm.com/FF7:CC



Stuff below has yet to be archived (much of it incomplete):

---------------------------
ATEL = Event data, story and game progression
---------------------------
- RAW Files:
- Not much else known about , except that they contain various different File-Type chunks:
Code: [Select]
    MBD    - Script (dialogue) Font table
    FEP     - ?
    ANM    - ?
    TEX     - Textures (fully decoded)
    MDL     - ?
    VTL     - ?
    [ASCII]- ?
    &&      - ?
    SSCF   - Audio (dialogues and/or music)

ignitz
Quote
The scripts is in this files ATEL headers and some other I don't remember
Inside these files have headers MBD in after these have the scripts then in Japanese and American have a table of 4 bytes
Here this table in format Thingy
80000000=A
80000100=B
80000200=C
80000300=D
80000400=E
80000500=F
80000600=G
80000700=H
80000800=I
80000900=J
80000A00=K
80000B00=L
80000C00=M
80000D00=N
80000E00=O
80000F00=P
80001000=Q
80001100=R
80001200=S
80001300=T
80001400=U
80001500=V
80001600=W
80001700=X
80001800=Y
80001900=Z
80001A00=a
80001B00=b
80001C00=c
80001D00=d
80001E00=e
80001F00=f
80002000=g
80002100=h
80002200=i
80002300=j
80002400=k
80002500=l
80002600=m
80002700=n
80002800=o
80002900=p
80002A00=q
80002B00=r
80002C00=s
80002D00=t
80002E00=u
80002F00=v
80003000=w
80003100=x
80003200=y
80003300=z
80003400=0
80003500=1
80003600=2
80003700=3
80003800=4
80003900=5
80003A00=6
80003B00=7
80003C00=8
80003D00=9
80003E00=!
80003F00=?
80004000=
80004100="
80004200='
80004300=,
80004400=.
80004500=:
80004600=;
80004700=--
80004800=/
80004900=(
80004A00=)
80004B00=$
80004C00=<
80004D00=>
80004E00=%
80004F00=[
80005000=]
80005100=YEN
[email protected]
80005300=#
80005400=_
80005500=|
80005600=^
80005700=º
80005800=-
80005900=+
80005A00==


---------------------------
! = Model Data (only textures found so far)
---------------------------
- RAW Files: 02012 - 02457, maybe other places too
- So far only Textures have been decoded, mesh-data remains elusive

Code: [Select]
Header: 127 bytes
   0x00   6 bytes: magic hex values ( 00 00 00 00 21 27 )
   0x10   DWORD: offset to [unknown chunk 1]
   0x14   DWORD: offset to [unknown chunk 2]
   0x1C   DWORD: offset to [TEXTURE Chunk]
   0x34   DWORD: offset to [unknown chunk 3]
   0x3C   DWORD: offset to [unknown chunk 4]
   0x40   DWORD: offset to [unknown chunk 5]
   0x48   DWORD: offset to [unknown chunk 6]

[Unknown Chunk 1]
   (seems to always start at 0x80)

   for each data-chunk (32 byte-block):
   - 1 short    always: 00
   - 1 short    bone id?
   - 2 bytes   always: FF FF
   - 1 byte    sometimes: 00, sometimes: 01 -- maybe polygon type (quad / tri)
   - 1 byte    always: 00
   - 1 long    always: 00
   - 4 bytes    ???
   - 4 floats    polygon normal?

[Unknown Chunk 2]
   Possibly Vertex data

   Header: 32-bytes
   - 1 long (0x00)  = number of polygons (data-blocks) in [Unknown Chunk 1]
   - 1 long (0x04)  = offset to data start

   Data: ?


[TEXTURE Chunk]
   header: 16-bytes
0x00  = SHORT: image width
0x02  = SHORT: image height
0x08  = DWORD: offset to PALLETE-data start offset
0x0c  = DWORD: offset to PIXEL-data start offset

   PALLETE data:
255 4-byte [RGBA] chunks - Texture Colour Pallete [255 entries]

   PIXEL data:
bytes referencing to Pallete-Entry


[Unknown chunk 3]
   ?

[Unknown chunk 4]
   ?

[Unknown chunk 5]
   ?

[Unknown chunk 6]
   ?


==========================================
[eof]
==========================================

to be continued...
« Last Edit: 2009-05-20 20:22:42 by koral »

Zande

  • Cool newbie
  • *
  • Posts: 55
  • Karma: 0
  • 友情は武器よりも強し
    • View Profile
Re: FF7 Crisis Core - File Format Investigation
« Reply #1 on: 2009-01-21 08:27:10 »
Only thing I can remember right now is that SSCF chunks are audio samples.
Seems I have a different naming of the files, so..


edit:
Quote
00063 - 01349
Header: "RIFF" - possibly Audio files
Oh yes, I remember those also.. they are in vaild IFF format, it's all the voice acting file thingies. The files are stored in Sony's ATRAC3 format (Minidisk etc.), if you rename the files to .wav and have an ATRAC3 codec they should play in your prefered mediaplayer (like winamp).
« Last Edit: 2009-01-21 12:44:50 by Zande »

koral

  • Guest
Re: FF7 Crisis Core - File Format Investigation
« Reply #2 on: 2009-01-21 14:14:20 »
Hey cool, thanks info Zande  :-D

Now why would your file-naming be different? I used the data from a US-region UMD, but I can't imagine why there would be differences between different region versions of the game, unless theres actually differences in game-content?!
Can anyone confirm that my file-lists? I may have botched up somewhere with the ISO creation or something, although I doubt that should make a difference.

I'm going to look into some of the "potentially mesh-data" files [02012 - 02457], see what I can dig-up.

Zande

  • Cool newbie
  • *
  • Posts: 55
  • Karma: 0
  • 友情は武器よりも強し
    • View Profile
Re: FF7 Crisis Core - File Format Investigation
« Reply #3 on: 2009-01-21 14:45:15 »
Nono, there's no difference between the game data, it's just the way the files are extracted, and most precisly, the naming of the files extracted... The big pack of files doesn't have a filename list, hence all the numbers.. and my files are numbered slightly different (which is for I reason that's not very important). :)

koral

  • Guest
Re: FF7 Crisis Core - File Format Investigation
« Reply #4 on: 2009-01-21 19:44:57 »
Ah, OK thanks for clearing that up :)

I've had a go at parsing some of the files which I believe contian vertex data, but I don't seem to be getting anywhere. Lots of files contain lots of chunked values, yet I've found nothing yet which might be vertices.
Files [03353 - 08538] might contain something of interest, thousands of files, could be anything.
Also some files seem to contain different File-Types embedded within the same file, i've noticed some "MDB" file-types too.

Do you have any sort of information on the "IMG" files / chunks (or some way to view these PSP format images )?
I know Playstation used to use TIM image files almost exclusively, but I've got no idea what PSP images tend to be (especially for this game).

I'm guessing that Meshes would usually be located near to their corresponding Textures (or in the same file) so if I know which Texture belongs to which character / model, I could check out nearby files more thoroughly

Zande

  • Cool newbie
  • *
  • Posts: 55
  • Karma: 0
  • 友情は武器よりも強し
    • View Profile
Re: FF7 Crisis Core - File Format Investigation
« Reply #5 on: 2009-01-21 21:41:42 »
Quote
Do you have any sort of information on the "IMG" files / chunks (or some way to view these PSP format images )?
I know Playstation used to use TIM image files almost exclusively, but I've got no idea what PSP images tend to be (especially for this game).
The image palettes are 32bit format R8G8B8A8. Palettes for textures are fairly easy to notice because the alpha channel is pretty much all the time fully solid. For sprites it slightly harder as they use alot of transparency.

koral

  • Guest
Re: FF7 Crisis Core - File Format Investigation
« Reply #6 on: 2009-01-22 11:11:02 »
OK, thanks again zande  :-D

I was just looking through one of those "Atel" files, and I believe it may be a character or something, lots of different file-types embedded within.

For reference, I'll list out the Offsets and the Header (file-types) here, as well as "regions" of data-flow changes:


File: 01488
Header: Atel
Chunk Offsets:
   0x2fc40    SSCF   - Sound files
   0x43da0   FEP
   0x46f50    - Event scripts?
   0x47090    MBD
   0x47450    MBD
   0x507f0   FEP
   0x52fd0    MDL
   0x5a2c0    MBD
   0x5a7d0   FEP
   0x5b560    MDL
   0x5b6c0    VTL
   0x5b770    - script?
   0x5b790    FEP
   0x60d50    ANM
   0x60e00   TEX
   0x63240    MDL
   0x632f0      - "false" !!
   0x63310      - "true" !!
   0x633a0    MDL
   0x63600    MDL
   0x63920    VTL
   0x639d0    - script
   0x63a20    FEP
   0x6a2f0    ANM
   0x6a3a0    ANM
   0x6a5d0    TEX
   0x6ca10    TEX
   0x6ee50    MDL
   0x6efb0    MDL
   0x6f210   MDL
   0x6f530    VTL
   0x6f5e0    - script
   0x6f640    FEP
   0x74c00    ANM
   0x74cb0    TEX
   0x770f0    MDL
   0x77250    MDL
   0x774b0    MDL
   0x777d0    VTL
   0x77880    - script
   0x778d0    FEP
   0x7f3d0    ANM
   0x7f480    TEX

        .... [to be continued?]


its a big file, and I haven't even covered half the file (it got boring, and tiriing), but its clear to see a pattern of distinct data blocks, most importantly (in order they seem to occur):

    FEP     - ?
    ANM    - animation data of some sort?
    TEX     - texture data of some sort?
    MDL     - model data? guessing maybe skeleton or skin data
    VTL     - ?
    [ASCII] - script of some kind?

Vertex/face data may have been covered within the first 0x2fc40 bytes, but that's open to speculation.

Next step: Look into Texture-data (IMG and now TEX files) to try get an idea of WHAT is located where (character, backgrounds, sprites, menus, etc, etc)
fun times :lol:

koral

  • Guest
Re: FF7 Crisis Core - File Format Investigation
« Reply #7 on: 2009-01-22 17:19:53 »
(sorry for the double post, but I want to help differentiate the large content coming up from last time. won't happen again)


just been playing with an IMG file and have more-or-less got it figured, maybe around 50%

File: 01428

Contains 4 different IMG paletted textures

Header: 80 bytes

   0x4   - DWORD: offset to IMG-1 Pixel data
   0xC   - DWORD: offset to IMG-1 Pallete data

   0x14   - DWORD: offset to IMG-2 Pixel data
   0x1C   - DWORD: offset to IMG-2 Pallete data

   0x24   - DWORD: offset to IMG-3 Pixel data
   0x2C   - DWORD: offset to IMG-3 Pallete data

   0x34   - DWORD: offset to IMG-4 Pixel data
   0x3C   - DWORD: offset to IMG-4 Pallete data

Then starting with each "IMG" headed chunk:

        - 32 byte header (contains shorts for sizes etc)   [eg, for IMG-1, offset: 0x50]
        - actual Pixel-data (which pallete value)             [eg, for IMG-1, offset: 0x70]
        - pallete entries (4 bytes RGBA)                         [eg, for IMG-1, offset: 0x10070]


Textures aren't my strong point, and from what I've spotted so far, this all seems like menu-images or sprites or something, far too small and incoherent to be anything useful.

I'll have a look at the TEX data-chunks next, see how they match up with these IMG formats, maybe be able to get something useful from it.

koral

  • Guest
Re: FF7 Crisis Core - File Format Investigation
« Reply #8 on: 2009-01-23 21:58:19 »
TEX files are just like IMG images, palletized bytes of RGBA colors, not packed or encrypted AFAIK.

I haven't been able to get any nice images/textures though, maybe there's some kind of interlacing going on or something else altogether. Textures just aren't my cup-of-tea and I'm not going to waste any more time on them.

Examples of my handiwork:





Here's the structure of TEX files as far as I can tell:

-------------------------------------
TEX File Format
-------------------------------------

Header: 48 bytes [upto 0x2f]

   0x00   magic "TEX"
   0x1C   DWORD: size Width
   0x20   DWORD: size height

Pixel Data: 0x30 offset

   (Width * Height) number of bytes, 1 byte per-pixel index (to pallete entry color, below)

Pallete data: (immedietly following Pixel data)

   16 bytes of mostly zeros (don't seem important)
   255 4-byte chunks [RGBA] (per pallete entry)


next, going to investigate those "GT" files
« Last Edit: 2009-01-25 20:51:44 by koral »

koral

  • Guest
Re: FF7 Crisis Core - File Format Investigation
« Reply #9 on: 2009-01-24 14:28:16 »
GT Files are Textures too!  :|

---------------------------
GT File Format
---------------------------

Header: 16 bytes
   0x00   magic "GT"

Pallete Data: start offset = 0x10
   255 [RGBA] 4-byte chunks

Pixel Data: immedietly following Pallete [start offset = 0x410]
   256 * 256 bytes (index to pallete entry)


List of GT files I found:

   01458-01467
   01495
   01497-01498
   01563-01565
   01590-01592

Funnily enough, they were all 258 KB, 256x256 images. There might be another cluster of somewhere with larger or smaller sizes.

Definitly still something weird going on with how I'm trying to view them (image interpolation?), but IMO they look a LOT like SKY-BOX Textures, or background images of somesort.
Infact, I'd say they look more like TEXTURES than the TEX files seemed to be (which might have just been "effect sprites" or something).

Can someone please have a look into these GT files and see if they could interpret them properly? I'm VERY sure they're going to be proper images, but I totally suck with texture-data.

BTW, it really is lonely out here... I feel like I'm all on my own, even with the 200+ views this topic has had already  :?
Any help would be appreciated!
« Last Edit: 2009-01-25 20:52:33 by koral »

NameSpoofer

  • Guest
Re: FF7 Crisis Core - File Format Investigation
« Reply #10 on: 2009-01-24 18:15:25 »
I would love to help you, because I also want to rip the textures and models from crisis core. The problem is I don't have an ISO file of crisis core so theres nothing I can do *sigh*. Anyways I believe people like zande, darkwings or shinra inc. are interested in these, maybe you should contact them.

Quote
File: 01488
Header: Atel
Chunk Offsets:
   0x2fc40    SSCF   - Sound files
   0x43da0   FEP
   0x46f50    - Event scripts?
   0x47090    MBD
   0x47450    MBD
   0x507f0   FEP
   0x52fd0    MDL
   0x5a2c0    MBD
   0x5a7d0   FEP
   0x5b560    MDL
   0x5b6c0    VTL
   0x5b770    - script?
   0x5b790    FEP
   0x60d50    ANM
   0x60e00   TEX
   0x63240    MDL
   0x632f0      - "false" !!
   0x63310      - "true" !!
   0x633a0    MDL
   0x63600    MDL
   0x63920    VTL
   0x639d0    - script
   0x63a20    FEP
   0x6a2f0    ANM
   0x6a3a0    ANM
   0x6a5d0    TEX
   0x6ca10    TEX
   0x6ee50    MDL
   0x6efb0    MDL
   0x6f210   MDL
   0x6f530    VTL
   0x6f5e0    - script
   0x6f640    FEP
   0x74c00    ANM
   0x74cb0    TEX
   0x770f0    MDL
   0x77250    MDL
   0x774b0    MDL
   0x777d0    VTL
   0x77880    - script
   0x778d0    FEP
   0x7f3d0    ANM
   0x7f480    TEX

Anway what type of file is this? It is like a model file or sth? I believe MDL is shortcut of model, ANM is shortcut of animation and TEX is shortcut of texture. I see good signs now. Thanks for doing this.

Edit: I might risk myself to download the ISO from bittorrent to help you since I am also interested. What version of crisis core do you have? Can you send me G's extractor as well?
« Last Edit: 2009-01-24 18:22:56 by NameSpoofer »

Zande

  • Cool newbie
  • *
  • Posts: 55
  • Karma: 0
  • 友情は武器よりも強し
    • View Profile
Re: FF7 Crisis Core - File Format Investigation
« Reply #11 on: 2009-01-24 20:08:39 »
The GT files are chapter images or whatnot (I have no idea myself is I've never played CC but my friend said something like that when I asked him a while ago), the image data's slightly interleaved.

koral

  • Guest
Re: FF7 Crisis Core - File Format Investigation
« Reply #12 on: 2009-01-24 22:32:52 »
wow, thanks for the support guys!

my goal is to try find and extract the characters mainly, but anything else is a bonus I suppose. There's so many darn files, its going to take a while, but we'll definitily find some interesting things along the way

don't worry about things too much namespoofer, I could always upload some files for you to have a look at

but yeah, I thought those were MoDeL, ANiMation, etc too, but the IMG files have been pretty small and I don't think those files will contain much useful. They might be small monsters or something though...


zande:

those are chapter images?!  :-o
your friend's image has helped me a lot, because I've realised where the header now contains the image width and height sizes, and also how close I got to extracting it.

Here's that very image [GT file 01495] showing how the data appears if read with no modifications (ie, read like scanlines from first x first y, going across and down):



Its not really interlacing thats the problem, its how the pixels are organised for the PSP to show them efficiently. You can tell that the white pixels on the left are supposed to be the window behind Genesis, and roughly see Sephi's sword, but the pixels are totally confusing for me... :?

the GT file is here if anyone is interested: http://www.mediafire.com/?mohmdhz43zj


I'll update the first post to try organise my findings a little more, and any other information you guys discover or suggest.
I've already got a handful of extractors/viewers made for these files, but I'll upload them for everyone once they're perfect (not good enough yet, as you can tell by my screens).

I've stumbled across another potential group of character/model files which also contain palleted textures.
The pixel-data is probably organised in the same way everywhere, so if we can crack this, I'm sure ALL the other textures will also fall into place nicely too!

« Last Edit: 2009-01-25 20:53:25 by koral »

Zande

  • Cool newbie
  • *
  • Posts: 55
  • Karma: 0
  • 友情は武器よりも強し
    • View Profile
Re: FF7 Crisis Core - File Format Investigation
« Reply #13 on: 2009-01-24 23:27:22 »
I should've written also:

For the GT files, the 4th byte in the header tells the colour depth format, if this byte is 3 the palette have 16 colours and 1 nibble per pixel. If the byte have a value of 4, there's 256 colours and 1 byte per pixel. All GT files are the later format anyway (but not all textures).

The 5th byte in the header I'm unsure about but I think it's the pixel data format, but I haven't really look much at it. The only thing I know is that both the GT files and alot of other textures stores the image interleaved as smaller images, usually as 16x8 pixels (this would represent foramt 1).

koral

  • Guest
Re: FF7 Crisis Core - File Format Investigation
« Reply #14 on: 2009-01-25 14:49:28 »
Ammended the first post, thanks again zande  :-D

I'm getting close to deciphering the interlacing with the images, now it looks something like this:

its a lot more like how it should be, with the two distinct halves of the image really helping to make things easier to figure out

they're definitly split into 16x8 blocks (as zande pointed out) but there's some additional interlacing going on between the blocks themselves. The image is still quite literally all over the place, just need to re-order the pixels some more

This is actually my first time trying to de-interlace anything, so if any video/image experts can give me a hand then it'll speed things up a bit!
 :lol:

dziugo

  • No life
  • *
  • Posts: 1474
  • Karma: 3
    • View Profile
    • A new copy of FF7 thanks to Salk. Pack (zip/rar/etc) your saved game before sending it to me.
Re: FF7 Crisis Core - File Format Investigation
« Reply #15 on: 2009-01-25 15:53:33 »
The way I think it is:
If you number the strips (16 pixels in width, 1 pixel in height) from left to right, and from up to down like that:
Code: [Select]
  0,   1,   2,   3,   4, ...  30,  31
 32,  33,  34,  35,  36, ...  62,  63
 64,  65,  66,  67,  68, ... 126, 127
128, ................................
.....................................

On the image it should be placed like this:
Code: [Select]
   0,   8,  16, ... 240, 248, | 256, 264, 272, ... 496, 504
   1,   9,  17, ... 241, 249, | 257, 265, 273, ... 497, 505
   2,  10,  18, ... 242, 250, | 258, 266, 274, ... 498, 506
   3,  11,  19, ... 243, 251, | 259, 267, 275, ... 499, 507
   4,  12,  20, ... 244, 252, | 260, 268, 276, ... 500, 508
   5,  13,  21, ... 245, 253, | 261, 269, 277, ... 501, 509
   6,  14,  22, ... 246, 254, | 262, 270, 278, ... 502, 510
   7,  15,  23, ... 247, 255, | 263, 271, 279, ... 503, 511

So basically, each 512x8 strip on the source image is a scrambled version of a 512x8 strip on the resulting image.

Edit: Or something like that :P
« Last Edit: 2009-01-25 15:59:50 by dziugo »

koral

  • Guest
Re: FF7 Crisis Core - File Format Investigation
« Reply #16 on: 2009-01-25 20:51:12 »
I CRACKED IT!!

dziguo, your suggestion was close but it gave me weird results. It got me thinking on the right track though and I managed to solve it eventually.

The data is divided into 16x8 pixel-chunks, but is arranged in a strange sort of way:
Code: [Select]
ABCD  MNOP  YZ... 
EFGH  QRST   ...
IJKL  UVWX   ...

Then reading each chunk (in that order), grab rows of 16-pixels (like 8 scanlines, top to bottom of chunk) and plot them like so:
Code: [Select]
  1,  17,  33,  49,     5,  21,  37,  53,     9,  25,  41,  57,    13,  29,  45,  61,
  2,  18,  34,  50,     6,  22,  38,  54,    10,  26,  42,  58,    14,  30,  46,  62,
  3,  19,  35,  51,     7,  23,  39,  55,    11,  27,  43,  59,    15,  31,  47,  63,
  4   20,  36,  52,     8,  24,  40,  56,    12,  28,  44,  60,    16,  32,  48,  64


Here's some extracted images [the most important FF7 characters no-less]:

   

And my [GT]-File-Viewer: http://www.mediafire.com/?2ez4jmmmlki

I'll update the first post sometime later, probably look into some of the other texture image files (TEX and IMG) to see how the Format-interlacing differs.

Enjoy! :wink:

NameSpoofer

  • Guest
Re: FF7 Crisis Core - File Format Investigation
« Reply #17 on: 2009-01-25 20:53:43 »
theres no words to describe your greatness

keep up the work

dziugo

  • No life
  • *
  • Posts: 1474
  • Karma: 3
    • View Profile
    • A new copy of FF7 thanks to Salk. Pack (zip/rar/etc) your saved game before sending it to me.
Re: FF7 Crisis Core - File Format Investigation
« Reply #18 on: 2009-01-26 10:33:48 »
Yay. Glad you worked it out.

koral

  • Guest
Re: FF7 Crisis Core - File Format Investigation
« Reply #19 on: 2009-01-26 22:28:47 »
YAHOO!!! [TEX] Files now decoded!

Examples:
     

I'm not sure what they represent exactly, possibly just misc textures used throughout the game, for various events, locations, etc. Nothing too characteristic of exactly WHAT they belong to, but still interesting nevertheless.

The pixel and pallete locations are different (see header data in first post) but the pixel packing is exactly the same as GT files (1 byte per pallete entry).

Interlacing is exactly the same, except that the interlacing in columns (how I numbered above: 1,17,33,49,   5,21,37,53, ... ) is determined by the width of the image.
So,
if width = 512, then you get 4 interlaced columns
if width = 256, then you get 2 interlaced columns
if width = 128, then you get 1 interlaced column
(ie, divide image width by 128)

Remember that TEX files are embedded within ATEL files (and possibly other kinds of files too) so they're not as easy to simply "view" like [GT] files were.

I'm working on a unified FF7-CC File Loading program right now which would be able to find and view ANY recognisable file-format from any of the thousands of FF7-ISO extracted data files.
When I get it going, I'll be sure to upload it for y'all.

I can't be bothered with IMG files though (GUI images with weird bit-palleting) so I'm going to jump the gun and check out some textures from possible character files.

Roxas

  • Crazy poster
  • *
  • Posts: 157
  • Karma: 0
    • View Profile
    • ReBoot Revival
Re: FF7 Crisis Core - File Format Investigation
« Reply #20 on: 2009-01-27 01:47:21 »
Wow!  :-o I'll definitely be watching how this progresses. 8-)

Jonnylossus

  • Crazy poster
  • *
  • Posts: 115
  • Karma: 0
    • View Profile
Re: FF7 Crisis Core - File Format Investigation
« Reply #21 on: 2009-01-27 01:57:07 »
Great work!

koral

  • Guest
Re: FF7 Crisis Core - File Format Investigation
« Reply #22 on: 2009-01-27 23:48:13 »
 :-D
As promised, I've uploaded the first public version of my FF7-CC File-Viewer proggy. Check out the first post, the link's there right on top.

I've browsed through some of the ATEL files [01457 - 02011] but haven't spotted anything tooo important. Seems like story-event related storage mainly, especially if you consider that the GT Chapter-End images were also mixed in.
But obviously I haven't been through ALL these ATEL files, so if anyone manages to spot anything REALLY interesting then do let me know here.

So far its only been Textures, but until I find the Character Textures I'm not going to get distracted with all the other chunks in those ATEL files (sound, mesh, animations?).
Unless someone wants to help out  :-P

ShinRa Inc

  • Insane poster
  • *
  • Posts: 416
  • Karma: 0
  • The Ascension of the Ordinary Man
    • View Profile
    • http://shinrainc.org
Re: FF7 Crisis Core - File Format Investigation
« Reply #23 on: 2009-01-28 00:43:25 »
Anyways I believe people like zande, darkwings or shinra inc. are interested in these, maybe you should contact them.

Very interested, and in Dirge of Cerberus as well.  Unfortunately, my technical knowledge is somewhat lacking, so I'm not sure what I could do to help at this stage.  In the past I've mostly helped with testing out some of the viewers and editors that were developed here (Ultima, Ifalna, Leviathan, Zangan come to mind), and also with sorting/identifying the extracted data.

I'll try and keep up with what's going on as best I can here, but feel free to drop me a line on AIM if there's something you think I can help with.

koral

  • Guest
Re: FF7 Crisis Core - File Format Investigation
« Reply #24 on: 2009-01-28 20:26:45 »
Thanks for offering ShinRa, so far things seem to be going smootly though, so feel free to just sit back and enjoy  :wink:


GOOD NEWS ALL!
I found the CHARACTERS!  :-D

Not just Characters, but most (if not all) the Models too, including Monsters, weapons, NPCs, some sky-dome images and other stuff (which came out garbled, probably just sprites or particles or something).



There's a good 360 files, and what you see above is only a small selection  :-P


Some more information on how I got these:

Files: 02012 - 02457 = what I call [!]-Files

These files seem to contain a number of different chunks too, similar to ATEL files except not as obvious. I managed to decipher the offsets in the header and load the Texture-data from within each file.
So what you see above are ONLY the textures from within each of these [!] files, seemingly one texture per file.

[!] FILE
Code: [Select]
Header: 127 bytes
   0x00   6 bytes: magic hex values ( 00 00 00 00 21 27 )
   0x10   DWORD: offset to [unknown chunk 1]
   0x14   DWORD: offset to [unknown chunk 2]
   0x1C   DWORD: offset to [TEXTURE Chunk]
   0x34   DWORD: offset to [unknown chunk3]

Unknown Chunk 1
   14 (0xE) 32-byte chunks - vertex info? bones?

Unknown chunk 2
   header: 32-bytes
       0x02   DWORD: offset to data start
   data: 40 bytes chunks - vertex data? lots of data available

TEXTURE Chunk
   header: 16-bytes
0x00  = SHORT: image width
0x02  = SHORT: image height
0x08  = DWORD: offset to PALLETE-data start offset
0x0c  = DWORD: offset to PIXEL-data start offset

   PALLETE data:
255 4-byte [RGBA] chunks - Texture Colour Pallete [255 entries]

   PIXEL data:
bytes referencing to Pallete-Entry


Unknown chunk 3
mostly 0xff

The texture-data is exactly like [TEX] files (minus the TEX header) so there were no problems viewing those.
Mesh data HAS to be present in the same files too, just a matter of finding the correct chunk and understanding the face-indices to be able to render them.
Not long now :lol: