Author Topic: FF7 Crisis Core - File Format and Data Investigation  (Read 142856 times)

BlitzNCS

  • Freak
  • *
  • Posts: 891
  • Karma: 2
  • Master of nothing in particular
    • View Profile
    • My Youtube
Wow, you really seem to know what you're talking about, X-Dina! it all seems to make quite a lot of sense.
Koral, that's a magnificent job you're doing here, just look at what you've accomplished! A few weeks ago, i would have thought the textures would just be a dream, let alone the actual models! you're so close, so keep going!

Satoh

  • Insane poster
  • *
  • Posts: 386
  • Karma: 6
  • Assuming this statement is correct, I'm alive.
    • View Profile
Wow, you really seem to know what you're talking about, X-Dina! it all seems to make quite a lot of sense.
Koral, that's a magnificent job you're doing here, just look at what you've accomplished! A few weeks ago, i would have thought the textures would just be a dream, let alone the actual models! you're so close, so keep going!

I don't know as much as you think... I'm just making a lot of observations....

MrAdults

  • Guest
This thread just randomly caught my eye - nice work, koral. I thought I'd mention that the data you are looking at is bone-relative, meaning the vertex positions need to be cumulatively multiplied by bone matrices to end up with the correct base pose position. There should also be some bone index data and weight values in there with the X-Y-Z's, as per common PS2/PSP vertex blending block arrays. Though that isn't necessarily the case, if there are bone indexes in there they should be pretty easy to spot.

Hope that helps, good luck.

spidergaby

  • Fast newbie
  • *
  • Posts: 22
  • Karma: 0
    • View Profile
Anyway, nice work koral ;) continu !

ignitz

  • Fast newbie
  • *
  • Posts: 44
  • Karma: 0
    • View Profile
if I have knowleged of grafics, but I don't, then I don't have to help, but Koral make a excelent work and I hope to mek progress XD.

koral

  • Guest
I really should check here more often  :-P

ignitz, don't worry about it, I think I have almost solved this  :-D
your help with the scripts was useful, so if you want you could try looking into other things, like Monster data maybe?

X-Dina and MrAdults thanks for your observations, but I doubt that those vertices are bone-positions, or in pre-bone-heirarchy-multiplied positions because all the vertices are actually present and parsable.
But I do agree that some kind of skinning-data might also be mixed into those Type-2 vertices.

This was how Tifa looked when only Type-1 vertices are parsed:


And like this when I attemtpted to parse Type-1 vertices out of the Type-2 chunks:


As you can see, all the vertices are there in correct positions, and it's only the polygon indexing which is messed up. The vertices are actually listed as a huge Triangle list within these [!]-files, listed one after the other in approximatly 24-byte chunks.

I have discovered that there seems to be a sort of 0x1C-byte "header" before the beginning of Type-1 or Type-2 vertex groupings, with a short at 0x14 giving a count of how many Type-1 vertices follow. This doesn't seem to work with Type-2 vertex grouping "headers" though, and I am guessing that those are probably multiple "headers" listed one after another until a Type-1 vertex comes along. Or maybe skin data with bone weights.

Or something which isn't imporant at any rate. :lol:

Satoh

  • Insane poster
  • *
  • Posts: 386
  • Karma: 6
  • Assuming this statement is correct, I'm alive.
    • View Profile
Well, I'm not good at reading headers, and as such I won't even attempt it just now...

But I do have an idea about them...

IF the header is structured like this...

Code: [Select]
File sections
       vertex pool size
              type one vert amount
                            actual verts


then it could be that the type one verts are counted with the type one vert amount, and everything in the vertex pool AFTER the end of the type one's end offset is a type 2.

EX:
Code: [Select]
Vertex type one amount = offset 0x01A


Vertex pool starts at 0x010 and ends at 0x01F



0x010 This is a Type 1
0x011 Type 1
0x012 Type 1
0x013 Type 1
0x014 Type 1
0x015 Type 1
0x016 Type 1
0x017 Type 1
0x018 Type 1
0x019 Type 1
0x01A Type 1
0x01B This is after the specified size... everything after this is Type 2, even though Type 2's size isn't listed
0x01C Type 2
0x01D Type 2
0x01E Type 2
0x01F Type 2
0x020 This is after the end of the vertex pool... Everything from here on is something else...


This is all speculation...

MrAdults

  • Guest
Pre-multiplied positions would actually be parsable and appear just as your mog screenshot depicted. In the last model you show, I believe the case might be that it happens to have relatively few bone weights, so most vertices are already in the correct position. However, I still notice some verts in that shot appear out of place, like the one next to the left leg.

It seems likely that type-2 vertices are just vertices with an implicit bone weight reference. The problem is that even though the 1-weight vertices happen to be in triangle list order, that is not the "proper" way to draw them, as there is probably an index key which reference both 1-weight and multi-weight vertices according to face. Again though, totally speculating, as I have no actual data to reference.

koral

  • Guest
I guess you guys have a point :oops:
What I should do is take a couple of these files and upload them for you guys to view at your leisure, I am sure I may be missing the obvious.
"1-weight vertices" does sound like a more accurate description of TYPE-1 vertices, but I can't seem to decipher any possible indices anywhere obvious.

Right now, my spare time is going elsewhere.
I have started to Wikify all my findings into Quimms wiki: http://wiki.qhimm.com/FF7:CC
It is still far from complete, but I should manage to get it all filled up in the next few days.

Huge thanks to The SaiNt for giving me access and getting me started!
 :-D

koral

  • Guest
Sorry for the double-post, but whilst I was updating the Wiki I noticed something I should have noticed a long long time ago:

click http://wiki.qhimm.com/FF7:CC
then goto the "Miscelenous Findings" section


Quote
<TEXT>
 Materia has leveled up!
<END>

<TEXT>
 Nothing more to do, kupo!
<END>

in all ATEL files this script is present, and because the Moogle text is related so strongly with the Debug-Menu data in file 00000.RAW, the first line about the Materia-Levelling-Up can be assumed to also be linked to that.

In other words: Maybe there is a "hidden" Materia to use the Debug-Menu!

All the Materia-Data is present in files 08628 - 08690, so all we have to do now is put two-and-two together!
 :-D

MrAdults

  • Guest
Cool, I'd be happy to have a look, if you get the chance to put some binary up somewhere. My Crisis Core disc is currently at a friend's house, and I don't have an early-firmware PSP to assist in ripping the contents myself. :)

koral

  • Guest
Here are a couple of the !-Model files that you or anyone else could look at: http://www.mediafire.com/?yiyjj22nm0d

02373.RAW is Moogle-san (representing a simple character sample) and 02190.RAW is Reno (a more complex character).

The actual vertex data begins after a line containing nothing but "wwwwwwww" (or "777777777" hex)
The offset of the Texture-data is located at 0x1C from the beginning of the file, so the size of the entire block can be determined pretty easily.

Best of luck!  :-D

koral

  • Guest
Apologies for the double-post (yet again  :oops:) but I would like to share some new interesting findings.

I quickly ran through the [MAGIC] files and they all have ASCII names (or some other kind of visual tag) at the end of the files, describing what they are.

Here is a list of all the names exactly as I read them (starting from file 08628):
Code: [Select]
FIRE
FIRA
FIGA
BRIZD
BRIZRA
BRIZGA
THD
GRABIDE
ANKOKU
ULTIMA
?EMPTY?
?EMPTY?
?EMPTY?
REJNE
DREIN
GRABIDE
POISON
QUAKE
STOP
BARRIER
MABARRIER
WALL
?EMPTY?
LEAVES_WUTAI SMOKE_FALL, SMOKE_BOSS_FALL
LEAVES_SPIN, SPIN_SPIN, SPIN_KISEKI
LEAVES_HIT_TEST, HIT_FLASH, SWORD_KISEKI
?EMPTY?
?EMPTY?
HIT
HIT_01
?EMPTY?
LEAVES_GOD_HAND, STEAL_HAND, STEAL_ARM
RECOVERY, SWORD_KISEKI
JUMP_JUMP, LAND_HIT, KISEKI_SHINRA , KISEKI_BASTER, KISEKI_PRASOL
RAKURAI
FIRE_HIT, SWORD_FIRE
BRI_HIT, SWORD_BRI
THD_HIT, SWORD_THD
SHOOT_THROUGH_HIT
SWORD_KISEKI
ZENI_COIN, ZENI_HIT, ZENI_PITCH
LEAVES_STEAL, STEAL_HAND
SP_CHARGE
LEAVES_SAILESS, SAIRENT2, SAIRENT4
HIT01_DEATH
ASPIL, ASPIL_HIT, ASPIL_SIPPO
QUAKE
?EMPTY?
HIT_RECOVERY
HIT01, SEORD_KISEKI
EXPLOSION_HIT, KISEKI_SHINRA
LEAVES_TEKKEN_PUNCH
LEAVES_FLARE, FLARE_FILTER, FLARE_HIT
?EMPTY?
THDGA
LEAVES_GOBURIN_PUNCH
?EMPTY?
?EMPTY?
?EMPTY?
DESP

The "?EMPTY?" ones are just my own tags, because those are 1KB files which do not contain any ASCII text.
The comma seperated ones represent multiple ASCII texts following one another in the same file.

But everything else is exactly from the files, spelling mistakes included.

My observataions:

  • "R"'s are used instead of "L"s, as per Japanese fashion
  • Japanese words like "KISEKI" are mixed in too
  • References to "SHINRA" and "BASTER" and "PRASOL" refer to the 3 swords which Zack has during the game, namely the Shinra standard, Buster Sword and the parasol (used when on vacation at Costa De Sol)
  • The spelling mistakes suggest that these ASCII characters were typed in very quickly without much care, so they probably were nothing more than aides to the developers
  • Most of them seem to be obvious Materia magic, but others (like the "HIT" ones) are standard attacks used by Zack
  • "DESP" is probably DMW attack activation
  • "ASPIL" is probably items being used
  • There are TWO "GRABIDE" files, kindof strange, no?
  • "RAKURAI" and "ANKOKU" seem just as mysterious
  • The one called "LEAVES_WUTAI" is quite interesting too
  • "LEAVES_TEKKEN_PUNCH" :-P

These can probably be called "Zack's Battle Attacks and Actions" to be more precise, encompassing Materia as well but by no means stopping there.
Can anyone spot anything else?
 :-D

MrAdults

  • Guest
Thanks for putting those files up! I got up and running with them tonight and have textures working and have found pointers to some vertex data (with some extra vertex info, will document that once I have things working a bit more solid). I'm now working on trying to find some kind of index or skeletal data, and will let you know if I have any success.

BTW, you accidentally put up the ShinRa storage box model instead of Mog. :) However, in working with that model, I noticed the texture is 128x128, but it seems I need to treat it as though it has 2 interlace rows (just like the Reno texture which is 256x256) to decode it properly. Is this a specific thing to this one texture in this chunk, or is the "width/128" rule not reliable/correct?

ignitz

  • Fast newbie
  • *
  • Posts: 44
  • Karma: 0
    • View Profile
Apologies for the double-post (yet again  :oops:) but I would like to share some new interesting findings.

I quickly ran through the [MAGIC] files and they all have ASCII names (or some other kind of visual tag) at the end of the files, describing what they are.

Here is a list of all the names exactly as I read them (starting from file 08628):
Code: [Select]
FIRE
FIRA
FIGA
BRIZD
BRIZRA
BRIZGA
THD
GRABIDE
ANKOKU
ULTIMA
?EMPTY?
?EMPTY?
?EMPTY?
REJNE
DREIN
GRABIDE
POISON
QUAKE
STOP
BARRIER
MABARRIER
WALL
?EMPTY?
LEAVES_WUTAI SMOKE_FALL, SMOKE_BOSS_FALL
LEAVES_SPIN, SPIN_SPIN, SPIN_KISEKI
LEAVES_HIT_TEST, HIT_FLASH, SWORD_KISEKI
?EMPTY?
?EMPTY?
HIT
HIT_01
?EMPTY?
LEAVES_GOD_HAND, STEAL_HAND, STEAL_ARM
RECOVERY, SWORD_KISEKI
JUMP_JUMP, LAND_HIT, KISEKI_SHINRA , KISEKI_BASTER, KISEKI_PRASOL
RAKURAI
FIRE_HIT, SWORD_FIRE
BRI_HIT, SWORD_BRI
THD_HIT, SWORD_THD
SHOOT_THROUGH_HIT
SWORD_KISEKI
ZENI_COIN, ZENI_HIT, ZENI_PITCH
LEAVES_STEAL, STEAL_HAND
SP_CHARGE
LEAVES_SAILESS, SAIRENT2, SAIRENT4
HIT01_DEATH
ASPIL, ASPIL_HIT, ASPIL_SIPPO
QUAKE
?EMPTY?
HIT_RECOVERY
HIT01, SEORD_KISEKI
EXPLOSION_HIT, KISEKI_SHINRA
LEAVES_TEKKEN_PUNCH
LEAVES_FLARE, FLARE_FILTER, FLARE_HIT
?EMPTY?
THDGA
LEAVES_GOBURIN_PUNCH
?EMPTY?
?EMPTY?
?EMPTY?
DESP

The "?EMPTY?" ones are just my own tags, because those are 1KB files which do not contain any ASCII text.
The comma seperated ones represent multiple ASCII texts following one another in the same file.

But everything else is exactly from the files, spelling mistakes included.

My observataions:

  • "R"'s are used instead of "L"s, as per Japanese fashion
  • Japanese words like "KISEKI" are mixed in too
  • References to "SHINRA" and "BASTER" and "PRASOL" refer to the 3 swords which Zack has during the game, namely the Shinra standard, Buster Sword and the parasol (used when on vacation at Costa De Sol)
  • The spelling mistakes suggest that these ASCII characters were typed in very quickly without much care, so they probably were nothing more than aides to the developers
  • Most of them seem to be obvious Materia magic, but others (like the "HIT" ones) are standard attacks used by Zack
  • "DESP" is probably DMW attack activation
  • "ASPIL" is probably items being used
  • There are TWO "GRABIDE" files, kindof strange, no?
  • "RAKURAI" and "ANKOKU" seem just as mysterious
  • The one called "LEAVES_WUTAI" is quite interesting too
  • "LEAVES_TEKKEN_PUNCH" :-P

These can probably be called "Zack's Battle Attacks and Actions" to be more precise, encompassing Materia as well but by no means stopping there.
Can anyone spot anything else?
 :-D

This is a Debug DATA, it's normal this unique "lenguage" that some the programmer only "understand"   :evil:
The wrong translate leave to this

MrAdults

  • Guest
I'm still perplaxed by the lack of face indices - I've tried all kinds of bruteforce searches through the file for every type of index I could think of. The most promising thing I found between both files you put up were from a DWORD pointer at 0x48, it leads to an array of data that looks very much like triangle indices. However, the size and stride does not appear correct, so if it is index data, there is probably another key somewhere else into the file pointing into it. I have not tried getting offsets to the specific "faces" and scanning the file for them yet.

I have to get back to a contract project for a couple days here, so I won't have time to look further until I'm done. In the mean time, good luck! :)

koral

  • Guest
Woops, sorry for the missing Mog!  :oops:
I have uploaded it here now, along with the other two as well: http://www.mediafire.com/?yzzmoiandee

But thankyou so much for trying to help!  :-D
The file-data is really quite unusual, and as inefficient as un-indexed triangle-lists are, it seems to be the method they chose to store the data, possibly just directly dumping the contents into RAM for the GPU (although I am not so sure about the internal functioning of PSP's myself).

I haven't fully understood the texture-interlacing, but the rule of dividing it by 128 seems to work majority of the time. Zande had pointed out before that there might be different interlacing modes determined by a value in the header, so it is worth keeping in mind that my interlacing rules are temporary.

The skeletal data probably does follow on after the vertices, in [unknown-chunk-3] and [unknown-chunk-4], because the ShinRa box and the barrel dont contain any data at those points.
If I had to choose, I would say that any face-indexing is in [Unknown-chunk-1], before all the vertices.
But the values are mostly zeros, so maybe they are just initial bone rotation values?

Just do what you have to do, we are not on any strict kind of deadlines here!  :-P

ultima espio

  • No life
  • *
  • Posts: 1359
  • Karma: 20
    • View Profile
Just do what you have to do, we are not on any strict kind of deadlines here!  :-P

Yeah you are :-P :-P Only joking lol

Satoh

  • Insane poster
  • *
  • Posts: 386
  • Karma: 6
  • Assuming this statement is correct, I'm alive.
    • View Profile
Apologies for the double-post (yet again  :oops:) but I would like to share some new interesting findings.

I quickly ran through the [MAGIC] files and they all have ASCII names (or some other kind of visual tag) at the end of the files, describing what they are.

Here is a list of all the names exactly as I read them (starting from file 08628):
Code: [Select]
FIRE
FIRA
FIGA
BRIZD
BRIZRA
BRIZGA
THD
GRABIDE
ANKOKU
ULTIMA
?EMPTY?
?EMPTY?
?EMPTY?
REJNE
DREIN
GRABIDE
POISON
QUAKE
STOP
BARRIER
MABARRIER
WALL
?EMPTY?
LEAVES_WUTAI SMOKE_FALL, SMOKE_BOSS_FALL
LEAVES_SPIN, SPIN_SPIN, SPIN_KISEKI
LEAVES_HIT_TEST, HIT_FLASH, SWORD_KISEKI
?EMPTY?
?EMPTY?
HIT
HIT_01
?EMPTY?
LEAVES_GOD_HAND, STEAL_HAND, STEAL_ARM
RECOVERY, SWORD_KISEKI
JUMP_JUMP, LAND_HIT, KISEKI_SHINRA , KISEKI_BASTER, KISEKI_PRASOL
RAKURAI
FIRE_HIT, SWORD_FIRE
BRI_HIT, SWORD_BRI
THD_HIT, SWORD_THD
SHOOT_THROUGH_HIT
SWORD_KISEKI
ZENI_COIN, ZENI_HIT, ZENI_PITCH
LEAVES_STEAL, STEAL_HAND
SP_CHARGE
LEAVES_SAILESS, SAIRENT2, SAIRENT4
HIT01_DEATH
ASPIL, ASPIL_HIT, ASPIL_SIPPO
QUAKE
?EMPTY?
HIT_RECOVERY
HIT01, SEORD_KISEKI
EXPLOSION_HIT, KISEKI_SHINRA
LEAVES_TEKKEN_PUNCH
LEAVES_FLARE, FLARE_FILTER, FLARE_HIT
?EMPTY?
THDGA
LEAVES_GOBURIN_PUNCH
?EMPTY?
?EMPTY?
?EMPTY?
DESP

The "?EMPTY?" ones are just my own tags, because those are 1KB files which do not contain any ASCII text.
The comma seperated ones represent multiple ASCII texts following one another in the same file.

But everything else is exactly from the files, spelling mistakes included.

My observataions:

  • "R"'s are used instead of "L"s, as per Japanese fashion
  • Japanese words like "KISEKI" are mixed in too
  • References to "SHINRA" and "BASTER" and "PRASOL" refer to the 3 swords which Zack has during the game, namely the Shinra standard, Buster Sword and the parasol (used when on vacation at Costa De Sol)
  • The spelling mistakes suggest that these ASCII characters were typed in very quickly without much care, so they probably were nothing more than aides to the developers
  • Most of them seem to be obvious Materia magic, but others (like the "HIT" ones) are standard attacks used by Zack
  • "DESP" is probably DMW attack activation
  • "ASPIL" is probably items being used
  • There are TWO "GRABIDE" files, kindof strange, no?
  • "RAKURAI" and "ANKOKU" seem just as mysterious
  • The one called "LEAVES_WUTAI" is quite interesting too
  • "LEAVES_TEKKEN_PUNCH" :-P

These can probably be called "Zack's Battle Attacks and Actions" to be more precise, encompassing Materia as well but by no means stopping there.
Can anyone spot anything else?
 :-D

Rather than treating this as materia with other attacks thrown in, I would suggest that this is all of the battle commands... just some of them happen to be item based like materia and item usage. While this doesn't change anything, it gives a little more insight into the programmers' mindset.

an interesting note about the game that may be useful... or not, is that no matter when you do it, the fight with Minerva ALWAYS uses the Buster Sword. (This may point in some way to how scripted events work in the game at a later point.)

I finally got my custom firmware now, so at any point if you all figure out where the 'DEBUG materia' is if it exists, and you can give me an ISO offset for the memory address materia is in... I can test it out with CWCheat.

I haven't looked into the model data just yet, but I'll post anything that piques my interest once I do.

AH! and about the type 1 and type 2's MrAdults mentioned they might be 1 weight verts. This would mean that the other verts, type 2, are weighted to multiple bones. If that is the case, then we would need to figure out how the weight is displayed...

Or rather, another way to look at it... the type 1 and type 2 verts are exactly the same but the bone data for the type 2's continues after the 1st bone index.

This means, we simply need to figure out how to determine the difference between vertex data and bone relations. I imagine the bone indeces wouldn't extend past 4... and more likely would only hold 2 bones.

Now, there are 2-3 things bone indeces would need...

Vertex in question's XYZ orientation + texture UV + :
1. how many bones are weighted...
2. bone's name (probably just a number)...
3. Bone's actual amount of influence...

here is how I would set it up if I were making the format.

Code: [Select]

x,y,z,u,v,#bones,bone1,weight1,bone2,weight2

20 32 12 20-42 02 10 32
11 32

This means vertex 32, 50, 18
has a UV of 32, 66
and has 2 bones
Bone 1 is bone number 16, and has a 50% weight.
Bone 2 is bone number 17, and has a 50% weight.

(I used a 100% scale on the bone weight, because a decimal number would be harder to program. the scale goes from 00(0) to 64(100) )


This is only my thoughts on how it might be worded for efficiency. So, looking at that, the type 2 verts might be as long as 10 offsets if they only have 2 bones, and up to 14 if they had as many as 4 bones weighted.(this is unlikely, but possible...)

Type 1 verts mightn't need the bone amount mini header, and would be implicitly converted.

Still, this is all speculation- Educated speculation, but speculation nonetheless.


In short... We need to decipher the bone data.

EDIT:
Ok, Upon examining I find that the verts are separated by a series of 00 chunks... between 00 00 00 and 00 00 00 00 00 00 long... what are these for? I could speculate that the length of this padded 0 data is used in determining the type of vertex it is. I notice the very first vert begins with a 4 byte long 00 00 00 00.
(In Reno)

So, my guess is that the verts begin with the padded 0's to determine their type, and then use actual coords to determine their location. I could be wrong, but that is my first thought.
« Last Edit: 2009-03-07 16:26:31 by X-Dina »

MrAdults

  • Guest
One other thing that occurs to me is, even though the models seem to have only a single texture map, the models would need to be drawn in sorted/batching order for the purpose of alpha blending. For example, Reno's hair has a very soft-edged alpha, and as I recall playing the game, those surface did use real alpha blending instead of hard-edged alpha tests.

So, there would have to be a way to batch those verts into separate "surfaces", for depth-tested non-alpha (or hard-edged alpha) surfaces (like the whole body), and post-sorted alpha blended surfaces (like the hair). That dictates that there must be some kind of container with references to groups of verts, at least, if not an index list. Maybe if we can find these containers, it will shed more light on the vertex drawing order, or at least provide some clean absolute offsets into the vertex list.

Edit: Although, on second thought, it's possible that it was all hard-edged and I just could not tell because of the small screen and size of characters on the screen. :) My copy of the game is still at my friend's house, but if anyone has it handy, it would be worth looking to see if the character models do indeed use alpha blending.

Second edit: No, they are definitely blended. :) Take a look at Genesis's hair:
http://www.product-reviews.net/wp-content/userimages/2008/03/crisis-core-final-fantasy-vii-montage-for-the-sony-psp.jpg
« Last Edit: 2009-03-07 19:18:19 by MrAdults »

koral

  • Guest
I really like you two, speculative thinkers with the game-programmers mindset :-D
Although I have divided my reply, I hope both parts will be read and considered!

X-Dina:
Yes, understanding those "padding" values would be the key to solving this enigma. I have discovered some interesting patterns to help determine the vertex-data from the "extra" data:

  • Firstly, all "real" vertices begin with two floats in the range 0 to 1. Never entirely 0, never entirly 1. The remaining values could be anything (4 unknown bytes, 3 shorts XYZ, and 2 unknown bytes almost always zeros).
  • Secondly, following the definitive vertex tag "0000803f" there may be any number of floats with values of zero, which should be ignored as padding.
  • Thridly, if the second byte of the 4-unknown-bytes within a possible vertex is zero, then that vertex isn't a vertex (will either have a zero UV or messed up XYZ value).


These three are the most "concrete" rules, because they seem to work through all the files I have examined. The other tags which i had posted about a while back are unreliable.

Also, I noticed an interesting value at offset 0x14 (from the first byte after the "77777777777777" chunk seperator).
If you read it as a SHORT and minus one from it, it gives you the count of how many 24-byte (TYPE-1) vertices are listed starting from offset 0x1C.

So for the Moogle file (02373), the (absolute) file offset is 0x714 (local chunk offset 0x14), which gives a value of 0x42 (66). Then starting from (absolute) file offset 0x718 (local chunk offset 0x1C), you will find the first TYPE-1 vertex (tagged by "0000803f"), followed by 64 more such TYPE-1 vertices.

But the pattern breaks down after that, reaching TYPE-2 vertex data. I am sure there must be some kind of offset here too to determine where the next batch of TYPE-1 vertices begin, but it isn't easy to spot.

The Reno file (02190) has a value of 0x1 at the 0x14-offset (absolute:0xcb4), and contains zeros at 0x1c (ie, no TYPE-1 vertices), leading upto the first occurance of the "0000803f" tag (in this case it is an "embedded" TYPE-1 Vertex within the TYPE-2 data chunk).


MrAdults:
Interesting observation :-)
I know that z-sorting of transparent polys has to be done when developing for PC's, but dont games-console PPUs handle that sort of thing automatically? I am not an expert on console-hardware, so I will assume that the vertices would need to be stored in batches for optimal rendering of transparent objects.

Could we not conclude then, that we would expect to find similar transparancy-parts (eg, hair clumps) grouped together, not with any other vertices belonging to "solid" parts?
For example, for Reno, the vertices making up his hair would be distinct from the vertices from everything else (indexed or not).

The Moogle texture file is interesting.

The only alpha it contains is within its wing-segment, but I can recall that the vertices belonging to the wings were a seperate batch of TYPE-1 vertices from its head. Likewise, Moogle-san's ears and nose were separate batches of TYPE-1 vertices from each other too.

Each of those seperate parts (batches of TYPE-1 vertices) has its own UV-space on the texture-map.
Maybe they just modelled (and later stored) these model parts in such a way that the transparant-parts would be differentiated on their own just as a "side-effect" of seperating the parts to better fit the UV map?

In other words, no discreet form of "transparent-part-tagging" or indexing would have been neccessary (or present)?

Satoh

  • Insane poster
  • *
  • Posts: 386
  • Karma: 6
  • Assuming this statement is correct, I'm alive.
    • View Profile
I think I know why they were separated into different groups.

When I import a 3DS file from 3DS Max, all of the edges that correspond to separate texture chunks, are always cut open and I have to re-sew them together again.

I believe the engine that saved the files had to separate the verts so there wouldn't be any texture warping on the model when rendered.

This would mean that when read by the PSP itself, it looks solid, only because the verts are touching each other, but in actuality there are sliced open edges all over.

Can you confirm this?

As for transparency, I believe the alpha is simply drawn as solid on the first pass over the entire model, then the engine does a secondary render that multiplies the alpha to create the transparency(the transparent rendering is kept in memory and is written to the screen from there rather than directly from the disc).

Again... speculation... based on past experience.

I believe, in a shorter phrase, the entire models are treated as alpha transparent enabled unless the texture image has NO alpha at all.


As for the 0x14 data.... that's very interesting...I had a feeling there was a header that noted the size in there somewhere...


and !!! the unknown data you mentioned that is 'will either have a zero UV or messed up XYZ value' could be the bone weighting info we've been guessing about!

MrAdults

  • Guest
Yeah, it is possible that they just leave both blending and depth writing enabled, and have the verts that reference part of the uv that is actually transparent ordered after the non-transparent verts. However, they would still have to software-sort their models by distance to camera in this case, or you would have cases where you would not see another character model behind a transparent surface, as that kind of sorting can't be done on the hardware level (due to the fact that it would need to defer rasterization to support it). It also assumes the PSP GPU has blend optimizations to save fillrate on blend operations if the written depth fragment is previously clear, which it may, but I'm unsure.

In either case, it would make a lot more sense to me to, at the very least, have some kind of marker between vertices to signify a change in blend mode, unless they were very desperate to cut down on primitive batches, without regard to the cost of software distance sorting or potentially greater fill expense. However, it doesn't make a lot of sense to do a post-depth alpha blend pass if you have rendered your alpha surfaces to the depth buffer, and the information in the framebuffer is stomped on whenever a depth fragment passes, which is why the hardware itself can't support an alpha post-multiply.

I'll have a look around some of those verts that reference transparent areas of the texture map, and see if I can find anything that would be indicative of a change in blending between batches. It seems more and more like the verts are more like a "draw command list" of some sort, which would explain some of the seemingly arbitrary tagging values thrown in there. However, I don't know enough about programming for the PSP, particularly with commercial GPU interface libs, to know if there are any official display list formats this data could resemble.

halkun

  • Global moderator
  • No life
  • *
  • Posts: 2113
  • Karma: 23
  • NicoNico :)
    • View Profile
    • Q-Gears Homepage
I awake..... from my slumber....

When I helped reverse the FF8/CC models, it was no unheard of for the programmers to put GPU opcodes in the model data.

Sadly, I only know how the GPU works in the PSX/PS2. I have not have any hands on experience with the PSP.

Keep in mind the models probably were made from object trees (waist, torso, head, left_upper_arm, left_frearm, left_hand, etc). You probably need to work on parsing one body part at a time as the vertex pools are usually grouped together by body part. The Mog head was a good start.


Aurenasek116

  • Crazy poster
  • *
  • Posts: 155
  • Karma: 0
    • View Profile
All I want to say is - this is one great piece of work. I'm looking forward to this project being finished and able to fully render the models.