Author Topic: Tech issues (Read 20218 times)

ficedula · « **on:** 2001-03-22 03:41:00 »

OK. I know we're keeping most tech discussion in the open forum, but there are a few things we need to discuss related specifically to the remake. I know that most of these have probably been brought up in the public tech forum already, but I'd like to keep this discussion separate - it is, after all, 100% relevant to the remake only.

The first - main - problem we face is: game engine. For the sake of completeness (been said already, but...) we have 2 options:

1) Use the existing FF7 engine.

Pros:
-Less likely to attract adverse attention from Square - it makes our remake a patch to the existing FF7 game.
-Reuse all the existing graphics, spells, models, just adding the new ones we need as well.
-Everybody likes it!

Cons:
-We NEED to figure out *at least* the "script" part of the level files. As anyone following Cosmo will know, we currently have about 90% of the text and backgrounds decoded but nothing whatsoever from the rest of the file - and since the rest could be in absolutely ANY format, it's gonna be hell to decode. I'm not sure if it'll ever be done.

2) Use another engine.

Pros:
-We can start work NOW. No delays waiting for people to hack file formats.
-Change anything in the engine we need to.

Cons:
-Can anyone say "Copyright violation?". Sure, any remake is a breach of copyright - but a "mod" to the game is less likely to be contested. We don't need Square pressuring our website hosts to cancel accounts.
-It's not gonna look quite the same. People might be put off.
-MAJOR downloads. Gonna have to recode lots of files.

You'll probably know, I'm somewhat dubious over whether we can decode the script format. (I think we'll never decode every format in the game unless Square release source code, which won't ever happen). Bitmaps were relatively easy - there's only so many ways to store a bitmap, and Square (understandably!) reused an existing format. Same with sound effects. Text - well, you can read some of it yourself once the file's decompressed! Scripts - it's a format invented just for FF7, totally binary (or near enough) - REAL problems.

So where does this leave us?

My preference: Those of us involved in the tech side of things should try to decode the script format. Yes, it's hard, but we should *try*. If we haven't got anywhere by, say, June? then we use our own engine. There's really no other choice.

Something to bear in mind: We can code up our own engine. Why not? That has the BIG advantage that we can program it to reuse *some* of the files from FF7 - like the music, sound effects, text, and some of the 3d data (basically everything we know how to decode!). Major advantage: requires an existing install of FF7, hence making less problems over Square getting p***ed off with us. Don't think it won't happen; IF we ever release this remake (I'm not trying to be awkward here, but it's by no means 100% it'll get finished) then Square could well get stroppy.

So: comments, anyone?

halkun · « **Reply #1 on:** 2001-03-22 06:24:00 »

You have so little faith in us. (^_^)

Seriously though, use the original. I've been kicking it about now for the last week and find it a good, stable engine to work with. I've only been working with the status module right now and have 99.999% of the data structures worked out. The only thing missing is the flags, but they are only important when I start working with fields next.

Let me give you guys some advice in decoding the field files.

1) Don't be afraid to corrupt a data file to see what it does. Just remeber to keep a backup of whay you changed. The ff engine had most of its sanity checking stripped before it went gold for space. The debug rooms are your friends, it does an integraty check on every aspect of the game.

2) Debug room is your friend. Did you know that the debug room displays every type of animation in the game. Swap an .a file with another and check all nine debug rooms to see the change you made. If you can, try swapping an .a file of simialar size but differnt data. Pull out your hex editor and make a change to *one byte* in the file, record the change or make a backup and look again. Yes, you will see that edios startup movie every single time, but it works. If the change isn't apparent, try another location. Remeber to fix your original change first. You can also access any field file from the Debug room too.

3) Look at what you have, then fill in the blanks. If I had a sentence "Mary ____ to the store" what's missing? "went" works "gone" does too, kind of. If you know whats where in a data file, look to see whats left. Every thing is important in a data file, that's why it's there. Qhimms FF7 save editor marks a data memeber as "random" but I don't beleieve that is true. Unless it's truely a random seed used in things like battles so you don't wind up with the same monsters every time you load the game. I'll work with status and I bet whatever is missing, will fit perfectly in that "random" spot. (like flags)

4) Don't be afraid to ask why. When I swap cloud's model with sephs model I get seph but his long hais is messed up. Why? well seph has an extra bone for his hair. Try swapping seph with tifa, she has long hair too. Hey, I wonder if the animation is compatible...

anything else?

The SaiNt · « **Reply #2 on:** 2001-03-22 10:00:00 »

Quote

You have so little faith in us. (^_^)

I thing the only reason fice feels so is cause the only real people who actually make progress in this tech arena are Qhimm, halkun and fice himself. Everyone else, no hard feelings, ok?

About the debug room, i would suggest you guys use a debug room guide so that you'll know what each option is for. I would recommend using Andrew Dean's guide for the debug room. It's listed on http://www.gamefaqs.com" TARGET=_blank>http://www.gamefaqs.com and on my site too http://www.finalfantasyfanatics.net" TARGET=_blank>http://www.finalfantasyfanatics.net
But considering that my site has horrible international access speeds, i would recommend getting the guide from gamefaqs.

Honestly speaking, most of FF7PC's files are exactly the same as the PSX counterparts.
Since, the playstation has only THAT many ways of displaying stuff(i just know halkun is going to comment on me for this), I'm sure that finding out what Square decided to use SHOULDN'T pose any problems. Even for the file formats that the PC porters decided to change, I believe they've changed a minimal amount of things, becuase using a whole new system would only result in causing more problems.

Quote

4) Don't be afraid to ask why. When I swap cloud's model with sephs model I get seph but his long hais is messed up. Why? well seph has an extra bone for his hair. Try swapping seph with tifa, she has long hair too. Hey, I wonder if the animation is compatible...

Are you talking about switching them in the EXE or the LGP files?
Anyway, here's what I have to say. So far, switching the HRC references seems to be more likely to be possible when both the char's have the same amount of bones. Unfortunately, this isn't the only thing that has to be taken into consideration. It seems that the skeleton has different stuctures for the males and the females too. Thus when you replace seph with tifa or vice-versa, there seems to be a gap where the hips are supposed to be(IIRC).

There are two problems I've encountered while editing the HRC files.
1. If you change the RSD references within the HRC file, you will find that cloud doesn't have enough bones to attach the RSD files too, so the replaced cloud will not have Seph's hair.

2. If you decide to add the bones extra bones to the HRC file however, you will be able to display Seph's hair. Unfortunately, there are 2 complications here:-

a)If you add the reference to his hair anywhere in between two RSD file references, everthing above the reference to his hair displays correctly, but anything that precedes it will display in an akward manner. Try it, and you will see Seph floating in the air with his legs doing impossible degrees of movement.

b)There is a small solution to the problem above though. The trick is to add the additional RSD references after the ones originally listed in the HRC file. In other words, put your modifications at the bottom most of the file. This will enable the whole character to be displayed correctly and the hair polygon will be displayed as well too. The only problem is, the hair will not appear in the right orientation. It will appear at the right coordinates, but not it the right orientation. In the Sephiroth case, his hair appears like a dunce hat on his head.

This has led me to believe that there is some other information that controls the orientations of each RSD file somewhere in the .A(most likely) or .P files(possible) or FF7.EXE(unlikely)

halkun · « **Reply #3 on:** 2001-03-22 12:54:00 »

I was just making up an idea, I see you guys are a bit ahead of me on that. Now that I think about it, girls have hips, it suprises me that even with the SD characters that square will include correct body proportons even though the proporton on the whole is outta whack. wierd.

You have narrowed something down however, it would seem that the .a file would *only* contain orintation data, each devided by the frame of animation. Let me theroise a little more. Why is sephs hair bouncing on his head? obvoisly it's reading data from somewhere. I bet the program does this...

The program reads in the bone defs and finds that there are, say 9 bones on the model. The .a file looks like this: (o=orentation data)

o1,o2,o3,o4,o5,o6,o7,o8,o9|o1,o2,o3,o4,o5,o6,o7,o8,o9|....

each set of orintations represents a frame of aninmation. This means all you have to use is a simple FIF0 buffer for the animation. What is happing is say that seph has 10 bones, not 9, because of his hair. When the data is coming from the o1 of the next frame (slight overrun) ooh, this might be a breakthrough.. (^_^)

-halkun (still working in status)

The Skillster · « **Reply #4 on:** 2001-03-22 18:20:00 »

fice, u said desode the game script, how? where? when? like where is it (most likly in the EXE) but where abouts in the EXE?

Qhimm · « **Reply #5 on:** 2001-03-22 22:36:00 »

The game script is not stored in the EXE, except for the initial reference to the starting map. The scripts are stored in the field files, and finding out which of the sub-blocks is the actual script shouldn't be that difficult, a lot of it would be text-display bytecodes.

ficedula · « **Reply #6 on:** 2001-03-22 22:47:00 »

Qhimm's exactly right.

The "script" for each location is stored in the same file as the other location-specific data, i.e. one of the files in FLEVEL.LGP. Each of those "location" files contains:

-Text (Cosmo)
-Background graphics (Cosmo, Gast)
-A "script" for how objects act, when to trigger text, etc...
-Possibly other data

SaiNt has also said pretty much what I meant; I'm not intending to slag anyone off here (including myself!) but decoding the script *is* harder than doing graphics, text or sound, plus there aren't that many of us experienced in programming & hex editing.

Like I said, we should *try* and decode the scripts. I'm just pointing out it's by no means 100% certain we'll succeed!

Sephiroth 3D · « **Reply #7 on:** 2001-03-22 22:48:00 »

I'm lost, but I want to help.

Could someone find or make me a tutorial or somethng so I could learn how to do the stuff you guys do so I can help? I probably have most the programs I need, but a list of those should also help.

Thanks!

On the scripts: Since they are all in the field, my guess is that there are triggers within the fields seperating the text which say what and what happens to activate this, and in between lines, it tells who's talking, plus what animation of theirs to play.

Sephiroth 3D

"One who seeks knowledge from another person, doesn't learn half as much as the one who seeks knowledge for himself." - Vincent Valentine, The Sephiroth Chronicals, Book 1: Resurrection

[email protected]
http://www.sephiroth3d.com" TARGET=_blank>Sephiroth 3D's Final Fantasy
http://members.tripod.com/~EmperorTim/index.html" TARGET=_blank>Sephiroth's 3D Lair
http://members.tripod.com/~Sephiroth3D/index.html" TARGET=_blank>Sephiroth 3D's Promised Land

The Skillster · « **Reply #8 on:** 2001-03-22 23:23:00 »

im getting abit lost, how about if we do it like this?
someone figure out where the script MIGHT start, tell me which file they have seen it in, and ill get onto it and make allsorts of hex editing experiments with it to dectate if a can CHANGE an event?

ficedula · « **Reply #9 on:** 2001-03-23 00:41:00 »

Seph3d: Unfortunately, all the text in a location file is stored in one big block with nothing between the various text items. The script has to be stored somewhere else in the same file. Obviously it'll *reference* the text, but it's not stored *in between* the different text items, if you see what I mean.

On a side note, Cosmo 0.76 uploaded. Give it a go (assuming Tripod is working...)

halkun · « **Reply #10 on:** 2001-03-23 07:32:00 »

I wrote something a while back when I was poking at the ff8 file formats. I have to go to work now, so loo for it.

The SaiNt · « **Reply #11 on:** 2001-03-23 07:56:00 »

The scripts are definately in the field files.
Which section?
I'm not too sure about that.
But we could try to take a look at the PSX version of the field files first. I'm sure the PSX programmers purposely seperated the files to make it easier for them to change something and not have the chance of messing up something too easily.

Quote of what I said earlier

Quote

One more thing, they also defined these file extentions(i think) for flevel.lgp
.mim (map backgrounds)
.map (This is some PSX format I believe)
.ate (script files)
.bsx
.sta(status files)
.inf
.enc (encounter rate???)
.fcr (i believe the f in the following
.fda files stand for field something)
.fpl
.fcc
.mcc

Quote of what fice said earlier

Quote

Block 0 (if we're starting at 0) contains the text at the end of the block, dunno about the start.
Block 3 is palette.
Block 8 is background

So, the thing is that we have to guess what the shortform of the format extentions stand for. Then we can match the extentions with the files. Honestly, we should look for anything that is usually repeated in each field file to help quicken the workings.

and

halkun,

Quote

The program reads in the bone defs and finds that there are, say 9 bones on the model. The .a file looks like this: (o=orentation data)
o1,o2,o3,o4,o5,o6,o7,o8,o9|o1,o2,o3,o4,o5,o6,o7,o8,o9|....

I believe you are wrong!
Why?
This is because the files are named like this:-(note this is not the real files, I'm making it up)

aaaa.hrc (The first file always .hrc)
aaab.RSD (the next file will always be .RSD)
aaac.p (this file references to the RSD file)
aaad.tex (any .tex files referenced in the .RSD file will appear later)
aaae.A (the animation files will appear after .RSD files that require animation)

In other words, each .A file only has instructions for one RSD file or in other words just one polygon.

Sephiroth 3D · « **Reply #12 on:** 2001-03-23 11:48:00 »

What's in the RSD files? I know that in normal PSX games, that's the geometry, but FF7 geometery is in the .p files, so what's in the RSD files? Could it be the orientation info?

This is just a thought... The RSD files ARE around 20k each or so, thats plenty for orientation data for each anim or every part, plus referances to the .p files...

Am I wrong about this?

Also... Anything about the info so I could help guys? Right now I'm shooting in the perverbale dark here. I'd like to know what I'm talking about before I say it.

Sephiroth 3D

"One who seeks knowledge from another person, doesn't learn half as much as the one who seeks knowledge for himself." - Vincent Valentine, The Sephiroth Chronicals, Book 1: Resurrection

[email protected]
http://www.sephiroth3d.com" TARGET=_blank>Sephiroth 3D's Final Fantasy
http://members.tripod.com/~EmperorTim/index.html" TARGET=_blank>Sephiroth's 3D Lair
http://members.tripod.com/~Sephiroth3D/index.html" TARGET=_blank>Sephiroth 3D's Promised Land

Srethron Askvelhtnod · « **Reply #13 on:** 2001-03-23 19:13:00 »

Sephiroth 3D: The other's already know this, so I guess I'll fill you in. Basically, the .HRC file is the skeleton. If you load the .HRC file, it will load all of the other parts of the model and put them together for you. The .HRC lists the coordinates and .RSD files for each individual piece of the model. When you load up the .RSD file, it tells you which .P file to use. Simple as that. Also, the .RSD for the head of the model also tells you which 2 .TEX files to load for the eyes. Got it?

The SaiNt · « **Reply #14 on:** 2001-03-23 22:56:00 »

You guys might want to check out this pagehttp://www.finalfantasytactics.org/

ficedula · « **Reply #15 on:** 2001-03-23 23:51:00 »

Actually, the .A files do contain bone data for a whole model (according to Qhimm's 3dS plugin. I haven't tested it out myself, though).

The Skillster · « **Reply #16 on:** 2001-03-24 00:14:00 »

been there about a year ago

The SaiNt · « **Reply #17 on:** 2001-03-24 11:46:00 »

ficedula :Exactly, what I suspected

The SaiNt · « **Reply #18 on:** 2001-03-25 10:42:00 »

Ok, guys I thought I'd just add to what fice said

Each field file should contain:-
-The background information (.MIM)
-encounter rates (should this be a non-zero, the enemies available should be specified)
-scripts for automated sequences
-text displayed in dialogue boxes
-areas "walkable" and "non-walkable"
-scripts to load the appropriate midis
-flag to enable to disable save
-scripts to load objects (treasure chests etc)
-scripts to define how one field is connected to another

BTW, we can really discuss everything here, cause I think I have access to move topics from this forum to the tech forum.

The Skillster · « **Reply #19 on:** 2001-03-25 13:34:00 »

u forgot NPCs and stuff

halkun · « **Reply #20 on:** 2001-03-25 15:14:00 »

Actully, there are several "stroy" flags in there as well, I was playing with the honeybee inn data, then there exactly 4 of those. One when Cloud comes in for the first time, and arith waits on the way left, the second one is when arith waits outside the honeybee inn door and everyone is pestering her. The 3rd is when cloud is in a dress and they crowd around him not alloing him to enter the field, and last is when the shinra solder is blocking the door and says that shinra took over. It appears that when a field is "Called" from the last field, it also passes these "story flags" as well. (You can pass these flags from a dialog choice too)

^_^

ficedula · « **Reply #21 on:** 2001-03-25 15:56:00 »

Obviously there are some flags defining how far through the story you've got ... I always assumed they were 'global' though, in the sense that any level *could*, if it wanted, modify any flag.

The SaiNt · « **Reply #22 on:** 2001-03-25 16:39:00 »

Hmmm, I too always thought that those flags were global.
Are you trying to say that those flags also define when you access a field?
You could be right though...........

Anyway, I just wanted to point out some things here.

The save games files actually do save which disc you are on so I'm sure this info is important in some way. Thus, I tried to make the first field file(the railway scene at the start of the game) into 3 savefiles. Each as Disc1,Disc2 and Disc3 respectively. Then I copied my movies to my hard drive. What's odd is when I get to a movie scene, where I first see the mako reactor, the game will crash if i use the save games from disc2 or disc3 but not with disc1. So, I guess the movies in the game only run on a per-disc basis. Even with all the avi files stored locally, the game still jammed.

2nd, thing. Assume I load an edited savegame with a party of lets say Barret, Cait Sith and Aeris(Seems stupid and likely to crash right) The thing is, the game will not crash. In fact it will still run normally. One thing which is special or rather what I'm trying to point out is your main character will usually not appear in the field screen. The field character is not missing, he's only just invisible; you can still exit the screens and interact with everything. The most important part is this: Upon entering certain field files, the character will actually be displayed and you will see him appear. Doesn't this mean that the possible people to appear in a field file is also defined?

[This message has been edited by The SaiNt (edited March 25, 2001).]

ficedula · « **Reply #23 on:** 2001-03-25 16:52:00 »

Yes.

If you extract a field file from FLEVEL.LGP and decompress it, one of the sections right at the start is a list of 'names' - things like "cloud", "tifa", "reno", plus others. It's only a guess, but it seems reasonable to assume it's a list of the models required by the scene.

halkun · « **Reply #24 on:** 2001-03-25 19:35:00 »

Aha! That answers everything In the field files the movies are not refrenced by name, but by *number* so the program loads it *knows* what disk it's on and assumes that if it's on disk three, it the movie video must be movie #1 on that disk. But you can fool the program by changing the disk number. but keeping the other disk in at load. (looks like the only time the disk is checked is when you are told to change it)

Wait enlighenment!

On disk 3 there is the metero in the sky and the gun in junon has been moved...Check to see of, by switching the disks in the dave data, the world is changed. It may use the same data from the disks, but the game thinks "Hey, I'm on disk 3, put a metero in the sky" even through you have disk one actully in the drive. you follow?