Making an Open Source FF7 Engine

[mirex]

However, the greatest help you or anyone can be to me is to work out the battle animation file format.
I myself am giving it another go (or will be soon) and I can share what we have already uncovered about it.

Hard thing to do man, I already tried long time ago, when Phareon's info came out. Its bit information .. i think it was 12 bits for a float ? And it does not work afterwards on next frame ... but it works on some next frames ... so i think either a) its somehow bit-aligned and i missed that b) those next frames are delta values, and some of the frames are not. c) maybe those are deltas or not, but some bones (which haven't moved) are left out from the frame data d) homm i had one idea but ... it just flew out of my mind.

I can give you information from Biturn, which can read at least some of the frames if you want.

Another thing to do, if you don’t want to work on the battle animations, is finish the field file formats.
The images drawn in the fields are not finished. Although mostly complete, there are obvious flaws in many of them.

What's missing on field file formats ? You mean field 3d models or something else ?

[L. Spiro]

i think it was 12 bits for a float ? And it does not work afterwards on next frame

Err, didn’t you remember it was I who provided more information about those files?
Remember, I redid the header structure, including number of frames in the information, and also the special byte that details how many bits per float.
0 = 12 bits, 2 = 10 bits, and 4 = 8 bits per float.

After that follows the position offset, then the rotations.
Then yes, there are some frames where the second animation from is decoded also.  I don’t know why yet because I haven’t studied any animation sets where this is actually the case.

c) maybe those are deltas or not, but some bones (which haven't moved) are left out from the frame data

I have confirmed that they are definitely deltas.  I have figured out the offset information for the second frame, which is delta information.  After that, I can change certain bits and cause his arm to move too much in a direction or something like this.  Everything points to offset data mixed with flags that determine which frames are left out and how many deltas there are.
I just can’t find the exact system.

I am hoping if bulk_4me gets the windowed patch up again I can use it to debug Final Fantasy VII and actually follow, step-by-step, the routine that decodes the battle animation file format.

What's missing on field file formats ? You mean field 3d models or something else ?

I am talking about the background files.  The 2-D animated backgrounds for every area where you can walk, aside from the main map.
To the best of my knowledge, LGP Tools has the most recent decoder for these files, but even with it you can see plenty of disfigured areas.

Then there is the walk mesh, which Micky (?) decoded, but I haven’t put any of that information to use yet.
I really need to back up that topic so it doesn’t get lost…


L. Spiro

[Cyberman]

Well
L. Spiro has a nice long diatrab on his working on the animation format.
Anyhow it appears to be some sort of nifty bit packed scheme.
12 10 8 bit angles (initial) then shifting to dx dy dz and the change in angle for each bone.
unfortunately with bit pack schemes like that it's never obvious how they work.  If it were people would have had the animation format decoded by now

Probably what would be helpful is a set of decoded (IE results) values (preferably integers) to work from a singular animation.

Cyb

[bulk_4me]

I am hoping if bulk_4me gets the windowed patch up again I can use it to debug Final Fantasy VII and actually follow, step-by-step, the routine that decodes the battle animation file format.

I'll but I am rewriting the whole thing.. stupid Sony discs.

[mirex]

Err, didn’t you remember it was I who provided more information about those files?

Eh sorry, I forgot about that I was using the specs from Phareon. But now I remember something that you were saying something about this ... now when I see the thread I remember ... sorry

Then there is the walk mesh, which Micky (?) decoded, but I haven’t put any of that information to use yet.

iirc it was 'Kero' who posted the screenshots of walkmesh

[L. Spiro]

I have compiled some information regarding Cloud’s first 5 animations (in rtda).
I can compile more but I have to sleep.


NOTE: THE .RAR and .ZIP ARCHIVES CONTAIN THE SAME FILES.  YOU DO NOT NEED TO DOWNLOAD THEM BOTH.
Cloud.rar
Cloud.zip


What this is:
Cloud’s first 5 animations in 3 files each and in 4 versions.
Each animation has a .bin file.  This is the raw data extracted from rtda.
This is the animation as it appears inside the files.  Every animation BEGINS on 0x11 (because the header is 0x11 bytes!).
Each animation begins with 6 bytes of offset information (3 shorts) followed by the root rotation (almost always 0 0 0).
Consult my “nice, long, hard, throbbing, ‘diatrab’” here for more format information.

Each .bin file has a .txt file (.bin.txt).  This is the above file, only translated into bits and hexadecimal.
I have put a break in these files where the second frame begins (and so begins the mostly non-decoded data).

Finally, each animation has a .txt version.  This is the final list of rotations for each frame/bone.  The goal is to take what is in the .bin files and turn them into what is in the .txt files.
The .txt files have two versions of the animation information.
A series of raw floats and then the same floats listed in frame/bone structure.  Offset information has been added also.  I wrote a script to do all of this.


Hopefully this will be enough to get some people interested in it.

I am thinking it is best to start with very simple animations though.
I want to get the animation information for the helicopter during the Rufus fight.  It is surely very simple and it would be very easy to see any patterns in that animation, but getting it is the hard part since I lost my files that had a save at every save point (hard drive crash).

Also, I would recommend this program specifically for the converter:
Memory Hacking Software
I am not sporting my own software here; that will come another day.
But in this case it really does have the most useful converting tool.
Just run it and press Alt-T/C, or go to Tools/Converter.
It shows things in both big and little Endean which makes it especially useful for hacking Final Fantasy® VII.
As well it shows many data types at once, which helps to recognize between data stored as floats or longs.


L. Spiro

[bulk_4me]

FF7w Mirror:

http://aaronserv.dyndns.org/!!stuff/qhimm/FF7w.zip

Thanx to Aaron

[MaTiAz]

anti-cheat stuff........

Wouldn't those anti-cheat measures be pretty CPU/Memory intensive, since all the "hard work" is done by the server?

[L. Spiro]

Not at all.
The anti-cheat by design simply means giving less information to the client and having the server keep track of your stats (which is only logical).


This won’t be at all process-intensive on the client end, and it won’t be packet-intensive on the server end.
Your stats change after every battle and when you change your equipment or use an item.
These are the times when the server will send your stats (and only the stats that have changed) to your client.
This could mean about 16 bytes of data (assuming a 10-byte header) sent when you give yourself a Potion, or up to 60 (spread out in chunks which your client will receive when you are looking at the items/gold you gained) after a battle.
60 is a rough guess, but I think that number should mostly stay down near 20 or so for an average after-battle gain (Gil + 2 or 3 items = [10-byte header + 2-byte specific header + Gil amount (4) + 2-byte specific header + item amount(s) (3 bytes per item)] = 10 + 2 + 4 + 2 + 6 (2 items) = 22.



As for the client, it is currently running at over 1,000 frames-per-second on my AMD 2.8 GHZ computer with a Radeon 9700.
I sent it to a friend some time ago who has a lower-end computer but he was also getting over 1,000.
You must understand that the models, textures, etc., were all designed for low-end machines (so low even a PlayStation® can handle it).
To completely remake the engine for modern machines leaves plenty of room for adding fancy effects or any type of mathematical computation.

I only plan to add a little; I don’t want to ruin the spice of the original game.  It isn’t the same unless the polygons are jagged.

While your client will have a huge frame rate, the server will also have room to spare.

The server has nothing to do but verify.
When formulating packets, it has nothing to do but add something to your current stat and then create a packet and send it to the client.
Client overwrites the old value with the new value, thus if you change your Gil in the client to 99999, when you get a packet that tells you your Gil is 10, your client will now have 10 for Gil.
When running a battle, it only needs to work about 2 or 3 times per second (when something attacks something else).

The anti-cheat measures aren’t active devices, running constantly to verify the client is clean.
It simply involves limiting information the client gets and never asking questions when it comes to stats.



L. Spiro

[rmco2003]

What about botting? If PVP is involved in this online rendition then players can use bots to increase their stats tenfold overnight like in Gunbound. That game has anti-cheat design but it still doesn't know when bots are running and when they're not.

[L. Spiro]

Bots are hard to detect and they were a problem in one of my previous games (a game my company is remaking now for Nintendo DS).

But actually there is one simple method that is a bit awkward, but works.
It’s essentially the same method forums use to control bot spamming.

After 1 or 2 hours of straight gaming, the server can send a message to the client which causes the client to display a graphical picture, which contains text.  The user would then need to type the text he or she see in the image before he or she is allowed to continue gaming on the server.
The person could just be AFK, so he or she won’t be kicked for not answering at all, but while the server is waiting for an answer, the player will not be able to move or fight in any battles.

The correct answer will not be sent to the client at all, so no hooks, RAM hacks, or packet sniffers will be able to find out the answer.
The server can generate the image and store the correct answer itself.
Then send only the image.
Client is ordered to display it (which can be ignored but it won't work since the server won’t let you battle until you answer the question).
When you type the text, it will send what you typed to the server.
The server will verify it and if it is correct, it will release the lock, allow you to continue, and order the client to stop showing you the picture.



It seems awkward to have to do this, but it isn’t unreasonable, and it will fix the bot problem.
It would only occur after a player has been playing for 2 hours (or so), which is pushing the limit of human endurance anyway.



But anyway, making a bot for this type of game is risky business.
Unless you put a lot of effort into the bot, it will just get you killed.
At least in medium+ battles.  Maybe it can fight MP’s all night but who is going to gain a level from that…


L. Spiro

[Qhimm]

Actually, real players might get more than a little miffed being constantly asked to verify their species like that. Also, the standard captcha's are starting to be cracked anyway, if we look at current anti-spamming measures.

[bulk_4me]

Bots are hard to detect and they were a problem in one of my previous games (a game my company is remaking now for Nintendo DS).

Which game?

[rmco2003]

You could always code for it to detect if keys are being pressed at regular intervals for more than 40+ minutes, I doubt human players can keep up constantly pressing a key at exactly the same time difference for that long, so it might be something to look in to rather than having to verify yourself all the time. Is this an official remake with official Nintendo devkits or are you using homebrew devkits for it?

[L. Spiro]

Actually, real players might get more than a little miffed being constantly asked to verify their species like that.

Yes, this is what I want to avoid, but at when it comes time I will just have to weigh the two sides.
If I can find a good lengthy time period that won’t bother too many people but at the same time won’t give bots as much time as they need to be useful, that would be best.

Also, the standard captcha's are starting to be cracked anyway, if we look at current anti-spamming measures.

Well nothing is ever going to be 100%.  I just have to deal with the 99% of script kiddies who are not skilled enough to get past these checks.  Eliminating them is worth it.

Which game?

It was a personal project of mine a long time ago.
An online fighting game where each player selects his or her moves in secret, then the bot (mIRC bot) would take their moves and deal damage, then let them both attack again.
Had a lot of magic and customization of your character and became quite popular for being just a text-based IRC battle bot.
The battle style is like Pokémon in terms of both players select their moves (secretly) and then attack.
The rest of the game and battle mechanics are really their own styles.
I borrowed some things I liked from some good games and mixed in some new things of my own to make sure the battles are very interesting and strategic.  This is important, since “just attack until you win” is dull and pointless.  I want magic and strategic combinations of spells to decide the winner, with pure stats being more of a minor factor, but still enough factor that a player 10 levels higher than you will almost always win, unless he is a moron or TRYING to lose.
Each spell needs more value besides just “deal more damage than a regular attack would”.  Spells need to actually do things to somehow change the state of the character.
So, it is actually pretty fun and most of the people who played the original are very excited about it.
Several of them have decided this game is the only reason they will buy a Nintendo DS.

You could always code for it to detect if keys are being pressed at regular intervals for more than 40+ minutes, I doubt human players can keep up constantly pressing a key at exactly the same time difference for that long, so it might be something to look in to rather than having to verify yourself all the time.

When you make a bot, you don’t necessarily have to put keypresses on timers that repeat.
You can hook in-game functions to find out when things are happening and then only hit keys in reply to those events.
Especially during battle, where the bot would have to wait for its turn to attack.

Is this an official remake with official Nintendo devkits or are you using homebrew devkits for it?

It is an official remake that, when completed, you should be able to go to the store and buy.  We are using official devkits.


L. Spiro

[bulk_4me]

What's the name of the game? Where can I download the PC version of it? Or this was just a personal game that it's somehow ported to a console?

[L. Spiro]

It was a personal game of my own that I was able to convince my company to make as a project.
I called it Shiny Star, but we will rename it for the Nintendo DS.

I don’t always have it running but you can go to #FF7 on irc.dynamix.com (I own the room) and request to play it.
I will load it and you can play it from mIRC (or any IRC client you like).

Simple text-based input to play, but somehow fun even from the start.


L. Spiro

[jc2k300]

Bad Idea, Square Enix will close your project down in a snap of a finger.

[Decayrate]

not if evry one keep quite and dont blow the wissel, it wont go down.....so don't be the one that blow the wissel. I would NOT do that cuz then I had to go and make you dont do it

sorry if you are scared by me

[yoshi314]

nobody said you need to write your own engine from scratch in order to write a new one for ff7.

if someone would use OGRE rendering engine it would save ppl tons of work. it is GPL licensed and there is also a GPL physics engine and stuff.

http://www.ogre3d.org

actually i was thinking of rewriting vagrant story more

i'm currently looking for spare time to take some tutorials on ogre (their wiki is great in that matter).

of course Square will not allow to make a custom ff7 engine :/

then again, why does ficedula work on his remake?

[Micky]

nobody said you need to write your own engine from scratch in order to write a new one for ff7.

if someone would use OGRE rendering engine it would save ppl tons of work. it is GPL licensed and there is also a GPL physics engine and stuff.

Actually the rendering side on FF7 is quite simple: Just some static backgrounds and some rigid skinning. Most people already have code for that in their drawer.
The difficult thing is all the scripting, and of course reverse-engineering the so-far unkown bits (for example the battle animations, worldmap, field NPCs...)

[L. Spiro]

Battle Animation Format

I will decode the rest when I am ready to use them.

Right now that may be a while since I am working on my GoldenEye 007â„¢ and Perfect Darkâ„¢ projects, and since I have officially decided to rewrite my engine.

I have found myself to be highly uninspired when I think about what to do next on my engine, and it is basically because the code in it is 3-4 years old and frankly “newbie”.

Even though all the code works and it is stable, I am just not happy with it.
It needs to be cleaner and faster.
But anyway it won’t take long to restart it and catch up to where I am already.
Probably a week or so.
Once I am working with faster, cleaner, and more organized code, I will become much more energetic with the project.
Strange how my inspiration can be directly related to the tidiness of the code rather than the functionality?


L. Spiro

[PapSnatchBrim [SwornEnemy] (DMR)]

Oh well... I guess it's all up to me then, I have plenty of free time in my hands.

Expect it to be finished in around 10 years time, when no one cares about FF7 anymore

its nearly been 10 years hows it going? lol

[Covarr]

Reviving a ten-year-old thread outside of Completely Unrelated? That's a paddlin'.

Anyway, there is such an engine in public development right now. It's called Q-Gears. I'm gonna go ahead and lock this thread now. ~Covarr