Author Topic: ReWriting the FF7 Engine  (Read 25154 times)

Alhexx

  • *
  • Posts: 1894
    • View Profile
    • http://www.alhexx.com
ReWriting the FF7 Engine
« on: 2004-06-24 14:35:26 »
I know there have been topics related to the before, however, I want to hear your opinion.

I don't want to knwo if it is possible to rewrite the engine; of course it is.
I want to know if you think that our team would be able to do this.

Would we be able to write a new engine that could use all FF7 files and play the game?

 - Alhexx

Darkdevil

  • *
  • Posts: 728
    • View Profile
    • Http://darkdevil177.5u.com
ReWriting the FF7 Engine
« Reply #1 on: 2004-06-24 15:56:56 »
Personally I think its gonna be a bitch of a task if it is started.  I seem to recall that Fice started re-writing it and i couldn't see much difference in the screenies to the normal game.  It will take alot of time, but If its going to happen, I would like to be a part of it.

Given a big enough team, alot of hard work, ****loads of spare time and real passion for this project I reckon it can be done.

halkun

  • Global moderator
  • *
  • Posts: 2097
  • NicoNico :)
    • View Profile
    • Q-Gears Homepage
ReWriting the FF7 Engine
« Reply #2 on: 2004-06-24 17:48:35 »
rewriting the engine for modern platforms was a goal of mine too. I can't code very well, and absoluty suck at c++

We also don't have everything yet. I need to figure out the scripting commands and where they are in the PC field files. Right now I only have ther names and bearly any argument sentax. I bearly know what they do.

We also don't have the walkmeshes for the field files, not the cam movements or the battle animations. I'm still unifying the PSX and PC versions so we can have a common codebase.

BUT!

We have the source code directory listing so we can see how it all fits togeather. I'll probably make a Dia map of the program code files and probable functions inside. We can use that as a filler to start with the low-level kernel functions.

Me will have to make the kernel first, and that's the least understood out all of the modules. Keep in mid it *IS* a multitasking kernel with threading abilities.

But I will contine. I only ask you guys help...

Right now if someone can dig through the STARTMAP field file and try and find the tolkens listed in my scripting post and what they do. I would be greatful.

(I listed the entire STARTMAP script by the way)

ragnarok13

  • *
  • Posts: 24
    • View Profile
ReWriting the FF7 Engine
« Reply #3 on: 2004-06-24 18:07:07 »
A project, well worth the effort. It'd make the best game of all times work decently once again.
If you made it, perhaps it wouldn't be too complicated to enable/include movie and background filtering too? Since that could count as a full remake in terms of quality improvement :wink:
Makes me wish I wasn't just a beginner programmer :(

Magus Spirit

  • *
  • Posts: 35
    • View Profile
ReWriting the FF7 Engine
« Reply #4 on: 2004-06-24 19:32:21 »
Know what ? I've been thinking about that recently. That would be great if we could do that. I know C++ and the DirectX API (by "to know", I do not mean I'm a pro or something like that, just that I have some knowledge of it and made some programs that use DirectX, such as a 2D RPG Engine - still under work), and would like to work in such project. The main problem is that I won't have much time during these holidays. And next year, I'll be in a Computer & Engineering university - no time for distraction. (yeah, I'm someone scared by time :lol:)

EmperorSteele

  • *
  • Posts: 933
    • View Profile
ReWriting the FF7 Engine
« Reply #5 on: 2004-06-25 06:22:08 »
If anything, re-write it so that it's easier to create and impliment modifications without overwriting original models and material, and so that they actualy work.

As it is now, 3-d models really can't be edited too much.  They have to retain the same information.  It would be nice if completly new models could be implimented, models with mroe than like, 8 polys per part =P

Cyberman

  • *
  • Posts: 1572
    • View Profile
ReWriting the FF7 Engine
« Reply #6 on: 2004-06-25 07:29:24 »
Well Alhexx all things are possible

So you need to decide what it is you want.

My thought is go for portability.  This means disembodying platform specific code from the main engine.  I think Halkum has a good idea, recreate the base set of data.  Now the PC version has a number of crashes and the PSX version has a few bugs.

The portability concept would allow for example Direct X or Open GL to be supported.  Different Audio, and more importantly any movies could be in a format like XVID with an OGG wraper.  Again non proprietary formats and data lend to portability.  The orginal movie sequences are pure unobtainium for it so the old movies repacked should be fine.

This is what I want though, what goal do you want?  Multiplatform compatibility and a functional game engine?  Toolset for compiling FF data and working with it's engine?

Set goals, makes it easier to acomplish something I've heard. :)

Cyb

ragnarok13

  • *
  • Posts: 24
    • View Profile
ReWriting the FF7 Engine
« Reply #7 on: 2004-06-25 10:13:49 »
Cyberman: the movies should work fine in any format as it is. Mine are divx5 with mp3 audio and they work perfectly (and yes, I do own 3 original copies of the game, in case anyone wondered :D - I'm not joking here :oops:). We only need the filtering and perhaps higher color depth 8)

halkun

  • Global moderator
  • *
  • Posts: 2097
  • NicoNico :)
    • View Profile
    • Q-Gears Homepage
ReWriting the FF7 Engine
« Reply #8 on: 2004-06-25 11:39:37 »
Here are forseeable problems and isssues:

1) The new engine should support both the PSX and PC datasets. The PC version is simply too rare while you can pick up the PSX verion new as a "greatest hits". The PSX disks however are not in ISO-9660 format and will not mount in linux. (At least not on my computer, I don't know how to mount a "Mode 2" disk) This means we *MAY* have to write a direct filesystem accessor within the new kernel. (Which, now that I think about it, we will have to do anyway as most of the datasets have to be extracted/decompressed/decoded to even be looked it) If not dual format right away, al least make the system modular enough where support can be added later

2) It should also be cross-platfrom, (Win Mac Linux) which causes some other problems.

3) The kernel is going to be difficult to code, as most of it's fuctionality is invisable

4) To keep from beating our heads aginst the wall, we must use a cross platform libary. Linux can't run DirectX, but windows can run OpenGL

5) The PSX sound data will have to be *re-rendered* on the fly. This includes both the SEQ music files and the sound data. (Which is why Square simply used MIDI and rendered WAV files in the PC version) Linux can't use MIDI out of the box. Timidity only comes with a piano voice. That's it. I had to download 128 meg of sound data from my keyboard just so I can have proper sound. It can use soundfonts in the PC version, but you have to look out for people who don't have the PC version availible to them.

6) The Duck codec is propiatary. It can't play on linux and you must have a licence to use it in you game. That's a legal issue we have to solve. Because duck seems to not even want us asking about it, as you need a password to even look at thier website. That poses a real problem.

7) There is no 7

8) Speaking of legal hurdals, a demo of *ORIGINAL CONTENT* will have to be created in order to make the new engine an original work. It needs to be able to stand on it's own without any support from Square at all.

mirex

  • *
  • Posts: 1645
    • View Profile
    • http://mirex.mypage.sk
ReWriting the FF7 Engine
« Reply #9 on: 2004-06-25 12:05:25 »
Yes the team will be able to do it, time is only issue ... i'd say it will take few years to complete it (Yea i know im born pesimist : )

I can say that from my personal experience, i'm stuck in few projects and im just-in-the-middle ... complex things like this allways take a long time.

halkun

  • Global moderator
  • *
  • Posts: 2097
  • NicoNico :)
    • View Profile
    • Q-Gears Homepage
ReWriting the FF7 Engine
« Reply #10 on: 2004-06-25 12:12:14 »
Hey Mirex, how long have we been working on this now ^_^

Look at your join date and mine, we've been here a while...

Goku7

  • *
  • Posts: 1301
    • View Profile
ReWriting the FF7 Engine
« Reply #11 on: 2004-06-25 18:54:07 »
Quote from: halkun
Here are forseeable problems and isssues:

5) The PSX sound data will have to be *re-rendered* on the fly. This includes both the SEQ music files and the sound data. (Which is why Square simply used MIDI and rendered WAV files in the PC version) Linux can't use MIDI out of the box. Timidity only comes with a piano voice. That's it. I had to download 128 meg of sound data from my keyboard just so I can have proper sound. It can use soundfonts in the PC version, but you have to look out for people who don't have the PC version availible to them.


Well, the PSF patch should be up to that task.

Not to mention, the PSX-emu scene has a decent amount of attention given to Linux, so I wouldn't be surprised if there are Linux-compatible players out there, that can be adapted to use with the PSF patch....

Darkdevil

  • *
  • Posts: 728
    • View Profile
    • Http://darkdevil177.5u.com
ReWriting the FF7 Engine
« Reply #12 on: 2004-06-25 20:23:19 »
Quote from: halkun
8) Speaking of legal hurdals, a demo of *ORIGINAL CONTENT* will have to be created in order to make the new engine an original work. It needs to be able to stand on it's own without any support from Square at all.



If this is done anytime after the 14th of august (When I get Broadband) I will be able to send it in to PC Gamer, PCZone and PCFormat to see what they think of it *IF* thats what you mean.

EmperorSteele

  • *
  • Posts: 933
    • View Profile
ReWriting the FF7 Engine
« Reply #13 on: 2004-06-25 22:39:49 »
Well, the thing is, ff7 and all the characters, etc, are also the INTELLECTUAL copyright of square.  We'd have to be very specifi about this being a "Fan-project" and that Cloud, Sephiroth, etc are still copyright of Square.  I'm sure that's a no-brainer, but still.

My point is, I don't think it'd be too wise to ditribute a demo to gaming magazines.  I mean sure, plenty of mods go onto gaming disks, but this is above and beyond any normal mod. Basicalyl, y'all would ahve to be careful how you present it, that's all.

halkun

  • Global moderator
  • *
  • Posts: 2097
  • NicoNico :)
    • View Profile
    • Q-Gears Homepage
ReWriting the FF7 Engine
« Reply #14 on: 2004-06-25 23:35:43 »
Quote from: Darkdevil

If this is done anytime after the 14th of august (When I get Broadband) I will be able to send it in to PC Gamer, PCZone and PCFormat to see what they think of it *IF* thats what you mean.


I highly doubt we will have *AYNTHING* soon. We can bearly swap models. We can't even create our own without crashing the engine. After four years of work, we are more at the beginning than the end.

Cyberman

  • *
  • Posts: 1572
    • View Profile
ReWriting the FF7 Engine
« Reply #15 on: 2004-06-26 00:55:05 »
I'm not sure about anyone else but I like making tools.  IE need something to dump out a particular format, I can do that. It's something I like to do, heck I wrote a tool that took assembly for one processor and recoded for another processor (In PERL of all languages).

Reading in Mode 2 is a big issue period, maybe you need to seperate the CD/DISK IO out as a section instead of incorperation. Hmmm anyhow.

PSX sound engine is well emulated and documented, PeopsSound toy is a good example.

Your project will die by a thousand paper cuts if you don't break it down into seperate problems.  Like the original game engine is.  Perhaps that's the best method to follow, use the original game concept to run the game. 6 functional programs 1 kernal and 5 sub programs that run from the kernal.

In the PC version do they include all the characters as part of the movie sequence or are they like the PSX 3d characters over an animated background?


Cyb/Stephen

Goku7

  • *
  • Posts: 1301
    • View Profile
ReWriting the FF7 Engine
« Reply #16 on: 2004-06-26 02:20:17 »
Quote from: halkun
Here are forseeable problems and isssues:

1) The new engine should support both the PSX and PC datasets. The PC version is simply too rare while you can pick up the PSX verion new as a "greatest hits". The PSX disks however are not in ISO-9660 format and will not mount in linux. (At least not on my computer, I don't know how to mount a "Mode 2" disk) This means we *MAY* have to write a direct filesystem accessor within the new kernel. (Which, now that I think about it, we will have to do anyway as most of the datasets have to be extracted/decompressed/decoded to even be looked it) If not dual format right away, al least make the system modular enough where support can be added later.


I'm guessing that when you are refering to having to extract/decompress/decode the data files, you're refering to what both the PC and PSX versions of game have to do to access their files, since they're both using compressed formats, right?

If so, then if what I'm about to say is old ground that was already covered, feel free to beat me with the "OLD!" stick -- mainly because I haven't been very attentive in regards to this board's progress regarding the reverse-engineering of FF7pc. :roll:

But hear me out anyway, in case I wind up giving any of you ideas, even if this a bunch of old ground....heck, for all we know, this might allow us to get at the the specific models inside the LGP without needing to decompress to separate files first. :P

What if there is no actual large-scale decompression going on in the PC version (and possibly the PSX version)?  I mean, we know that the PC version was coded in C++....so what if they decided to use C++ member functions like "seekp" and "seekg" to do non-sequencial read access?

Now think about this for a moment.  If they tried to model on the PC how the PSX does its data lookups (direct sector reading, and not using CDFS format) -- which really only works when you can be positive that the data on the disc isn't gonna move around -- then using seekp and seekg to point toward specific bytes of whatever LGP file they are using, then they can essentially copy that portion of the file into a struct array, and then decompress it without needing to decompress the whole LGP file.

Now, this would also explain why things can go wacky when you try to alter the file and come up with a different size -- for example, let's say the game tries to pull up Cloud's battle model, so it goes to the appropriate LGP file and uses the "seekg" function.....I'll do some code off the top of my head to show what I mean:

(please note that all the data types, byte offsets, and file sizes have been grabbed out of thin air and do not accurately reflect what types they should be...this is more half-psuedocode, basically)

Code: [Select]


#include <iostream>
#include <fstream>
using namespace std;

//define model data structure
struct model
{
  int Polys[InsertArraySizeHere];
  int Vertices[InsertArraySizeHere];
  int Textures[InsertArraySizeHere];
};

//function prototypes
void getPolyData(model *);
void getVertexData(model *);
void getTextureData(model *);
void decompressData(model *);

int main()
{
  *ptrCloud = new model;

//grab cloud's poly, vertex and texture data
  getPolyData(*ptrCloud);
  getVertexData(*ptrCloud);
  getTextureData(*ptrCloud);

//now decompress everything into memory -- and pray the
//struct that you've set up that will hold the decompressed
//data is large enough :)
  decompressData(*ptrCloud);

  return 0;
}

void getPolyData(model *cloud)
{
  fstream file;
  fstream *position;
  fstream *end_struct;

//open file for random access of data
  fstream file("battle.lgp", ios::in | ios::binary);

//jump to end of Cloud's model data, and flag it with a
//pointer
  *endofstruct = file.seekg(40L, ios::beg);

//we know that Cloud's compressed poly data occupies bytes 20
//through 40, so we jump to byte 20
  *position = file.seekg(-20L, ios::cur);

//increment 'till we hit the end of this model's compressed
//poly data, or the end of the LGP

  do
  {
     for (int i = 0; position < end_struct; i++)
    {
       file.get(cloud.Polys[i]);
       position++;
    }
  } while (!file.eof());

   file.close();
}

void getVertexData(model *cloud)
{
   blah blah blah...
}

void getTextureData(model *cloud)
{
   yada yada yada.....
}

void decompressData(model *cloud)
{
   *insert file bloating sounds here*
}


....using numbered offsets to set the read position to the correct part of the file -- or what it THOUGHT was the correct part of the file, if we are using a drastically changed file whose size is totally different, and would thus make the game convulse if it doesn't have a way of dynamically recalculating the byte offsets in a changed file.

Anyway...thoughts on my code?  Oldness, or am I on to something? :P

-edit-
Ack, the comments got all out of line when it resized the text boarders.  Fixed it -- I think.... :wink:

-2nd Edit-
Did general code cleanup and formatting that I felt like doing. :P

Darkdevil

  • *
  • Posts: 728
    • View Profile
    • Http://darkdevil177.5u.com
ReWriting the FF7 Engine
« Reply #17 on: 2004-06-26 09:22:33 »
Quote from: halkun
Quote from: Darkdevil

If this is done anytime after the 14th of august (When I get Broadband) I will be able to send it in to PC Gamer, PCZone and PCFormat to see what they think of it *IF* thats what you mean.


I highly doubt we will have *AYNTHING* soon. We can bearly swap models. We can't even create our own without crashing the engine. After four years of work, we are more at the beginning than the end.


A good point.  Well made too.

Alhexx

  • *
  • Posts: 1894
    • View Profile
    • http://www.alhexx.com
ReWriting the FF7 Engine
« Reply #18 on: 2004-06-26 18:54:02 »
Well, actually, we *can* create new FF7 Models that the engine does accept. However, this has only been tested with untextured files, but with a little bit of hacking (I hope I will have time for this after my army time...) we'll be able to create new textured models, too.
Of course, the other file formats are a big problem. Battle animations, cameras and so on have to be hacked first.

 - Alhexx

Cyberman

  • *
  • Posts: 1572
    • View Profile
ReWriting the FF7 Engine
« Reply #19 on: 2004-06-27 02:12:23 »
Well 'we' know scene.bin contains all the nifty little battle beasties yes we do!

the battle engine is a beats to itself is it not? Since it's loaded by the kernal module a good debugger would be handy at that point indubidably.


Cyb

mirex

  • *
  • Posts: 1645
    • View Profile
    • http://mirex.mypage.sk
ReWriting the FF7 Engine
« Reply #20 on: 2004-06-28 08:45:42 »
Uhhh I would gladly try to work on creating P files that FF7 engine accepts, only if i had some free time. Too many projects are on my hands.

Darkdevil

  • *
  • Posts: 728
    • View Profile
    • Http://darkdevil177.5u.com
ReWriting the FF7 Engine
« Reply #21 on: 2004-06-28 12:55:42 »
Quote from: mirex
Uhhh I would gladly try to work on creating P files that FF7 engine accepts, only if i had some free time. Too many projects are on my hands.


Mirex, If you could teach me how to do stuff like that, I will be able to do some stuff as Im doing nothing untill september now...Im out of school so i have ALOT of free time.

mirex

  • *
  • Posts: 1645
    • View Profile
    • http://mirex.mypage.sk
ReWriting the FF7 Engine
« Reply #22 on: 2004-06-28 14:59:09 »
Quote from: Darkdevil
Mirex, If you could teach me how to do stuff like that...

Teaching this kind of hobby would take even more time imho.
Nothing personal.

Darkdevil

  • *
  • Posts: 728
    • View Profile
    • Http://darkdevil177.5u.com
ReWriting the FF7 Engine
« Reply #23 on: 2004-06-28 15:07:09 »
Oh, Ok.

mirex

  • *
  • Posts: 1645
    • View Profile
    • http://mirex.mypage.sk
ReWriting the FF7 Engine
« Reply #24 on: 2004-06-28 16:08:40 »
But if you really want to, then i could give you few hints. Are you skilled in programming ? If not then that is the best way to go. C++, Pascal or Visual Basic (which is not much recommended, but works ;) ) ... choose any and go for it. Try to program something that messes with files, anything.