Engine building. If we make it, will they come?

[Qhimm]

You can certainly make custom models for FF7 already, with enough knowledge of the file format. The new engine will not provide any additional functionality to make it easier than that. It should do the exact job the original engine does (but better), not take on any new "additions" to the game. halkun: ...right?

This is, as someone may or may not have said, more of a technology demostration. Sort of like writing an emulator, but emulating the game environment instead of the console environment. I'm still not sure about the reverse-engineering issues though, even the current base of FF7 information is certainly in many places the result of reverse-engineering/disassembly. We (at least I) base the new code on the functionality of the original, not the code of the original; but it's a bit murky since a lot of the existing knowledge of the functionality comes from observing the original code...  

[ragnarok13]

I think the guys making CT:R made a serious mistake: they were about to, with their project, make freely available a game, that is in all aspects better copy of the original, AND (if I remember correctly?), it being functional did not require the original game.
So if you can make the new engine unconditionaly require the original game (data?) to function... But then again, big companies sometimes just shut down projects for no good reason at all, so I can't say anything for sure

This is, as someone may or may not have said, more of a technology demostration. Sort of like writing an emulator, but emulating the game environment instead of the console environment.

One thing: the PS emulator needs the BIOS, which isn't supplied with it. So officialy, the thing doesn't work and is not supposed to. That makes it pretty safe from being shut down, don't you think? An emulator of a software enviroment might not have that luxury, though. Except if you successfuly translate the concepts: BIOS >>> original game data?

[halkun]

Here are the goals I would be intrested in.

1) Update the engine for modern computers and graphics cards.
Why? beause the old one has atrophied. It can't run on XP, and now Duck has dropped support of TrueMotion.

2) Create a cross-platfrom engine
A native engine will always prefrom better than one in an emulated environment. Using FF7 on a PSX emu example, even if you play the game at 2048x1024, you are stil using a 320x224 "grid" and the polygon vertexs "snap to the grid" even though the edges are in higher resolution. It would be nice to be truely scaleable. I also, selfishly, want to play FF7 as a native Linux game.

3) Create an extendable engine that you can write your own content for
What the difference between FreeSCI running Sierra's "Hero's quest" and a new engine running the FF7PC? Geep in mind there is also the "SCIStudio" where you can make your own content. I'll jus be happ if you could use somewhat sane formats and real directory tree.

I have to get to work. I'll probably whine later, I have to dig my car out of the snow now....

[Qhimm]

First of all, I'd like to say I like this discussion. Both because it's important, but because the people in it have very well-formulated opinions.

3) Create an extendable engine that you can write your own content for
What the difference between FreeSCI running Sierra's "Hero's quest" and a new engine running the FF7PC? Geep in mind there is also the "SCIStudio" where you can make your own content. I'll jus be happ if you could use somewhat sane formats and real directory tree.

Now this is where we it starts getting dangerous... that being getting close to a point where the FF7 original data is just a "game pack" and our engine and future development tools (?) allows people to toy with Square's original content. They're leaving people who do small-scale hacking alone, but while the distinction between the small-scale "providing a new engine for FF7" and the slightly larger-scale "providing the ability to use FF7 data in a new, extendable context" is a small one, it's a most important difference of motive. We're talking about taking something entirely based on SquareEnix's ideas and property (the engine) into entirely new uses. Let's not kid ourselves that any soul out there would create a new game using this highly primitive and restrictive engine, the main public interest would clearly be to make (half-assed) customizations to the FF7 game itself. And since I'm assuming we'd have to figure out a way to prevent such use of the original data just as a "please don't sue" gesture for SquareEnix, what would then be the point to spend even more huge amounts of work to build a game customization engine that either no one will or can use legally?

The challenge is to build something that will run FF7 as it was intended. That's a rather specific and unique goal, with plenty of work involved. If we want to make a general-purpose engine (which would also run FF7), we're talking incredibly more work for very little reason. Any team willing to spend the work to create the amount of content needed for a game running in the FF7 engine would most likely want to write their own engine too, to get exactly what they want. Sure we could start a new project later for a RPGMaker-style engine that would let you make games similar to FF7, but I really feel we should keep the two concepts entirely separate. If we actually get together and create the latter engine (using what we've learned from this project about engine design), we would actually have a viable product. It would be a real shame then not to be able to do anything with it or even protect it, due to some legal issue regarding its origin because it was developed from the FF7 engine.

My point, which could probably have been written much shorter, is mainly that the new FF7 engine and the extensible RPG engine are and should be two entirely separate projects. If nothing else, than for the simple reason that the FF7 engine by necessity is messy, while a new engine (especially an extendable one) should be written "by the book" so to speak. You would not want to try and combine the two philosophies in a single software development project, trust me.

[Alhexx]

It seems like we've got a two-problems-discussion in here:
1. Argueing about who ported FF7 to PC and the Quality of the Port.
2. Writing our own Engine

I'l try to concentrate on the second one...

I've got to agree with Qhimm. I thought that making self-made additions to the engine would be an option, however, I've been tought here that this is going to be a risk for the whole project. So let's forget about this one first.
A "Naked Engine" is what we have to work on.

This is one of the points that makes my head ache:

1 make sure that users have orygiinal ff7 so Square earned money

The engine *should* have a cd-check or any sort of pseudo-copy-protection to ensure the end-user has a legal copy of FF7.
Problems:
1. If this is going to be an open-source project, then we can totally forget that. Everyone could download the source and recompile it without the protection. (what means that Open-source isn't such a good idea at all)
2. AFAIK FF7 did not have any copy protection on their original discs, so there won't be a way to check if the end user has a legal copy. But this is Square's Problem I think...


And something else:
Some months ago I was trying to write a 2d-jump-and-run engine, however, the project was cancelled.
But I was writing some sort of "Virtual Directory Manager", and I think this maybe could be helpful for that FF7 engine...
That manager consist of 2 Classes: A Directory and a File class.
Once you specify a path to the directory and "mount" it, the directory class scans the path for files, and creates an array of File classes.
Then, it checks if there are any archive files (LGPs in FF7), and if there are, it reads out the Header of the archives and add the files inside of that archive to the File class array. Of course, this only works when the archives are not compressed, but AFAIK, all LGPs are uncompressed in FF7 PC.

Advantages:
1. You do not to decompress those archives to tempory files, you can access them directly from the archive (as use it as a usual file that is not packed into an archive). Of course, this is only working well if you want to have Read Access to those files - it's tricky when you want to change a file inside an archive.
2. Because of that File array, you do not need to search for the files you want to open - you just get them from your array.

Disadvantage:
Once a directory was "mounted" it does not recognize any changes in that directory until it is "remounted", which means closing all files, destroying the file array and mounting again.
(However, I was thinking about making an "Silent Remount" which will not require closing and destroying *all* objects...)

Unfortunately, my last file backup f**ked up, so I lost the source files.
However, I have already started rewriting those two classes. It will take some time until I can send you some code, however, maybe this would be a way to solve some of our cross-platform problems...

What do you think?

 - Alhexx

[Cyberman]

First Cyberman . I see that you like saying everything precise. anything said generaly is wrong

If you say things generally they generally are wrong.  It sounds odd but it's true. It's best to be specific so people won't be misled or get wrong ideas.

Maybe not Square but someone made ff for pc for Square and Square aproved that ff version

Ummm Ok in any case we agree it's a crummy port.

I was speaking here about media,graphic. When you have high resolution backgrounds,movies,textures on pc then it is easy to convert them to low
resolution psx. From low resolution media you can't make high one.

Mmmmm maybe there is a detailed discusion about FF7's porting to the PC somewhere in this forum read that.  Porting is very difficult.  Even with highly portable code (See Nethack as an example) is quite difficult to port, I can attest to that. It really depends on what you are developing for.  PC games about the time FF7 was released were not as good as playstation games in terms of developement and quality.  Keep in mind you are looking backwards on the progression and not what the authors had to do.  It's not what people can do now it's what was available then.  At the time of the initial game writting the playstations video hardware did much better than the PC's.  That's just the way things were.  Now it's reversed.

You need to render them again in high resolution. People creating pc version has got generaly psx media . (only movies in ff8 were rendered again in 640x480) or maybe they had high res movies in first place and encoded them to psx native resolution.

Please read the discusion of the porting of FF7 it's somewhere in this forum.  I think you'll understand things a lot better.  I would also like to point out that if you don't make a game with the intent to port to begin with, this will be the typical situation.

I am also playing ff7 models for long time and i was speaking generaly. I was not speaking about exact numbers.  From watching ff7 models and few ff8 models i got felling that they could be better on pc becouse pc is not limited by strict specifications like psx (of course pc is limited by economical boundries).  For example - today no one can release game that is playable only on geforce 6800 ultra .( one can release such game but will not earn much from it)

Well I've not been manipulating models I've been rendering them from the original content.  As for if they could be better sure, Qhimm posted some rather startling images from FF8 content rerendered.  We know enough to grab the PSX data from the disk and give an OK rendering of it as a result. This is without enhancements.  The whole point of the engine is to actually get the data and make it work correctly.  Doing additional things to the data won't help unless the engine can run the original content to start with.  That's the primary focus of the project.

Cyb

[Chesso]

ummmm i doubt this is any help but....... what if this newly made engine just applied special effects on top of the original gfx to get a nicer or different look then there's no content modification.....

Im not sure what you would call it and whether it would work for this but you might be able to enhance the graphics and stay out of trouble somewhat.

[L. Spiro]

Some special effects such as distance blur and stencil-depth shadows could be applied (assuming there was a light source, and in the battle scenes there are none), and I personally have little opinion about whether it is a good idea.
I consider it with my own engine but never get around to doing it, primarily because I need more done.

Changing the graphics is not acceptable, but applying modern graphics-enhancing effects, well, depends on what the makers of the engine want.

I am completely against modifying the models or game content but for my own engine I am considering making shadows and blur to be options the user can select if he or she wants to see a more modern game.

Besides that, right now, with a fully-loaded battle scene and all characters on the field, I am getting over 1,000 frames-per-second at 1280×1024.
I need something to slow that down.


L. Spiro

[Qhimm]

Enhancing as in how emulators enhance (2xSAI filtering, increased resolution, anti-aliasing) should all be relatively OK. These are merely ways of showing the original content, and do not depend or interact with the content in question. We could, for example, apply a gauze effect (a la Prince of Persia: SoT) for the world map, but say adding motion blur to the Bahamut summon would be a bit closer to the edge of "manipulating original content" -- the only difference being that we let the engine do it in real-time without modifying the Bahamut summon files. I don't know, but this is in any case definitely something to think about late in the project, not during the planning stage.

[Chesso]

Ahhh yes of course i just wanted to suggest it as it doesn't brake any laws or atleast not to my knowledge and it shouldn't.

[Cyberman]

A few thoughts on portability:
If we want it more portable, then OpenGL is probably a good starting point. The other option is to keep the Rendering engine seperate from the content extraction.  Essentially each localized rendering engine is optimized for whatever system it's running on.  All optional enhancements should however are NOT be part of the rendering engine. Doing this would make a lot of descrepency problems.  The prefered method is querrying and passing the options to the engine.  This method prevents splintering of the base code and it's option handling.  It also makes the code for option setting universal instead of each option system different per platform.  Consistancy is important.  In fact it's more important that the UI be consistant for every port than how cool it looks. This leads to a lot less frustration to the poor guy having to use it.

A list of base options I would prefer for graphics:
Full screen
Windowed
Image resolution selection
'Driver' selection (IE OGL GDI DirectX X software etc.) This is a compatibility issue with the hardware and is obviously indicating the possibility of multiple rendering engines for each binary.

Audio support is another issue.  It might be a bit complicated to say the least.

[Micky]

Audio support is another issue.  It might be a bit complicated to say the least.

You could try OpenAL. That should work fine for soundeffects, maybe even better than the original.
It will become tricky for the music. You could convert it internally to MIDI and pass it to the operating-system's synth, or write a soft-synth and write it out as a sound stream.

[Chesso]

Why would you want to use OpenGL it is slow compared to DirectX?

EDIT: Oh right for other OS's i gotcha now.

[J*** H*******]

Message

[sfx1999]

Why would you want to use OpenGL it is slow compared to DirectX?

EDIT: Oh right for other OS's i gotcha now.

That isn't necessarily true.

[Cyberman]

Why would you want to use OpenGL it is slow compared to DirectX?

EDIT: Oh right for other OS's i gotcha now.

It's all relative mostly it is faster but it can be slower depends on how cheap a card you are using is.  It's mostly dependant on the vendor's implementation of the standard.  OpenGL in general is simpler to deal with.  The big area of contention these days is the fact DirectX has become more like OpenGL.. ironic isn't that?

Back to the main subject, anyone have thoughts about the interface between the script engine and graphics engine?  Or maybe we should proceed with more discovery

Cyb

[L. Spiro]

When the battle-field animation format is fully decoded I can finish my project’s battle engine (mostly; particle effects and camera motions are also not yet hacked, as well as sequence scenes for summons and spells), and I will at that time also begin the server code, which will make menus functional and drive the battle engine.

Because my progress depends on how much and how quickly we hack through the game, I have had plenty of time for planning and making sure that each new addition is done correctly and with stability.

After that, the 2-D backgrounds for the non-battle fields need to be more fully understood.

Then there is the over-world map/tree.

Particle effects and camera motion need to be hacked as well, as already mentioned.

This is my checklist.  I believe we have everything else we need to have.

And I think it could be more efficient to have some working on the engine while others are hacking.
And some who do both.


L. Spiro

[Cyberman]

When the battle-field animation format is fully decoded I can finish my project’s battle engine (mostly; particle effects and camera motions are also not yet hacked, as well as sequence scenes for summons and spells), and I will at that time also begin the server code, which will make menus functional and drive the battle engine.

I'm getting close to starting with the battle animations with the PSX again.  A few more things to do with my current project and that will be up for working on.

Because my progress depends on how much and how quickly we hack through the game, I have had plenty of time for planning and making sure that each new addition is done correctly and with stability.

Stable is a good thing.  Hopefully I'll be done with this project soon enough to my satisfaction, my real goal of dealing with the battle information will get attention.  Taking it steadily and working it out is always the best method.

After that, the 2-D backgrounds for the non-battle fields need to be more fully understood.

Then there is the over-world map/tree.

Particle effects and camera motion need to be hacked as well, as already mentioned.

The background information on the PSX version is entertaining to say the least to find  If you resolve the world format for the PC version the PSX version is resolved as well. Which is a good thing no?

This is my checklist.  I believe we have everything else we need to have.

And I think it could be more efficient to have some working on the engine while others are hacking.
And some who do both.

It might be but getting some coordination between the two might be challenging.  I know my 3d OGL usage isn't as swift as I would like.  I think however it's more an issue of knowing how to read the data properly than anything else.

Cyb

[ShinRa Inc]

There are many times I knew anything useful about coding and such.

Wish I could contribute.  :\

Also,

After that, the 2-D backgrounds for the non-battle fields need to be more fully understood.

What needs to be understood, if I may ask?  There's already a program out that seems to be flawlessly interpreting the Pre-rendered backgrounds for the PSX version already (ff7mimic), including the various triggered animations (Doors open/close, smoke, lights, etc)

[halkun]

A google for this lead me to a frech webboard with an expired article. Where, pray tell, is this?

[Qhimm]

There are many times I knew anything useful about coding and such.

Wish I could contribute.  :\

Also,

After that, the 2-D backgrounds for the non-battle fields need to be more fully understood.

What needs to be understood, if I may ask?  There's already a program out that seems to be flawlessly interpreting the Pre-rendered backgrounds for the PSX version already (ff7mimic), including the various triggered animations (Doors open/close, smoke, lights, etc)

One thing that's always been bothering me is that in the PC version, our current understanding of the backgrounds leave little black dots in the output. I don't know if this phenomenon showed up with the PSX files as well during 7mimic's development, maybe the author could shed some light on that.

halkun: 7mimic can be found here. He made a similar program for FF8 (PSX), I think.

What's the current status of the various field file sections, anyway? I've got the first section covered (although not yet executable), and the background and walkmesh are reasonably understood. What about the other sections? Has any progress been made on the "unknown" ones (according to Gears' list)?

Also, one tiny bit of trouble for making the engine portable: how are we going to handle the movies? Write our own decoder?

[halkun]

For christmas I got a new RAID card. I figured i would use it as an extra IDE controller and plugged up a new hard drive....

Sadly, it would seem when I was writing to my MP3 drive, the contents were also being written to my root drive and, shortly after, Naru went into a flaming taildive and smashed.

Oops! (^_^);

Anywho, I have a muchly updated server now, (However my suck bandwith is still a constent) I've been forced to rebuild my home directory from a backup so everything is there, but not in the correct place. I did move gears to my home's WWW space and you can grab it here, but I haven't found a definitive place for it yet....

http://the-afterm.ath.cx/~halkun/

As far as writing our own encoders, we do have the source code for reading PSX movies. (Though you have to read the data by raw sectors as it's a mode 2 disk). I can play the PC movies in linux via Mplayer, (Abet upside down and backwards)

Keep in mind that on Linux, the CD-ROM driver will have to support mode 2. Many linux CD-ROM drivers can't mount those disks correctly, like DVD drivers and CD-R drivers. Mode 2 is only used for PSX games, CD+G disks, and PhotoCDs so there really isn't much support for them. The good thing is adding support is very trivial as all you need to do is tell the drive in kernel-space to read more bytes per sector. The bad thing is you have to patch the kernel to do it.

I'm emailing the 7mimic guy for some colloberation.

**-- EDIT--**

Either way, an installer will probably be the best method to deal with the game.

[ShinRa Inc]

One thing that's always been bothering me is that in the PC version, our current understanding of the backgrounds leave little black dots in the output. I don't know if this phenomenon showed up with the PSX files as well during 7mimic's development, maybe the author could shed some light on that.

Out of curiousity, have you been able to make the backgrounds animate as well, or just display in a default form (as Fice's LGPtools and Cetra display them)?  If not, might those dots be a side effect of the animation coding being wrongly interpreted as graphics information, or something like that?

[Chesso]

Movies why would you need to worry about them? FF7 uses direct show which handles all that doesn't it? Providing the person has the needed decoder depending on what the movie was encoded in it'll be fine.

It's just like embedding a media player component or equivalent into your project and using it to play midi/wav/mp3/avi etc as long as the correct decoder for the file is installed no problem.

[L. Spiro]

Chesso, their goal here is portability to Linux. I think that should answer your question.


ShinRa Inc, the backgrounds we have animate (optionally).
The block dots and even the disarrayed pixels are still a problem.
The current decoding process for us leaves the black dots and occasionally blurring of pixels half-way across the image. Highly distorted images indeed, on occasion.
This is shown in the most recent version of LGP Tools.


It is a good idea to get some information from that guy. He may even know more about the other sectors of those files.
halkun stated once that shop prices and general shop information are not yet hacked. Surely that is in those files.

It would be great if he can shed the light here; I would really like to make more progress on my own project, which has been collecting dust for 3 months now.
I would be very excited to get those 2-D images running in my engine because I could then proceed with the walk-meshes and get online interaction going.


I still work on the battle animation format sometimes but I make no ground.


L. Spiro