Engine fantasy.

[halkun]

I've been contemplating writing the framework for an engine this coming summer. Here's the rub. I suck at coding. But I don't think that's a huge problem ^_^

Here's some bare bones...

1)Platfrom: Cross platfrom

2) Language - C - (GCC)
Rational: It's mature, portable, cross-platfrom and the language I know. It will lend itself to more help if I need help as C++ people can do C, but not the other way around.

3) 3D Graphics - OpenGL
Rational: It's mature, portable, cross platfrom, and the only 3D API I know. It's pretty simple to pick up and whole books to learn it are downlaodable for free.

4) Display Management - SDL/Allegro
Rational: Both of these are a much better manager than GLUT. Allegro is a really pretty API, but I don't know how AllegroGL can co-operate with OpenGL. SDL has lots of pretty add-ons including MIDI and stuff. Both are also cross platform. I don't know which one to use. I've done SDL, but it's a little too "low level" for me. I'm used to the Allegro API

5) Repository: CVS
For a project like this a code repository would be cool. Sourceforge gives them away.

6) Licence: GPL
I'll need the help ^_^

----------------

My first task will be to load kernel.bin and initalize the savemap. After that, a framwork menu system for starting the system and then seeing if we can get a field file to "boot"

small steps

Then again, I might just be playing pretend again.

[Micky]

I'll have a look, maybe I can donate some text rendering code that uses the ff7 font. I'll have to rip it out of my test framework, first.

[Akari]

2) Language - C - (GCC)
Rational: It's mature, portable, cross-platfrom and the language I know. It will lend itself to more help if I need help as C++ people can do C, but not the other way around.

I think we shoud use C++ instead of C.
Rational: It's not much harder than C. It can do everything that C can do.
It has a lot of standart containers and algorithm in STL. The C++ code easily reusable in other projects (Xenogears engine for example or something else we may want to write) unlike the C implementation. And we can find more pluses of uses C++ instead of C.

[Cyberman]

Another less obvious point is C++ can be used like C if need be

If you are going for a generic system.  SDL might be OK. I would prefer the use of GTK since it would allow the frame work and I believe you can still use SDL with it.  GTK will make portability easier as well as developement of the GUI easier. It is highly portable to Win32 platforms as well as win?? (XP is the ?? Longhorn is going to be a problem I fear).

I recomend looking at this from a virtualization perspective also.  IE what is the best way to make everything function the same from perspective of the engine across multiple platforms.  I know you like linux so do I.

I also recomend looking at Eclipse.

What MEDIA are you planing on using for the engine? There is PC and PS1 media for FF7. this will be an important decision since it will affect how the engine deals with data.  I don't believe the original FF7 operates quite the way we think it might to be honest.  I'm fairly certain that the ISO system information is just there because they didn't bother obscuring it. I will check if it has something similiar to Xenogears hidden in it

As for handling the source code, I think Source forge will give you the tools, whatever they are using I recomend using. I use to have a sourceforge account which I couldn't find the information for if my life depended on it.

[Micky]

Oh, and feel free to use the battle background rendering code I posted ages ago.

[Sukaeto]

If you want to release it GPL, I would recommend using QT.

As much as I love C, the code tends to get very messy after awhile.  Plus, it'll be tougher to keep it cross platform.  Since you already know C, you won't have a problem using QT, beings that it's C++.

[Micky]

I'm not sure what you'd use QT or GTK for? FF7 is a full-screen game, so there is little use for all the stuff that a UI library provide.

[Qhimm]

@ C vs. C++: Why not a mix then? I've found that C++ is most useful at the low levels of programming. Smart types/objects, templates, containers, all the "base" code that you write once and then just use, greatly simplifying the mundane tasks. That having been said, using C++ at the higher levels of programming can be bothersome and present unnecessary problems, unless you have a very clear code design document from the start (which I believe a project like this won't have, since so much of the wanted functionality is ...well, yet unknown). Therefore, you could have a bunch of C++ people writing the "tools" which the C people can then use without too much adjustment (learning to write "object.method()" instead of "function(objectptr)" is the easy part).

Just my two cents, since IMHO where C++ really shines is being able to make your code more abstract, more functionality and less form, and being able to do so at neglible or no performance loss.

[Sukaeto]

I'm not sure what you'd use QT or GTK for? FF7 is a full-screen game, so there is little use for all the stuff that a UI library provide.

QT is FAR more than a UI library.  It's an entire API, complete with it's own ADTs, pointer types, Sockets implementation, SQL and OpenGL plugins among other things.  GTK in no way BEGINS to compare to QT.

[halkun]

WOW! Talk about over-engineering!

Ok let me backtrack. I'm glad we got this discussion going. Let's indulge.

Here's the goal: make a small, cross platform engine that is compatible with FF7PC, FF7PSX, and user created data.

Here's the problems I have, and why some of this discussion became very silly.

First.

I absolutely loathe OO Programming. Period. I've given C++ more than a few shots, but I just don't like the abstraction. I come from the days of programming when computers were considered procedural systems. They still are. I just don't like the concept of placing a function within a structural entity. (Classes) I've tried. I really did. But I simply don't grok the OO discipline as a whole. I can perfectly concede that there are those who swear by at, and use it very elegantly. I can't. My roots are in Machine Code and C sits on it really well. It's human enough to work with the system, but Machine-y enough to see what's going on under the hood.

Another facet is the C++ is what the original FF7PC was written in. The last thing I would like is have undue stress if Square tries to pull some kind of copyright heartache. It's best to write it in another language. I just want to do the engine. No data unless it's user supplied. (I.E. SummVM) The "big" project is to make an RPGMaker kind of system. (Think FreeSCI) In the end, I don't want to use square's data at all.

I also know C. It's silly to start a project in a laguage I simply can't speak.

++++++++++++

On the display Management side. I saw GTK+ and QT tossed around. To quote Tycho from Penny Arcade... "That went to a weird place"

What I mean by a "display manager" is the ability to make a 2D surface for OpenGL to draw in. GLUT is the "Native" manager, but it doesn't have any ability to do other cool things like thread support, MIDI and wave support, and it's 2D texture support is a little clunky. SDL is cool, but a little low level. There are modules available that could fill the gap. Allegro is "higher" but has no threading support and can't do GL management without a special extension. I want this to be cross-platform. (Think PDA or PSP) I'm not going to recompile GTK or QT just to get a display surface hook for OpenGL.

That's silly.

Keep in mind the ORIGINAL FF7 ran on 2MB of ram, with a very tiny BIOS interface (512k)

QT is FAR more than a UI library. It's an entire API, complete with it's own ADTs, pointer types, Sockets implementation, SQL and OpenGL plugins among other things. GTK in no way BEGINS to compare to QT.

We don't need ANY of this crap. Why does FF7 need sockets of *any* kind. SQL? I need a OpenGL *surface* that's it. GLUT will work, but I'd like to use a gaming lib to gave me access to MIDI and stuff along with a GL surface. Something small, modular, and compact.

I don't know where you guys are getting a GUI idea from. The FF7 GUI is in the system iteslf and uses mostly 8 buttons. (up, down, left, ok[O], cancel[X], menu[triangle], and.... that other button[square]
++++++++++++++++++

Right now simply writing a program that allocates memory for the savemap and initializes the "new game" data should be fine. This information is found in Kernal.bin and the only lib needed is zlib to make it happen.

Here's some problems.

I have found that my linux computers can't read the FF7PSX disks. They were written in mode 1 and my CD-R/DVD-R simply barfs on them. This is going to be an issue with Linux users as that means you have to loop-mount an ISO or do direct hardware reads, bypassing the OS. That will suck.

Let's work out problems like this first.

I'm also going to be making a UML flowchart with Dia to get a framework down on FF7's execution strategy. I'll be putting this up online. You can download Dia yourself and we can play with psudocode before committing anything to a compiler.

Comments?

[Cyberman]

First.

I absolutely loathe OO Programming. Period. I've given C++ more than a few shots, but I just don't like the abstraction. I come from the days of programming when computers were considered procedural systems. They still are. I just don't like the concept of placing a function within a structural entity. (Classes) I've tried. I really did. But I simply don't grok the OO discipline as a whole. I can perfectly concede that there are those who swear by at, and use it very elegantly. I can't. My roots are in Machine Code and C sits on it really well. It's human enough to work with the system, but Machine-y enough to see what's going on under the hood.

Ok
Well don't get all bothered about C++ you can use a C++ compilor and write in C. I suggest you follow Qhimm's logic a bit.  The C++ abstraction of low level things makes your C simpler.  Let me explain, I write for embeded systems. I use prefer using C++.  Why? because low level code using C is complicated and eventual ends up what C++ would do anyways. Otherwise working with low level hardware requires a special version of ansi C (IE with extensions and #pragma's to make things work cleanly on the hardware). And I'm talking lower level than a device driver here.  Abstract your low level interface using C++ it makes things significantly more portable OK? That's as simple as it gets. Everything else can be in C.

Another facet is the C++ is what the original FF7PC was written in. The last thing I would like is have undue stress if Square tries to pull some kind of copyright heartache. It's best to write it in another language. I just want to do the engine. No data unless it's user supplied. (I.E. SummVM) The "big" project is to make an RPGMaker kind of system. (Think FreeSCI) In the end, I don't want to use square's data at all.

I also know C. It's silly to start a project in a laguage I simply can't speak.

No problem it can use C++ without you even knowing it. It's as simple as that.

What I mean by a "display manager" is the ability to make a 2D surface for OpenGL to draw in. GLUT is the "Native" manager, but it doesn't have any ability to do other cool things like thread support, MIDI and wave support, and it's 2D texture support is a little clunky. SDL is cool, but a little low level. There are modules available that could fill the gap. Allegro is "higher" but has no threading support and can't do GL management without a special extension. I want this to be cross-platform. (Think PDA or PSP) I'm not going to recompile GTK or QT just to get a display surface hook for OpenGL.

OK I can see QT... why are you hell bent on no UI to manage the game outside the game? You expect people to launch this from the CLI with the options they want?

We don't need ANY of this crap. Why does FF7 need sockets of *any* kind. SQL? I need a OpenGL *surface* that's it. GLUT will work, but I'd like to use a gaming lib to gave me access to MIDI and stuff along with a GL surface. Something small, modular, and compact.

That I agree too We don't need QT's features.  GTK just allows a nice GLUE the darn thing together interface.

I don't know where you guys are getting a GUI idea from. The FF7 GUI is in the system iteslf and uses mostly 8 buttons. (up, down, left, ok

• , cancel[X], menu[triangle], and.... that other button[square]

The original system is an embeded system a PC is not.  I suggest not trying to make a UI like those anoying games that possess your PC

Right now simply writing a program that allocates memory for the savemap and initializes the "new game" data should be fine. This information is found in Kernal.bin and the only lib needed is zlib to make it happen.

Hmmm OK this is doable right now no?

I have found that my linux computers can't read the FF7PSX disks. They were written in mode 1 and my CD-R/DVD-R simply barfs on them. This is going to be an issue with Linux users as that means you have to loop-mount an ISO or do direct hardware reads, bypassing the OS. That will suck.

Ummm Mode 2 Form 1 and 2 primary form1 though. (See xenogears playing I've been doing).  

Let's work out problems like this first.

I'm also going to be making a UML flowchart with Dia to get a framework down on FF7's execution strategy. I'll be putting this up online. You can download Dia yourself and we can play with psudocode before committing anything to a compiler.

Comments?

Well for the Media handling a RAW iso is the only way for Linux to run the media.  
This is because most of the data is recorded in Mode 2 form 1 and form 2 (this includes XA audio).
The sectors are as a resullt 2352 bytes of raw data. By examing the first 16 bytes of the sector header (12 bytes sync 1 minute 1 second 1 frame 1 mode), you get the mode. All PS1 CD's are mode 2 and mode 1. So you MUST read this header.  Mode1 sectors are 2K data 32bit CRC 8 bytes unused and 276 bytes error correction data (reed solomon parity data) This breaks the data up as block size 4 bytes. 79 blocks repair data (ECC data) and 512 blocks data for the ability to repair 15% corruption in the data. Mode 1 form 2 I believe has No error correction data.
Mode 2 has 2 forms
Form 1 and 2 have a subheader  1 byte FileNumber 1 ChannelNumber 1 SubMode 1 CodeInformation.  (there are 2 copies of this data).
Mode 2 form 1 is almost identical to Mode 1 form 1 (save it doesn't have 8 unused bytes because of the subheader).  Form 1 is used for Data on the PS1. Form2 has the subheader and 2334 bytes this is used for audio on the PS1. The Ps1 setup is such that things are setup in an almost nonsensical manner however it's a result of the PS1 hardware requiring 2x speed for reading all data so audio files etc are formated in odd ways.  For example 1 file might have the audio for 8 different conversations.  An FMV may have 7 video and 1 blank sector or 3 video and 1 audio interleaved throughout the disk.  Again a result of the PS1 2x requirement and the system expiring if you do anything else.

Bottom line is the PS1 media is complicated because of the hardware requirements.  You have to use an ISO 2352 sector image for Linux because Linux doesn't do raw data.

Cyb

[halkun]

I'm hell bent on no UI because managing the game outside the game can be handled by another program altogether. I'm happy simply to read the configuration from an .ini file and call it that.  If someone wants to write a front-end to manage this before the program executes (ala an installer), more power to them.

Keep in mind that the menu module in the engine is going to be tricky. The original FF7 used modules that seem to be scriptish, (the .mnu files) but I can't tell if that's native code that's banked in (most likely) or some sort of script language ala fieldscript.

Either case, the menu module was rewritten in the PC version and placed within the executable. (Which leads me to believe that the PSX .mnu files are compiled machine code banked in from the CD-ROM) At a later date we can debate if we want to hard-code the menu system or make some external TCL script thingy to make it more end-user configurable.

Now about mouting the PSX disks. When I put in a "normal" ide cdrom. I can mount the PSX files just peachy, or at least get a directory. I have a feeling the mode 2 problems  (oops I got my modes messed up) is because I'm running my DVD-R/CD-R through a virtual SCSI driver, even though the devices are IDE. The kicker is even when I do a direct /dev/hdd hexdump, the device hangs after a few sectors, leading me to believe that it's a driver issue. For some reason, however, epsxe can read the disk just fine.

I'm not much on CD-ROM authoring modes. I bearly know what a "mode" is other than the PSX uses something different than "normal" data CDs. (hense why I call the mode by the wrong name). I know red book (audio) and yellow book(mode 1). That's it. It would seem, that's all my writers support as well.


The other modes, in linux speak are these...

XA Form 1 and Green/White Book Mode 2
XA Form 2

I'm guessing these are what you are talking about, I don't think my driver supports these. (which is the same story on any CD-R drive I've owned.)

I'm going to look into cdfs for fun. That might be helpful .

On another level, I have no idea how to do all kinds of low-level stuff lin linux. My code-fu is weak.

Very weak.

The extent of my coding ability is I can somewhat use pointers in C. Use that as a gauge. In a perfect world, I would like to just use an open("filename") command to get the data and not muck about with raw device access.

I know much more about the PSX side of things. I hack via savestates and epsxe. In that case, I've never had a disk issue as the format the disk was in wasn't important. I would like to support both the PSX and PC versions. I need to be able to mount both under linux.

Why linux? It's the only OS I own.

[Cyberman]

I'm hell bent on no UI because managing the game outside the game can be handled by another program altogether. I'm happy simply to read the configuration from an .ini file and call it that.  If someone wants to write a front-end to manage this before the program executes (ala an installer), more power to them.

Hmmm Ok not a big deal.  I suppose unless you want to  pause the game execution and examine things. Not sure, perhaps a UI and an Engine. The engine being the portable part and the UI being the non portable part.  Not a big deal as far as I see it.

Keep in mind that the menu module in the engine is going to be tricky. The original FF7 used modules that seem to be scriptish, (the .mnu files) but I can't tell if that's native code that's banked in (most likely) or some sort of script language ala fieldscript.

I had not thought of that possibility.  I had assumed the menu's were scripted and the engine did stuff from there.

Either case, the menu module was rewritten in the PC version and placed within the executable. (Which leads me to believe that the PSX .mnu files are compiled machine code banked in from the CD-ROM) At a later date we can debate if we want to hard-code the menu system or make some external TCL script thingy to make it more end-user configurable.

Or DLL's not terribly important I suppose at the moment

Now about mouting the PSX disks. When I put in a "normal" ide cdrom. I can mount the PSX files just peachy, or at least get a directory. I have a feeling the mode 2 problems  (oops I got my modes messed up) is because I'm running my DVD-R/CD-R through a virtual SCSI driver, even though the devices are IDE. The kicker is even when I do a direct /dev/hdd hexdump, the device hangs after a few sectors, leading me to believe that it's a driver issue. For some reason, however, epsxe can read the disk just fine.

That's a bit odd unless the game only uses Mode 1 most seem to use both.

I'm going to look into cdfs for fun. That might be helpful .

All right but I've only been able to run games by using ISO cdrom plugins under linux.

The extent of my coding ability is I can somewhat use pointers in C. Use that as a gauge. In a perfect world, I would like to just use an open("filename") command to get the data and not muck about with raw device access.

Would be nice however with any PS1 data it's not pretty like that

I know much more about the PSX side of things. I hack via savestates and epsxe. In that case, I've never had a disk issue as the format the disk was in wasn't important. I would like to support both the PSX and PC versions. I need to be able to mount both under linux.

Why linux? It's the only OS I own.

I don't see Linux being a problem. Some thinking obviously needs to go into this though.  Ignore a Jihad of personal favorite concepts is always good. However as long as there are people there will be different ways of looking at things I suppose
You can directly read the PS1 data from an image file under linux until something better comes up. that would be the simplest approach at the start. Actually both Win32 and Linux varients should start this way as it's rather portable to open("filename") and read the disk image.  Brute Force is always the start. Things like that can be improved later.

So back to the start of things.
First order of business would be load kernal.bin?

Cyb

[halkun]

Well, It's getting late, so here's what I know about FF7's init system.

When you start the game, during the various "splash screens" (the squaresoft logo... etc) the system is actually the loading of the base kernel system and the supporting low level functions. For now let's just say that the engine is loaded, along with window.bin

A range of memory 0x10F4 bytes in length is allocated. A 256 byte scratchpad is also allocated. This is the savemap. Then file 3 of kenel.bin is uncompressed, (offset 0x063A) and placed in the savemap. This is the base structure of the map.

From here, on the psx, begin.mnu is loaded, this is (I think) the credits program. When you press "start" save.mnu is loaded in a special mode that loads and paints the buster sword and gives you the choice to have a New game or continue. When you Select "New Game", kernel.bin is opened and file 3 once again opened, uncompressed and copied to the savemap. After this file 4 (offset 0x0F7F) is uncompressed and copied at the top of the savemap to "reset" the character data. The menu system then hands over execution to the fieldscript engine and STARTMAP is executed. We then see the opening movie.

On a "Continue" The file picker is used to select a savegame. (save.mnu is still banked in and executing) Only the very head of each savegame is read for speed's sake. (Which is why the very top of the savemap is cosmetic, the only "real" data kept there is the window colors). When you select the game, CRC check is done, and the savemap is copied from the memory card to ram and the fieldmap that was saved in the savegame is executed.

That's how it all starts anyways ^_^

[Micky]

Halkun, when I was looking for something cd-related I found this:
http://www.gnu.org/software/libcdio/
That is a library for low-level cd access on Linux that maybe is some help for you. You'll need raw access to load the movie files.
If you use it through a wrapper then other people on Windows or MacOS can just plug their own code in later.

[Sukaeto]

We don't need ANY of this crap. Why does FF7 need sockets of *any* kind. SQL? I need a OpenGL *surface* that's it. GLUT will work, but I'd like to use a gaming lib to gave me access to MIDI and stuff along with a GL surface. Something small, modular, and compact.

You're absolutely right.  I realize you don't need any of that.  I got carried away and went off topic, and for that I apologize.

If you like low-level, SDL is the way to go (then again, you said that was a little TOO low level.)

As for the OO thing goes:  there are some cases where it's approprate, some cases where it's not.  The choice is yours, here.  I wouldn't try to convince you to change your mind.

Sorry I don't have more to add to this topic.

[sfx1999]

I would suggest using FreeGLUT or OpenGLUT to handle opening the surface, if that's what you need. They can do keys, also. They are under a the MIT license, IIRC.

To load images, I have used corona (http://corona.sourceforge.net/). It worked very well for me. Just make sure you know how to use it.

I haven't found a sound library that I liked. I was going to use OpenAL, but I couldn't get it to work with Dev-C++.

If you want to be portable, I suggest that you either not use MIDI, or use a software synthesizer. I never got MIDI working in Linux.

I also suggest C++ as a language. 3D engines greatly benefit from objects.

[Micky]

Guys, do me a favour and read this short article on bike sheds:
http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING
And yes, it does matter for this topic.

[halkun]

Guys, do me a favour and read this short article on bike sheds:
http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING
And yes, it does matter for this topic. :)

My name is Halkun, and I approve of the above link.

[sfx1999]

How about this:

Do what is the easiest for you, and don't just hack something together. Make sure your code is clear by commenting and using appropriate variable names so I can understand it.

I guess your stuck using structures instead of objects, unfortunately.

Make sure you learn about things like transparency and frustum culling if you are going to make it so you can walk around.  An FF7 style engine doesn't really need the culling, but you should definitely research transparency.

Another thing, you might want to peak at the code for Quake to see how Id did it.

[Cyberman]

Halkun:
I'll see if I can get something to work with an ISO image, I suppose ye olde CLI will be a start. I think an @<filename> setup method for testing is appropriate. We do need debugger output on STDERR as well. (IE what line was executed and what were the results).

Cyb

[Akari]

We do need debugger output on STDERR as well. (IE what line was executed and what were the results).

We could use this class (http://server.titansoft.ru/akari/logger.rar). It cross platform (It uses only standart library). It log info together with time (Maybe we need to do it optional?).
Of cource only if we deside to use C++  

[Akari]

Does anyone already writing something?

[Cyberman]

Well I've been fighting/installing Eclipse on my Linux box.

I'm also writting up a Plan Of Attack for it.
Has anyone here installed the CDT for eclipse? For some reason it's complaining about permissions.  I installed it as root but I'm adding the CDT as a user. I think this is causing a few problems.  Should I run and install the CDT for Eclipse SU root? (CDT is the C/C++ Eclipse plugin).

Erstwhile the plan is this.
[list=1]Read Config File
Mount Media - by this I mean locate the media image and set up to read the ISO.
proceed with Halkun's operations[/list:o]

I'm using Eclipse specifically since it's cross platform.
I have it for Windoze and Linux.

Makes things more consistant for me. Halkun it's OK you don't HAVE to have Eclipse to build the binaries of the code (LOL) It creates a makefile that's all you should need.

I suppose this is the SFFVM (S_quare F_inal F_antasy Virtual Machine)

Cyb

[halkun]

This was going to be a summer project... Like when I was done with school (Almost there)

I haven't had time copy so much as Kernel.bin from my CD yet, much less write a quick a dirty zlib decompressor.