Messages

1

General Discussion / Final Fantasy VII’s Surprisingly Advanced AI System

« on: 2023-10-19 00:12:52 »

https://youtu.be/S-8PVydb9CM?si=yL6Nu_EvCfEPvdco

How are enemies scripted in Final Fantasy VII? What is the AI system capable of? In this video I give a technical overview before diving into a simple script, to tour how the system operates.

We reflect on the reasons behind the system's design and the surprising similarities between FFVII and modern videogames.

2

General Discussion / What do we know about the FFVII battle script language?

« on: 2023-05-08 21:10:32 »

FFVII's battles run on a fairly powerful scripting system that has been explored in serious depth on these forums. We know a lot about

- the script format;
- data addressable within scripts;
- the runtime that actually executes the scripts

But I don't think we know a lot about how scripts were authored.

I think it's reasonable to say the FFVII battle planners did not work in the bytecode format, manually entering numeric opcodes or calculating jump offsets. These people were designers, script writers, and artists. They needed something far more human-friendly.

For a bit of context, here's some FFVII battle bytecode

Code: [Select]

70 0123 ; jump to 0123 - I have to keep on top of this offset
12 2070 ; push address of "target mask"
02 20A0 ; read value of "opponents mask"
90 ; store, clear stack
60 03 ; push "summon" command
60 3F ; push "Bahamut" ID
92 ; summon bahamut on action queue, clear stack

Some things I think we know about these scripts:

- Square probably had a DSL (domain specific language) adapted from previous titles, and managed by the core programmers of BATTLE.X for assisting with battle scripts
- These scripts could probably be authored with and linked to other battle data, making it simpler to re-use them
- Square anticipated some things going wrong with these scripts - thus the DATA ERROR fallback condition - but only implemented rudimentary error detection in BATTLE.X (example: if you run 86 "MP cost" for an attack ID not in the scene, the game just crashes)
- Debugging was enough of a concern that there's an opcode (A0) that pushes sprintf format lines to the debug console (I think dev PlayStations had a mechanism for the BIOS to push this text to an attached TTY)

Things we could speculate:

- Square had existing tooling coming off of FFVI, but their goals for VII were much more ambitious. They may have sought a ground-up rethink of battle authoring
- Square wouldn't have had the benefit of modern language toolkits, but that wouldn't stop them implementing simple languages. FORTH in particular is a natural fit for a stack-based programming model
- They may have used latin characters in their scripts just for convenience with the operating systems and tooling of the time
- As far as I know, there is no documentation around about how Square wrote their battle scripts for FFVII, or any of the other PSX era games
- The focus of a scripting language is rapid iteration and ease of use. There would be only simple validation, basic control flow, and a lightweight compilation step.

Some ideas in my head:

- it is not very clear to me how battles exactly were developed. Was it possible to execute an ad-hoc battle on a running debug console? Was it even possible to flash battle and character scripts over the serial port? It would seem risky to do "hot reloading" as the battle engine would not tolerate scripts being rewritten in-place, but perhaps some other form of rapid dev loop was possible
- would the DSL have been upfront about the stack based data structure, our would it have attempted to disguise it with variables? The latter seems to add complexity and disguise the constraints of the platform.
- authors would have probably not have had to worry about "types" (bits / counts of bytes). And the DSL probably provided something like if conditions over jumps

What do we know? What can we speculate? It's something I've been thinking about a lot

3

Completely Unrelated / Re: I now own a Net Yaroze!

« on: 2023-05-08 20:01:27 »

Managed to find it on eBay on the cheap! A local seller as well.

I'd love to have a go at some homebrew, but I think the best way to do that is with an emulator. If I wanted to work on the Yaroze I'd need to jury rig a special serial cable. Also the Yaroze only has 2mb memory, not the 8mb of a dev console, so you can't compile with debug symbols 🙂

However, there is the small matter of me not really knowing C. Well, not well enough to know what I'm doing, anyway. But perhaps this could be a good hobby project for getting my head around C?

4

Completely Unrelated / I now own a Net Yaroze!

« on: 2023-05-01 17:14:31 »

Very pleased with this - I have wanted one since, like, forever



I'm also fortunate enough to receive the original SDKs and Sony Yaroze documentation. This attempts to explain 3D PSX development to an audience of non-gamedev C programmers, with a bunch of info about PSX internals.

It doesn't have the serial cable, alas. I believe I can re-make one by taking a normal PSX serial cable and adding an FTDI bridge to invert the CTS, RTS, DSR and DTR lines. And to use a 3.3 voltage and not default RS232 12v power. However, I don't intend to do any actual development on the machine - I can use an emulator to do that with much more convenience.

I'm just pleased to own a bit of PlayStation history!

5

Scripting and Reverse Engineering / Re: [FF7 PSX] What should be removed after recompressing the BATTLE.X file?

« on: 2021-05-31 23:03:47 »

It's been a long time since I worked on the BATTLE.X file. However, I did find the following README I wrote to myself, in a file from 2014. It was for running an assembler patch for long range enemy attacks:

Here's the procedure:

- unpack the BATTLE.X by removing the initial eight header bytes (copy them somewhere)
- decompress the file as a gzip archive
- run the ASM file (enemy_long_range_attacks_asm.txt) through ARMIPS
- recompress with gzip (using the Cygwin version of gzip with the flags -9 -n will work best)
- re-add the initial header bytes
- reinsert your BATTLE.X file using whatever means you like

6

Completely Unrelated / Re: Cloud: "...A lot has happened"

« on: 2021-05-09 15:00:02 »

Welcome back, alot has happened. Its once again a whole new crowd around here. Some of the older users do pop on from time to time. Lots of progress has been made w/ FFNx (new driver based on aali's old code). They can always use more eyes if you want to dip your toes in to gfx. On the mod front there is a lot a nice new backgrounds and for those that use it I've heard the new version of 7th heaven works well.There are also quite a mods few tools being worked on still If your looking for other things to code.

Hi Sithlord! Thanks for the pointers, I'll look into those.

(Also - OT for FF, but I'm also thinking about working on this: https://github.com/AliveTeam/alive_reversing)

Join the Discord, much more activity there.

I will do just that!

7

Completely Unrelated / Cloud: "...A lot has happened"

« on: 2021-05-08 20:47:34 »

Hello everyone. It's been quite a while, hasn't it? I don't think I've been a regular member of this forum in - well - close to ten years now.

What has happened to everyone?

I remember Halkun was working for a console vendor - maybe it was Nintendo? Is he still in the reverse engineering scene at all? A lot of the FFVII internals YouTubers have gone dark. KudistosMegistos seems to have erased himself from the internet; Terence Ferguson too. I sometimes wonder what they're up to.

I saw Seifer/Dan/DLPB mentioned in a article in... Vice, I think? And of course, he's gotten himself banned again. I guess some things never change.

I'm a software engineer now, living in London. I don't work in games but I have done bits of VR work - I wouldn't mind getting into graphics programming. It would be nice to work on something that isn't just shuffling strings in and out of databases! I've got other priorities this year first, though: I'm currently losing a lot of weight, dating again after the pandemic, and trying to start writing again. Maybe once that's stable I can look at other development careers.

I never did finish my big FFVII mod, Rebirth. The scope of the project just grew and grew, and every time I wanted to rebalance something, I had to start playtesting all over. The irony is, ten years later, I actually would have the knowhow to finish it.

I can see there's been a lot of progress elsewhere, though. I just picked up the Lunatic Pandora mod for FFVIII, and the beta for Angelwing. I think it's absolutely amazing where we are nowadays. What are people working on these days?

8

General Discussion / Re: Non-battle materia interactions in Final Fantasy VII

« on: 2019-04-22 19:57:49 »

No.  The designers simply give you materia because it's a game.  It has nothing whatsoever to do with the storyline.

You're saying that Cait Sith carries a Manipulate materia by plain coincidence? It just seems a bit too much of a smoking gun to me. If I were designing that section I would definitely consider concealing hints in character equipment to intrigue and delight the player.

Current mystery - why Sephiroth throws the Destruct materia at Cloud.

Destruct is a dark, mean-sounding materia; Sephiroth is a dark, mean-kinda guy. I'm not sure what else would be appropriate for him to throw at this stage in the game (he can hardly hand over an Ultima or Contain).

I do think the destruct materia is linked to his Barrier/MBarrier use as a final boss, but in a more prosaic way: it gives the player an 'aha' moment when they remember that the materia Sephiroth gave them holds the key to beating him at the end of the game.

Relationships like that give the player clues what to do and pleasure when they recognise the callback. It's like when you use a brick joke in stand-up.

9

Scripting and Reverse Engineering / Re: What was the development context for FFVII's battle scripts?

« on: 2019-03-27 22:02:34 »

Thanks MysticLord - if I dig anything up, I'll post it here.

10

Scripting and Reverse Engineering / Re: What was the development context for FFVII's battle scripts?

« on: 2019-03-25 21:47:06 »

I suspect you're right. It's a project that finished over two decades ago. No-one close to the development of this module is likely to remember much or feel particularly motivated to talk about the development process. It's a shame; I find that kind of development archaeology fascinating.

11

Scripting and Reverse Engineering / What was the development context for FFVII's battle scripts?

« on: 2019-03-24 17:59:31 »

What were Square's motivations and design goals when implementing the battle scripting system?

I assume the main objective was to create something that could be easily hotswapped on a dev unit - as recompiling modules was presumably quite slow - but I wonder if there were other intentions too. Were battle scripts designed to be written by artists rather than programmers (perhaps in a simple intermediary format)? Did debug builds allow developers to step over / breakpoint AI code whilst beta testing?

How much do we know about how FFVII's battles were implemented? How different is the demo BATTLE.X from gold? Is much really known about this, or is it generally treated as a Square trade secret?

12

Scripting and Reverse Engineering / Re: [FFVII] Enemy AI Script questions

« on: 2019-03-11 20:06:22 »

In as much as it was probably a programming oversight. In fairness FFVII's battles are actually quite hard to manually test - would most QAs know that an attack was supposed to hit for 210 damage rather than 290? Or that a section of the AI script never fired? It's fairly opaque.

I'm not sure how modern RPG projects program battle scenarios, but I expect there's much better tooling these days. Even just being able to run the AI scripts in a test framework would have been a massive boon, but automated testing wasn't on the radar of anyone in 1997 outside of finance, defence or academia.

13

Scripting and Reverse Engineering / Re: [FFVII] Enemy AI Script questions

« on: 2019-03-11 19:54:22 »

There's a lot of bugs like that. For instance: all enemy attacks do half damage at long range, unless they're actual magic commands. The more one looks into FFVII's battle system the more rushed and incomplete it appears. Fights are fun but are only a shadow of what they could have been, had certain ideas been developed further.

Look at all the obscure status effects and strange damage formulae. Look at how big and complicated Eligor's AI script is - when 80% of players will miss him completely. Read through the nuances of how the MPs react to Cloud's party position. It's clear Square had huge ambitions for VII's battle scenarios but not nearly enough time to realise them.

14

Scripting and Reverse Engineering / Re: [FFVII] Enemy AI Script questions

« on: 2019-03-11 16:49:57 »

So, I'm left still unsure as to what exactly is disabling the end of battle rewards. I'm aware in the back of my mind that there is an easy fix by just assigning Heli Gunner's EXP AP and GIL value to match that what it should have been if Hundred Gunner did leave rewards, but I'd rather identify the problem instead of creating work-arounds. I'm pretty sure there are a few examples of the Spoil values being toggled manually in-battle; take the Trickplay for example (increases gil earned by 800 with every use of Gold Mountain).

I think your best bet is to just to fire up an emulator with a debugger, replay the end of a battle (e.g. using saved states) and just trace where the exp total comes from.

Personally I'd be tempted to just modify the exp reward values in the unit struct directly from a post-death script, but if you need a solution that's easy to invoke and 'robust' then the ideal would be to find an unused status flag and patch BATTLE.X to use a custom exp calculation function with the help of e.g. ARMIPS. You can then distribute the patch separately as a basis for other mods.

15

Scripting and Reverse Engineering / Re: [FFVII] Enemy AI Script questions

« on: 2019-03-11 00:53:41 »

Also, what is 2120 specifically? The Wiki says"Elements of last performed action." is that the last action performed *by* the actor; last action performed *on* the actor; or the last action in general by any actor?

The latter. I can't remember if it's set by the time the pre-action script runs, or you can only reliably read the element of incoming attacks in counter scripts. It can be handy in mods when implementing things like an 'oil' status.

In my opinion, having the Soldier:3rd only attack players physically when they do not have sleep, or only attacking them magically when they do have sleep is a clever AI design. It's the type of thing that I want to focus on when I write my mod, instead of just 50% do this 50% do that. The Solder:3rd will never cure a character of sleep; you have to cure them yourself by attacking them or wait it out.

I've done this kind of thing before. It's not that easy to balance. Enemies that hone in your weaknesses and exploit status effects sound interesting but aren't always fun to play against. Remember how FFIX enemies always seemed to target party members who you'd just Phoenix Downed? It's quite cool to see now and again but the player needs options to adapt and counter the strategy mid-battle, or it just feels like the AI is cheesing.

16

Scripting and Reverse Engineering / Re: [FFVII] Enemy AI Script questions

« on: 2019-03-11 00:45:59 »

Do we think that the Hundred/Heli Gunner battle should have had the Still More Fighting theme play?

I think the Cloud / Rufus fight might have lost something if it simply repeated the music from the Heli Gunner.

Also, the EXP AP and GIL for Hundred Gunner are not earned, I'm guessing this is because of the fact that the game uses "Next Arenas" for the battle? Or is it because the battle doesn't end upon killing that enemy, the same as other enemies that "transform"? In fact, thinking on it, what exactly is controlling the deactivation of Spoils earned by Enemies after a transformation? Why exactly is it that the Hell House (for example) does not give spoils for killing both versions?

There are quite a few questions here. It's been a long time since I looked into FF7's scripts (5+ years), so my memory is quite hazy, but I'll do my best to answer.

Hundred Gunner / Heli Gunner don't work the way Battle Arena does. The arena is implemented via field scripting: field code can start battles and pass a bit flag to enable various battle mechanics features, one of which is a continuous multi-formation battle with player prompts every victory ("Off course!"). By contrast the Gunners (and things like Hell House) are implemented via battle AI scripts which enable / disable units in the same formation and copy over a struct of associated data (so that the game can simulate continuity in terms of statuses, stats multipliers - etc)

I don't remember researching how exp is gathered at the end of a battle. I don't think the Exp screen is actually part of the menu modules, but I may be wrong. If it is not a menu then it will presumably be powered by BATTLE.X. Looking at Proud Clod, there seem to be some actor flags (e.g. 4022/4024) that typically get set when those 'transforming' enemies die - my guess is that the battle engine uses these to determine whether a unit should be included in the exp calculation. I'd suggest should sniff around Battle.x to see what functions get called at the end of a battle.

Curiously enough, you can actually alter exp / ap / gil rewards for an enemy with AI scripts directly, but I'm not sure if any enemies actually do so.

17

FF7 Tools / Re: FFVII Enemy Database & Calculator V1.0.0

« on: 2019-02-19 11:31:51 »

Yeah, great work Dynamix. I would have absolutely loved this whilst developing Rebirth.

18

Completely Unrelated / Re: Considering learning programming

« on: 2019-02-14 12:48:54 »

@QuantumPencil - how do you feel about Rust?

19

Completely Unrelated / Re: Considering learning programming

« on: 2019-02-12 09:44:14 »

There's a reason why [C] remains the only choice for real-time (deterministic) programming tasks, or embedded systems programming.

Not true. Lots of scientific simulation and maths code is still written in Fortran. And safety-critical embedded systems are often written in Ada - I have a friend who was writing new Ada code for satellite systems as recently as three years ago.

However C does not force you to rely on higher level "safety" features present in most other compiled language which means it does allow you to write much faster code than in most other languages.

Yeah, maybe, but it is also almost impossible for 99% of human beings to write large applications in C that are totally secure, unless you've got some kind of cybernetic implants in your brain. For instance - quick q, what's wrong with this snippet?

static int podhd_try_init(struct usb_interface *interface,
        struct usb_line6_podhd *podhd)
{
  int err;
  struct usb_line6 *line6 = &podhd->line6;

  if ((interface == NULL) || (podhd == NULL))
    return -ENODEV;
  ....
}

That second null check won't do what you think it will, at least at first glance.

Of course, fiendishness like this is exactly why I think C is rewarding to learn. But actually using it is a different story!

20

Completely Unrelated / Re: Considering learning programming

« on: 2019-01-28 11:59:18 »

Yes, I'm aware of intermediate languages. As far as I'm concerned, though compiling to an IL or bytecode that then still needs to be interpreted still qualifies as an interpreted language, for the sake of a performance comparison to languages that compile straight to machine code.[/spoiler]

Yes, but remember that most runtimes / language virtual machines are able to do recycle generated machine code and perform 'hotspot' AOT optimisation on heavily used functions.

Even the JavaScript V8 runtime does it these days: Chrome will spot that functions keep getting invoked with a particular call signature and begin not only caching inline machine code, but actively running background recompiles to speed them up. Once "warmed up" things like V8 and the JVM can achieve some very impressive results.

As for speed, C++ must be the worst of all the 'fast' languages. Its design is so gangly, it inspired another language to fix its mistakes (Java). Here's hoping that Rust becomes the systems programming language du jour.

(And btw, DLPB, they don't end up as assembly. They end up as machine code.

They do, but don't forget, machine code isn't even the lowest level instruction set. Most processors actually unroll their instructions into microcode. How those get mapped onto permutations of NAND gates etc. I don't really know.

21

Completely Unrelated / Re: Considering learning programming

« on: 2019-01-26 22:24:33 »

Dropping my five cents into the ring: I don't think C++ is a great first language for programmers. Not because it is too low level, but because it is too hairy and inconsistent. It is very much a language that has been grown rather than designed; using C++ in the workplace tends to mean carving out a sane subset your team are all happy to work in (e.g. C++ 11).

C, on the other hand, is a great language. It's small, it's fast to compile, and it teaches all the basic concepts. You can "get" C within a couple of weeks. You won't get much work writing C, but if you're learning to program to reverse engineer things I can hardly think of a better way to encounter all those low level ideas whilst still being productive in a relatively high level language.


If employment is a motive...

Then the most obvious choice is Java, because pretty much everyone uses Java for systems programming. Then again you can do a lot with JavaScript these days - you can even call modules written in C if you're using Node (or a runtime that embeds Node, like Electron). This can be a nice way of threading together key modules that are run as native code with a very terse scripting language (JS). I know SithLord despises Electron, and it has its flaws, but there aren't many competitors for sheer cross-platform productivity. Node and NAPI also provide a surprisingly good C/C++ buildchain.

For my day job I tend to write mostly Java, Kotlin, TypeScript and JS. WebGL here and there, because I'm quite interested in things like WebVR. I do sometimes write a bit of C++ but it's not commonplace. I might get an opportunity to use Clojure soon, which is quite exciting.

22

FF9 Graphical Releases / Re: [FF9PC] Background upscaling mod

« on: 2019-01-26 22:15:46 »

This is truly excellent work. It would never have occurred to me to use ML to upscale the background layers. The results are phenomenal in a game as beautiful as FFIX.

My only concern - and I don't really want to add a sour note to this discussion - what's the position of anyone who distributes artwork that's transformed from SE's original assets? Are ML manipulated backgrounds treated as a transformed work? I wouldn't want any HD pack to be stymied by legal problems.

23

General Discussion / Re: FF7 : certain enemy skills break the game

« on: 2016-10-17 20:41:00 »

What we have is a polished up version of the setup system they used to test the job classes. It's very likely that enemyskills were meant for Red XIII (Cait Sith maybe too). The game would make even more sense with Job Class Materia, because you would use the abilities of a Cetra or Cetras.

That's a really interesting idea: that the original plan was to have materia bound to archetypal character classes. Where did you hear that?

24

General Discussion / Re: The voice in clouds head ....

« on: 2016-10-17 20:38:45 »

Yes, 'clones' is a very unfortunate choice of terminology indeed. When Sephiroth reveals Cloud was never the man he thought he was, I'm sure a lot of players interpreted 'clone' to mean Cloud was some kind of artificial, genetic copy, grown in a vat somewhere.

I'm not sure what translation I'd use myself. 'Derivative'? 'Imitation'? 'Analogue'?

25

General Discussion / Re: FF7 : monster damage drops dramatically after you leave Midgar?

« on: 2016-10-17 20:24:22 »

FFVII's monsters are probably the weakest element of its design. Now, I have an interest in saying that - I've worked on a mod for several years (on and off) that drastically modifies enemy stats and formations - but the world map encounters are the most clearly insipid of the bunch. Enemies are slow, hit for trivial amounts of damage, rarely inflict status effects and are essentially interchangeable. Their graphic design is vibrant and varied, but that becomes a problem in itself when that creature design itself is so random and arbitrary.

Enemies are rarely explained or have much to do with their surroundings. They generally do not follow any consistent rules for status or elemental vulnerabilities, and their statistics are mostly interchangeable. We can all remember what a Zuu looked like, or a Yin-Yang, or even a Vlakorados, but can you remember what their attacks were? Any special tactics they demanded? Even the bosses were mostly copy-pastes of each other with incremental stat raises.

Edit : Interestingly, Jenova-birth's standard laser attack does about 175 damage to characters in the front row and half that to characters in the backrow, even though it is NOT listed as a short range attack and should not be affected by the row modifiers

All enemy attacks are treated as short-range. See here: http://forums.qhimm.com/index.php?topic=14975.0