Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Topics - Bosola

Pages: [1] 2 3
1
https://youtu.be/S-8PVydb9CM?si=yL6Nu_EvCfEPvdco

How are enemies scripted in Final Fantasy VII? What is the AI system capable of? In this video I give a technical overview before diving into a simple script, to tour how the system operates.

We reflect on the reasons behind the system's design and the surprising similarities between FFVII and modern videogames.

2
FFVII's battles run on a fairly powerful scripting system that has been explored in serious depth on these forums. We know a lot about

- the script format;
- data addressable within scripts;
- the runtime that actually executes the scripts

But I don't think we know a lot about how scripts were authored.

I think it's reasonable to say the FFVII battle planners did not work in the bytecode format, manually entering numeric opcodes or calculating jump offsets. These people were designers, script writers, and artists. They needed something far more human-friendly.

For a bit of context, here's some FFVII battle bytecode

Code: [Select]
70 0123 ; jump to 0123 - I have to keep on top of this offset
12 2070 ; push address of "target mask"
02 20A0 ; read value of "opponents mask"
90 ; store, clear stack
60 03 ; push "summon" command
60 3F ; push "Bahamut" ID
92 ; summon bahamut on action queue, clear stack

Some things I think we know about these scripts:

- Square probably had a DSL (domain specific language) adapted from previous titles, and managed by the core programmers of BATTLE.X for assisting with battle scripts
- These scripts could probably be authored with and linked to other battle data, making it simpler to re-use them
- Square anticipated some things going wrong with these scripts - thus the DATA ERROR fallback condition - but only implemented rudimentary error detection in BATTLE.X (example: if you run 86 "MP cost" for an attack ID not in the scene, the game just crashes)
- Debugging was enough of a concern that there's an opcode (A0) that pushes sprintf format lines to the debug console (I think dev PlayStations had a mechanism for the BIOS to push this text to an attached TTY)

Things we could speculate:

- Square had existing tooling coming off of FFVI, but their goals for VII were much more ambitious. They may have sought a ground-up rethink of battle authoring
- Square wouldn't have had the benefit of modern language toolkits, but that wouldn't stop them implementing simple languages. FORTH in particular is a natural fit for a stack-based programming model
- They may have used latin characters in their scripts just for convenience with the operating systems and tooling of the time
- As far as I know, there is no documentation around about how Square wrote their battle scripts for FFVII, or any of the other PSX era games
- The focus of a scripting language is rapid iteration and ease of use. There would be only simple validation, basic control flow, and a lightweight compilation step.

Some ideas in my head:

- it is not very clear to me how battles exactly were developed. Was it possible to execute an ad-hoc battle on a running debug console? Was it even possible to flash battle and character scripts over the serial port? It would seem risky to do "hot reloading" as the battle engine would not tolerate scripts being rewritten in-place, but perhaps some other form of rapid dev loop was possible
- would the DSL have been upfront about the stack based data structure, our would it have attempted to disguise it with variables? The latter seems to add complexity and disguise the constraints of the platform.
- authors would have probably not have had to worry about "types" (bits / counts of bytes). And the DSL probably provided something like if conditions over jumps

What do we know? What can we speculate? It's something I've been thinking about a lot

3
Completely Unrelated / I now own a Net Yaroze!
« on: 2023-05-01 17:14:31 »
Very pleased with this - I have wanted one since, like, forever



I'm also fortunate enough to receive the original SDKs and Sony Yaroze documentation. This attempts to explain 3D PSX development to an audience of non-gamedev C programmers, with a bunch of info about PSX internals.

It doesn't have the serial cable, alas. I believe I can re-make one by taking a normal PSX serial cable and adding an FTDI bridge to invert the CTS, RTS, DSR and DTR lines. And to use a 3.3 voltage and not default RS232 12v power. However, I don't intend to do any actual development on the machine - I can use an emulator to do that with much more convenience.

I'm just pleased to own a bit of PlayStation history!

4
Completely Unrelated / Cloud: "...A lot has happened"
« on: 2021-05-08 20:47:34 »
Hello everyone. It's been quite a while, hasn't it? I don't think I've been a regular member of this forum in - well - close to ten years now.

What has happened to everyone?

I remember Halkun was working for a console vendor - maybe it was Nintendo? Is he still in the reverse engineering scene at all? A lot of the FFVII internals YouTubers have gone dark. KudistosMegistos seems to have erased himself from the internet; Terence Ferguson too. I sometimes wonder what they're up to.

I saw Seifer/Dan/DLPB mentioned in a article in... Vice, I think? And of course, he's gotten himself banned again. I guess some things never change.

I'm a software engineer now, living in London. I don't work in games but I have done bits of VR work - I wouldn't mind getting into graphics programming. It would be nice to work on something that isn't just shuffling strings in and out of databases! I've got other priorities this year first, though: I'm currently losing a lot of weight, dating again after the pandemic, and trying to start writing again. Maybe once that's stable I can look at other development careers.

I never did finish my big FFVII mod, Rebirth. The scope of the project just grew and grew, and every time I wanted to rebalance something, I had to start playtesting all over. The irony is, ten years later, I actually would have the knowhow to finish it.

I can see there's been a lot of progress elsewhere, though. I just picked up the Lunatic Pandora mod for FFVIII, and the beta for Angelwing. I think it's absolutely amazing where we are nowadays. What are people working on these days?

5
What were Square's motivations and design goals when implementing the battle scripting system?

I assume the main objective was to create something that could be easily hotswapped on a dev unit - as recompiling modules was presumably quite slow - but I wonder if there were other intentions too. Were battle scripts designed to be written by artists rather than programmers (perhaps in a simple intermediary format)? Did debug builds allow developers to step over / breakpoint AI code whilst beta testing?

How much do we know about how FFVII's battles were implemented? How different is the demo BATTLE.X from gold? Is much really known about this, or is it generally treated as a Square trade secret?

6
Completely Unrelated / Crash Bandicoot disassembly
« on: 2016-03-03 22:31:24 »
It turns out there's a small group of developers who've taken to reverse engineering the Playstation-era Crash Bandicoot games.

There's a very thorough Crash 1 disassembly document over on Github, some file format docs for both the released games and the Crash 1 prototype, a Crash 2 port in C and even - and this is really impressive - a Crash 1/2/3 level editor.

7
Completely Unrelated / Reverse Engineering WipEout
« on: 2015-04-17 09:12:51 »
Someone has reverse-engineered WipEout's (PSX 1995) texture format and written an in-browser WebGL track viewer in JavaScript.

The blog post: http://phoboslab.org/log/2015/04/reverse-engineering-wipeout-psx

The viewer: http://phoboslab.org/wipeout/

The viewer's source: https://github.com/phoboslab/wipeout/

8
Completely Unrelated / Golang?
« on: 2014-12-02 19:59:15 »
Has anyone here ever played with Golang? What's people's opinions with it? It looks quite interesting at first glance; kind of like a cleaned up C++. I was going to look at Node for web development but it seems to me that Go's Goroutines are a lot more intrinsically attractive than the various workarounds for callback hell in server-side JavaScript (promises etc.)

I was going to have a play with Scala over the holidays but maybe Go might be more interesting.

9
The site hosting binaries of the excellent pSX emulator, http://psxemulator.gazaxian.com/, appears to have gone down.

I've added a mirror on my own domain and sent a message to the owner over Twitter.

10
We've had Hojo. We've seen Heidegger, later Proud Clod. White Choco and Black Choco have been claimed. Meteror, Palmer, and Ochu have been used as editor names too. There's one big name missing, and I'd like to fill that gap. That name is Sephiroth. And rather appropriately for someone who manipulates people, I think he'd be suited to an AI editor of some kind.

My proposal is a GUI AI script builder broadly based around the AI templating snippets I wrote way back when, taking a similar concept to FFXII's gambits. You build a script by creating a 'decision flow', essentially a list of 'possibilities'. Each possibility has a condition and an associated action. We build an AI script by giving each type of possibility a template, which can produce 'meta script' with symbolic addresses which we can fill in fairly easily in a single pass, as the templates each have a fixed length. Each template leaves two values on the stack - a target and an action ID - and each script ends by executing those items and calling the standard script exit function.

I've got a pretty decent idea how I'd implement this. I'd probably use web technologies as I did with the FFVII calculator, meaning a JavaScript application that can be packaged as something desktop-shaped if needs be.

I managed to get a basic UI up and going just now using a few of the frameworks I expected to use. That's the easy part, but perhaps it shows exactly how something like this might fit together:



What do people think? If it's something people are really keen on I reckon I could get something up in maybe a couple of weeks.

11
I owe my career to these forums.

I'd always been interested in computers growing up. Since the age of 8 or 9 I was obsessed with tinkering and exploring, and I took every opportunity I could to play with any computer I saw. Growing up, I kept a keen interest in programming and understanding how pieces of hardware and software 'fitted together'. I wrote several (bad) QBASIC programs, most notably 'Space Warrior', a repetitive space shmup with limited mechanics and flickery graphics (I don't think QBASIC's graphics API let me do any kind of double-buffer rendering). For my GCSE in electronics, I took the unusual route of programming a PIC controller in what was, retrospectively, a crude, stack-based assembly language.

But I stopped. Somewhere along the way I convinced myself that programming was outside of my capabilities. Perhaps I was burned by my attempts to program in C++ without the help of the internet and only a few snippets of tutorials I managed to read up on the school computers. Or maybe my other academic goals took over; by the time of my A-levels I'd decided I wanted to go into academia in the humanities. In any case, software was something I talked about passionately, but never pursued.

My plans took a diversion in 2010, though. With the recession in tow and trouble finding work before my planned (and abandoned) postgraduate studies, I found myself with a room, a poorly specced laptop and a lot of spare time. I'm not sure exactly when, but at some point the thought of modding FFVII crossed my mind and I made my first, tentative post on these forums. Somewhere along the way I learned I could gain a great deal from AI scripting, and began programming again for the first time since my teens. With the release of LiBrE (embarrassingly written in retrospect), I'd taken baby steps in my first 'proper' language, Python.

This didn't lead directly into a career as a software developer. But when I'd exhausted my options as a technical writer, then UX designer, my hobby led me to the conclusion that I always wanted to be a programmer. It was with some trepidation, as I still believed I might not succeed, but once I started (as a front end developer) my progress was rapid and my job satisfaction transformed my life. It's egotistical to say, but it became clear pretty quickly that I was an exceedingly strong engineer - much, much more than I'd written myself off as - and once that was recognized my confidence and self-esteem - and much more, my sense of worth - could grow properly.

And so, here we are now. As a self-taught developer I've still a lot to learn and do. And the field I work in has turned upside down over the last three or four years, with new technologies emerging every day. But wherever I go, my path will have started here - a little spark, in 2010.

12
If you've ever flirted with the idea of modifying the PSX FFVII executable, you've probably struggled to find much info on exactly how to do it. I know I did. That's why I'd like to post a brief how-to on the procedure for performing assembly edits on the PSX release.

Understanding the executable

Unlike FFVII PC, the PSX does not have a singular 'executable'. It is made up of overlays, representing the different parts of the engine (field, battle, menu, minigame etc.) and a small kernel storing state shared between the overlays. Overlays are so-called because they are 'painted over' memory space as they are loaded. So you are not going to edit 'the executable', but rather the particular overlay relevant to the behaviour you'd like to hack.

Overlays sit in their own individual files, and may even be themselves separated further still. For example, whilst the battle engine is simply BATTLE\BATTLE.X, the menu module is split into multiple .MNU files defining behaviours for different aspects of party management logic (SHOPMENU.MNU for shops; EQIPMENU.MNU for equipment; etc.). I don't know all the files for all the logic; I've only ever really taken an interest in BATTLE.X

Extracting and inserting the files on disc

The easiest way to do this is to use CD Mage beta 5. Hit the 'export file...' and 'import file...' dialogs to extract and insert your executable files.

Turning a .X file into readable assembler

FFVII executables are compressed, so you can't just run them straight through a disassembler. You have to unpack them first. You can do this by removing some header bytes, then treating the file as a gzip archive (.gz). You will need a hex editor (I like Frhed; it has quite a few features and it's freeware) and a utility for unpacking gzipped files (7-zip can do this for you, and it's open source)

Here's the process:
  • Using the hex editor, remove the first eight bytes from the file (store them in a document somewhere; you'll be reinserting them later). 'Remove' means remove, not 'zero out'.
  • Save your edited file. If you save with the extension .gz, 7-zip should be able to pick up that it's in gzip format.
  • The output of the gzip is a plain file which contains your executable binary
Disassembling the file

A disassembler will let you view an executable binary as a sequence of assembly opcodes - so you can read the code. I like to use LemASM for this job. Remember that the files are little-endian in both bytes and words, so you will need to use LemASM's 'swap bytes' and 'swap word' functions to view the file properly. You will also need to hit 'MIPS disassembly' under the File menu.

Reversing the assembler

I can't really provide a guide to this. The best reference for the PSOne's opcodes and registers is Halkun's own "Everything you wanted to know about the PlayStation but were afraid to ask". Otherwise, take a look at this reference. Remember that quite a few MIPS opcodes have a one-instruction delay, so when you're trying to read the logic around 'lw' (load word) or branch instructions, remember that they don't take effect until the next opcode has been executed.

Writing new assembler

I would recommend using ARMIPS for this job. You give it a file which contains its own insertion instructions (e.g. BATTLE.BIN 0x800AB00), and it'll encode your assembler for you (it'll handle the endianness for you, too). You can use labelled jumps, you can use macros, and you can make it spit exceptions if your assembler is larger than the area you want to replace. It's a very handy tool.

Re-packing your executable file

This is basically the decompression step in reverse, but because you're usually trying to avoid file size increases (unless you fancy updating file lookups), it's a tad more complicated. You are going to need the Unix gzip (or some gzip utility with the same functionality). If you're on Linux or OS X you should be able to just call that from bash; if you're on Windows, you're probably going to want a Cygwin-compiled version like the one that comes with FF7dec.exe.

The procedure is as follows:
  • Turn your updated assembly file into a .gz by calling, at the command line, gzip -n -9 . The -n switch stops gzip inserting the file's name and compression timestamp into the archive, and -9 forces gzip to compress at tightly as it can, even if it means performing the compression slowly. This helps stop the filesize from increasing
  • Reinsert the eight bytes you originally removed from the .X file into your .gz file. 'Reinsert' means prepend, not 'overwrite'. Save with the ending .X, .MNU, or whatever the original was.
  • Reinsert your file using CDMage Beta
And that should be enough to get playing.

13
Releases / [FF7PSX NTSC-U] Enemy long range attack mod
« on: 2014-02-12 22:07:20 »
You might not be aware of this, but in the vanilla FFVII, all enemy attacks are treated as short range. This includes attacks that use the 'magical' upper formula and attacks that lack the 'short range' flag. If it's an enemy attack (command type 0x20) and it uses the X1 formula, it's always treated as range-sensitive, with halved damage if either actor or target is in the back.

This might not be desirable. If you're creating a 'rebalance' mod that relies on a few enemy attacks being able to counter the use of the back row, it won't work as you expect. This was the problem I faced for my own PSX rebalance, Rebirth.

As such, and following the discussion here, I've written a little ASM hack to fix this behaviour. As I imagine most users will be modders themselves, I've chosen to distribute this as both a binary patch for the BATTLE.X file, and the assembler itself (for use with ARMIPS). You can download the BATTLE patch here.

Source

The assembler itself is as follows:

Code: [Select]
.open "BATTLE",0x800A0000
.psx

.org 0x800ADA28

.area $9C

//check_long_range_attack
   lw   v1, $0050(a1)
   nop
   andi   v1, v1, $0020
   bgtz   v1, check_self_backrow ; if short range attack, check self/target backrow
   nop
   j   0x800ADAC8 ; else skip to next sub

check_self_backrow:
   lw   v0, $0000(a1)
   nop
   sll   v1, v0, 1
   addu   v1, v1, v0
   sll   v1, v1, 2
   addu   v1, v1, v0
   sll   v1, v1, 3
   lui   at, $8010
   addu   at, at, v1
   lw   v0, $83E4(at)
   lw   v1, $0208(a1) ; start loading word for check_target_backrow (save space)
   andi   v0, v0, $0040 ;v0 now contains self.isBackRow

//check_target_backrow   
   sll   at, v1, 1
   addu   at, at, v1
   sll   at, at, 2
   addu   at, at, v1
   sll   at, at, 3
   lui   v1, $8010
   addu   v1, v1, at ;v1 contains address for lookup
   lw   at, $83E4(v1)
   nop
   andi   v1, at, $0040 ;v1 now contains target.isBackRow

//do_comparison
   or   at, v0, v1
   bgtz   at, reduceDmg ; if target.isBackRow || self.isBackRow, go to reduceDmg branch
   nop
   j   0x000ADAC8 ;skip to next sub
   nop

reduceDmg:
   j   0x000ADABC ; sends to damage reduction subfunction
   nop

.endarea

.close

You will need to use ARMIPS to compile this with a decompressed BATTLE.X, then you'll need to repack with gizp compression and re-add the header bytes to create an X file. The procedure is detailed elsewhere (I'm about to post a thread on it, in fact).

This file was made with the NTSC version, but as I believe the European PAL version uses the same BATTLE.X, it should work fine. I will need to investigate.

I have tested this patch myself, but it's always possible I missed something. If you have any issues, let me know in the thread below.

14
Edit: This issue has a patch. See http://forums.qhimm.com/index.php?topic=14998.0

Only recently, I discovered that all enemy attacks are treated as close range by the FFVII battle engine. Attacks that are not tagged close range, and attacks that are magical, will still perform half damage against player characters in the back row.

This is a bit of a problem for the balance of certain mods; allowing certain characters (e.g. those with long range weapons) to take half damage for the rest of the game is pretty broken.

The question is: what's the best way to handle this? I can think of two ways to try this:

  • Put everyone in the front row, disable the row change effect on the mag-rod enemy attack, and find a way of disabling the 'change' and 'order' commands
  • Force a check in the damage calculation for the short-range flag; if the attack doesn't have the flag, skip the row check function
What is best and most feasible? My gut instinct tells me that opting for 2 is the best choice, because it fixes the intended behaviour and creates a patch for other modders to use in other versions of the game.

What I don't know is exactly how to do this. I believe that the battle executable is in batini.x, and that the file is basically gzipped. I could probably get the asm out of it. Finding the relevant functions might be hard, though. I know the location of the enemy stats in the PSX's memory, so I guess I could look for a read and assume that read is the first part of the initial base damage equation. I guess I could try and follow what the game does from there. If I can locate the data for the enemy attack power, and all the attack stats are together, I could probably work out how to lookup the flag. If I need to make the binary bigger, it looks to me like the LBA+Size of batini.x are in the YAMADA.BIN (see the string 63 45 00 00 40 16).

How feasible is this? Is there anything I've missed? Has anyone else already looked at this problem and have any additional insights?


Another thing - how do I use the call stack info of the PSX emulator's debugger? What are the values in the stack viewer? Are they the locations the CPU are expected to return to once the function has finished? Is the most immediate function call at the top (as per a stack?)

15
Gameplay / Making modding less painful (FFVII)
« on: 2014-01-26 18:03:22 »
As someone who's been spending an inordinate amount of time over the last couple of years modding Final Fantasy VII, I thought I'd share some tips on testing and playing changes and making the debug process a whole lot less painful.

Testing battle changes (enemy stats, AI, formations, attacks)

If you haven't touched the battle scene lookup table in the KERNEL.BIN, you shouldn't need to reset your emulator to test these changes. You can just go ahead and instigate a fight on the world map. If you want to test a particular scenario, I would use the gameshark 'force enemy encounter' code (80116274 xxxx for NTSC), filling in the encounter ID you can get from looking at the formation data in Proud Clod.

Testing kernel changes (character stats, spells, etc.)

KERNEL changes require a reload of the game. For this reason, it's easiest to work with if you have a 'save anywhere' cheat enabled, particularly if you're tweaking things at the initial part of the game and you don't want to go through the intro video a thousand times. I have found the code 8009D2A6 0000 fairly reliable at giving me this.

Tweaking enemy damage

Use a damage calculator to tweak attack power rather than using trial-and-error with real battles. The relationship between power and final damage will not scale in the way you expect it to. I have one here: http://www.breck-mckye.com/final-fantasy-7-damage-calculator/calc.html

Testing new attacks

Don't use save states to try and get different outcomes for randomized attacks - obviously, loading a state will load all the pseudo-random number generator states.

I would also shove some Master Materia in your save with Black Choco etc., so you can activate attacks without any fussing around.

Skipping things

You can achieve some limited sequence breaking with a 'walk though walls' code (D00A99A4 A979; 800A99AC 0013; D00A99A4 A979; 800A99AE 1000). You should also see if your emulator has a skip frames / fast forward feature. On ePSXe, you can activate this with F4. On the PSX emulator, hit backspace.

Testing with arbitrary characters

The USO code can help you with this - it's a bit like a universal PHS. I believe the NTSC code is D0062D78 0022; 30049450 000B; D0062D78 0020; 30049450 0001.

Testing field changes

I believe fields are loaded on the fly, so simply entering an area should be enough to force a reload from disk. If you want to test things without getting bothered by enemy encounters, use a no-encounter code (8007173C 0000).

General tips
  • Keep a folder you can easily reach on your desktop with shortcuts to all your tools
  • Use this folder to quickly reach your mod source folders when doing imports with CDMage
  • Keep revisions of your old files
  • Use something like Wiki on a Stick to keep track of changes, reported bugs, documentation etc.

Hope that helps - and please, if you've any tips of your own, do contribute.

16
FF7 Tools / [FF7] Damage Calculator
« on: 2013-11-02 23:35:53 »
I have released an FFVII damage calculator as a web app. You can access it at http://www.breck-mckye.com/final-fantasy-7-damage-calculator/calc.html. You will need JavaScript enabled to use it.

Enter the statistics of the attacker and defender, set flags for battle conditions like berserk and critical hits, then hit the 'Calculate' button to estimate the damage. You can view a damage 'breakdown' to see exactly how the value was reached and whether there is any chance of causing overflow.

It's still in beta, so I would appreciate any reports if it acts in odd ways. I've tested it and it seems accurate, but there's always the chance of something slipping under the rug. I'd also appreciate any requests for features; there's no point to something like this if it misses something obvious.

If you'd like a local copy, you can pull a version of the source from Github. I've licensed the source under GPL2 in case anyone wants to re-use anything from it (not that I'd understand why; it's fairly simple).

17
A long time ago, Ficedula created a save editor for the FFVII pocketstation minigame 'Chocobo World'. The download location has been dead for quite a while, but I've managed to find a copy and mirror it on my own domain.

The file is located at http://www.breck-mckye.com/final-fantasy-modding/Ficedula-Mirror/chole100.zip

The .zip file contains a Windows binary and documentation in the form of a readme file.

18
Completely Unrelated / Cool / funny pictures
« on: 2013-04-09 20:46:00 »
Got a pic or animated .gif you'd like to share? Post it here.

Adding images saved on your computer
If you'd like to post an image saved on your machine, upload it to Imgur and copy the 'direct link'. Then, when you post, click 'Insert image' and paste the link URL as the image location.

19
Completely Unrelated / Random Links
« on: 2013-04-07 12:36:31 »
Got a link to an article, web page or YouTube video you'd like to share, but which probably doesn't justify a thread of its own? Post it here.

Please mark anything NSFW for those of us surfing from work or school, as well.

21
General Discussion / MOVED: re-release ff7 black textures
« on: 2012-08-21 20:41:24 »
This topic has been moved to Troubleshooting.

http://forums.qhimm.com/index.php?topic=13437.0

Note: All FF7-2012 topics go into the same categories that equivalent FF7-1998 topic would. Re-release threads don't need to stay in General Discussion.

Pages: [1] 2 3