Qhimm.com Forums
Miscellaneous Forums => Scripting and Reverse Engineering => Topic started by: halkun on 2006-09-17 02:00:59
-
So I've been probing more into Fieldscript's form can say pretty much with 95% probably that the original language was a TCL extension.
I'm creating my own "compiler" of sorts using TCL and extending the opcodes myself. I've only worked on it for 5 hours last night and the whole thing fell into place. Because of the way the an interpreter is implemented in C, there can't be any other way this wasn't a TCL system to begin with.
The framework I have so far puts all the opcode handlers in a separate C module. Each opcode is represented by a single function. How TCL works is that the whole system is based on substitution. There are no keywords and everything is defined by commands. You can easily extend TCL by adding a custom command (It's a line of code) The TCL syntax is the following
proc function {arg arg arg}
{
code
}
It's trivial to implement fieldscript
proc tifa{ }
{
on_click
{
battle {1 999}
ret
}
}
Using TCL also allows you to define global vars, and then hook them directly into the engine. If you change a varible in the TCL script, it's changed within the C code as well.
You call a proc from C and on return, control goes back to the engine. It's exactly how fieldscript works. In the script itself, a proc can call another proc without giving control back to the engine.
The "compiler" I'm making will allow anyone to take the handlers and hook it directly into q-gears, as opposed to spitting out bytecode.
I'll be working on this this weekend, after I learn a little more TCL.
-
So our worst fears are realized we've been had by TCL!
AHEM.
I remember 3d engine I was colaborating with circa 1992 that used TCL. Yes it's extensible. So it all makes sense.
I suppose what we might want to do is implement all the field script based on that principle so we have something very close to what the original was?
IE
PMOVIE(index); etc.
The movie indices aren't unique which, is quite a downer but I suppose that's life.
Anyhow looks fun. As for me and my ISO file system... it's almost here. ;)
Cyb
-
Sorry, but it's really time for a reality check here. Really.
You essentially argue that since TCL is an interpreted language, and since an interpreter is usually implemented in a certain way in C, and since the FF7 scripting language must be run through an interpreter, the FF7 scripting language must be based on TCL! Amazing deduction here. You've incidentally also proven that all interpreted languages are TCL-based. Guess we should notify someone?
So for the reality check. TCL is an extremely generic text substitution based scripting language, which can relatively easily be tied into other programs via simple hooks. It allows you to dynamically define and redefine just about anything, and it has built-in support for various high-level concepts. TCL is based on the same sort of thinking that's behind LISP, i.e. maximum genericity.
This is what we know about the FF7 scripting language: It is extremely specific and minimalistic. It is not extensible. It is very low-level with minimal support for even basic program flow control constructs. It is compiled into bytecode (and the bytecode allocation shows clear signs of emergent development, i.e. the command set has changed during development). The FF7 scripting language is based on the same sort of thinking that's behind assembly language, i.e. maximum performance through simplicity.
So essentially claiming FF7's language to be based on TCL is like claiming that assembly language uses LISP.
Speaking as someone with some experience with both FF7's actual low-level internals and with compilers, I can state with calm certainty that the FF7 scripting language isn't a "TCL-based extension", and furthermore I can be relatively certain it wasn't derived from, compiled from or compiled with anything TCL-related. Here's what we do know:
- The opcode names, by a pure fluke. The fact that they're named and arranged in a consistent manner (especially the language-related control codes) suggest that the language (or at least the compiler+interpreter) was designed from scratch. Anything premade or reused would certainly show signs of more general (and in FF7 unused) constructs littered throughout the opcode table, or signs of "computer thinking" in the allocation order. The unused/unimplemented opcodes also suggest that the language (or rather, the command set) evolved over time. The opcode allocation table was done by a human, I have no doubts about this.
- Repeating occurances of certain control opcode patterns, and the complete lack of "variations". This suggests a compiler with some support for higher-level constructs such as if-else and while, which would then be automatically translated into and compiled as their simpler multi-opcode constructs.
- The relatively few kinds of higher-level constructs found. Anything more advanced than if-else and while tend to be written individually to suit the current situation, with frequent variations. This suggests the compiler (and hence language) did not offer any higher-level constructs than those established above.
- The great variation and "cleverness" in the use of other opcodes. This suggests that the majority of all opcodes were accessible and used directly (hence the names).
- The design of "if" opcodes and the if-else/while constructs. The fact that these both use jump-aheads/jump-backs in a clear and consistent manner suggest that these were compiler-handled, which in turn means that the compiler had simple scope management support. We know nothing of the syntax of these scopes, if they were {}, if-end, indent-based or whatever.
- The extra "if"-style opcodes found later in the opcode table. This suggests that not only did they have control of the opcode allocation, but also of the language and compiler (since these opcodes are designed used in the same fashion as the "core" if opcodes). The fact that the language was extended in this low-level manner later in the development suggests that there was no high-level (read: TCL-style) assistance in extending the language or feature-set, rather that it was done by hand.
Presumably the compiler was rather minimalistic, parsing text and helping with simpler argument syntax (and the higher-level constructs mentioned earlier), but did nothing else. Are you suggesting that they went through the trouble of using a feature-rich scripting language like TCL, then went through the huge effort of disallowing and removing all of its useful features?
No, this is clearly a custom-made language, possibly with some inspiration drawn from earlier Square engines, but definitely still written from scratch. The original language they wrote the scripts in might have borrowed some simple syntax (such as pseudo if-else/while constructs, labels, comments, variable names etc.) from some other language, but this has since been compiled away and we know nothing about it. Though one can guess that since there are no "for" and "do" constructs, they probably didn't borrow from C syntax (since then it'd be natural to include these). The only thing we can guess with some certainty is that the code would probably look like:
CHAR 1
PXYZI 25 -13 14 2
SOLID 1
RET
IF #plotprogression >= 582
 PXYZI 48 -12 18 5
END
RET
That is, very simple and minimalistic, just enough to get the job done.
-
partypooper :P
-
How about BASIC?
RETURN is a normal abuse in it. It's interpretive it IS event driven as well.
ON <instance> GOTO/GOSUB
etc.
-
Read what Qhimm said ... it's probably not based on any language, really; at most it will share very, very basic syntax with a particular family of languages; mostly, block delimiters.
Also, given that it appears nothing more advanced than IF really existed, it hardly matters what syntax the original script used either; there's practically no substance there that's interesting, however it was implemented...
-
Oh well, I'm leanring TCL coding anyway, so it's not a total loss.
-
If I may interject an observation..
Perhaps it might be prudent to investigate the assembly codes for the PSX RS3000 MIPS (sp?) engine. I have a sneaking suspicion that everything was simply re-compiled to run in assembly with the PSX PSU, so the original language wouldn't matter. It might be worthwhile to look into the PC version and see if it has some kind of interpreter/recompiler to convert the original PSX text and scripting to a PC standard language or micro-code.
It might also be worth investigating the original PSX CPU, for instruction sets and data pathing sets. I know they use to sell the Yaroze (sp?), which was basically designed for testing and de-bugging software and for do-it-yourself programmers (yes, Sony actually allowed people to manufacture their own programs, as long as they didn't SELL them). Their might even be some web sites associated with this.
Just a few thoughts..
-
In my view it is irrelevant what they used, TCL from my studies appears to be perfectly good for the purpose of Q-gears. It doesn't matter really what square did now that I think about it. Obviously they did something, but what? Only they know, and I doubt they still do considering the PC port of FF7 as an example.
Side note, it would be very easy to implement a byte code assembler in the short term (with the ussual linguistical cheats of EQU etc). If that's what they did well I don't blame them. Since many of there programmers were still in the assembly mode of thinking at the time.
Opinions and perspectives vary .. whatever way the script system was implemented, TCL is as good as any coding route to go, but more importantly it's a well documented route, that is far more important than whatever was originally done. As for decompilation of script code, it can be pretty much generated how one wants it in terms of syntax. The best method to determine who well that works is to compile some test code and see how it decompiles. Learning etc. will be significantly easier the TCL route than any other. TCL has significant use on Unix boxs for a reason. I suggest reinventing a wheel is a waste of time. So I'm in favor of a few early headaches with TCL than bigger ones from a more custom route.
So getting back to what is truely important, is TCL is useable, well documented, easy to learn, and useable for the purpose of Q-gears. A compilor can be made for it, and a Decompilor for the byte code.
It would be good to think of what we do know in terms of the engine.
Global
256 Unsigned 8 bit variables
256 unsigned/signed 16bit variables.
Local
256 variables of the Global varietys, It would be good to check the execution scope of these.
What other variables are there (I'm a bit unclued on this)
For each field location we have 32 scripts for each field object (some are just returns).
The first script on each object has an initiation section.
Does every field location have at least 1 object in it?
etc.
A thorough compilation of what is known in terms of the engines structure along with variable access might be prudent. That way it's possible to lay out a clear course of action for Q-gears.
Cyb