Messages

51

Scripting and Reverse Engineering / Re: FF7 World/Field Map Scripts and Encounters

« on: 2019-04-09 12:04:58 »

The idea that makes the most sense to me, would be to build a "dependency map/graph". Each "node" in the graph represents a milestone like reaching a location in the game, or setting a Game moment variable. (For example, suppose there's an "if block" in a field script that only executes if some bank variable is set, but no scripts in the game set it. Then that part is unteachable.)

That sounds like a very fun project, and I might even pursue it.

Maybe the three of us could all discuss ideas on Discord or something.

52

Scripting and Reverse Engineering / Re: FF7 World/Field Map Scripts and Encounters

« on: 2019-04-07 22:51:30 »

Very cool, our goals are similar but different, and we could probably benefit by helping each other.

The overall goal behind Kujata is to translate the original assets from FF7 format to web-friendly formats like JSON.

So, if you wanted to display field models and animations in your website, you could use the output of those translations. (Not sure if that's practical for the kind of guide you're making, but just something to think about.)

53

Scripting and Reverse Engineering / Re: FF7 World/Field Map Scripts and Encounters

« on: 2019-04-06 18:45:15 »

codemann8, this is super funny, because I've also been working on a "graph" of how the field levels and world maps are connected together, then today I just read the first page of this thread, and I saw that you're working on a very similar thing.

I've already got a script that generates a graph very similar to yours. I have one node for each field ID (1-64 are wm nodes, the rest are field nodes), and I have different kinds of links, one for gateways, one for mapjump codes. I draw my graph using d3 force simulation, but my next goal is to anchor the wm nodes so that they don't go all over the place.

Anyway, I plan to share my work soon and will update this thread when that happens.

54

Scripting and Reverse Engineering / Re: FF7 World/Field Map Scripts and Encounters

« on: 2019-04-06 15:37:14 »

Just found this thread.

Also, I just found a project that I worked on a long time ago to parse the world map script. I basically started with this project: https://github.com/cebix/ff7tools

Then I made some enhancements. The output parses the opcodes and outputs them in a readable format, very similar to what codeman shared, except in a more "pure" format.

The reason there are the extra bytes like 01 10 in between the data is because these are opcodes like PUSH. World map script operates with data on a STACK.

DLPB said:

wm0.ev (event file in world_us).  0x7e0

My script outputs this around that address:
offset opcode (params) interpretation
07d1   0100      CLEAR
07d2   011b   0006   PUSH GetSpecial(0006)
07d4   0110   0037   PUSH #0037
07d6   0070      PUSH POP2 == POP1
07d7   0201   07eb   JUMPZ(POP1) 07eb
07d9   0100      CLEAR
07da   0110   000b   PUSH #000b
07dc   0110   0016   PUSH #0016
07de   0308      ActiveEntity.SetMeshCoordsXZ(POP2, POP1)
07df   0100      CLEAR
07e0   0110   0b50   PUSH #0b50
07e2   0110   0247   PUSH #0247
07e4   0309      ActiveEntity.SetCoordsInMeshXZ(POP2, POP1)
07e5   0100      CLEAR
07e6   0110   0000   PUSH #0000
07e8   0304      ActiveEntity.SetDirectionAndFacing(POP1)
07e9   0200   083d   JUMP 083d

This is a more "pure" translation of what codeman provided, which was:

07d7      IF ((GetSpecial(0006) == #0037)) {
07de        ActiveEntity.SetMeshCoordsXZ(#000b, #0016)
07e4        ActiveEntity.SetCoordsInMeshXZ(#0b50, #0247)
07e8        ActiveEntity.SetDirectionAndFacing(#0000)
07e9        GOTO 083d

Not sure if you guys already figured that out, but just wanted to share. I only skimmed the conversation above, but will read it more carefully now, because I'm working on a personal project to figure out how to find the world map coordinates when a field scripts sends you to the world map.

55

Scripting and Reverse Engineering / Re: Field IDs

« on: 2019-04-06 15:00:49 »

Wow, thanks for pointing me to that thread, I read it once all the way through, looks like codeman and DLPB figured it out, or at least got 90% of the way there. Will digest it again.

56

Scripting and Reverse Engineering / Re: Field IDs

« on: 2019-04-05 22:43:58 »

I bet DLPB or someone else could answer this pretty easily...

When a field script sends you to the overworld (world map), it does so by sending you to a Field ID between 1 and 64, which corresponds to field names "wm0" through "wm63". The destination x,y,z in this case is 0,0,0.

Where can I find the world map coordinates that correspond to "wm0" through "wm63"?

57

Scripting and Reverse Engineering / Re: Field IDs

« on: 2019-04-04 17:50:06 »

Confirmed. File format is very simple. First 2 bytes is the numFieldMaps. Then there are numFieldMaps * strings, where each string is 32 bytes, null-terminated.

58

Scripting and Reverse Engineering / Re: Field IDs

« on: 2019-04-04 17:33:02 »

Hmm, I think I found it. There's a file called "maplist" inside flevel.lgp. I didn't notice it before because it has the same file extension as the ~700 other field files in the same archive (i.e. no extension).

59

Scripting and Reverse Engineering / Field IDs

« on: 2019-04-04 17:21:01 »

Where in the original game files can I find a mapping of field ID (like 116) to field name (like md1stin)?

Couldn't find this. Looks like it's hard-coded in Makou Reactor. Didn't check Ochu source code yet. I'm guessing maybe in flevel.siz but I couldn't read it.

60

Scripting and Reverse Engineering / Re: FF7 animations import and export

« on: 2019-03-25 17:34:05 »

Finally got a release out. So I've created an official post in the FF7 Tools forum: http://forums.qhimm.com/index.php?topic=18724.0

61

FF7 Tools / [PC] Translate graphics models to glTF - Kujata (v1.0)

« on: 2019-03-25 17:15:12 »

Kujata is a suite of tools that translates FF7 assets to web-friendly formats like JSON and glTF, and also provides metadata and statistics.

• https://github.com/picklejar76/kujata
• Translates field models to glTF
• Translates field data to JSON, and field entity/group scripts to Javascript
• Provides metadata like field map links, op code usage statistics, etc.

The Kujata webapp is a website that illustrates examples of what could be done with the data.

• https://picklejar76.github.io/kujata-webapp/

Browse every field model in the game:


Browse every field map in the game:


See field map details, including links to other maps, field models used in the map, scripts, etc.:


See op-code statistics, sort and filter by specific fields, and export to CSV.
Example: See every place $GameMoment is set:



Edit: Made the screenshots above into links
[/list]

62

Scripting and Reverse Engineering / Re: FF7 animations import and export

« on: 2019-03-17 20:33:29 »

BTW, after talking with people on Discord and working with glTF, I've decided to pursue that as the universal data model. I've already started developing the new translator and got vertexes, normals and vertex colors working, outputting a mix of json and binary OpenGL data. Will keep posting here occasionally with progress. The main discussion medium for this will be Discord (FF7 #tools thread).

63

Scripting and Reverse Engineering / Re: FF7 animations import and export

« on: 2019-03-15 18:05:57 »

Regarding the FF7 field model data, does anyone know:
Q1: Does ff7 consider positive y direction to be up, or some other axis+direction?
Q2: Does ff7 use a right-handed coordinate system or a left-handed coordinate system?

The reason I ask is, after importing the field model+anim data (e.g. for Cloud) and displaying it, Cloud is upside-down.

The way I "fixed" this is to just manually apply a 180-degree rotation to the "root bone". I looked at the Kimera code, and although I don't understand it fully, Kimera seems to do something similar (e.g. in FF7HRCSkeleton.bas, line 247, it does "AAnim.Frames(0).Rotations(0).Gamma = 180"). But this seems "hacky" to me.

So, I was wondering if maybe we've been interpreting the data wrong, and just living with it in some way?

So, I actually ran an experiment to vary the following factors:

• whether the coordinates in the file are in x, y, z order or some other order (6 possible combinations)
• whether to apply rotations in y, x, z order, or some other order (6 possible combinations)
• whether to apply rotations in counterclockwise or counterclockwise order (2 possible combinations)
• which direction to move child bones from parent bones, e.g. "in the negative-z direction" (6 possible combinations)

The results were very interesting...

Oops, my lunch break is over, and I need to run to a work meeting now, but... I'll be back soon to share the results...

64

Scripting and Reverse Engineering / Re: FF7 animations import and export

« on: 2019-03-15 17:41:27 »

sounds sensible to me!

65

Scripting and Reverse Engineering / Re: FF7 animations import and export

« on: 2019-03-14 22:18:18 »

Yep, browsed the spec earlier today. Looks extremely promising.

The problem is, although glTF importers/exporters exist (e.g. for Blender, Unity, etc.), they seem to be in "beta" and have tons of issues created in their respective github repos.

And I already just developed a translator for FBX (and before that, for COLLADA, and before that, for Java3D). I know FBX will work, and it's 95% done.

So, I'm very torn. (FBX is so prevalent, and it's done, but it's proprietary. glTF is "open" and probably going to be "the future", but will take more time investment.)

Damn you, halkun, I was happier before you told me about glTF. Ignorance is bliss. You just had to give me useful information, didn't you?

It's okay, I see that you have already thought on your sins, and took a lesson from Vincent, and went back inside your coffin to atone!

66

Scripting and Reverse Engineering / Re: FF7 animations import and export

« on: 2019-03-14 03:23:57 »

Wow, thanks halkun, I don't know how glTF escaped my initial searching around for potential formats. It's pretty much exactly what I wanted in the beginning: a universal format in JSON. I'll definitely be looking into it.

67

Scripting and Reverse Engineering / Re: FF7 animations import and export

« on: 2019-03-10 23:30:17 »

Update: I have decided to go with FBX instead of COLLADA, because Unity can't import COLLADA animations.

I've made great progress and plan to soon-ish provide a GitHub repo with the following assets:

* program to translate field files to JSON, for db/stats/debugging/browsing purposes
* program to translate to FBX (models, textures, animations)
* sample output FBX and JSON for others to test

68

Scripting and Reverse Engineering / Re: FF7 animations import and export

« on: 2019-03-06 18:40:13 »

screenshot:

69

Scripting and Reverse Engineering / Re: FF7 animations import and export

« on: 2019-03-06 13:02:55 »

Update: Tool can now translate "polygons with vertex colors" and "polygons with textures" for a single *.RSD file (e.g. AAAF.RSD = Cloud's head) to COLLADA and import into Unity.

Next step: translate and import entire skeletons (*.HRC) and animations (*.A). After that, will probably publish to GitHub and start testing Blender import, displaying models on the web, start adding export support, etc.

70

Scripting and Reverse Engineering / Re: FF7 animations import and export

« on: 2019-03-02 14:58:03 »

Thanks, great info! First link will definitely help jump-start the "database". 2nd link, I tried, but the owner will need to approve my access. Third link was a pretty good reference, too.

In discord we've had some great discussion. We're going to try COLLADA for an intermediate format. I've been able to export a simple mesh of Cloud's head to COLLADA and import into Unity. I'll also be testing Blender import. Anyway, will keep people posted.

71

Scripting and Reverse Engineering / Re: FF7 animations import and export

« on: 2019-03-01 11:36:28 »

BTW, one of the challenges with the models and animations is that they aren't labelled. So it's not obvious that AAAA is Cloud, nor that BYAB is a "running animation for skeletons that have 7 bones".

Kimera mentioned a "database" for this stuff. Where can I get that, and how complete is it? If it's not complete, I was considering using a tool like Makou Reactor to track which animations are used in which situations, to figure out the more mysterious ones. Was even considering writing a tool to let others contribute to this effort by writing a simple web app to display the animation and let people write a description of what it is.

72

Scripting and Reverse Engineering / Re: FF7 animations import and export

« on: 2019-03-01 11:33:07 »

BTW, I found RedXIV and quantumpencil in Discord (ff7 #tools), and I look forward to other people joining. We're in different time zones, so discussion is not always real-time.

Meanwhile, I've been thinking about what Kaldarasha said:

Could you join our discord server please and go in contact with Red XIV? He wanted to write a new Kimera, though after a discussion with quantumpencil we decided it would be better to port the files to a much more universal file format for better editing.

By "universal format", initially I was thinking of something like FBX or COLLADA. But now I'm thinking it would be better to do a more generic intermediate format like YML, JSON, XML. That way, others could easily write tools to translate from this generic intermediate format to any format they want. (Since importing from FF7 format or FBX format or COLLADA might be prohibitive, depending on what programming language the author wants/needs to use.)

I encourage everyone to participate in Discord, and/or to contribute "longer" ideas to this thread...

73

Scripting and Reverse Engineering / Re: FF7 animations import and export

« on: 2019-03-01 02:09:52 »

Thanks for the replies, guys! Glad to hear there's a discord now! (IRC is boss, but old.) Anyway, I'll probably hop into Discord in a couple of days.

I'd like to go ahead and share some thoughts in the meantime. Especially around tool design decisions.

First, let's consider two different goals:
Goal #1 is "play custom models in FF7"
Goal #2 is "use FF7 models as content for creating new games to play on a modern platform"

Now, let's consider some workflows:
Workflow #1 = import FF7 models into Kimera, edit them within Kimera, export from Kimera < provides updated FF7 models
Workflow #2 = import FF7 models into "FF7 model translator", export them to universal format < provides translated FF7 models
Workflow #3 = import universal models into "modern model editor program", export as universal format < #2 + #3 = updated FF7 models in universal format
Workflow #4 = import universal models into "FF7 model translator", export as FF7 format < #2 + #3 + #4 provides updated FF7 models

The great thing about Workflow #1 is, you don't have to worry about universal formats, and it's the shortest path to playing FF7 with updated models. But the big disadvantage that I don't like is, you have to maintain Kimera, which requires expertise in models and animations and programming.

So, what if we don't want to reinvent the wheel and develop and maintain software involving editing 3D models? That's where Workflows #2, #3, #4 come in. You don't need a graphical editor. You can just a modern editor. So, we only need to be experts on translating back and forth between the FF7 file format and modern file format(s).

So, I'm leaning towards Workflows #2, #3, #4.

I've chosen FBX as the first universal format to support. So far, I've already successfully created a model of Cloud's head in FBX format. Next steps would be skeleton, textures, animations. At that point, they'd be usable in just about any modern game engine, but not FF7. If anyone really wants to use universal models in FF7, then we just need Workflow #4.

Curious to know others' thoughts. But, yeah, maybe Discord would be best. On that note, sorry for the long post.

74

Scripting and Reverse Engineering / Re: FF7 animations import and export

« on: 2019-02-26 19:40:11 »

BTW, after some research, the OBJ format does not store animations. So, I'm likely to go with something like FBX or COLLADA.

The main requirement for my personal short-term needs is, I need the ability to translate FF7 models + textures + animations into another format that can be easily imported into *Unity* with minimal adjustments. However, other people will have different needs (and I might have different needs long-term), so I'm trying to choose a file format that can also be easily imported into other popular programs like Blender, Sketchup, 3DSMax/Maya/AutoDesk, etc.

75

Scripting and Reverse Engineering / FF7 animations import and export

« on: 2019-02-26 17:55:01 »

So, I know there are a few tools out there that can import FF7 models, and even a couple that can display the animations, but are there any tools that can export the animations to a more popular format like obj or fbx?

If not, I am seriously considering writing such a tool. Many years ago I already wrote a program, using the great data on this forum about the file formats, that could read FF7 model and animation files and display the animations. So all I'd need to do is learn a popular file format (perhaps obj or fbx) and write the export logic.

But before I go down this path... are there any tools out there that already do this?

And if I end up writing such a tool, does anyone have any feature requests?