Author Topic: making 3D object extractor... (please help)  (Read 8368 times)

ils

  • Guest
making 3D object extractor... (please help)
« on: 2004-12-23 00:28:22 »
ok, i've been browsing this forum for a while and noticing someone already made FFX model viewer (Kingdom Hearts).

since nobody share it, i would like to make one myself.. :(

i never learn 3D modelling (or anything related to 3D).
so can you give me "direction" which i should learn/read first before i can extract 3D data from iso/cd/file ?

err.. so far i only code with Visual Basic (most of them is database programming). the only apps i wrote that close with "data extraction" is PSS/FMV extraction from PS2 games...

so which site/link should i visit ?
oh, and this is my first post. nice to meet you all
(i'm using 28.8Kbps, and rarely online **very very slow connection**, but please reply)

thanks alot!

Qhimm

  • Founder
  • *
  • Posts: 1996
    • View Profile
    • Qhimm.com
making 3D object extractor... (please help)
« Reply #1 on: 2004-12-23 08:36:14 »
You might be in for a rough ride aiming so high for your first entry into the 3D world, but since you've already done some work in data extraction I suppose I could at least give you a few pointers.

[list=1]
  • Study the hardware platform. FFX is running on the PS2, and no doubt its data files are specifically designed to be effective on that platform (common data structures and formats).
  • Learn how vertex lists and face lists typically appear in a hex editor. Vertices are usually the easiest to spot, then faces and textures, after that you should begin to see the surrounding file structure.
  • Learn OpenGL (or DirectX, or some good 3D file format). You need a good platform to visualize your output and test theories. OpenGL is usually a good start here.[/list:o]
    Piece of the proverbial mammoth.

Alhexx

  • *
  • Posts: 1894
    • View Profile
    • http://www.alhexx.com
making 3D object extractor... (please help)
« Reply #2 on: 2004-12-23 10:08:12 »
I was going to give those FFX models a try next year.
Let's see if I'm able to write an useful File Format doc...

 - Alhexx

ils

  • Guest
making 3D object extractor... (please help)
« Reply #3 on: 2004-12-23 11:07:54 »
thanks for replying..
is there any link to learn those 3 point ? (like tutorial or tech explanation..)
i currently downloading gears.doc/pdf and psx.pdf from the "sticky" post. hope that file help a bit...

alhexx : i'm supporting 100% :)

Alhexx

  • *
  • Posts: 1894
    • View Profile
    • http://www.alhexx.com
making 3D object extractor... (please help)
« Reply #4 on: 2004-12-23 11:58:07 »
I doubt that gears will help you, since it is a very "technical" file... if you don't know how to work with the formats, it'll lead you to nothing but confusion...  :(

 - Alhexx

Zande

  • *
  • Posts: 55
  • 友情は武器よりも強し
    • View Profile
making 3D object extractor... (please help)
« Reply #5 on: 2004-12-23 12:41:36 »
Here's a linky with nice OpenGL tutorials, it might come in handy: http://nehe.gamedev.net/. The language used in the tutorials is C++, but in the end of each tutorial there's conversions to alot of different languages and platforms. :)

Alhexx

  • *
  • Posts: 1894
    • View Profile
    • http://www.alhexx.com
making 3D object extractor... (please help)
« Reply #6 on: 2004-12-23 13:31:01 »
I can say that a direct step from VB to C++, especially in 3d programming is hard - I can tell b'cause I did that with Ultima.
Besides, Nehe's Tutorials are lessons for 3d programming, not for the basics. You should know ow to build, or even more important in model hacking, you should know how to disassemble those model formats.
This requires to study a lot of other model formats. Like FF7, FF8 models and even PSX/PS2 formats...
In other words: You've got a long and hard to go...

 - Alhexx

ils

  • Guest
making 3D object extractor... (please help)
« Reply #7 on: 2004-12-23 16:28:03 »
thanks for the link zande :)

actually i can write code in C/C++ too..  only i already forget most advanced syntax (like using pointer, linked-list, struct, etc, etc).. for "normal" syntax, i still remember since my last assignment was PHP coding (C a-like language)

but as Alhexx says i still have a long and hard way to go :), since i don't have basic in 3D :(
is disassambling model you're saying is "looking in HEX and *guessing* which is 3D model from those HEX value" ? how can i know which value is THE 3D model ?

mirex

  • *
  • Posts: 1645
    • View Profile
    • http://mirex.mypage.sk
making 3D object extractor... (please help)
« Reply #8 on: 2004-12-23 18:19:27 »
Quote from: ils
is disassambling model you're saying is "looking in HEX and *guessing* which is 3D model from those HEX value" ? how can i know which value is THE 3D model ?
Yes thats one way. Hard long way but it works. I think that it can be done also by disasembling the code and seeing how the code loads the data but I never did it that way (well I tried once, but it didn't work for me :) ).

ils

  • Guest
making 3D object extractor... (please help)
« Reply #9 on: 2004-12-24 10:24:02 »
hmm seems disassmbling code is out of option :) since my ASM skill is very low.

errr... between million of HEX value, how should i know it is THE 3D model or movies or just only an executeable program ?
(like everyone say, there is a pattern.)  but for a beginner like me, how can i learn to "know" the pattern (any link)?

thanks

L. Spiro

  • *
  • Posts: 797
    • View Profile
    • http://www.memoryhacking.com/index.php
making 3D object extractor... (please help)
« Reply #10 on: 2004-12-24 14:38:25 »
If you are looking at RAM (which I assume so, since you mention executable area) then you should study how RAM is organized into chunks, etc.

A chunk is is contiguous group of 0x1000 bytes with the same memory attributes.

A chunk can be executable, mapped, unmapped, inaccessible, etc.
When it comes to locations where data it stored, it depends on how the data is loaded.
In old games, most of the space required to hold the data was declared in the executable and filled in by external files (Final Fantasy® VII for example).
This is why most of Final Fantasy® VII’s data structures can be mapped so easily, since they never move in RAM.

Unfortunately these days most things are loaded entirely from external files with the memory declared in real-time as required.  This causes data to shift around in RAM and makes it often very difficult to track.
In the case of 3-D models that are only in RAM when they are being used, they are always declared in real-time and almost never end up in the same location in RAM.


In order to track them, you have to find the pointers to them.
Pointers, even in advanced games, are almost never more than 2 deep.  This means finding a pointer to a pointer, which is usually because one class has an array of models (one pointer) and then that class is instantiated with a pointer.
So you would find the pointer to the class, map that class, and find the member of that class that is a pointer to the 3-D models.
Also, most graphics cards store the vertices, mesh, and texture information for 3-D models.  This is an area of RAM you can not access (not through the Windows® API).  Even DirectX® can not interact directly with graphics devices, which is why Microsoft® requires all graphics card to have a HAL (Hardware Abstraction Layer).


I have written a program with a Hex Viewer with many useful features.
It shows you where chunks begin and end, and it uses color-coding to display the hex graph.

It shows you where pointers are and shows you executable areas in RAM (blue).
It shows areas that are reserved for data (free store ) and it has a feature to help you map structures in RAM.

It allows you to build structures to depict complicated data types, including 3-D models, and to paste those structures over RAM to see how that area in RAM measures up to your structure.
It shows you if the area is a valid area for the structure you have defined, and if it is, it shows you the names of each of the structure elements and their values.

It can even scan your existing projects for structures you have already defined inside code files (.cpp, .h, and .txt) and convert them into the structures you can paste over RAM.


I have posted the link here a few times but nobody cares.
They are either too busy or they aren’t hacking right now, or whatever else.
I request feedback but get none.

mirex is now hosting my site, so here is the link to my Windows® 9x version.

Memory Hacking Software Compatible
And in order to know how to use the templates (only helpful if you already know how to use classes/structures) you need the tutorial.
Tutorial


L. Spiro

mirex

  • *
  • Posts: 1645
    • View Profile
    • http://mirex.mypage.sk
making 3D object extractor... (please help)
« Reply #11 on: 2004-12-25 03:55:59 »
Yes that looks like an interesting way of reading the data structures, I have to try it sometime.
L.Spiro:  I dont use it ... i was not ripping anything recently and also looking into static data files looks more simple to me. You can see there all without any movement, and also you usually approximately guess what is in which file (from size and extension).

How to I do it when looking into file... hm .... you have to look for the patterns ;) ... you can usually spot few things when looking at file in hexviewer ... for example you can see there values like:  
00 00 00 01  00 00 00 03  00 00 00 0A  00 00 00 08
 - repeating three zero bytes and one byte with low number ... from that you can see that it is array of long integers (4 byte integers), which have values of 1, 3, 10, 8  ... those are usually some indexes of something. Indexes of vertices or materials or stuff.
Well than another hint is to count the longs ... and try seeing if there number similair to the count at the start of the array ... that gives you number of items to read. Sometimes the number can be multiple of the count ... for example if those are vertex indexes of polygons, then the number is divided by 3 and it keeps number of polygons to read.

Then another hint is that floats (mostly vertex and texture coordinates) are usually stored as 4 byte float, which you can sometimes recognize by usual sign - they end by 3F or 3C byte mostly. Like:
 ?? ?? ?? 3F   ?? ?? ?? 3C   ?? ?? ?? 3F   ?? ?? ?? 3C

Some data are more complex but you can find them by visible patterns in file.

Well and than after some time it will make your eyes 'see the data inside the file by glance' ... like guys in matrix saw the world on their screens in their ship :-)


* sorry if I used bad language, im quite drunk and stuff *

Cyberman

  • *
  • Posts: 1572
    • View Profile
making 3D object extractor... (please help)
« Reply #12 on: 2004-12-25 21:40:45 »
Quote from: ils
hmm seems disassmbling code is out of option :) since my ASM skill is very low.

errr... between million of HEX value, how should i know it is THE 3D model or movies or just only an executeable program ?
(like everyone say, there is a pattern.)  but for a beginner like me, how can i learn to "know" the pattern (any link)?

thanks

Actually disassembling the code will not do much good for what you want. Sorry, it's a lot more work than finding the data structures etc. that the model consists of.  You might be well off using a emulator with a debugger to set up a break point at the place where it calls DISK IO to load data.  This will help you learn where it looks for data for various scenes.  Sadly I keep getting distracted.  Like my current Ashiron NWN module, to complete one of the thirty things I started :D

The problem with the PS2 is Sony didn't make a defined developement format so every game is not the same since there isn't a set image format even.  PS2 developement as a result is VERY difficult and laborious.  However as a result the data is very hard to hack, all around everything ends up being more expensive and difficult.

Cyb

ils

  • Guest
making 3D object extractor... (please help)
« Reply #13 on: 2004-12-27 17:54:42 »
hi, my phone line was damaged (and internet been down along with it), so i was not here lately..

btw, in FFX case, they don't have "real file" in DVD (they only PS2 routine file, etc). where should i look those 3D ? (so far i rip into ISO file and open the *large* 4GB files)

hmm.. maybe i should try FF7/FF8 first huh?

oyeah, THANKS! for everyone :)