Decent FF7 Model Viewer

[Cyberman]

Well I have some better images.

I discovered that the QUADS follow PSX ordering of vertices (yea me ).
Quadrics on the PSX are split into 2 trinagles, and that's how the PSX processes a quad (not a joke.  Even though there is a quadric command, this is how the PSX internally handles them. What has this to do with the model data? Well it helps to know how the vertices are actually ordered.

Code: [Select]


Triangle Quad
B--C    B--D
|  /    |  |
| /     |  |
|/      |  |
A       A--C

These are the ordering of the vertices given to have a correct triangle or quad draw. The normal follows the left hand rule. The vertices are the vertices as they appear in the model data (in order).
Any how I also found out how to make the objects into meshs.  I broke the quads into triangles (because meshes do not support quads in POV).  Currently it creates a huge file however the results speak for themselves.

Images removed Site is gone
ENEMY011 <WEAPON>
ENEMY010 <WEAPON>
TIFA <reprise>

[Qhimm]

Since FF7 models use only vertex colors for shading, I think you'll find the quality of your renders will increase if you turn gouraud/phong shading off (or set self-illumination to 100% or similar).

[Kislinskiy]

Sorry I had not much time the last weeks. Yes I know about the ordering of the quad vertices -> look at the one in ENEMY000.BIN (the bottom of the pyramid). =)

Textured objects must have vertex colors AND U/V-coords. Look at the face textures. They have transparent areas => textures have to be mapped over colored vertices.

I'll write a small D3D application that can display our progress this month I think.

Vertices, colors, textures, objects are no mysteries anymore. We should look at animation and bone/hierachy data.  

[Cyberman]

I'm a little baffled about the UV data so far..  what it looks like is that ALL numbers are used.
I believe it goes this way now:

Code: [Select]


Vertice Counts
Vertices
UV Count
UV objects (they are 16 bytes each)
Line Count
Lines (16 bytes each as well)
Triangle Count
Triangles
Quad Count
Quadds
UV exist if the lower word is > 0
It decodes better now that I have that added in.
However I found Lines in a model and it's still not decoding line data correctly yet.. I don't know why.


Qhimm:
I'm using a raytracer so things will look odd. because there is no vertex normal data I can't make them smooth triangles.  In any case the fact I can get it to export model code for a raytracer is a good start Yufi decodes quite nicely safe her head. Still working on the UV data.

Cyb

[Kislinskiy]

I don't know whether line data is really line data.    That was an assumption, because I had looked on ENEMY000 and 012 only.

[Cyberman]

I don't know whether line data is really line data.    That was an assumption, because I had looked on ENEMY000 and 012 only.

Ahh.. well look at ENEMY019.lZS that is the first one with any texture to it. I noticed that the 'line' data is used and another one you ignored was used whenever textures were added.  I need to recompile my OGL components for BCB in order to add function OGL commands. It's been a while since I've messed with it. I guess better now than never?  One anoying thing the author of FF7meV did was use a black background for model viewing, a lot of the FF7 models are REALLY dark so you can't see the shape very well. The second one is that you can't change your POV or the scale of the model being viewed.

I am working on FF8 Models Qhimm as well, I have a scanner for the PSX version that finds valid TIMs I've noticed that all objects begin on XXXXX800 or XXXXX000 boundaries in the *.IMG files. This to me indicates The file is broken up into 2K sectors which makes sense because FMV on the PSX version are 2K video JPEG compression with 256bytes per sector added for the audio Data.  The files must be written in MODE2 with blank data for regular files in the extra 256 bytes and for Movies has the audio stuffed into 256 extra bytes per sector.  So I'm going to try using the model data you have posted and corelate that to the begining of the model file.  This means an FF8 model viewer for the PSX version is possible. I haven't tried FF9 yet.. but my suspicion is they used the same engine and model format with better modeling (because a lot of the characters look quite good in FF9).

Cyb

[Kislinskiy]

Here are a few teaser screenshots of my D3D9 application   :

[The Skillster]

nice keep up the good work!

[mirex]

Yeah looks nice

[Cyberman]

Ok Fine fine.. I'll post my shots from my internal viewer for TV too (woo).

It took me a couple hours to remember my opengl code and to fix up a few mistakes I made. Anything with textures still can't be read sadle so here is weapon (same as you Kislinskiy) without an FPS being displayed. ( rather unimportant to me hehehe).

Images removed Site is gone

Yep a few hours work and poof.. no textures yet which sucks.. maybe with some more abuse.

[Kislinskiy]

Looks nice.  

This week I will try to handle/unterstand texture data.

One more thing:

I have noticed that the most models have one more visible object than non-zero offsets in the "unknown"-structures. E.g. ff7vM reads 18 objects from ENEMY010. My app can only read 17 objects. Any idea?

We should look at ENEMY145 (mirror) and ENEMY165 (airplane) because they have only 3 visible objects, no textures and 2 offsets in the "unknown"-structures.

[Kislinskiy]

Here is the visible object format   :

SizCoords
Coords[SizCoords / 2]

NumTexturedTriangles (LOWORD only)
TexturedTriangles[NumTexturedTriangles]

NumTexturedPlanes (LOWORD only)
TexturedPlanes[NumTexturedPlanes]

NumTriangles
Triangles[NumTriangles]

NumPlanes
Planes[NumPlanes]

[Kislinskiy]

Now we can read HICLOUD (ff7vM can not).  
It should be easy to implement the textured triangles and planes to our applications. I will do it tomorrow, because I have not much time today. Then I'll look at the "missing-last-visible-objects-problem".  

I guess the animation and bone data are harder to decrypt.   More on this later.

[Cyberman]

Cool data!

I'll see if I can abuse it as well.  I'm wondering if the textured areas are drawn 2 times, IE first the nontextured versions of the polygons. Then with the textured layer and black being transparent.   Just a thought

Oh some more on the information contained in the Unknown area in FF7 I was looking at the bone data Qhimm has listed in FF8's format. The section I called flags has very similar settings in the FF8 data. I believe that might be a good clue as to what it is.  I do believe the unknown area is bone data but I think it might be the data heirarchy information as for actual skeletal data that hasn't poped out yet to my eyes.

I assume by Coords you are refering to vertices

Cyb

[Kislinskiy]

Coords -> vertices. I call it coords because it is only position data. I call the coords + colors/uv's "vertices".  

Your assumptions about the transparent textures are right. You must render the colored vertices before the textured vertices.

[Kislinskiy]

Can someone post a link to a good TIM file format description please?

[Cyberman]

Scarfed from the Internet

Code: [Select]


TIM Graphic Formats (PSX 2D Graphics)
By Klarth ([email protected])
http://rpgd.emulationworld.com/klarth

Version 0.7 - January 1, 2001
-- Totally rewritten for extra clarity
-- Removed Opinions/Ideas on Editing Utilities Section
Version 0.5 - August 22, 2000
-- Initial Release


I. Table of Contents

1. Introduction and Terminology
2. TIM Graphic Formats (4BPP, 8BPP, 16BPP, and 24BPP)
3. TIM Header Formats (4BPP, 8BPP, 16BPP, and 24BPP)
4. Color Conversion Algorithms (15bit BGR <-> 24bit RGB)

_______________________________________________________________________________________________


1.   Introduction and Terminology

-----A. Introduction

     Hmm, I started this doc awhile ago when I first started to get interested in PSX hacking.
     I then concluded that many tools needed to be coded before the PSX translation "scene"
     could really get started.  So I decided to finish it and release it so that all you
     coders out there can start making some graphic tools to make PSX translations remotely
     possible for anybody willing to try to tackle one.  These formats all have headers so I
     decided to make a seperate document for them only since they're so different from other
     console graphics formats.

     If you wish to put this document on your site, please contact me for authorization
     *before* you post this file on your site.  If I seem to "magically" disappear from the
     scene, you may post this document *only* in its original form.  If you use this doc
     please credit me in a thanks section since I just like to see who reads and uses them.


-----B. Terminology

     CLUT - Color LookUp Table (The Sony term for Palette)

     Image Org - Sets the coordinate of an image zero point, used for TIM files
          in the VRAM of the Playstation.  Not particularly necessary from an editing point
          but could be useful for a program that creates TIM images and their headers.

     Palette Org - Sets the coordinates of a CLUT image zero point.  Not particularly necessary
     from an editing standpoint but could be useful for a program that creates TIM images and
     their headers.

     BPP:  Bits per pixel.

     [pA-B rC]: pixels A-B (leftmost pixel 0), row number C (topmost row 0).
          This won't have a bitplane value because it's a linear format.

     [pX rX]: The last pixel of row X.  The last pixel's number is equal to the correct width
          of the image as defined in the header.

_______________________________________________________________________________________________


2. TIM Graphics Data (A. 4BPP, B. 8BPP, C. 16BPP, D. 24BPP)
   All the graphics are stored in Big Endian order.


-----A. 4BPP PSX TIM
        Each pair represents one byte
        Format:

  [p0-1 r0], [p2-3 r0], [p4-5 r0], [p6-7 r0], ..., [pX r0]
  [p0-1 r1], [p2-3 r1], [p4-5 r1], [p6-7 r1], ..., [pX r1]
  ...
  And it continues until the row number equals the height (from the header).  It gets its
  colors from a 16 entry CLUT in the header of the file, there may be multiple CLUTs.


-----B. 8BPP PSX TIM
        Each pair represents one byte
        Format:

  [p0 r0], [p1 r0], [p2 r0], [p3, r0], [p4, r0], [p5, r0], [p6, r0], [p7, r0], ..., [pX r0]
  [p0 r0], [p1 r0], [p2 r0], [p3, r0], [p4, r0], [p5, r0], [p6, r0], [p7, r0], ..., [pX r0]
  ...
  And it continues until the row number equals the height (from the header).  It gets its
  colors from a 256 entry CLUT in the header of the file, there may be multiple CLUTs.


-----C. 16BPP PSX TIM
        Each pair represents two bytes
        Format:

  [p0 r0], [p1 r0], [p2 r0], [p3, r0], [p4, r0], [p5, r0], [p6, r0], [p7, r0], ..., [pX r0]
  [p0 r0], [p1 r0], [p2 r0], [p3, r0], [p4, r0], [p5, r0], [p6, r0], [p7, r0], ..., [pX r0]
  ...
  And it continues until the row number equals the height (from the header).  It doesn't need
  a CLUT because it's basically a type of 15bit color.


-----D. 24BPP PSX TIM
        Each pair represents three bytes
        Format:

  [p0 r0], [p1 r0], [p2 r0], [p3, r0], [p4, r0], [p5, r0], [p6, r0], [p7, r0], ..., [pX r0]
  [p0 r0], [p1 r0], [p2 r0], [p3, r0], [p4, r0], [p5, r0], [p6, r0], [p7, r0], ..., [pX r0]
  ...
  Continues until the row number equals the height (from the header).  It doesn't need a CLUT
  since it's 24bit color.  I've never actually ripped a 24BPP TIM from a PSX game but I found
  one 24BPP TIM image and took this info from it.  May or may not actually be used on the PSX.

_______________________________________________________________________________________________


3. TIM Headers (A. 4BPP, B. 8BPP, C. 16BPP, D. 24BPP)
   All two byte values are in Little Endian.  The rest are in Big Endian.


-----A. 4BPP TIM Header:      

  [1-4]   - 10 00 00 00: ID Tag for TIM Format
  [5-8]   - 08 00 00 00: ID Tag for 4BPP
  [9-10]  - ?
  [11-12] - ?
  [13-14] - Palette Org X
  [15-16] - Palette Org Y
  [17-18] - ?
  [19-20] - Number of CLUTs
  [??-??] - CLUT Data.  16 Colors per CLUT, 32 bytes per CLUT.
  [21-22] - ?
  [23-24] - ?
  [25-26] - Image Org X
  [27-28] - Image Org Y
  [29-30] - Image Width (Multiply by 4 to get actual width)
  [31-32] - Image Height


-----B. 8BPP TIM Header:

  [1-4]   - 10 00 00 00: ID Tag for TIM
  [5-8]   - 09 00 00 00: ID Tag for 8BPP
  [9-10]  - ?
  [11-12] - ?
  [13-14] - Palette Org X
  [15-16] - Palette Org Y
  [17-18] - ?
  [19-20] - Number of CLUTs
  [??-??] - CLUT Data.  256 Colors per CLUT, 512 bytes per CLUT.
  [21-22] - ?
  [23-24] - ?
  [25-26] - Image Org X
  [27-28] - Image Org Y
  [29-30] - Image Width (Multiply by 2 to get actual width)
  [31-32] - Image Height


-----C. 16BPP TIM Header:

  [1-4]   - 10 00 00 00: ID Tag for TIM
  [5-8]   - 02 00 00 00: ID Tag for 16BPP
  [9-10]  - ?
  [11-12] - ?
  [13-14] - Image Org X
  [15-16] - Image Org Y
  [17-18] - Image Width (Stored as actual width)
  [19-20] - Image Height
  There is no CLUT data.


-----D. 24BPP TIM Header:

  [1-4]   - 10 00 00 00: ID Tag for TIM
  [5-8]   - 03 00 00 00: ID Tag for 24BPP
  [9-10]  - ?
  [11-12] - ?
  [13-14] - Image Org X
  [15-16] - Image Org Y
  [17-18] - Image Width (Divide by 1.5 to get actual height)
  [19-20] - Image Height
  There is no CLUT data.

_______________________________________________________________________________________________


4. Color Conversion Algorithms (15bit BGR <-> 24bit RGB)

-----A. 15bit BGR to 24bit RGB Color Conversion Algorithm
        Two bytes per 15bit BGR color, stored in Little Endian.
        Miscellaneous Note - Same as a SNES Palette Color.

  Initial 15bit BGR color in Little Endian:
  {ggg{rrrrr}  {0}{bbbbb}gg}
  Swap the two bytes to change to Big Endian from Little Endian.
  The color order will look like this now: (b-Blue, g-Green, r-Red, 0-mystery bit)
  {0}{bbbbb}{gg  ggg}{rrrrr}
  Take each of the color pairings and multiply by 8 to get a 24bit BGR color:
  {bbbbbbbb}  {gggggggg}  {rrrrrrrr}
  Rearrange from 15bit BGR to 24bit RGB:
  {rrrrrrrr}  {gggggggg}  {bbbbbbbb}
  This algorithm expands the color to 3 bytes instead of the original 2 bytes.


-----B. 24bit to 15bit Color Conversion Algorithm
        Three bytes per 24bit RGB color, stored in Big Endian.

  {rrrrrrrr}  {gggggggg}  {bbbbbbbb}
  Integer Divide (DIV) each color by 8
  {rrrrr}{ggg  gg}{bbbbb}{0}
  Rearrange into 15bit BRG
  {0}{bbbbb}{gg  ggg}{rrrrr}
  Swap the bytes to change from Big Endian to Little Endian (so the PSX can read it proerly)
  {ggg{rrrrr}  {0}{bbbbb}{gg}


-----C. Notes on PSX 15bit BGR Colors

  These are the same as the SNES 15bit BGR Colors.  There is a color loss whenever converting
  from 24bit RGB <-> 15bit BRG.  Also, there's probably a more advanced and quicker algorithm
  to convert the colors, however this is explaining the basic idea behind the algorithm.


My TIM Headers from my code

Code: [Select]


typedef  short int      INT16;
typedef  unsigned short UINT16;
typedef  char           INT8;
typedef  unsigned char  UINT8;

typedef struct
{
   UINT32   Magic;
   UINT32   Type;
} base_tim_hdr;

typedef struct
{
   UINT32   WCount;
   UINT16   IMGX, IMGY; // origin of image
   UINT16   Pitch;   // in 16 bit words per horizontal line
   UINT16   Height;  // number of lines;
} base_tim_img;

typedef  struct
{
   base_tim_hdr   INFO;
   UINT32   Bytes;
   UINT16   PALX, PALY;
   UINT16   ZZ01;
   UINT16   CLUT_CNT;
} tim_4bpp;

typedef  struct
{
   base_tim_hdr   INFO;
   UINT32   Bytes;
   UINT16   PALX, PALY;
   UINT16   ZZ01;
   UINT16   CLUT_CNT;
} tim_8bpp;

typedef  struct
{
   base_tim_hdr   INFO;
   base_tim_img   IMAGEDAT;
}  tim_16bpp;

typedef  struct
{
   base_tim_hdr   INFO;
   base_tim_img   IMAGEDAT;
}  tim_24bpp;

#define  TIM_4BPP    8
#define  TIM_8BPP    9
#define  TIM_16BPP   2
#define  TIM_24BPP   3
#define  TIM_MAGIC   0x10

// Pallette/Color Look Up Tables for TIM files
typedef  UINT16 CLUT_4bpp[16];
typedef  UINT16 CLUT_8bpp[256];


[mirex]

Cyberman: great info, thank you, i could not find anything on TIM so far !

[Cyberman]

Small adendum

You see 'Bytes' in the structure that's the size in bytes of the whole TIM.  Useful I suppose if you want to know how big it is.
WCount also means WORD count.  ALL image data is stored as 16bit words and broken up from those That is the whole image structure though in word sized units.

I've seen images with as many as 16 CLUT's, this is important to remember as the CLUTS <palettes> are prior to the actual image header in 4BPP and 8BPP TIMS.

That's all

[Cyberman]

Kislinskiy:
These are the vertices

Code: [Select]


[  1] <   39,    52,   -99>
[  2] <    0,    72,   -96>
[  5] <   39,    47,   -79>
[  6] <    8,    65,   -68>
[  7] <    0,    69,   -78>
[ 12] <   37,    49,   -64>
[ 14] <    5,    70,   -57>
[ 22] <    0,    55,   -29>
[ 52] <  -28,    48,   -47>
[ 57] <   -5,    70,   -57>
[ 58] <  -37,    49,   -64>
[ 62] <   -8,    65,   -68>
[ 63] <  -39,    47,   -79>
[ 64] <  -39,    52,   -99>


Texture Triangle information

Code: [Select]

TexturedTri[11] = {
[  0]   2,   7,  62
     D0080|1600 7800 2D00 390D
// negative direction above nose
[  1]   7,   2,   6
     D0080|2D00 7800 1600 390B
// Positive direction above nose
[  2]   2,  63,  64
     D0080|1600 7800 2C38 1338
// positive ebrow area
[  3]  63,   2,  62
     D0080|2C38 7800 1600 390D
// positive eye area
[  4]  63,  62,  58
     D0080|2C38 7800 390D 3E34
// positive cheek area
[  5]  57,  22,  52
     D0080|006D 7840 3579 1046
// positive mouth/right
[  6]  57,  14,  22
     D0080|006D 7840 0082 3579
// mouth center
[  7]  22,  14,  21
     D0080|3579 7840 0082 10A9
// negative mouth/left
[  8]   6,   5,  12
     D0080|390B 7800 2C38 3E34
// negative cheek/left
[  9]   2,   5,   6
     D0080|1600 7800 2C38 390B
// negative eye/left
[ 10]   5,   2,   1
     D0080|2C38 7800 1600 1336
// negative eye brow/left
}

These are from TIFA.LZS

Triangle 0 goes right to left < ( negative offset) where as Triangle one goes left to right (positive). I don't know what the numbers do (these are read as unsigned little endian ints currently.)  However it looks as if they are rotated 180 degress as 1600 and 2D00 are swaped in location in both. the first 2 triangles are part of the face/eye area thus should use the same pallete and image. Also these are 'oposite' parts of TIFA's eye.Triangle 1 is sweeping across texture left to right.. the other is as well only the texture is pasted in the reverse direction. I gleaned this from giving the varirious triangles a different color intensity for each triangle number made it easier to figure out what triangle was which

I hope this helps maybe you can figure it out some more than I can.

[Kislinskiy]

Oh, I have forget to post the format of the textured polygons.  

Code: [Select]

Textured Triangle:

UINT16 Index to Coordinate 1
UINT16 Index to Coordinate 2
UINT16 Index to Coordinate 3
       [80 00]
UCHAR8 U 1
UCHAR8 V 1
       [00 78]
UCHAR8 U 2
UCHAR8 V 2
UCHAR8 U 3
UCHAR8 V 3


Textured Plane:

UINT16 Index to Coordinate 1
UINT16 Index to Coordinate 2
UINT16 Index to Coordinate 3
UINT16 Index to Coordinate 4
UCHAR8 U 1
UCHAR8 V 1
       [80 78]
UCHAR8 U 2
UCHAR8 V 2
UCHAR8 U 3
UCHAR8 V 3
UCHAR8 U 4
UCHAR8 V 4
       [80 00]


U/V-coords are X/Y-coords in the TIM image.

To convert the U/V-coords to float[0.0f ... 1.0f] just
divide 1.0f by TIM-width/Height and multiply this with the U/V-coord:

fU = (1.0f / static_cast<float>(TIM.Width)) * static_cast<float>(ucU);
fV = (1.0f / static_cast<float>(TIM.Height)) * static_cast<float>(ucV);

THX for the TIM data.  

[Cyberman]

Well I had gotten to this:

V0 U0,  PALETTE,  V1 U1, V2 U2

Palette is actually information about which CLUT to use for the texture (yes it's time to groan because this is where the PSX system gets high efficiency but makes reading this data a pain).  
0x7800 CLUT one
0x7840 CLUT two

In TIFA.LZS
I've seen as many as 3 CLUTS in FF7's data.

I looked at all of them too. That being said.
Only a few of the TIM's support 8bpp these are in ENEMY6 and are the hirez Sephiros  and Cloud models.  I believe other information is in amongst the UV data, namely if the texture is transparent or not. In a few cases the texture is not (actually quite a few) Right now I'm going to assume the alpha channel on pixel of black pixels is 1.
The only way I can think of generating the textures is when I count 'parts' I can use this to find which sections of the TIM are which CLUT then create the texture acordingly in OGL.

Of course counting the parts in done first.
I believe the other sections are all the animation data etc. So I will be inspecting those after I finish making the model pieces look pretty

Cyb

[Kislinskiy]

Yes, it seems to be palette data between the U/V-coords. I have not checked all models but I assume black (0, 0, 0) is always transparent.

There are two options to render the models with textures:

1. Create one palettized texture (8 bit) and up to 3 palettes

2. Create a High/True-Color texture for every TIM-CLUT

[Cyberman]

Yes, it seems to be palette data between the U/V-coords. I have not checked all models but I assume black (0, 0, 0) is always transparent.

There are two options to render the models with textures:

1. Create one palettized texture (8 bit) and up to 3 palettes

2. Create a High/True-Color texture for every TIM-CLUT

for 1 I'll have to see if I can do that in OGL
2 I didn't really think of but is a possibility.  It would still work with FF8 data and FF9 data from what I've seen of that.

There is a third option which I am currently trying to use.
3. Create a texture using the palette and TIM data to generate a complete image from rectangular sections used from the polygon data.  This is iffy if it will work but I've noticed all textured regions do not overlap in rectangular coordinates. This is the most likely to work, but complicated.  Since the polygons are not ordered based on there palettes (and also cannot be)  I use the rectangular max min for X and Y directions then scan the tim generating the 24 bit resulting color data to create the data.  My problem is what to do with the alpha channel. At the moment, I am clearing everything and setting the alpha channel on just the sections used / read by the polygons.  Messy but for now most likely to work.  Since it's only done ONCE per for each model It shouldn't have too much of an overhead.

I think the single texture in 32bit is the 'best' however as you've seen how I am doing it, well it's tricky

Cyb

[Kislinskiy]

We have to correct the u/v-coords a little bit.

Change this:

Code: [Select]

fU = (1.0f / static_cast<float>(TIM.Width)) * static_cast<float>(ucU);
fV = (1.0f / static_cast<float>(TIM.Height)) * static_cast<float>(ucV);

to something like this:

Code: [Select]

fU = (1.0f / static_cast<float>(TIM.Width + 4)) * static_cast<float>(ucU);
fV = (1.0f / static_cast<float>(TIM.Height + 6)) * static_cast<float>(ucV);