OLD FF7 Field file data from Ficedula

[The Skillster]

Hi I dug up an old Field Format Doc from Ficedula, might be some help:

FF7 Field File documentation
----------------------------

Written by Ficedula ([email protected], http://www.legacy-ovwp.org/~ficedula/)

This document's based on what I and (mostly) Qhimm have discovered about the FF7 field files. I don't guarantee it's correct (there's almost certainly *some* mistakes in it), and it doesn't work on *all* field files either. Email me with any comments!

Qhimms site: http://www.qhimm.com


General file format
-------------------

Field files are always found in FLEVEL.LGP. They are always LZS compressed (see my other documents/tools for details of LZS compression and tools to do it).

The first two bytes of each (decompressed) field file are blank (zero).
The next four bytes is an integer indicating how many sections are present in the file.
Then a number of 4-byte integers follow, giving the starting offset for each section.

All field files should contain 9 sections; it's what FF7 expects.

So, normally you'll find:

00              Blank                   2 bytes
0009            NumSections             4 bytes
????            Section 1 location      4 bytes
????            Section 2 location      4 bytes
etc.            etc.                    etc.

This document is going to explain sections 4 (PALETTE) and 9 (BACKGROUND). I understand a *bit* of section 1 since that contains - among other things - text, but not all of it (ie. I only understand the text portion).

Each section generally starts with a four byte integer indicating the length of the section. You could just work this out by comparing offsets (how much space until the next section/end of file, etc) but FF7 stores the length at the start of the section anyway. After that the actual data follows. So the first bit of data for a section is actually 4 bytes after the point given in the section header (since the first four bytes are actually the length marker).


Section 4: PALETTE data
-----------------------
After the length indicator comes another integer, also indicating length. Useless, but it's there.
Then there's one more integer; unknown purpose.
Then one byte; unknown (blank often).
Then a word; number of colours in the palette plus one. No idea why. You can work numcolours out from the section length, but FF7 stores it anyway; why add one? Dunno.
One more byte; unknown (blank often).

Then the actual palette data.

Each palette entry is a 16-bit colour. This is unusual - normally palettes store as high quality data as possible, usually 24/32 bit. However since FF7 only ever runs in 16 bit I guess there isn't much point storing any other kind of data. Actually, the data is 15-bit (5-bit Red, 5-bit Green, 5-bit Blue, 1 bit unused. I think it's unused).

That's it for the palette! Only other thing you need to know is, palettes generally contain a number of colours that's a multiple of 256. This is because the palette is split up into 256-colour 'pages' internally. So the first colour is page 0/colour 0. Colour 256 is page 1/colour 0. Colour 628 is page 2/colour 116. You'll see why in the background section.


Section 9: BACKGROUND data
--------------------------
Firstly, a number of variables.
At offset $28, a Word = background width (BGWidth)
At offset $2A, a Word = background height (BGHeight)
At offset $2C, a Word = number of background sprites (NumBGSprites)
At offset $32, the background sprite data. See below for format (each sprite is 52 bytes long)
After the background sprite data, another $7 bytes, unknown purpose.
Then (ie. at offset $32 + NumBGSprites*52 + $7)  a Word = number of 2nd layer background sprites (NumBG2Sprites)
Then another $12 bytes, unknown purpose.
Then (ie. at offset $32 + NumBGSprites*52 + $1B) the background layer 2 sprite data. See below for format.
Then another $3D bytes, unknown purpose.
Then (ie. at offset $32 + NumBGSprites*52 + NumBG2Sprites*52 + $58) the raw image data.


Background paradigm
-------------------
(It isn't *really* a paradigm, but it sounds impressive if you say it is.)

FF7 stores its backgrounds in a rather complex format. Basically, you have the data split up into various sections:

1) Palette. List of colours.
2) Background sprites, layers 1 and 2. Just references to other bits of data.
3) Raw image data. Palettized data (ie. "grayscale" if viewed directly).

Each background sprite represents a 16x16 pixel block on the finished background. The sprite essentially contains the following information:

    -"Target" block, ie. where on the background to draw this 16x16 square
    -"Source" block, ie. where on the raw image data to take the pixels from
    -Palette "page", ie. which 256-colour palette block to apply to the raw image data

This is a very efficient way to store the image; on the one hand, it's in 16-bit colour, far better than just palettizing the whole image (ie. 256 colours over the *whole* background). On the other hand, each 16x16 pixel block takes much less space than if you'd stored it directly in 16-bit colour format. It isn't, however, easy to decode or (especially!) encode.


Format of a background sprite:

Type
  TFF7BgSprite = packed record
    ZZ1,X,Y:    Smallint;
    ZZ2:        Array[0..1] of Smallint;
    SrcX,SrcY:  Smallint;
    ZZ3:        Array[0..3] of Smallint;
    Pal:        Smallint;
    Flags:      Word;
    ZZ4:        Array[0..2] of Smallint;
    Page,Sfx:   Smallint;
    NA:         Longint;
    ZZ5:        Smallint;
    OffX,OffY:  Longint;
    ZZ6:        Smallint;
  end;

ZZ1,2,3,4,5,6:      Unknown data
X,Y:                Target position
SrcX,SrcY:          Source position
Pal:                Which palette page to use
Flags:              Indicate special effects ... not really understood properly.
Page:               Which image source page to use
Sfx:                More special effects?
NA:                 Unknown
OffX,OffY:          Unknown

The image source data is split up into 256x256 pixel pages; that's why as well as a source X and Y, you also have a source page (which 256x256 block to take data from). On the other hand, the destination background is stored as one big bitmap with no limits on size, so there, you just have a target X/Y position which can be used directly.

Also, note that each source image data "page" is preceded by 6 bytes of header.

So, say the raw image data starts at offset ImageData. Given a background sprite, the offset where that sprites data starts is:

StartOffset := (Page shl 16) or ((SrcY shl 8) or SrcX) + (Page+1)*6;

this is equivalent to

StartOffset := (Page * $FFFF) + (SrcY * $FF) + SrcX + (Page+1)*6;

(Page shl 16), (Page*$FFFF): Each page takes up 256x256 = $FFFF bytes, so skip that many for each page.

(SrcY shl 8), (SrcY*$FF): Each pixel row takes up 256 = $FF bytes, so skip that many to get to the right row.

SrcX: Taken directly.

(Page+1)*6: Skip 6 bytes of header per page. (Page+1) since even page 0 has 6 bytes preceding it.


Incidentally, the shifts are used in preference to multiplication since shifting is more efficient. Shifting on a computer is equivalent to multiplying/dividing by 2, 4, 8, .....

For the destination, note that you can use X/Y directly; however 0,0 *appears*, I *think*, to be at the image *centre*, not at the top/bottom left corner like with most programs.

So, now you know:
    -Where the raw image data for that sprite starts
    -Where you're drawing it to
    -Which palette page to use

Now, you just copy the pixels across, filtering the palette into it. IE:

Read a pixel from source image (one byte).
Set current colour to that colour in palette.
Draw onto target.

So, if you read a byte = 55 from the source image, you'd draw colour 55 in the selected palette to the target bitmap.


Other points
------------
Currently, Qhimms (and therefore mine too) source code doesn't draw a sprite if the Sfx is non-zero; this is because we don't understand what it does.

All variables above (Page, Pal, X, Y) start from ZERO; palette page zero is the first one, page one is the next one, etc.

The image data in FF7 palettes is stored in reverse order; ie. on Windows, data is stored Red first, then Green, then Blue. FF7 stores it the opposite way around, so you need to exchange the red/blue data. Here's how I do it in Cosmo:

      DCol := 0;
      DCol := DCol or ( (Col^ and $1F) shl 10 );
      DCol := DCol or (Col^ and $3E0);
      DCol := DCol or ( (Col^ and $7C00) shr 10);

Converts Col, an FF7 colour, into DCol, a (16-bit) Delphi colour.

The first background sprites are drawn "behind" the layer 2 sprites.

Variable conventions; I'm using Delphi names, which are as follows:

Byte:           8 bit, unsigned, integer
Word:           16 bit, unsigned, integer
Smallint:       16 bit, signed, integer
Integer/Longint:32 bit, signed, integer

(Unsigned = positive values only. Signed can hold positive or negative values).

Also, whenever I use numbers with $ signs above, it means I'm using hex values (hexadecimal).


Delphi code to do *all* of this (taken straight from Cosmo; cut-n-paste job) is below:


procedure TfrmCosmoBackground.FillBitmaps;
var
K,J,TOffset,TDest,I,B3Off,BOff:               Integer;
DCol,bgnsprites2,bgwidth,bgheight,bgnsprites: Word;
bgpsprites2,bgpsprites,psprite:               PFF7BGSPRITE;
Col,Dest,Pal:                                 PWord;
Source,Image,Comb,Picture,PB:                 PByte;
PI:                                           PInteger;
Bmp: TBitmap;
begin
Data.Position := 2 + (8+1)*4;
Data.ReadBuffer(BOff,4); Inc(BOff,4);
Data.Position := 2 + (3+1)*4;
Data.ReadBuffer(B3Off,4); Inc(B3Off,4);
Data.Position := BOff + $28;
Data.ReadBuffer(bgwidth,2);
Data.Position := BOff + $2A;
Data.ReadBuffer(bgheight,2);
Data.Position := BOff + $2C;
Data.ReadBuffer(bgnsprites,2);
GetMem(bgpsprites,Sizeof(TFF7BGSPRITE)*bgnsprites);
Data.Position := BOff + $32;
Data.ReadBuffer(bgpsprites^,Sizeof(TFF7BGSPRITE)*bgnsprites);
Data.Position := BOff + bgnsprites*52+$39;
Data.ReadBuffer(bgnsprites2,2);
GetMem(bgpsprites2,Sizeof(TFF7BGSPRITE)*bgnsprites2);
Data.Position := BOff + $32 + bgnsprites*52 + $1B;
Data.ReadBuffer(bgpsprites2^,Sizeof(TFF7BGSPRITE)*bgnsprites2);
PB := Data.Memory;
Inc(PB,B3Off + $C);
Pal := Pointer(PB);
//Pointer to raw palette data; needed later


begin {Palette creation}
  Palette := TBitmap.Create;
  PB := Data.Memory;
  Inc(PB,B3Off);
  PI := Pointer(PB);
  I := PI^ - $C;
  Palette.Width := 256;
  Palette.Height := (I div 2);
  For J := 1 to (I div 2) do begin
    Col := Pal;
    Inc(Col,J-1);
    K := ( (Col^ and $1F) shl 3 ) or ( ( (Col^ and $3E0) shr 5) shl 11) or ( ( (Col^ and $7C00) shr 10) shl 19);
    Palette.Canvas.Brush.Color := K;
    Palette.Canvas.Brush.Style := bsSolid;
    Palette.Canvas.FillRect(Rect( (16*(J mod 16)),(16*(J div 16)),(16+16*(J mod 16)),(16+16*(J div 16))));
  end;
end;
//Above section isn't really necessary, but it's nice to see a visual palette in the program.


Picture := Data.Memory;
Inc(Picture,BOff + bgnsprites*52 + bgnsprites2*52 + $58);
//Setup picture to point to the start of raw image data
GetMem(Image,bgwidth*bgheight*2);
ZeroMemory(Image,bgwidth*bgheight*2);
//Setup a target bitmap to draw to

For I := 0 to bgnsprites-1 do begin
  PSprite := BGPSprites;
  Inc(PSprite,I);
  TOffset := (PSprite^.Page shl 16) or ( (PSprite^.SrcY shl 8) or (PSprite^.SrcX) ) + (PSprite^.Page+1)*6;
  TDest := ((PSprite^.Y + (bgheight shr 1))*bgwidth)+(PSprite^.X+(bgwidth shr 1));
  TDest := TDest shl 1;
  If PSprite^.Sfx <> 0 then Continue;
  For J := 0 to 15 do begin //Copy 16 rows of data
    Source := Picture;
    If TOffset > Data.Size then Continue; //Sanity check! Don't read beyond end of file (=crash!)
    Inc(Source,TOffset);
    PB := Image;
    If TDest > (BGWidth*BGHeight*2) then Continue;
    Inc(PB,TDest);
    Dest := Pointer(PB);
    For K := 0 to 15 do begin //Copy 16 pixels in current row
      If Source^=0 then begin
        Inc(Source); Inc(Dest); Continue;
      end;
      If (Integer(Dest) > ( Integer(Image) + 2*BGHeight*BGWidth - 2 )) or (Integer(Dest) < Integer(Image)) then Continue; //Sanity check! Avoid crashes
      Col := Pal;
      Inc(Col, (PSprite^.Pal shl 8) + Source^ ); //Find correct palette entry
      If Integer(Col) > ( Integer(Data.Memory) + Data.Size ) then Continue; //Sanity check!
      Inc(Source);
      DCol := 0;
      DCol := DCol or ( (Col^ and $1F) shl 10 );
      DCol := DCol or (Col^ and $3E0);
      DCol := DCol or ( (Col^ and $7C00) shr 10);
      Dest^ := DCol; //Swap colour ordering and write to destination bitmap
      Inc(Dest);
    end;
    TDest := TDest + (BGWidth shl 1); //Skip one row of pixels in destination data
    Inc(TOffset,256); //Skip one row of pixels in source (always 256 pixels)
  end;
end;

Background := TBitmap.Create;
Background.PixelFormat := pf15Bit;
Background.Width := BGWidth;
Background.Height := BGHeight;
PB := Image;
For I := 0 to BGHeight-1 do begin
  Source := Background.ScanLine;
  CopyMemory(Source,PB,BGWidth*2);
  Inc(PB,BGWidth*2);
end;
//Do the actual drawing to a target bitmap


//Essentially this is the same as the previous code, except we're copying layer 2 this time.
Dest := Pointer(Image);
For I := 1 to bgwidth*bgheight do begin
  Dest^ := $1F shl 5;
  Inc(Dest);
end;

For I := 0 to bgnsprites2-1 do begin
  PSprite := BGPSprites2;
  Inc(PSprite,I);
  TOffset := (PSprite^.Page shl 16) or ( (PSprite^.SrcY shl 8) or (PSprite^.SrcX) ) + (PSprite^.Page+1)*6;
  TDest := ((PSprite^.Y + (bgheight shr 1))*bgwidth)+(PSprite^.X+(bgwidth shr 1));
  TDest := TDest shl 1;
  If PSprite^.Sfx <> 0 then Continue;
  For J := 0 to 15 do begin
    Source := Picture;
    If TOffset > Data.Size then Continue;
    Inc(Source,TOffset);
    PB := Image;
    If TDest > (BGWidth*BGHeight*2) then Continue;
    Inc(PB,TDest);
    Dest := Pointer(PB);
    For K := 0 to 15 do begin
      If Source^=0 then begin
        Inc(Source); Inc(Dest); Continue;
      end;
      If (Integer(Dest) > ( Integer(Image) + 2*BGHeight*BGWidth - 2 )) or (Integer(Dest) < Integer(Image)) then Continue;
      Col := Pal;
      Inc(Col, (PSprite^.Pal shl 8) + Source^ );
      Inc(Source);
      DCol := 0;
      DCol := DCol or ( (Col^ and $1F) shl 10 );
      DCol := DCol or (Col^ and $3E0);
      DCol := DCol or ( (Col^ and $7C00) shr 10);
      Dest^ := DCol; //!!!
      Inc(Dest);
    end;
    TDest := TDest + (BGWidth shl 1);
    Inc(TOffset,256);
  end;
end;

Foreground := TBitmap.Create;
Foreground.PixelFormat := pf15Bit;
Foreground.Width := BGWidth;
Foreground.Height := BGHeight;
PB := Image;
For I := 0 to BGHeight-1 do begin
  Source := Foreground.ScanLine;
  CopyMemory(Source,PB,BGWidth*2);
  Inc(PB,BGWidth*2);
end;
FreeMem(Image);
FreeMem(BGPSprites);
FreeMem(BGPSprites2);
end;

[ficedula]

Heh, that document is still on my website, although it's also a bit out-of-date...there's versions of LGP Tools around that do deal with some of the more complex features in the field files like transparency, although they don't do it very well...