FF8 Remaster new compression algorithm

[myst6re]

Hi, file formats in FF8 Remaster have changed a little. First step: open *.zzz archives wich contains data and movies.
For that I already coded a tool http://forums.qhimm.com/index.php?topic=19206.0

Next step, try to open field.fs/fl/fi with Deling. Unfortunately it does not work, because now files are compressed.
Example: bccent12.fl should contains a list of filenames, like this:

Code: [Select]

C:\ff8\Data\fre\FIELD\mapdata\bc\bccent12\bccent12.inf
C:\ff8\Data\fre\FIELD\mapdata\bc\bccent12\bccent12.mim
C:\ff8\Data\fre\FIELD\mapdata\bc\bccent12\bccent12.map
C:\ff8\Data\fre\FIELD\mapdata\bc\bccent12\bccent12.id
C:\ff8\Data\fre\FIELD\mapdata\bc\bccent12\bccent12.msd
C:\ff8\Data\fre\FIELD\mapdata\bc\bccent12\bccent12.ca
C:\ff8\Data\fre\FIELD\mapdata\bc\bccent12\bccent12.jsm
C:\ff8\Data\fre\FIELD\mapdata\bc\bccent12\bccent12.mrt
C:\ff8\Data\fre\FIELD\mapdata\bc\bccent12\bccent12.rat
C:\ff8\Data\fre\FIELD\mapdata\bc\bccent12\bccent12.pmd
C:\ff8\Data\fre\FIELD\mapdata\bc\bccent12\bccent12.pmp
C:\ff8\Data\fre\FIELD\mapdata\bc\bccent12\chara.one

On remastered version:



Filenames are still present, but several times, it seems that the file is now with multiple sections (starting with the size of the section). And each section is compressed.
The compression format always start with the magic code 4ZL_, followed by an unknown 32-bit integer, and followed by compressed data.

Is someone know about this 4ZL format?

[Maki]

I'm trying to RE this but no luck so far mostly due to debugger issues.
In FFVIII_EFIGS.dll at 015DE64B there's function that performs fread on files. Are you able to break on this? I'm not sure whether the issue is on me or the whole remaster has some anti-debugging techniques- tried with IDA, IDA+ScyllaHyde, CE and OllyDbg doesn't even show the process on the list

edit: to add to this it doesn't seem to use any of the zlibwapi.dll functions- tested with procmon- this is a stacktrace for reading the content of field.fl:

[myst6re]

Thanks for trying! I wonder if it's a homemade compression format, or a known algorithm.

[Maki]

Thanks to Hoodium- they lifted the anti-debugger/emulator 0xf00000f1 exception overwrite

Code: [Select]

int __usercall sub_115D72C0@<eax>(unsigned int maybeOutBuffer@<edx>, _BYTE *compressedBuffer@<ecx>, int firstDwordMinusOne, int secondDword)

Code: [Select]

int __usercall probably4LZDecompress@<eax>(unsigned int maybeOutBuffer@<edx>, _BYTE *compressedBuffer@<ecx>, int firstDwordMinusOne, int secondDword)
{
  _BYTE *v4; // eax@1
  unsigned int v5; // edi@1
  int v6; // edx@1
  int result; // eax@4
  unsigned int v8; // ebx@6
  size_t v9; // ebx@6
  int v10; // esi@7
  unsigned int v11; // esi@10
  unsigned int v12; // edx@10
  _WORD *v13; // esi@11
  int v14; // eax@12
  unsigned int *v15; // ebx@12
  unsigned int v16; // ecx@14
  int v17; // esi@14
  int v18; // edi@15
  int v19; // ecx@16
  unsigned int v20; // edi@20
  unsigned int v21; // ebx@20
  char *v22; // esi@21
  char *v23; // esi@21
  int v24; // ecx@22
  _BYTE *v25; // edx@23
  _BYTE *v26; // edi@26
  int v27; // ebx@26
  unsigned int v28; // edi@29
  int v29; // ecx@29
  unsigned int v30; // ebx@32
  char v31; // cl@33
  unsigned int v32; // edi@36
  int *v33; // ebx@36
  int v34; // ecx@37
  _BYTE *v35; // [sp+0h] [bp-60h]@1
  unsigned int v36; // [sp+4h] [bp-5Ch]@1
  int v37; // [sp+8h] [bp-58h]@1
  int v38; // [sp+Ch] [bp-54h]@1
  unsigned int v39; // [sp+10h] [bp-50h]@20
  unsigned int v40; // [sp+14h] [bp-4Ch]@1
  unsigned int v41; // [sp+14h] [bp-4Ch]@14
  char v42; // [sp+18h] [bp-48h]@6
  unsigned int v43; // [sp+18h] [bp-48h]@24
  __int128 v44; // [sp+1Ch] [bp-44h]@1
  __int128 v45; // [sp+2Ch] [bp-34h]@1
  int v46; // [sp+3Ch] [bp-24h]@1
  int v47; // [sp+40h] [bp-20h]@1
  int v48; // [sp+44h] [bp-1Ch]@1
  int v49; // [sp+48h] [bp-18h]@1
  int v50; // [sp+4Ch] [bp-14h]@1
  int v51; // [sp+50h] [bp-10h]@1
  int v52; // [sp+54h] [bp-Ch]@1
  int v53; // [sp+58h] [bp-8h]@1

  v4 = compressedBuffer;
  v35 = compressedBuffer;
  v5 = maybeOutBuffer;
  v36 = maybeOutBuffer;
  v6 = (int)&compressedBuffer[firstDwordMinusOne];// first byte of compressed data
  v40 = v36;
  v38 = (int)&compressedBuffer[firstDwordMinusOne];
  v37 = secondDword + v36;
  v46 = 0;
  v47 = 1;
  v48 = 2;
  v49 = 1;
  v50 = 4;
  v51 = 4;
  v52 = 4;
  v53 = 4;
  _mm_storeu_si128((__m128i *)&v44, _mm_load_si128((const __m128i *)&xmmword_116C4370));
  _mm_storeu_si128((__m128i *)&v45, _mm_load_si128((const __m128i *)&xmmword_116C2380));
  if ( secondDword )
  {
    while ( 1 )
    {
      v8 = *v4++;
      v42 = v8;
      v9 = v8 >> 4;
      if ( v9 == 15 )
      {
        do
        {
          v10 = *v4++;
          v9 += v10;
        }
        while ( (unsigned int)v4 < v6 - 15 && v10 == 255 );
        v5 = v40;
        if ( v9 + v40 < v40 || &v4[v9] < v4 )
          return v35 - v4 - 1;
      }
      v11 = secondDword + v36;
      v12 = v9 + v5;
      if ( v9 + v5 > v37 - 12 )
        goto LABEL_41;
      v13 = &v4[v9];
      if ( (unsigned int)&v4[v9] > v38 - 8 )
      {
        v11 = secondDword + v36;
LABEL_41:
        if ( &v4[v9] == (_BYTE *)v38 && v12 <= v11 )
        {
          memcpy((void *)v5, v4, v9);
          return v5 + v9 - v36;
        }
        return v35 - v4 - 1;
      }
      v14 = (int)&v4[-v5];
      v15 = (unsigned int *)(v9 + v5);
      do
      {
        *(_DWORD *)v5 = *(_DWORD *)(v14 + v5);
        *(_DWORD *)(v5 + 4) = *(_DWORD *)(v14 + v5 + 4);
        v5 += 8;
      }
      while ( v5 < v12 );
      v16 = *v13;
      v4 = v13 + 1;
      v17 = v12 - v16;
      v41 = v16;
      if ( v12 - v16 < v36 )
        return v35 - v4 - 1;
      v18 = v42 & 0xF;
      *v15 = v16;
      if ( v18 == 15 )
      {
        while ( 1 )
        {
          v19 = *v4++;
          if ( (unsigned int)v4 > v38 - 5 )
            return v35 - v4 - 1;
          v18 += v19;
          if ( v19 != 255 )
          {
            if ( (unsigned int *)((char *)v15 + v18) < v15 )
              return v35 - v4 - 1;
            v16 = v41;
            break;
          }
        }
      }
      v20 = v18 + 4;
      v21 = (unsigned int)v15 + v20;
      v39 = v21;
      if ( v16 >= 8 )
      {
        *(_DWORD *)v12 = *(_DWORD *)v17;
        v24 = *(_DWORD *)(v17 + 4);
        v23 = (char *)(v17 + 8);
        *(_DWORD *)(v12 + 4) = v24;
      }
      else
      {
        *(_BYTE *)v12 = *(_BYTE *)v17;
        *(_BYTE *)(v12 + 1) = *(_BYTE *)(v17 + 1);
        *(_BYTE *)(v12 + 2) = *(_BYTE *)(v17 + 2);
        *(_BYTE *)(v12 + 3) = *(_BYTE *)(v17 + 3);
        v22 = (char *)(*(&v46 + v41) + v17);
        *(_DWORD *)(v12 + 4) = *(_DWORD *)v22;
        v23 = &v22[-*((_DWORD *)&v44 + v41)];
      }
      v25 = (_BYTE *)(v12 + 8);
      if ( v21 <= v37 - 12 )
        break;
      v43 = v37 - 7;
      if ( v21 > v37 - 5 )
        return v35 - v4 - 1;
      if ( (unsigned int)v25 < v37 - 7 )
      {
        v26 = v25;
        v27 = v23 - v25;
        do
        {
          *(_DWORD *)v26 = *(_DWORD *)&v26[v27];
          *((_DWORD *)v26 + 1) = *(_DWORD *)&v26[v27 + 4];
          v26 += 8;
        }
        while ( (unsigned int)v26 < v43 );
        v21 = v39;
        v23 += v43 - (_DWORD)v25;
        v25 = (_BYTE *)(v37 - 7);
      }
      v28 = 0;
      v29 = v21 - (_DWORD)v25;
      if ( (unsigned int)v25 > v21 )
        v29 = 0;
      if ( v29 )
      {
        v30 = v29;
        do
        {
          v31 = *v23++;
          ++v28;
          *v25++ = v31;
        }
        while ( v28 < v30 );
        v5 = v39;
        v6 = v38;
        v40 = v39;
      }
      else
      {
LABEL_39:
        v6 = v38;
        v5 = v21;
        v40 = v21;
      }
    }
    *(_DWORD *)v25 = *(_DWORD *)v23;
    *((_DWORD *)v25 + 1) = *((_DWORD *)v23 + 1);
    if ( v20 > 0x10 )
    {
      v32 = (unsigned int)(v25 + 8);
      v33 = (int *)(v23 + 8);
      do
      {
        v34 = *v33;
        v33 += 2;
        *(_DWORD *)v32 = v34;
        *(_DWORD *)(v32 + 4) = *(v33 - 1);
        v32 += 8;
      }
      while ( v32 < v39 );
      v21 = v39;
    }
    goto LABEL_39;
  }
  if ( firstDwordMinusOne != 1 || *compressedBuffer )
    result = -1;
  else
    result = 0;
  return result;
}I didn't tidy it up, but does it seem familar to you? I'm not well into compression so I see nothing actually

XMM and int128- some long long buffers going on?


UPDATE:
First DWORD is uncompressedSizeBuffer:

example:
mapdata/mapdata.fl
1stDWORD = 45-8= 37
c:\ff8\data\x\field\mapdata\maplist 0x0A 0x0D  == 37 bytes

vars used in decoding:

Code: [Select]

xmmword_116C4370 xmmword 0FFFFFFFF000000000000000000000000h
xmmword_116C2380 xmmword 3000000020000000100000000h

I copied and fixed the code to new project, here the source so far:

Code: [Select]

// ConsoleApplication2.cpp : Ten plik zawiera funkcję „main”. W nim rozpoczyna się i kończy wykonywanie programu.
//

#include <iostream>
#include <xmmintrin.h>

#define BYTE unsigned char;

int probably4LZDecompress(unsigned int maybeOutBuffer, unsigned char* compressedBuffer, int firstDwordMinusOne, int secondDword)
{

unsigned char* v4; // eax@1
unsigned int v5; // edi@1
int v6; // edx@1
int result; // eax@4
unsigned int v8; // ebx@6
size_t v9; // ebx@6
int v10; // esi@7
unsigned int v11; // esi@10
unsigned int v12; // edx@10
unsigned short* v13; // esi@11
int v14; // eax@12
unsigned int* v15; // ebx@12
unsigned int v16; // ecx@14
int v17; // esi@14
int v18; // edi@15
int v19; // ecx@16
unsigned int v20; // edi@20
unsigned int v21; // ebx@20
char* v22; // esi@21
char* v23; // esi@21
int v24; // ecx@22
unsigned char* v25; // edx@23
unsigned char* v26; // edi@26
int v27; // ebx@26
unsigned int v28; // edi@29
int v29; // ecx@29
unsigned int v30; // ebx@32
char v31; // cl@33
unsigned int v32; // edi@36
int* v33; // ebx@36
int v34; // ecx@37
unsigned char* v35; // [sp+0h] [bp-60h]@1
unsigned int v36; // [sp+4h] [bp-5Ch]@1
int v37; // [sp+8h] [bp-58h]@1
int v38; // [sp+Ch] [bp-54h]@1
unsigned int v39; // [sp+10h] [bp-50h]@20
unsigned int v40; // [sp+14h] [bp-4Ch]@1
unsigned int v41; // [sp+14h] [bp-4Ch]@14
char v42; // [sp+18h] [bp-48h]@6
unsigned int v43; // [sp+18h] [bp-48h]@24
__m128 v44; // [sp+1Ch] [bp-44h]@1
__m128 v45; // [sp+2Ch] [bp-34h]@1
int v46; // [sp+3Ch] [bp-24h]@1
int v47; // [sp+40h] [bp-20h]@1
int v48; // [sp+44h] [bp-1Ch]@1
int v49; // [sp+48h] [bp-18h]@1
int v50; // [sp+4Ch] [bp-14h]@1
int v51; // [sp+50h] [bp-10h]@1
int v52; // [sp+54h] [bp-Ch]@1
int v53; // [sp+58h] [bp-8h]@1

v4 = compressedBuffer;
v35 = compressedBuffer;
v5 = maybeOutBuffer;
v36 = maybeOutBuffer;
v6 = (int)& compressedBuffer[firstDwordMinusOne];// first byte of compressed data
v40 = v36;
v38 = (int)& compressedBuffer[firstDwordMinusOne];
v37 = secondDword + v36;
v46 = 0;
v47 = 1;
v48 = 2;
v49 = 1;
v50 = 4;
v51 = 4;
v52 = 4;
v53 = 4;

__m128 xmmword_116C4370;
__m128 xmmword_116C2380;
xmmword_116C4370.m128_u64[1] = 0xFFFFFFFF00000000;
xmmword_116C4370.m128_u64[0] = 0x0000000000000000;

xmmword_116C2380.m128_u64[1] = 0x0000000300000002;
xmmword_116C2380.m128_u64[0] = 0x0000000100000000;

/*_mm_storeu_ps
_mm_storeu_si128((__m128i*) & v44, _mm_load_si128((const __m128i*) & xmmword_116C4370));
_mm_storeu_si128((__m128i*) & v45, _mm_load_si128((const __m128i*) & xmmword_116C2380));*/

//looks like _mm_storeu_si128 stores to &v44, and _mm_load_si128 loads _m128, so that's just v44=xmmword...
v44 = xmmword_116C4370;
v45 = xmmword_116C2380;


if (secondDword)
{
while (1)
{
v8 = *v4++;
v42 = v8;
v9 = v8 >> 4;
if (v9 == 15)
{
do
{
v10 = *v4++;
v9 += v10;
} while ((unsigned int)v4 < v6 - 15 && v10 == 255);
v5 = v40;
if (v9 + v40 < v40 || &v4[v9] < v4)
return v35 - v4 - 1;
}
v11 = secondDword + v36;
v12 = v9 + v5;
if (v9 + v5 > v37 - 12)
goto LABEL_41;
v13 = (unsigned short*)& v4[v9];
if ((unsigned int)& v4[v9] > v38 - 8)
{
v11 = secondDword + v36;
LABEL_41:
if (&v4[v9] == (unsigned char*)v38 && v12 <= v11)
{
memcpy((void*)v5, v4, v9);
return v5 + v9 - v36;
}
return v35 - v4 - 1;
}
v14 = (int)& v4[-v5];
v15 = (unsigned int*)(v9 + v5);
do
{
*(unsigned int*)v5 = *(unsigned int*)(v14 + v5);
*(unsigned int*)(v5 + 4) = *(unsigned int*)(v14 + v5 + 4);
v5 += 8;
} while (v5 < v12);
v16 = *v13;
v4 = (unsigned char*)v13 + 1;
v17 = v12 - v16;
v41 = v16;
if (v12 - v16 < v36)
return v35 - v4 - 1;
v18 = v42 & 0xF;
*v15 = v16;
if (v18 == 15)
{
while (1)
{
v19 = *v4++;
if ((unsigned int)v4 > v38 - 5)
return v35 - v4 - 1;
v18 += v19;
if (v19 != 255)
{
if ((unsigned int*)((char*)v15 + v18) < v15)
return v35 - v4 - 1;
v16 = v41;
break;
}
}
}
v20 = v18 + 4;
v21 = (unsigned int)v15 + v20;
v39 = v21;
if (v16 >= 8)
{
*(unsigned int*)v12 = *(unsigned int*)v17;
v24 = *(unsigned int*)(v17 + 4);
v23 = (char*)(v17 + 8);
*(unsigned int*)(v12 + 4) = v24;
}
else
{
*(unsigned char*)v12 = *(unsigned char*)v17;
*(unsigned char*)(v12 + 1) = *(unsigned char*)(v17 + 1);
*(unsigned char*)(v12 + 2) = *(unsigned char*)(v17 + 2);
*(unsigned char*)(v12 + 3) = *(unsigned char*)(v17 + 3);
v22 = (char*)(*(&v46 + v41) + v17);
*(unsigned int*)(v12 + 4) = *(unsigned int*)v22;
v23 = &v22[-*((unsigned int*)& v44 + v41)];
}
v25 = (unsigned char*)(v12 + 8);
if (v21 <= v37 - 12)
break;
v43 = v37 - 7;
if (v21 > v37 - 5)
return v35 - v4 - 1;
if ((unsigned int)v25 < v37 - 7)
{
v26 = v25;
v27 = (unsigned char)v23 - (unsigned char)v25;
do
{
*(unsigned int*)v26 = *(unsigned int*)& v26[v27];
*((unsigned int*)v26 + 1) = *(unsigned int*)& v26[v27 + 4];
v26 += 8;
} while ((unsigned int)v26 < v43);
v21 = v39;
v23 += v43 - (unsigned int)v25;
v25 = (unsigned char*)(v37 - 7);
}
v28 = 0;
v29 = v21 - (unsigned int)v25;
if ((unsigned int)v25 > v21)
v29 = 0;
if (v29)
{
v30 = v29;
do
{
v31 = *v23++;
++v28;
*v25++ = v31;
} while (v28 < v30);
v5 = v39;
v6 = v38;
v40 = v39;
}
else
{
LABEL_39:
v6 = v38;
v5 = v21;
v40 = v21;
}
}
*(unsigned int*)v25 = *(unsigned int*)v23;
*((unsigned int*)v25 + 1) = *((unsigned int*)v23 + 1);
if (v20 > 0x10)
{
v32 = (unsigned int)(v25 + 8);
v33 = (int*)(v23 + 8);
do
{
v34 = *v33;
v33 += 2;
*(unsigned int*)v32 = v34;
*(unsigned int*)(v32 + 4) = *(v33 - 1);
v32 += 8;
} while (v32 < v39);
v21 = v39;
}
goto LABEL_39;
}
if (firstDwordMinusOne != 1 || *compressedBuffer)
result = -1;
else
result = 0;
return result;
}

int main()
{
FILE* f = fopen("D:\\ff8_R\\EXPORT\\world_fs_export\\mapdata\\mapdata.fl", "rb");
fseek(f, 0, 2);
auto position = ftell(f);
rewind(f);
unsigned int* buffer = (unsigned int*)calloc(position, 1);
fread(buffer, sizeof(char), position, f);
fclose(f);

unsigned int firstDword = (*buffer) - 8;
unsigned int secondDword = *(buffer + 2);
unsigned char * buffer_ = (unsigned char*)(buffer + 3);
unsigned int* outBuffer = (unsigned int*)calloc(firstDword, 1);
unsigned int firstParm = (unsigned int)outBuffer;
int testing = probably4LZDecompress((unsigned int)firstParm, buffer_, firstDword, secondDword);
}


and it didn't quite worked well on line 119- I had mismatch with data which I had on demastered code.
It turns out the casual field.fs contains compression method of ==2
there are some issues:
in field.fi the mapdata/maplist has 0x25 uncompressed size (that is true actually)- but absolute offset points to 0x25- that's true, but it's way after the header. You have to seek -4 to get secondDword which will give 0x25 (37) - this is how much data should be read- currently Deling reads the size correctly- but with the header missing 8 bytes afterward
So to summarize:
field.fl ->
c:\\ff8\\data\\x\\field\\mapdata.fl is entry 2

field.fi entry 2 (so 12) - that is correct:
uncompressed size - 37
location in FS file - 25
compression flag - 2

If we jump to 0x19- we get the DWORD+4ZL... this is wrong (more or less)
now Deling exports by size of uncompressed- that's okay, but there's 12bytes header data
So the content reads 37 bytes, while it should read 37+12 = 49

[Maki]

WIP:
https://github.com/MaKiPL/4ZL__decompressor

*pardon my C++ - I'm not into that language
anyway if someone wants to help/rebuild the algorithm so it's not dirty as now, then much appreciated

[RinoaIsDumb]

4ZL is LZ4 backward, could it be it ? (https://en.wikipedia.org/wiki/LZ4_(compression_algorithm)).

[myst6re]

WIP:
https://github.com/MaKiPL/4ZL__decompressor

*pardon my C++ - I'm not into that language
anyway if someone wants to help/rebuild the algorithm so it's not dirty as now, then much appreciated

That's great, I will make PR to this!

4ZL is LZ4 backward, could it be it ? (https://en.wikipedia.org/wiki/LZ4_(compression_algorithm)).

Seems not, I forced the magic number, it doesn't work with LZ4 implementation. But it could be very similar.

[Maki]

Oh, myst6re I've got so many bad news
I managed to get fields up to 'cc' extracted. Even after decompressing they are still useless with Deling:
example bgroad_6.fl:

Code: [Select]

c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6.ca
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6.id
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6_jp.inf
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6.inf
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6_en.jsm
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6_fr.jsm
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6_it.jsm
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6_de.jsm
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6_es.jsm
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6_jp.jsm
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6_en.map
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6_fr.map
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6_it.map
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6_de.map
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6_es.map
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6_jp.map
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6_en.mim
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6_fr.mim
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6_it.mim
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6_de.mim
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6_es.mim
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6_jp.mim
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6.mrt
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6_en.msd
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6_fr.msd
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6_it.msd
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6_de.msd
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6_es.msd
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6_jp.msd
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6.msk
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6.pcb
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6.pmd
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6.pmp
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6.pvp
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6.rat
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6.sym
c:\ff8\data\x\field\mapdata\bg\bgroad_6\bgroad_6.tdw
c:\ff8_6.sym
c:\ff8\data\x\field\mapdatchara.onethe fi size is 456/12.0 = 38.0, but as you can see there are 39 lines in fl - this is where Deling crashes on open, but on other fields the bare minimum is supported

The newest commit on that test soft of mine makes it to decompress a bunch of fields

[kruci]

Hi,
it is LZ4 block
https://github.com/lz4/lz4/blob/master/doc/lz4_Block_format.md
To convert it to LZ4 file, you need add header (create LZ4 frame).
https://github.com/lz4/lz4/blob/master/doc/lz4_Frame_format.md

To convert 4ZL to valid LZ4 file:

Code: [Select]

remove first 12 bytes from 4ZL (remove header, now you have LZ4 Block)
add to start 7 bytes "04224D18607073" and 4 bytes "compressed block size" (4ZL file size - 12)
add to end 4 bytes "00000000"

Code: [Select]

04224D18607073  4B_BLOCKDATASIZE  BLOCKDATA  00000000
04224D18 is LZ4 magic number
607073 is frame descriptor (minimal variant)
00000000 is end mark

Then LZ4 file can be decompressed:

Code: [Select]

lz4.exe -d "bgroad_6.fl.lz4" bgroad_6.txtAnd result bgroad_6.txt have 38 lines

[Maki]

Totally outplayed! Thank you @kruci - so the line difference/broken FL was actually a bad decompression algo implementation

[myst6re]

Awesome kruci, this is easier than I thought

[Maki]

@myst6re will you be updating Deling for new comp=2 (LZ4)?

[myst6re]

Sure

[rick]

you think that it will be possible to create something that allow to change the spawn location of the monsters and of the bosses? something similar to what is possible to in final fantasy 9 with hadesworkshop , the general editor created by trilititi

[IFireflyl]

you think that it will be possible to create something that allow to change the spawn location of the monsters and of the bosses? something similar to what is possible to in final fantasy 9 with hadesworkshop , the general editor created by trilititi

I think you're getting ahead of yourself. They still need to properly unpack everything and take a look at it first. We won't know what can be done until we have everything properly laid out to see our options.

[myst6re]

@myst6re will you be updating Deling for new comp=2 (LZ4)?

Back to the subject, I pushed changes in the Deling repository to decompress LZ4 files. Now I try to handle multi-language field files, basically there is now one field.fs archive for every languages (files like *.jsm are named *_en.jsm, *_fr.jsm...).

[kaspar01]

You're doing great guys!
I just can't wait to see if we're finally be able to change 3D models to higher polycount version..I'd really be into creation of HD assets for serious modding if that became a thing..