[Question]Texture formation of android version

[albert]

Hi everyone,

I try to do something to FF8R android version and I saw lot of texture file formation is astcz, I don't know how to decompress/decode this, is some one know about it?

thanks.

[Maki]

ASTC is texture compression format that is almost native to handheld/console devices. Personal computers with x86 architecture and typical Nvidia/AMD graphics cards have ZERO support for ASTC. Instead, the typical compression is DXT. Graphic cards in Nintendo Switch console, iOS and Androids have native support for ASTC. Now that's where it gets tricky. The consoles and mobiles read the ASTC files as-is. On Windows you rely only on CPU.
Anyway... ASTCZ is as far as I remember LZ4 compressed raw ASTC.
You have to decompress it via LZ4 algorithm by adding the LZ4 header to the file, then... I have no idea. There was a tool AMD's "Compressor" or something which was capable of re-encoding the ASTC files but I have no idea.
ASTC work on pixel blocks, sometimes even completely outside the typical 2-4-8 bit layout (like 6x6 bit blocks which produces non power of two numbers which are problematic)


Tl;dr - add LZ4 header, decompress and then I have no idea. That AMDs compressor might help.

EDIT:
https://gpuopen.com/compressonator/

EDIT2:
Found something better from ARM directly:
https://github.com/ARM-software/astc-encoder

It let's you decode texture to TARGA with both SDR and HDR support + re-encode back to ASTC

[albert]

ASTC is texture compression format that is almost native to handheld/console devices. Personal computers with x86 architecture and typical Nvidia/AMD graphics cards have ZERO support for ASTC. Instead, the typical compression is DXT. Graphic cards in Nintendo Switch console, iOS and Androids have native support for ASTC. Now that's where it gets tricky. The consoles and mobiles read the ASTC files as-is. On Windows you rely only on CPU.
Anyway... ASTCZ is as far as I remember LZ4 compressed raw ASTC.
You have to decompress it via LZ4 algorithm by adding the LZ4 header to the file, then... I have no idea. There was a tool AMD's "Compressor" or something which was capable of re-encoding the ASTC files but I have no idea.
ASTC work on pixel blocks, sometimes even completely outside the typical 2-4-8 bit layout (like 6x6 bit blocks which produces non power of two numbers which are problematic)


Tl;dr - add LZ4 header, decompress and then I have no idea. That AMDs compressor might help.

EDIT:
https://gpuopen.com/compressonator/

EDIT2:
Found something better from ARM directly:
https://github.com/ARM-software/astc-encoder

It let's you decode texture to TARGA with both SDR and HDR support + re-encode back to ASTC

Thank you for your help, I used Python and made a script to do that. this problem has solved.

They use Astcz file to make Android version can read high revolution texture, it's useful.

[Maki]

Found this python script in my files:

Code: [Select]

#!/usr/bin/env python3
# Author: AnalogMan + Marcin 'Maki' Gomulak
# Modified Date: 2023-06-03
# Purpose: Compress/Decompress DDSZ files for Switch
# Requirements: LZ4 (pip install lz4)

import argparse
import lz4.block
import os
import sys
from pathlib import Path

args = None

def RecurseDecode(folder_path):
    if not args.ext:
        args.ext = 'ddsz'
    for path in Path(folder_path).rglob('*.' + args.ext):
        with open(path, 'rb') as f:
            data = f.read()
        decodedData = lz4.block.decompress(data[4:])

        with open(str(path).replace(args.ext, args.ext[:-1]), 'wb') as fd:
            bytesWritten = fd.write(decodedData)
        if args.verbose:
            print('Decoded ' + str(path) + ' (' + str(bytesWritten) + ' bytes)')


def main():
    global args
    print('\n======== DDSZ Archive Tool ========\n\n')

    if sys.version_info <= (3, 1, 0):
        print('Python version 3.1.x+ needed to run this script.\n\n')
        return 1

        # Arg parser for program options
    parser = argparse.ArgumentParser(description='Process DDSZ files with LZ4')

    parser.add_argument('--decompress', '-d', action='store', help='Decompress DDSZ file', dest='filename_decomp')
    parser.add_argument('--recurseFolder', '-r', action='store', help='Recursive mode path', dest='folder_path_recurse')
    parser.add_argument('--verbose', '-v', action='store_true', help='Verbose mode', dest='verbose')
    parser.add_argument('--extension', '-e', action='store', help='Override default DDSZ extension', dest='ext')

    # Check passed arguments
    args = parser.parse_args()

    # Check if required files exist
    if args.folder_path_recurse:
        RecurseDecode(args.folder_path_recurse)
        return 0

    filename = args.filename

    # If DDSZ, decompress RAW block
    if (filename[-4:].lower()) == 'ddsz':
        try:
            with open(filename, 'rb') as f:
                compressed = f.read()
            with open(filename[:-4] + 'dds', 'wb') as f:
                f.write(lz4.block.decompress(compressed[4:]))
            print('DDS file decompressed successfully.\n\n')
        except:
            print('Could not decompress file.\n\n')

    # If DDS, compress to DDSZ and include filesize.
    elif (filename[-3:].lower()) == 'dds':
        try:
            with open(filename, 'rb') as f:
                decompressed = f.read()
            with open(filename[:-3] + 'ddsz', 'wb') as f:
                compressed = lz4.block.compress(decompressed)
                f.write((len(compressed) + 4).to_bytes(4, byteorder='little', signed=True))
                f.write(compressed)
            print('DDS file compressed successfully.\n\n')
        except:
            print('Could not compress file.\n\n')
    else:
        print('Unsupported file.\n')
        return 1


if __name__ == "__main__":
    main()


[rabsanek]

Thank you for your help, I used Python and made a script to do that. this problem has solved.

They use Astcz file to make Android version can read high revolution texture, it's useful.

can share your script please?

[albert]

can share your script please?

Here you are：



import sys, os, lz4.block

isDecompress=True
op='1'

astcenc=os.path.join(os.path.abspath(os.path.dirname(__file__)),'etcpack','astcenc-sse2.exe')

def get():
   print('input your choise：\n\t1.deachieve_Z\n\t2.achieve_Z\n\t3.png_trans_astc\n\t4.astc_trans_png')
   s=input()
   global op
   if s in ['1','2','3','4','']:
      
      if s=='':
         op='1'
      else:
         op=s
      #if s in ['1','']:
      #   isDecompress=False
      #else:
      #   isDecompress=True
   else:
      print('Inpurt error')
      get()

def get_path():   
   path=input('Please_input_the_path(don't_have_any_blankspace：')
   if os.path.isdir(path):
      pass
   else:
      print('Invalid_path'+path)
      get_path()
      return -1
   #print(os.getcwd())
   if op=='1':
      exts=['ddsz','pkmz','astcz']
   elif op=='2':
      exts=['dds','pkm','astc']
   elif op=='3':
      os.chdir(os.path.dirname(astcenc))
      exts=['png']
   elif op=='4':
      os.chdir(os.path.dirname(astcenc))
      exts=['astc']
   else:
      pass
   #input(os.getcwd())
   for root,dirs,files in os.walk(path):
      #for d in dirs:
      #   print(d)
      for fn in files:
         #print(os.path.join(root,fn),root,fn)
         ext=fn.split('.')[-1].lower()
         extLen=len(ext)
         #print(fn,exts,ext)
         if ext in exts:
            #print("1")
            p=os.path.join(root,fn)
            if op in ['1','2']:
               fi=open(p,'rb')
               data=fi.read()
               if op=='1':
                  print(p+'Deachieve...')
                  newData=lz4.block.decompress(data[4:])
                  fo=open(p[:-1],'wb')
                  fo.write(newData)
               else:
                  print(p+'Achieve...')
                  newData=lz4.block.compress(data)
                  newData0=(len(newData)+4).to_bytes(4, byteorder='little', signed=True)
                  fo=open(p+'z','wb')
                  fo.write(newData0)
                  fo.write(newData)
               fi.close()
               fo.close()
            elif op in ['3']:
               print('Transforming：'+p)
               os.system(' '.join([astcenc,'-cs',p,p[0:-3]+'astc','4x4','-exhaustive']))
            elif op in ['4']:
               print('Transforming：'+p)
               cmd=' '.join([astcenc,'-d',p,p[0:-4]+'png'])
               print(cmd)
               os.system(' '.join([astcenc,'-ds',p,p[0:-4]+'png']))

               
get()
get_path()

[rabsanek]

Thanks, but it show error
input your choise:
        1.deachieve_Z
        2.achieve_Z
        3.png_trans_astc
        4.astc_trans_png
1
Please_input_the_path_dont_have_any_blankspace:/home/alex2/2
/home/alex2/2/logo.astczDeachieve...
Traceback (most recent call last):
  File "/home/alex2/albert.py", line 87, in <module>
    get_path()
  File "/home/alex2/albert.py", line 64, in get_path
    newData=lz4.block.decompress(data[4:])
_block.LZ4BlockError: Decompression failed: corrupt input or insufficient space in destination buffer. Error code: 12