Author Topic: [Question]Texture formation of android version  (Read 3152 times)

albert

  • *
  • Posts: 52
    • View Profile
[Question]Texture formation of android version
« on: 2023-08-01 14:33:12 »
Hi everyone,

I try to do something to FF8R android version and I saw lot of texture file formation is astcz, I don't know how to decompress/decode this, is some one know about it?

thanks.
« Last Edit: 2023-08-01 14:40:58 by albert »

Maki

  • 0xBAADF00D
  • *
  • Posts: 624
  • 0xCCCCCCCC
    • View Profile
Re: [Question]Texture formation of android version
« Reply #1 on: 2023-08-26 20:03:47 »
ASTC is texture compression format that is almost native to handheld/console devices. Personal computers with x86 architecture and typical Nvidia/AMD graphics cards have ZERO support for ASTC. Instead, the typical compression is DXT. Graphic cards in Nintendo Switch console, iOS and Androids have native support for ASTC. Now that's where it gets tricky. The consoles and mobiles read the ASTC files as-is. On Windows you rely only on CPU.
Anyway... ASTCZ is as far as I remember LZ4 compressed raw ASTC.
You have to decompress it via LZ4 algorithm by adding the LZ4 header to the file, then... I have no idea. There was a tool AMD's "Compressor" or something which was capable of re-encoding the ASTC files but I have no idea.
ASTC work on pixel blocks, sometimes even completely outside the typical 2-4-8 bit layout (like 6x6 bit blocks which produces non power of two numbers which are problematic)


Tl;dr - add LZ4 header, decompress and then I have no idea. That AMDs compressor might help.

EDIT:
https://gpuopen.com/compressonator/

EDIT2:
Found something better from ARM directly:
https://github.com/ARM-software/astc-encoder

It let's you decode texture to TARGA with both SDR and HDR support + re-encode back to ASTC
« Last Edit: 2023-08-26 21:43:13 by Maki »

albert

  • *
  • Posts: 52
    • View Profile
Re: [Question]Texture formation of android version
« Reply #2 on: 2023-08-27 13:42:45 »
ASTC is texture compression format that is almost native to handheld/console devices. Personal computers with x86 architecture and typical Nvidia/AMD graphics cards have ZERO support for ASTC. Instead, the typical compression is DXT. Graphic cards in Nintendo Switch console, iOS and Androids have native support for ASTC. Now that's where it gets tricky. The consoles and mobiles read the ASTC files as-is. On Windows you rely only on CPU.
Anyway... ASTCZ is as far as I remember LZ4 compressed raw ASTC.
You have to decompress it via LZ4 algorithm by adding the LZ4 header to the file, then... I have no idea. There was a tool AMD's "Compressor" or something which was capable of re-encoding the ASTC files but I have no idea.
ASTC work on pixel blocks, sometimes even completely outside the typical 2-4-8 bit layout (like 6x6 bit blocks which produces non power of two numbers which are problematic)


Tl;dr - add LZ4 header, decompress and then I have no idea. That AMDs compressor might help.

EDIT:
https://gpuopen.com/compressonator/

EDIT2:
Found something better from ARM directly:
https://github.com/ARM-software/astc-encoder

It let's you decode texture to TARGA with both SDR and HDR support + re-encode back to ASTC

Thank you for your help, I used Python and made a script to do that. this problem has solved.

They use Astcz file to make Android version can read high revolution texture, it's useful.

Maki

  • 0xBAADF00D
  • *
  • Posts: 624
  • 0xCCCCCCCC
    • View Profile
Re: [Question]Texture formation of android version
« Reply #3 on: 2023-10-06 15:38:15 »
Found this python script in my files:

Code: [Select]
#!/usr/bin/env python3
# Author: AnalogMan + Marcin 'Maki' Gomulak
# Modified Date: 2023-06-03
# Purpose: Compress/Decompress DDSZ files for Switch
# Requirements: LZ4 (pip install lz4)

import argparse
import lz4.block
import os
import sys
from pathlib import Path

args = None

def RecurseDecode(folder_path):
    if not args.ext:
        args.ext = 'ddsz'
    for path in Path(folder_path).rglob('*.' + args.ext):
        with open(path, 'rb') as f:
            data = f.read()
        decodedData = lz4.block.decompress(data[4:])

        with open(str(path).replace(args.ext, args.ext[:-1]), 'wb') as fd:
            bytesWritten = fd.write(decodedData)
        if args.verbose:
            print('Decoded ' + str(path) + ' (' + str(bytesWritten) + ' bytes)')


def main():
    global args
    print('\n======== DDSZ Archive Tool ========\n\n')

    if sys.version_info <= (3, 1, 0):
        print('Python version 3.1.x+ needed to run this script.\n\n')
        return 1

        # Arg parser for program options
    parser = argparse.ArgumentParser(description='Process DDSZ files with LZ4')

    parser.add_argument('--decompress', '-d', action='store', help='Decompress DDSZ file', dest='filename_decomp')
    parser.add_argument('--recurseFolder', '-r', action='store', help='Recursive mode path', dest='folder_path_recurse')
    parser.add_argument('--verbose', '-v', action='store_true', help='Verbose mode', dest='verbose')
    parser.add_argument('--extension', '-e', action='store', help='Override default DDSZ extension', dest='ext')

    # Check passed arguments
    args = parser.parse_args()

    # Check if required files exist
    if args.folder_path_recurse:
        RecurseDecode(args.folder_path_recurse)
        return 0

    filename = args.filename

    # If DDSZ, decompress RAW block
    if (filename[-4:].lower()) == 'ddsz':
        try:
            with open(filename, 'rb') as f:
                compressed = f.read()
            with open(filename[:-4] + 'dds', 'wb') as f:
                f.write(lz4.block.decompress(compressed[4:]))
            print('DDS file decompressed successfully.\n\n')
        except:
            print('Could not decompress file.\n\n')

    # If DDS, compress to DDSZ and include filesize.
    elif (filename[-3:].lower()) == 'dds':
        try:
            with open(filename, 'rb') as f:
                decompressed = f.read()
            with open(filename[:-3] + 'ddsz', 'wb') as f:
                compressed = lz4.block.compress(decompressed)
                f.write((len(compressed) + 4).to_bytes(4, byteorder='little', signed=True))
                f.write(compressed)
            print('DDS file compressed successfully.\n\n')
        except:
            print('Could not compress file.\n\n')
    else:
        print('Unsupported file.\n')
        return 1


if __name__ == "__main__":
    main()

rabsanek

  • *
  • Posts: 2
    • View Profile
Re: [Question]Texture formation of android version
« Reply #4 on: 2024-11-06 15:54:24 »
Thank you for your help, I used Python and made a script to do that. this problem has solved.

They use Astcz file to make Android version can read high revolution texture, it's useful.

can share your script please?

albert

  • *
  • Posts: 52
    • View Profile
Re: [Question]Texture formation of android version
« Reply #5 on: 2024-11-07 13:16:01 »
can share your script please?


Here you are:



import sys, os, lz4.block

isDecompress=True
op='1'

astcenc=os.path.join(os.path.abspath(os.path.dirname(__file__)),'etcpack','astcenc-sse2.exe')

def get():
   print('input your choise:\n\t1.deachieve_Z\n\t2.achieve_Z\n\t3.png_trans_astc\n\t4.astc_trans_png')
   s=input()
   global op
   if s in ['1','2','3','4','']:
      
      if s=='':
         op='1'
      else:
         op=s
      #if s in ['1','']:
      #   isDecompress=False
      #else:
      #   isDecompress=True
   else:
      print('Inpurt error')
      get()

def get_path():   
   path=input('Please_input_the_path(don't_have_any_blankspace:')
   if os.path.isdir(path):
      pass
   else:
      print('Invalid_path'+path)
      get_path()
      return -1
   #print(os.getcwd())
   if op=='1':
      exts=['ddsz','pkmz','astcz']
   elif op=='2':
      exts=['dds','pkm','astc']
   elif op=='3':
      os.chdir(os.path.dirname(astcenc))
      exts=['png']
   elif op=='4':
      os.chdir(os.path.dirname(astcenc))
      exts=['astc']
   else:
      pass
   #input(os.getcwd())
   for root,dirs,files in os.walk(path):
      #for d in dirs:
      #   print(d)
      for fn in files:
         #print(os.path.join(root,fn),root,fn)
         ext=fn.split('.')[-1].lower()
         extLen=len(ext)
         #print(fn,exts,ext)
         if ext in exts:
            #print("1")
            p=os.path.join(root,fn)
            if op in ['1','2']:
               fi=open(p,'rb')
               data=fi.read()
               if op=='1':
                  print(p+'Deachieve...')
                  newData=lz4.block.decompress(data[4:])
                  fo=open(p[:-1],'wb')
                  fo.write(newData)
               else:
                  print(p+'Achieve...')
                  newData=lz4.block.compress(data)
                  newData0=(len(newData)+4).to_bytes(4, byteorder='little', signed=True)
                  fo=open(p+'z','wb')
                  fo.write(newData0)
                  fo.write(newData)
               fi.close()
               fo.close()
            elif op in ['3']:
               print('Transforming:'+p)
               os.system(' '.join([astcenc,'-cs',p,p[0:-3]+'astc','4x4','-exhaustive']))
            elif op in ['4']:
               print('Transforming:'+p)
               cmd=' '.join([astcenc,'-d',p,p[0:-4]+'png'])
               print(cmd)
               os.system(' '.join([astcenc,'-ds',p,p[0:-4]+'png']))

               
get()
get_path()

rabsanek

  • *
  • Posts: 2
    • View Profile
Re: [Question]Texture formation of android version
« Reply #6 on: 2024-11-08 08:07:55 »
Thanks, but it show error
input your choise:
        1.deachieve_Z
        2.achieve_Z
        3.png_trans_astc
        4.astc_trans_png
1
Please_input_the_path_dont_have_any_blankspace:/home/alex2/2
/home/alex2/2/logo.astczDeachieve...
Traceback (most recent call last):
  File "/home/alex2/albert.py", line 87, in <module>
    get_path()
  File "/home/alex2/albert.py", line 64, in get_path
    newData=lz4.block.decompress(data[4:])
_block.LZ4BlockError: Decompression failed: corrupt input or insufficient space in destination buffer. Error code: 12


albert

  • *
  • Posts: 52
    • View Profile
Re: [Question]Texture formation of android version
« Reply #7 on: 2024-11-13 11:41:08 »
Thanks, but it show error
input your choise:
        1.deachieve_Z
        2.achieve_Z
        3.png_trans_astc
        4.astc_trans_png
1
Please_input_the_path_dont_have_any_blankspace:/home/alex2/2
/home/alex2/2/logo.astczDeachieve...
Traceback (most recent call last):
  File "/home/alex2/albert.py", line 87, in <module>
    get_path()
  File "/home/alex2/albert.py", line 64, in get_path
    newData=lz4.block.decompress(data[4:])
_block.LZ4BlockError: Decompression failed: corrupt input or insufficient space in destination buffer. Error code: 12

There are original code:



import sys, os, lz4.block

isDecompress=True
op='1'

astcenc=os.path.join(os.path.abspath(os.path.dirname(__file__)),'etcpack','astcenc-sse2.exe')

def get():
   print('请输入你的选择:\n\t1.解压z文件\n\t2.压缩成z文件\n\t3.png转astc\n\t4.astc转png')
   s=input()
   global op
   if s in ['1','2','3','4','']:
      
      if s=='':
         op='1'
      else:
         op=s
      #if s in ['1','']:
      #   isDecompress=False
      #else:
      #   isDecompress=True
   else:
      print('输入错误。')
      get()

def get_path():   
   path=input('请输入文件夹地址(不能有空格):')
   if os.path.isdir(path):
      pass
   else:
      print('无效的文件夹地址:'+path)
      get_path()
      return -1
   #print(os.getcwd())
   if op=='1':
      exts=['ddsz','pkmz','astcz']
   elif op=='2':
      exts=['dds','pkm','astc']
   elif op=='3':
      os.chdir(os.path.dirname(astcenc))
      exts=['png']
   elif op=='4':
      os.chdir(os.path.dirname(astcenc))
      exts=['astc']
   else:
      pass
   #input(os.getcwd())
   for root,dirs,files in os.walk(path):
      #for d in dirs:
      #   print(d)
      for fn in files:
         #print(os.path.join(root,fn),root,fn)
         ext=fn.split('.')[-1].lower()
         extLen=len(ext)
         #print(fn,exts,ext)
         if ext in exts:
            #print("1")
            p=os.path.join(root,fn)
            if op in ['1','2']:
               fi=open(p,'rb')
               data=fi.read()
               if op=='1':
                  print(p+'正在解压...')
                  newData=lz4.block.decompress(data[4:])
                  fo=open(p[:-1],'wb')
                  fo.write(newData)
               else:
                  print(p+'正在压缩...')
                  newData=lz4.block.compress(data)
                  newData0=(len(newData)+4).to_bytes(4, byteorder='little', signed=True)
                  fo=open(p+'z','wb')
                  fo.write(newData0)
                  fo.write(newData)
               fi.close()
               fo.close()
            elif op in ['3']:
               print('正在转换:'+p)
               os.system(' '.join([astcenc,'-cs',p,p[0:-3]+'astc','4x4','-exhaustive']))
            elif op in ['4']:
               print('正在转换:'+p)
               cmd=' '.join([astcenc,'-d',p,p[0:-4]+'png'])
               print(cmd)
               os.system(' '.join([astcenc,'-ds',p,p[0:-4]+'png']))

               
get()
get_path()