Hi, I'm also working on an Update to my original Waifu HD Backgrounds for FFIX.
Nice ! I've come across your mod searching if it was possible to re-inject the background into the game, seing your mod was a deciding factor
What do you mean with rewritting HW to export more information? If you're extracting from PSX Source you just have to untick merge layers and you receive every layer (transparency, tiles and animations).
I need more informations than that in order to generate images that will be handled properly by the GAN. It was trained with full images, and doesn't cope well with uncomplete layers & transparencies (ie. a door on a black background, it will try to find pattern between the black & the door that are not suitable for the final result).
In order to do that, I don't only need the animation layers, I also need to know that a layer (in the tiff file) IS an animation layer (vs. static background), or if the layer has parralax, and if so, what is the depth of that layer.
For lighting effects and other stuff, I also need to know if a layer a different blending mode.
I've extraced all assets from PSX as well as Steam (as you need the light sources from Steam sadly) and I'm about 66% finished with everything.
I would not like to duplicate works here.
I don't know if you do that manually or not, but I wrote scripts that does that automatically (by reading the script field id from both versions in order to match them properly), and that the correct layer from both versions (as your said : blending layers from steam, real background from the PSX version).
I will try to release these (python) scripts ASAP (probably thursday or this week-end). I can't pack the final result due to copyright issues, so I need some time to be sure that they are easy enough to handle.
I'll also release all the PSX and Steam assets as well as every Folder with their respective Field# so no one have to worry about dubletts and Field#.
Yes, that is a good practice.
The seam issue is because the Steam Version uses one more pixel on each layer so they overlapp (the psx assets are seamless).
It's a classic layering issue. The main problem is that both version doesn't handle semi-transparent images, so the edges of each layer are sharp, instead of having some semi-transparent pixels in order to blend over.
Another issue is that, normally, the background layer shouldn't stop right where a layer on top start. That works on the PSX version because of the very sharp edges & low resolution, but I think they had some issues during the upscaling and used an extra pixel to avoid cover issues.
And on top of all that, their upscaling process had some black edge issues, related to bi-linear interpolation on single layer (that was I'm trying to avoid by recombining all the layers in a meaningful way first).
I resolved that with resizing the psx layers with 400% size & lanczos algorithm and get rid of the transperancy around the edges. Afterwards I just cut out the Layers on top of the new ESRGAN Image and voila -> No seam problems.
Actually, the transparency around the edge is not handle at all by the PC version. What is happening is that all pixels with a alpha < 127 result in a black alpha, and all pixels with an alpha <= 127 result in a white alpha.
I'm still unsure if I will use the PSX layers with a neighbooring algorythm, or something similar to what you did, but from early testing, both solutions seems workable.
How did you solve it? Maybe your art/mask is better than mine.
I'm not sure, but I think that the layer recombination is key to have a proper result.
ie .this kind of artefacts on the edge of the wood :
It's not due to the masking.
I've the same issue in photoshop is a perfect mask that I couldn't in-engine, but that I use for ground truth when testing.
It's due to the fact that the wood is interpolated near black pixels.
I've tried to extend the pixel borders to avoid that : I can reduce or increase the artifact effect depending of how it's done, but I'm only manage to get a perfect result when it's interpolated with a recombined image.