Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Topics - Ze_PilOt

Pages: [1]
1
FF9 Graphical Releases / [FF9PC] Background upscaling mod
« on: 2019-01-08 20:55:40 »
Hi,

While in vacation, in December, I gave a try with SR-GAN to upscale FF9 background, following a discussion with some friends that the original background where lost, and that DNN could help finding back these lost assets.
So I took all the exported background and gave it a go..

https://imgur.com/gallery/K1R3wJZ

This is with one of the trained generator (ESRGRAN4x).
The main problem here is that it's trained with a standard photoset (usually BSDS100) that is not suitable for artwork.
I think some people gave it a go mixing it with a set trained with the Manga109 dataset, but again not suitable : Photo & drawing are very different from artworks.

A photo from the set :

A drawing from manga109 :


What we are trying to achieve :


Think of the process like this : The DNN is trained to find pattern is small resolution images, and use patterns from the high resolution images it was trained for to fill the gaps.

BSDS or Manga109 dataset will generate too soft, not very detailed images due to the very simple nature of the images (manga109) or over-detailed and artifacts because of wrongly interpreted patterns (BSDS-100).
Also, the final image we want should have a "hand-painted feel" that none of these dataset offer.
This is often the mistake when I see people giving a go with DNN : You have to train your network with the kind of data you want to manipulate. There is sadly no silver bullet.

I've spend a couple of days and trained a GAN with my own dataset : HD artworks from
https://www.neogaf.com/threads/the-lost-art-of-final-fantasy-ix-mama-robotnik-research-thread.551612/
as well as artwork from different games that kind of look like FF9 style, ie :
http://conceptartworld.com/news/dragon-age-inquisition-environment-concept-art/
https://alishermirzoev.artstation.com/
https://www.pinterest.fr/andreamassa/tomb-raider-art/
https://www.videogamesartwork.com/games/witcher-3-wild-hunt
https://kaidash.artstation.com/projects/6kP9W

A selection of about 110 HD artworks, resulting in a 18.000 images dataset once tiled & randomized (for a DNN, a flipped/rotated image is a brand new image).




After 3 days of training, I've managed to get a better result out of the box.


(left is the original Manga109 / ESGRAN4x mix), right is mine, after a day of training. It's even better now that it's fully trained)

The next step is to automate the process of upscaling the whole game.
I've started from the PSX images, as the PC version is very poor quality wise, due to the de-dithering filtering applied :


(left is PC, right is PSX).

All was extracted using Hades Workshop.

I've of course had to apply my own filtering for decreasing the dithering without loosing too much details, through a photoshop batch script.

Then the masking problem.

https://twitter.com/Ze_PilOt/status/1080914390306643968

The masking was quickly solve, but I was still unhappy with the border artifacts from the different layers.
So I rewrite part of Hades Workshop to export me more informations (animations, light/effect layers, ...) so I could recombine layers to have a full image for the GAN generator, and to de-composite it back for Re-inserting it for Hades.

I'm happy to say that all these problem are now solved, and I'm in the process of converting the first CD (222 backgrounds) to do a test mod.


All the work -trained generator, automation scripts (normally no human/manual intervention), Hades modifications- (except the dataset for training due to copyright issues) will be available as open source ASAP.



Pages: [1]