Using AI - To get a small upgrade on background

[guyverthree]

Hey all,

I'm not graphic designer but I have done some thing with image manipulation and it got me thinking about image upscaling with AI.
Disclaimer, I'm not taking away from any of the great work that guys are doing and it's great and way outside my knowledge and skill.

Just a different approach to get some small measure of movement, it really got my that the new hi def figures don't really work with low res background, so I applied my knowledge:

Original Market shop:



Big market



I know it's not a patch on what the other guys are doing but it could be done on all the screens to get a semi working pack ?

[UpRisen]

It just takes to much detail away and make it look very flat and like plastic. There are 3 packs on the 7th heaven manager that use just upscaled versions and I would say that they all look better than this one because they dont flatten out as much of the depth.

[sl1982]

This took me less than 5 minutes to do

[UpRisen]

5 minutes for me aswell. I like to find a middle ground between grain and texture.

[guyverthree]

Fair enough,

I do take it the details do disappear. I was just trying a new approach, much thanks for considering it thou and being nice

Onward with the proper redesign, if you need any help scripting stuff or a tool to do something please let me know

[jasmer]

Has anybody tried Let's Enhance to upscale vanilla textures?

https://letsenhance.io/

[sl1982]

That is what i used. I am in the process of doing all the battle backgrounds with it. Some look great. Others look less great

[satsuki]

You may want to try https://github.com/xinntao/ESRGAN
I'm trying it for some days and can produce great results :





ESRGAN with some gimp cleanup:


On some other scenes it's harder to found a good value :



By the way if you're interest in a great simplication of fields resize i'm doing a software that will automate most of the process
1)i've generated all layers of all animations in full picture (with mixing layer if possible to lower the total number of picture to resize), that's the "base" pictures
2)i've generated all cut mask in 4x resize (with a vectorisation tools and imagemagik) for all single picture (right now i'm testing ingame for bug hunting ^^)
3)i've make a software for cuting automation
So when all will be tested i'll relase it and all you need to do is resize all the "base" with your favorite tool, put them in the soft, and it will generate all the folders ready to import in palmer with palmer frontend

So far the whole software is done, i need to test ingame and correct cutting layers by hand if i found some bug in autocut.
i might release the software and a howto in one or 2 month.

[Fischkopf]

You may want to try https://github.com/xinntao/ESRGAN
I'm trying it for some days and can produce great results :





ESRGAN with some gimp cleanup:


On some other scenes it's harder to found a good value :



By the way if you're interest in a great simplication of fields resize i'm doing a software that will automate most of the process
1)i've generated all layers of all animations in full picture (with mixing layer if possible to lower the total number of picture to resize), that's the "base" pictures
2)i've generated all cut mask in 4x resize (with a vectorisation tools and imagemagik) for all single picture (right now i'm testing ingame for bug hunting ^^)
3)i've make a software for cuting automation
So when all will be tested i'll relase it and all you need to do is resize all the "base" with your favorite tool, put them in the soft, and it will generate all the folders ready to import in palmer with palmer frontend

So far the whole software is done, i need to test ingame and correct cutting layers by hand if i found some bug in autocut.
i might release the software and a howto in one or 2 month.

Looks very interesting, how do I use this though? I also don't see a release there.

[satsuki]

To use this resize tool, just follow the instructions here : https://github.com/xinntao/ESRGAN
By the way i'm making now a new full upscale pack (video, battle, fields) with a more complex method and some manual tweaks (release maybe in 2 or 3 month):

[jusete]

This last one is looking amazing. Well done

[satsuki]

Thanks.
Most of the field's works is done with this method, now i need to do the game again to bug hunting and tweaks ^^ (then i'll release an upscale pack)

[Fraggoso]

Hi Satsuki I'm using esrgan as well for my FFIX background mod. I'm using the manga109 model though.
Maybe you can add me in steam or via PM so we can start a conversation. I think it can produce great values in esrgan and its way better that waigu and base esrgan.

[satsuki]

Thanks for the information.
I've tried the manga109 model (Manga109Attempt.pth) and i don't know witch method i like more.
My actuel method is not only esrgan, i use waifu to prefilter noise then some gmic then ersgan 0.8.

here's my version :


then manga109 :


i tried to prefilter before or after manga109 but the final resul il alway grainy and less sharp than my method, but is also more "natural" as there is let artfacts.
i'm trying interpolation now to try to found a good value

if you want to talk about it, mp me your stema and we can talk ^^

[Kaldarasha]

Can I make a suggestion? Couldn't you guys make two batches? One with full noise reduction and another batch with no noise reduction. What I have in mind is a manual merge of the two batches by the community. The merge wouldn't need any special skills only an eye for rough and smooth surfaces and a bit time. And if many do a bit of the work it would be done without to much stress and in a relative short time. I would like to do some images from time to time.

[satsuki]

Can I make a suggestion? Couldn't you guys make two batches? One with full noise reduction and another batch with no noise reduction. What I have in mind is a manual merge of the two batches by the community. The merge wouldn't need any special skills only an eye for rough and smooth surfaces and a bit time. And if many do a bit of the work it would be done without to much stress and in a relative short time. I would like to do some images from time to time.

I already do that for my pack.
I use 2 filtering, one "max detail" one "high noise reduction".
Then i layer them in gimp, with the "max detail" over, and i brush delete the too noisy parts so the "high noise reduction" appear.

I've tested a lots with the manga109 model (with and without pre/post filters) and in almost all cases my actuel filters are better, but in the cases of sharp edges, the manga one is way better on edge (with the 70% manga / 30% ersgan interpolated).

So i now i'll do:
-layer max detail and high noise removal and keep the best of 2 then merge them (mostly done, i'm checking it ingame for some days for optimisations)
-layer the "best of 2 details / noise removal" with the manga one to remove the "bad rendering edge" then merge them (need to it from begining)
-final game test


@Fraggoso
I've done some tests with FFIX background and the 70% to 80% manga109 with light noise filtering (or none, both look good) seems to be the way to go, my method look un-natural and to washed
To my ming FFIX as better quality backgrounds than FFVII so a lighter filtering seems good.

[Kaldarasha]

Yeah, FFIX has a 16bit colour palette and FF7 has only 8bit. They probably have added a colour shader to the BGs for FFlX before they converted the image for the game.

[Fraggoso]

Hi Satsuki, thanks for that information.
I'm still experimenting with my pipeline.

My steam name is Fraggoso. Add me.

I don't know if FFVII has sometimes objects on top of a Background layers?
If yes, do you know a way way of Filtering those properly?
I filter them with the entire scene so the output is better but cutting it manually is a pain in the Ass...

I tried to filter only the object, but the results are sub par.

[satsuki]

I found 3 Fraggoso on steam ^^'

I don't know if FFVII has sometimes objects on top of a Background layers?
If yes, do you know a way way of Filtering those properly?
I filter them with the entire scene so the output is better but cutting it manually is a pain in the Ass...

In FF7 each background use up to 5 layers for background itself and one layer for each step of each anymation.
The only way to do a proper upscale is to generate all full background of each animation step, then resize, then cut them back to layers.
I've automated most of the process with personnal vb6 software + imagemagick + potrace, but some background need manual tweaks, not so simple to do.

Here one exemple of the full layer pocessing :

The first step is to generate all the step of all layer to get base pictures, for this i used imagemagik "-layers merge" option with a custom VB software i made to generate the commands lines (one exemple to get one picture assembled):

Code: [Select]

cmd.exe /c start /B /wait /low " " "D:\Docs\Programation\VB\Sources\microbatclayersff7fields\imgk\convert.exe" -background none -layers merge +opaque black  "D:\Docs\Bureau\blin70_1_batch\blin70_1_0_00000128.png" "D:\Docs\Bureau\blin70_1_batch\blin70_1_0_00000129.png" "D:\Docs\Bureau\blin70_1_batch\blin70_1_0_00000130.png" "D:\Docs\Programation\VB\Sources\microbatclayersff7fields\provo\0.layersmix.1x.png"
Then i generate the 400% vectorized (to avoid strong aliasing and get smooth layers transition ingame) upscaled layers mask (to get the layers cuting and alpha channel after upscale) commands generated with a VB software and needs imagemagik and potrace, one exemple (from one layer to the 400% cutting layer):

Code: [Select]

cmd.exe /c start /B /wait /low " " "D:\Docs\Programation\VB\Sources\microbatclayersff7fields\imgk\convert.exe" -fill black -colorize 100%% -flatten "D:\Docs\Bureau\blin70_1_batch\blin70_1_0_00000128.png" "D:\Docs\Programation\VB\Sources\microbatclayersff7fields\provo\0blin70_1_0_00000128.png.bwavector.bmp"
cmd.exe /c start /B /wait /low " " "D:\Docs\Programation\VB\Sources\microbatclayersff7fields\potrace\potrace.exe" --turdsize 0 "D:\Docs\Programation\VB\Sources\microbatclayersff7fields\provo\0blin70_1_0_00000128.png.bwavector.bmp"
cmd.exe /c start /B /wait /low " " "D:\Docs\Programation\VB\Sources\microbatclayersff7fields\imgk\convert.exe" -density 288 -background #FF00FF -channel alpha -threshold 70%% -alpha remove -flatten "D:\Docs\Programation\VB\Sources\microbatclayersff7fields\provo\0blin70_1_0_00000128.png.bwavector.eps" "D:\Docs\Programation\VB\Sources\microbatclayersff7fields\provo\0blin70_1_0_00000128.png.vector4x.bmp"
cmd.exe /c start /B /wait /low " " "D:\Docs\Programation\VB\Sources\microbatclayersff7fields\imgk\convert.exe"  +transparent #000000 "D:\Docs\Programation\VB\Sources\microbatclayersff7fields\provo\0blin70_1_0_00000128.png.vector4x.bmp" "D:\Docs\Programation\VB\Sources\microbatclayersff7fields\provo\0blin70_1_0_00000128.png.vector4xblack.png"
cmd.exe /c start /B /wait /low " " "D:\Docs\Programation\VB\Sources\microbatclayersff7fields\imgk\convert.exe" -scale 400%% -background #FF00FF -alpha remove -alpha off -fill black +opaque #FF00FF "D:\Docs\Bureau\blin70_1_batch\blin70_1_0_00000128.png" "D:\Docs\Programation\VB\Sources\microbatclayersff7fields\provo\0blin70_1_0_00000128.png.raw4x.png"
cmd.exe /c start /B /wait /low " " "D:\Docs\Programation\VB\Sources\microbatclayersff7fields\imgk\convert.exe" +transparent #FF00FF "D:\Docs\Programation\VB\Sources\microbatclayersff7fields\provo\0blin70_1_0_00000128.png.raw4x.png" "D:\Docs\Programation\VB\Sources\microbatclayersff7fields\provo\0blin70_1_0_00000128.png.raw4x.coupe.png"
cmd.exe /c start /B /wait /low " " "D:\Docs\Programation\VB\Sources\microbatclayersff7fields\imgk\composite.exe" -gravity center "D:\Docs\Programation\VB\Sources\microbatclayersff7fields\provo\0blin70_1_0_00000128.png.raw4x.coupe.png" "D:\Docs\Programation\VB\Sources\microbatclayersff7fields\provo\0.layersmix.4x.png" "D:\Docs\Programation\VB\Sources\microbatclayersff7fields\provo\0blin70_1_0_00000128.png.raw4x.mix.png"
cmd.exe /c start /B /wait /low " " "D:\Docs\Programation\VB\Sources\microbatclayersff7fields\imgk\convert.exe" -alpha remove -alpha off +transparent black "D:\Docs\Programation\VB\Sources\microbatclayersff7fields\provo\0blin70_1_0_00000128.png.raw4x.mix.png" "D:\Docs\Programation\VB\Sources\microbatclayersff7fields\provo\0blin70_1_0_00000128.png.raw4x.back.mix.png"
cmd.exe /c start /B /wait /low " " "D:\Docs\Programation\VB\Sources\microbatclayersff7fields\imgk\composite.exe" -gravity center "D:\Docs\Programation\VB\Sources\microbatclayersff7fields\provo\0blin70_1_0_00000128.png.raw4x.back.mix.png" "D:\Docs\Programation\VB\Sources\microbatclayersff7fields\provo\0blin70_1_0_00000128.png.vector4xblack.png" "D:\Docs\Programation\VB\Sources\microbatclayersff7fields\provo\0blin70_1_0_00000128.png.fullblack.png"
cmd.exe /c start /B /wait /low " " "D:\Docs\Programation\VB\Sources\microbatclayersff7fields\imgk\convert.exe" -background #FF00FF -alpha remove -alpha off -transparent #000000 "D:\Docs\Programation\VB\Sources\microbatclayersff7fields\provo\0blin70_1_0_00000128.png.fullblack.png" "D:\Docs\Bureau\blin70_1_batch\blin70_1_0_00000128.pngcalquedecoupe.png"
Then i upscale

Last step is to get back the cutting and the alpha of layers from full pictures:

Code: [Select]

cmd.exe /c start /B /wait /low " " "D:\Docs\Programation\VB\Sources\microbatcutfieds\imgk\convert.exe" -define png:format=png32 -fuzz 01%% -fill #FE00FE -opaque #FF00FF "D:\Docs\Programation\VB\Sources\microbatcutfieds\work\Fields\sources4x\games_1_0_base_0000129.png" "D:\Docs\Programation\VB\Sources\microbatcutfieds\provo\games_1_0_00000128.4xnovi.png"
cmd.exe /c start /B /wait /low " " "D:\Docs\Programation\VB\Sources\microbatcutfieds\imgk\composite.exe" -gravity center "D:\Docs\Programation\VB\Sources\microbatcutfieds\work\Fields\layers4x\games_1_batch\games_1_0_00000128.pngcalquedecoupe.png" "D:\Docs\Programation\VB\Sources\microbatcutfieds\provo\games_1_0_00000128.4xnovi.png" "D:\Docs\Programation\VB\Sources\microbatcutfieds\provo\games_1_0_00000128.combi.png"
cmd.exe /c start /B /wait /low " " "D:\Docs\Programation\VB\Sources\microbatcutfieds\imgk\convert.exe" -define png:format=png32 "D:\Docs\Programation\VB\Sources\microbatcutfieds\provo\games_1_0_00000128.combi.png" -transparent #FF00FF "D:\Docs\Programation\VB\Sources\microbatcutfieds\work\Fields\final\games_1_batch\games_1_0_00000128.png"
not so simple to do ^^' took me lots of hours to get that but now i can do fast tests of upscales ingame because i have only to upscale then let my softs automate 99% of the process (http://forums.qhimm.com/index.php?topic=18500.0)

[Fraggoso]

Pick that one with the blond avatar that is eating an apple.

I also have an automatic way of doing my layers, what I meant was that sometimes on top of the static full backgrounds are layers with objects like a ghysal green in FFIX that's animated.

I also filter the whole scene with every animation as one whole picture. The problem now is that objects don't lign up with my generated alpha masks, derived from the psx backgrounds.

For those I would need a better way or manually cutting them to really fit as one object instead of taking subpixels from the nearby backgrounds.

[satsuki]

Added you in steam.
For this type of cutting you'll need to do it by hand, because (if it's like in ff7) the animated layer ins't only the animation but also some pixel around so no way to automate it to my mind

[Fraggoso]

Oh that really sucks but I guessed so.
How are you handling alpha layers as esrgan wipes them clean?

[satsuki]

As said first i make full picture (with black backgroud)
At the same time i generate a 4x inverted layer for each file (with the use of color FF00FF almost not used in ff7).
Then upscale.
Then merge the final resized picture with the 4x inverted layer.
Then delete FF00FF to get back the alpha channel
All that batched with the use of imagemagik (and potrace for the vectorisation part)

exemple:
base resized picture :


inverted 4x vectorized layer (white part is alpha channel):


merged:


delete FF00FF:


The hard part is to do a correct vectorised layer to get a smooth 4x layer and avoid missing part of the vectorised picture (for that i merge a vectorized and a scaled version of the 4x layer and it's avoiding most of bugs)

[satsuki]

After more tests, i'll use manga109 in some fields (i merge a manga109 upscale with some grain prefilter with a manga109 upscale without prefilter - 30%prefilter/70%standard).
If there's lots of rock or trees or vegetal forms le manga109 if better than my old method, but for any "human construction" my old method is better.
So let's gimp each picture to use both manga109 or my old method or a merge of both ^^

@Fraggoso :
Thanks for the manga109 model info, i haven't seen it before you tell me

[Fraggoso]

Satsuki it's great to hear so much dedication.
Glad I could help you out a bit.