Author Topic: [FF7PC] Remako HD Field Mod using AI neural networks (WIP)  (Read 37905 times)

AlphaAtlas

  • Fast newbie
  • *
  • Posts: 10
  • Karma: 1
    • View Profile
Re: [FF7PC] Remako HD Field Mod using AI neural networks (WIP)
« Reply #25 on: 2019-01-07 20:20:04 »
I use almost the same than for the esrgan background pack of ff7 i'm making:
1)Use waifu for noise reduction only (for this video i use noise level 2 with an old waifu build : "--scale_ratio 1 -m noise_scale --noise_level 2", i can upload you the specif exe if you can't get this result with modern waifu builds)
2)make a 25% downscale with imagemagik ("-filter box -resize 25%")
3)Upscale with esrgan (model used is an interpolation of Manga109 and RRDB_ESRGAN, ratio 80%Manga/20%ESRGAN)

I know that the "downscale then upscale again" process is a mathematic aberation but if you read the esrgan paper, they train the models with a set of high quality picture they donwscale with the bicubic algorythm (about the same as imagemagik box filter).
So the best way to get the full power of esrgan is to provide bicubic downscaled picture ^^
I tried looooots of methods with ff7 background and videos, the "noise reduction, downscale, upscale again" was always the best (off cource for the ff7 background, as a high resolution doesn't exist, i use waifu for noise reduction + upscale, before downscaling with imagemagic, and it's working great)

Waifu only looks at individual frames because it was programmed as an image upscaler. What you need a high quality temporal denoiser, right? I highly recommend V-BM3D or KNLMeansCL, as both are way better than waifu at temporal noise/grain like that.

You can actually do everything (artifact removal, denoise, upscaling, resize back down) in a single step with vapoursynth, and do intermediate steps like 2x or 3x... But that's a deeep rabbit hole to go down.

satsuki

  • Crazy poster
  • *
  • Posts: 203
  • Karma: 35
    • View Profile
    • My softs (french only)
Re: [FF7PC] Remako HD Field Mod using AI neural networks (WIP)
« Reply #26 on: 2019-01-07 22:05:41 »
I kown but i'm stuck with old avisynth, never take the time to go to vapoursynth.
I've done lots of video filtering long time ago (started with the first release of virtualdub to end with avisynth with its great features).
But honestly the hard part of this type of upscale is that esrgan (and other ia of the same type) is not designed to upscale cleaned up video picture but low resolution still picture.
With the original psx videos of ff7 you have to deal with low resolution (320*xxx) + temporal noise + low colors palette compression... not realy easy to work with ^^'
Right now i don't have much time to spend in it because i'm already do a test/correction off the fields of the game.. maybe when my pack will be done ^^

DLPB

  • No life
  • *
  • Posts: 9396
  • Karma: 323
  • For I realized that God's a young man, too.
    • View Profile
Re: [FF7PC] Remako HD Field Mod using AI neural networks (WIP)
« Reply #27 on: 2019-01-07 22:22:46 »
The thing is - if you couple these upscales with a good scanline shader, you'll probably find it works even better.  It definitely does with original field + shader smoother.  The ugly parts are masked. Your brain fills in the detail.

AlphaAtlas

  • Fast newbie
  • *
  • Posts: 10
  • Karma: 1
    • View Profile
Re: [FF7PC] Remako HD Field Mod using AI neural networks (WIP)
« Reply #28 on: 2019-01-08 02:54:40 »
I kown but i'm stuck with old avisynth, never take the time to go to vapoursynth.
I've done lots of video filtering long time ago (started with the first release of virtualdub to end with avisynth with its great features).
But honestly the hard part of this type of upscale is that esrgan (and other ia of the same type) is not designed to upscale cleaned up video picture but low resolution still picture.
With the original psx videos of ff7 you have to deal with low resolution (320*xxx) + temporal noise + low colors palette compression... not realy easy to work with ^^'
Right now i don't have much time to spend in it because i'm already do a test/correction off the fields of the game.. maybe when my pack will be done ^^

Yeah. I've tested a few algos, and ESGRAN DOES NOT like being fed dirty frames. It seems to amplify halos, noise and such even more than other AI upscalers.

As for the difficulty of Vapoursynth... This week, I'm working on packaging the Vapoursynth Fatpack and this amazing repo in one easy package: https://github.com/WolframRhodium/Super-Resolution-Zoo

So, theoretically, you'd download a big zip file, unzip some separate Nvidia (or Intel) files I'm not allowed to redistribute, run a .bat file to install everything, then open a custom vapoursynth script in VSEDIT, and you'd be able try dozens of different AI upscalers, temporal GPU denoisers, artifact removers and such and such by just changing a number in a text file. The setup could batch process basically any image or video file under the sun, no conversion needed, and previewing/tweaking would be fast.


« Last Edit: 2019-01-08 02:58:09 by AlphaAtlas »

BeanBagger

  • Newbie
  • *
  • Posts: 2
  • Karma: 1
    • View Profile
Re: [FF7PC] Remako HD Field Mod using AI neural networks (WIP)
« Reply #29 on: 2019-01-08 04:09:05 »
This is amazing.

OP, do you have Patreon or something to donate to for your time?


satsuki

  • Crazy poster
  • *
  • Posts: 203
  • Karma: 35
    • View Profile
    • My softs (french only)
Re: [FF7PC] Remako HD Field Mod using AI neural networks (WIP)
« Reply #30 on: 2019-01-08 08:28:37 »
Yeah. I've tested a few algos, and ESGRAN DOES NOT like being fed dirty frames. It seems to amplify halos, noise and such even more than other AI upscalers.

As for the difficulty of Vapoursynth... This week, I'm working on packaging the Vapoursynth Fatpack and this amazing repo in one easy package: https://github.com/WolframRhodium/Super-Resolution-Zoo

So, theoretically, you'd download a big zip file, unzip some separate Nvidia (or Intel) files I'm not allowed to redistribute, run a .bat file to install everything, then open a custom vapoursynth script in VSEDIT, and you'd be able try dozens of different AI upscalers, temporal GPU denoisers, artifact removers and such and such by just changing a number in a text file. The setup could batch process basically any image or video file under the sun, no conversion needed, and previewing/tweaking would be fast.

Cool !
i'll wait for it ^^

AlphaAtlas

  • Fast newbie
  • *
  • Posts: 10
  • Karma: 1
    • View Profile
Re: [FF7PC] Remako HD Field Mod using AI neural networks (WIP)
« Reply #31 on: 2019-01-08 17:41:26 »
Cool !
i'll wait for it ^^

Do you happen to have an Nvidia GPU?

The upscalers will work on a CPU, but they're slow. AFAIK, only Waifu2X has a version that's accelerated by AMD GPUs atm.

satsuki

  • Crazy poster
  • *
  • Posts: 203
  • Karma: 35
    • View Profile
    • My softs (french only)
Re: [FF7PC] Remako HD Field Mod using AI neural networks (WIP)
« Reply #32 on: 2019-01-08 18:52:46 »
Yes gtx1060

Heltonxl

  • Newbie
  • *
  • Posts: 1
  • Karma: 0
    • View Profile
Re: [FF7PC] Remako HD Field Mod using AI neural networks (WIP)
« Reply #33 on: 2019-01-08 20:09:33 »
Iam working on a Crono Kross remake using the same metods, but the game only exist for Ps1
Anyone could help me extractintg the raw textures of the game and put again in the iso file.
I will post what i already Made

BeanBagger

  • Newbie
  • *
  • Posts: 2
  • Karma: 1
    • View Profile
Re: [FF7PC] Remako HD Field Mod using AI neural networks (WIP)
« Reply #34 on: 2019-01-09 06:34:06 »
Iam working on a Crono Kross remake using the same metods, but the game only exist for Ps1
Anyone could help me extractintg the raw textures of the game and put again in the iso file.
I will post what i already Made

Like wise as per psx titles.

Id like to do this for Legend of Dragoon.

AuthenticM

  • Fast newbie
  • *
  • Posts: 9
  • Karma: 0
    • View Profile
Re: [FF7PC] Remako HD Field Mod using AI neural networks (WIP)
« Reply #35 on: 2019-01-09 19:42:48 »
Hey,

I just want to say that this is legit amazing. I was planning on playing FFVII with the release of R06, but I think I'm going to wait for this. Any estimation on when it would be complete?

Keep up the great work and Happy New Year!

satsuki

  • Crazy poster
  • *
  • Posts: 203
  • Karma: 35
    • View Profile
    • My softs (french only)
Re: [FF7PC] Remako HD Field Mod using AI neural networks (WIP)
« Reply #36 on: 2019-01-09 20:27:45 »
Mine is complete right now (fields, combats, worldmap), but i'im doing a full game trough to find bug and optimise filtering (because ingame rendering is not alway the same than picture rendering depending how much the game zoom the picture).
It's a long process to do but it'll ensure the best result a can do.
So maybe one or two months.
I'll also wait the AlphaAtlas tool to check i can do a better job on videos, can take some time too.
« Last Edit: 2019-01-09 20:29:48 by satsuki »

oraclex

  • Fast newbie
  • *
  • Posts: 26
  • Karma: 0
    • View Profile
Re: [FF7PC] Remako HD Field Mod using AI neural networks (WIP)
« Reply #37 on: 2019-01-09 20:45:50 »
A dutch webpage send be to this topic (long time member here) first time I have see them metioning a mod! It more a page for hardware and other technical related newbits
Well done.

guess also tnx is in order to you Satsuki. Been useing your work for quite some time. Can't play without it anymore.

AlphaAtlas

  • Fast newbie
  • *
  • Posts: 10
  • Karma: 1
    • View Profile
Re: [FF7PC] Remako HD Field Mod using AI neural networks (WIP)
« Reply #38 on: 2019-01-09 21:37:46 »
A dutch webpage send be to this topic (long time member here) first time I have see them metioning a mod! It more a page for hardware and other technical related newbits
Well done.

Yeah, someone linked this thread on /r/pcgaming, which PC Gamer is notorious for watching, then other tech media saw it on PC Gamer and reported on it.

Granfalloon

  • Fast newbie
  • *
  • Posts: 17
  • Karma: 0
    • View Profile
Re: [FF7PC] Remako HD Field Mod using AI neural networks (WIP)
« Reply #39 on: 2019-01-09 22:40:32 »
Mine is complete right now (fields, combats, worldmap), but i'im doing a full game trough to find bug and optimise filtering (because ingame rendering is not alway the same than picture rendering depending how much the game zoom the picture).
It's a long process to do but it'll ensure the best result a can do.
So maybe one or two months.
I'll also wait the AlphaAtlas tool to check i can do a better job on videos, can take some time too.
Pleaase post some screenshots of your play through in that time if you can, I'm dying to see more backgrounds in-game.

Thundersky309

  • Newbie
  • *
  • Posts: 1
  • Karma: 0
    • View Profile
Re: [FF7PC] Remako HD Field Mod using AI neural networks (WIP)
« Reply #40 on: 2019-01-10 03:59:51 »
Can you start doing more frequent posts about this project? I think more people would like to get involved with it. It's been in the news. Do you need any type of help?

satsuki

  • Crazy poster
  • *
  • Posts: 203
  • Karma: 35
    • View Profile
    • My softs (french only)
Re: [FF7PC] Remako HD Field Mod using AI neural networks (WIP)
« Reply #41 on: 2019-01-10 20:49:46 »
Can you start doing more frequent posts about this project? I think more people would like to get involved with it. It's been in the news. Do you need any type of help?
I made my own topic here : http://forums.qhimm.com/index.php?topic=18556.0

DLPB

  • No life
  • *
  • Posts: 9396
  • Karma: 323
  • For I realized that God's a young man, too.
    • View Profile
Re: [FF7PC] Remako HD Field Mod using AI neural networks (WIP)
« Reply #42 on: 2019-01-13 05:02:08 »
Can you start doing more frequent posts about this project? I think more people would like to get involved with it. It's been in the news. Do you need any type of help?

There is a lot of work involved.  Doing updates all the time isn't helpful.

mitzmitra

  • Cool newbie
  • *
  • Posts: 93
  • Karma: 24
  • one winged artist
    • View Profile
Re: [FF7PC] Remako HD Field Mod using AI neural networks (WIP)
« Reply #43 on: 2019-01-15 14:47:24 »
this is REALLY good news.. i hope this ends well and completed.. MORE POWER TO MODDERS AND TESTERS!  :)

nuada

  • Newbie
  • *
  • Posts: 1
  • Karma: 0
    • View Profile
Re: [FF7PC] Remako HD Field Mod using AI neural networks (WIP)
« Reply #44 on: 2019-01-18 11:27:20 »
i can olny say one about that. Perfect!

CaptRobau

  • Fast newbie
  • *
  • Posts: 19
  • Karma: 5
    • View Profile
Re: [FF7PC] Remako HD Field Mod using AI neural networks (WIP)
« Reply #45 on: 2019-01-22 21:17:32 »
This is amazing.

OP, do you have Patreon or something to donate to for your time?

i can olny say one about that. Perfect!

this is REALLY good news.. i hope this ends well and completed.. MORE POWER TO MODDERS AND TESTERS!  :)

I'm glad you all like it.

Yeah, someone linked this thread on /r/pcgaming, which PC Gamer is notorious for watching, then other tech media saw it on PC Gamer and reported on it.

I hadn't seen that. That's awesome! Being featured like that really helps push me past the finish line.

Can you start doing more frequent posts about this project? I think more people would like to get involved with it. It's been in the news. Do you need any type of help?

There is a lot of work involved.  Doing updates all the time isn't helpful.

Sorry for not responding in a while. I've been working on it in the background and went on a short trip this past week. I'm putting the finishing touches on it. I expect the beta release to arrive soon. I can understand you want to see more stuff, but so close to finish line I'm just going to focus on getting it out there. I'm sure you'd rather play it an evening sooner than later.

DLPB

  • No life
  • *
  • Posts: 9396
  • Karma: 323
  • For I realized that God's a young man, too.
    • View Profile
Re: [FF7PC] Remako HD Field Mod using AI neural networks (WIP)
« Reply #46 on: 2019-01-22 22:20:07 »
You've misread my post...  I am saying doing updates ISN'T helpful - to YOU.  I was responding to the other person.

Kaldarasha

  • No life
  • *
  • Posts: 2321
  • Karma: 146
  • Prince of Model Editing
    • View Profile
Re: [FF7PC] Remako HD Field Mod using AI neural networks (WIP)
« Reply #47 on: 2019-01-23 07:37:51 »

I hadn't seen that. That's awesome! Being featured like that really helps push me past the finish line.


And it is also a thing on Steam now.

CaptRobau

  • Fast newbie
  • *
  • Posts: 19
  • Karma: 5
    • View Profile
Re: [FF7PC] Remako HD Field Mod using AI neural networks (WIP)
« Reply #48 on: 2019-01-23 21:01:52 »
You've misread my post...  I am saying doing updates ISN'T helpful - to YOU.  I was responding to the other person.

I know. Just wanted to respond to both, that what you suggest is what I'm doing. Sorry for the confusion.

And it is also a thing on Steam now.


Very cool!  :mrgreen:

CaptRobau

  • Fast newbie
  • *
  • Posts: 19
  • Karma: 5
    • View Profile
Re: [FF7PC] Remako HD Field Mod using AI neural networks (WIP)
« Reply #49 on: 2019-01-24 22:28:37 »
Aside from rounding up the development work, I've also been working on my own little Blogger site to showcase the mod when it's released. Qhimm is great and everything, but forums just aren't a very casual-friendly way to show off a mod. I want to use the release thread (once it's been made) more for bug reports, technical discussions, updates, etc.

So while I said I wasn't going to do a big update here, as I wanted to work on the mod (and the accompanying webpage) I just realized that I needed to do a bunch of screenshots anyway for my Blogger site. So if you go to the link below and click on screenshots, you will be able to see some more in-game screenshots and renders. Aside from these, there will be no new media until release. It's just actual development work now.

https://captrobau.blogspot.com/