Qhimm.com Forums
Miscellaneous Forums => Graphical => Topic started by: yarLson on 2016-02-01 02:44:56
-
So I just can't leave well enough alone I suppose. I know someone already did an upscale using waifu2x but I believe his model choice was poor. He used the original "trained" model for anime art, whereas there is another "trained" model you can use with waifu2x that is based off of photography.
Using this "photo" model instead of the anime model I have gotten incredibly clean and sharp results without the blotched or smudgy look of the anime style waifu2x. I already modified Omzy's facepalmer script to work with waifu-caffe, trained with this new model and using cuDNN to make the processing time reasonable. As a result I am posting a demo of the bombing mission so you can get an in game taste of this new method. Also posting some sample images from an imgur album but keep in mind these are compressed a bit.
So here is the demo, just rename "demofin" to "field" and place in modpath.
http://www.mediafire.com/download/o8p04qwwaifgsms/demofin.7z
Also here are just a few samples to see what I am talking about
https://imgur.com/a/IjxFa
What do you think? Would it be worth processing the whole game again? With my new rig it probably wouldn't take nearly as much time as it did in the past, but then again even with CUDA and cuDNN the waifu2x resizing does take a fair bit of time to process even with a gtx 970, so just looking for feedback as to whether or not I should go for it?
-
I'm on my phone right now but from what I can tell it would definitely be worth it.
There are also a few battle scenes that haven't been remade yet that would probably benefit as well
-
Would you consider these better quality overall than your previous processing of the game? I have your first one in the 7H Catalog atm, but if you consider this one to be better, I could replace it. It's up to you.
-
that is a nice looking field.. 8)
EDIT:
sir yarlson, up to what stage of the game is this demo fields completed? i want to try it ;D i'm still in golden saucer level :D
-
Too me, this looks better than the previous waifu release, in the fact that there is less the "cel shade"effect that might have been a consequence of using the anime trained approach.
However, it would be good if you could have a comparison on the same field.
-
It looks like the best I've seen so far, regarding the upscaled backgrounds. Doesn't look anywhere near as plastic as some attempts or like what fractals does.
edit.
It's actually at a standard that I can say is "good" - which I couldn't with the rest. It's pretty impressive.
-
Wow, this is really impressive! I didn't think the original backgrounds could ever be this crisp and clear.
-
This is genuinely good. I've always avoided upscaled backgrounds in the past, because I felt like the scaling artifacts, the weird blotches, etc., looked worse than the pixellation of the originals. This is the first attempt I've seen that I would actually use myself.
It's not perfect; among other things, some of the higher contrast dithering still scales weirdly (most obvious with the back wall near the top in the first screen). Have you considered running a low-strength noise removal filter over it before scaling? I've found that Photoshop's noise removal is pretty good at blending dithering like that without eating away at detail. For example, using these settings (http://i.imgur.com/xMD3fQW.png), I processed that same screen and came up with this:
(http://i.imgur.com/PVbrswB.png)
At this size, the original size, the difference is virtually indistinguishable to the naked eye. But with the dithering blurred, I'd imagine this would scale much more nicely with that improved scaling system you've got set up. Just a thought, maybe worth trying. If you don't wanna put in the effort of preprocessing fields through Photoshop, I certainly won't blame you, and I can't guarantee that the results would even be as good as I suspect they would (I don't have waifu2x handy to test it myself right now). Still, it could be worth trying.
In any case, what you've got is already so much better than anything I've seen. I'm really excited to see where this goes. I'd use it.
-
I'm not agry with you with the "photo" preset.
here's a side by side samples of the waifu "anime" (left) and your waifu "photo" right (best looking at full size of course).
the waifu "photo" is simply less sharp than the "anime" one and way less precise in large colors areas (lock at the red in the warning sign)
also the omzy script is a good starting point but it's full of miss-cut in the layers witch generating black pixels areas (the warning sign for exemple).
(http://yatoshi.com/waifcomp.jpg)
-
I think anime looks good enough... Apart from the fact that the "photo" version is a little blurrier the difference seems really minor.
-
The only issue I have with the original waifu upscale was the creation of lines that weren't there originally. You can kind of see what I mean on the 3rd image that satsuki posted on the big wall on the right. I can't unsee them
-
Well this is just to suite my personal taste. I don't like distortion or stylized upscales. While the anime is decent in sharpness it definitely changes the style of the original scenes. This result with photo is the best and sharpest image I have ever gotten without completely botching the style. In addition I now have a 2k display and the 4x upscales aren't working anymore I may need to go as much as 6x. Also satsuki, if you have a better script that your willing to share that'd be great otherwise I'll just use omzy's and fix some scenes that need manual tuning. To answer some other questions the demo goes up to the point where you get off the train after the bombing mission with that scene not including.
Also I have considered doing some noise reduction on the scenes I just wanted to post the results of the raw upscales to showcase how crisp it looks.
I have even considered just training my own waifu2x model completely from scratch based on the FF7's style but I don't know where I would be able to find upwards of 6000 or so largescale images all in the style of the original ff7 with as little noise as possible to train the model. I know I could use some of the fields that have been upscaled manually as input but I would need a lot more data to be able to train an effective model
EQ2Alyza I wouldn't replace anything just yet until I decide on a finalized process but I definitely consider this higher quality from my previous work. When you compare the raws side by side there is really no comparison
-
I'm not going to replace anything until you're finished. You asked in the OP if it'd be worth processing the whole game again, and I wanted to motivate you to do it by saying I'd use your newest work in the 7H Catalog :)
-
Hey yarlson these are A-ff7ing-mazing! They are the best ive seen so far, amateur speaking mind you. Afai am concerned go for it full speed!
-
Also satsuki, if you have a better script that your willing to share that'd be great otherwise I'll just use omzy's and fix some scenes that need manual tuning.
It's not a script, it's 2 home made vb6 software generating .bat file to use ImageMagick + ImageResizer + a special version of waifu2x (cpu and no alpha channel to have better result).
most of the fields are ok with one of the 2 software but i had to fix about 50 field by and and adjust maybe 80 to have smoother layers transition, but if you test my version you'll see not a single black pixel area (done the whole game twice to be sure), with the onzy's script a found about 120 fields with missing pixels so i go with my own version.
If you want the software i could send them to you with a "howto" but if you're familiar with scripting you may have better result by modifying the omzy's one
If you want i could also say in a precise way how the software i made works so you counld adapt it to the omzy script to your taste
-
Thanks Satsuki for posting comparison.
This confirmed that the photo preset looks better to me, because it is less sharp than the anime one.
I am perfectly fine with the fact that some people would prefer the anime version, though.
So yarLson, please go forward with this project! I would love to use it!
PS: and if you need some computing power, I can run some processing.
-
If you want the software i could send them to you with a "howto" but if you're familiar with scripting you may have better result by modifying the omzy's one
If you want i could also say in a precise way how the software i made works so you counld adapt it to the omzy script to your taste
Yeah sounds like you put a lot of good work into the the software and a how-to would be very useful. I should have enough experience to be able to use it. Also knowing how it works in a precise way would also be useful if you have the time. Now also if you documented which scenes it was that you needed to edit manually that'd be great as well that way I already know what to work on. Basically whatever your willing to share I'll take. You have done a lot of good work.
On a side note. hope my statement about "poor" choice wasn't offensive. I would be more correct to say "poor to my taste" but I rather admire your work from a technical standpoint.
Now I am also serious about training my own waifu model but does anyone have any clear idea about where we could get that many images. I mean using google's advanced search might not be a bad way to find the images but how exactly could we define the original Final Fantasy 7's style from a purely objective standpoint, in order to collect only those images that conform to that definition? If we can define that clearly I may just start compiling images.
-
Yeah sounds like you put a lot of good work into the the software and a how-to would be very useful. I should have enough experience to be able to use it. Also knowing how it works in a precise way would also be useful if you have the time. Now also if you documented which scenes it was that you needed to edit manually that'd be great as well that way I already know what to work on. Basically whatever your willing to share I'll take. You have done a lot of good work.
Ok i'll do a howto when i'll have some time (need to do some clean up to make the forware more user friendly) but for the documentation of scene i've edited manualy i haven't done a list ^^'
Now I am also serious about training my own waifu model but does anyone have any clear idea about where we could get that many images. I mean using google's advanced search might not be a bad way to find the images but how exactly could we define the original Final Fantasy 7's style from a purely objective standpoint, in order to collect only those images that conform to that definition? If we can define that clearly I may just start compiling images.
I thought about it to when i started working on waifu but never found a good way to train waifu for FF7 design specificly
-
What would happen if you use the animie trained waifu images with full noise reduction and use these for training waifu?
-
Well theoretically you'd end up with the same or worse image quality. Basically you need something better to learn from. Using the same images but processed once through would just cause a reduction in quality most likely if any difference at all.
-
Now I am also serious about training my own waifu model but does anyone have any clear idea about where we could get that many images. I mean using google's advanced search might not be a bad way to find the images but how exactly could we define the original Final Fantasy 7's style from a purely objective standpoint, in order to collect only those images that conform to that definition? If we can define that clearly I may just start compiling images.
The closest I think you could get is using assets from newer HD games or original artwork of the same artists? What were the latest Square Enix titles that used static backgrounds?
-
Would these not work?
http://www.neogaf.com/forum/showthread.php?t=551612 (http://www.neogaf.com/forum/showthread.php?t=551612)
-
This project all the way... Mainly since I cant seem to get the other Waifu2x pack in it's entirety without dead links. But I'm more interested in accuracy vs stylized. :)
-
I just recently placed an order for the 980 ti. I am just waiting for it to arrive and I will get to work on process the full game with this method.
-
This project all the way... Mainly since I cant seem to get the other Waifu2x pack in it's entirety without dead links. But I'm more interested in accuracy vs stylized. :)
It's in the 7th heaven catalog.
-
You could take the intro movie and splice it into .png using vDub. In theory you could and tens of thousands of images in just a few movies. Some with exact color profiles as the fields.
I know it's rather the opposite of what I just said but the AAcolor shader looks fantastic in conjunction with the Anime setting.
-
You could take the intro movie and splice it into .png using vDub. In theory you could and tens of thousands of images in just a few movies. Some with exact color profiles as the fields.
I know it's rather the opposite of what I just said but the AAcolor shader looks fantastic in conjunction with the Anime setting.
I have definitely thought of this. We would just need a fairly high resolution video copy, similar to the raw video with as little noise or compression artifacts as possible. In order to train an effective model noise free images are crucial. Actually I have been looking for some videos along those lines haven't found anything yet but if anyone has any suggestions I'll give it a shot.
-
Could use the PS3 tech demo I believe that was released in 1080p not sure but definitely HD.
Or even the CC psp ending scene.
*Spoilers*
Its the intro to FFVII.
-
Could use the PS3 tech demo I believe that was released in 1080p not sure but definitely HD.
I have spent the past few years searching on and off, and was unable to find the PS3 tech demo in better than 480p 4:3 with letterboxing. Certainly nothing HD. The best I found was a 60fps version (http://archive.videogamesdaily.com/viewer/vidview_games.asp?games/e32005/kikizo_ps3_ffvii_demo_hd.wmv) buried in an obscure corner of the internet. I can believe that it was produced in HD, but I'm certain it was never actually released in HD.
-
Well IGN posted a 720p version floating around. Here:
https://www.youtube.com/watch?v=IVCYy8C5Av4 (https://www.youtube.com/watch?v=IVCYy8C5Av4)
mentioned before that the Crisis Core ending is the tech demo with a small addition. There are a number of fan converts that should be more than suitable to build a trainer... other then those I doubt very much your going to get anywhere close to actual rendered HD using original backrounds without artifacts I believe... This one is artifact free and should build an pretty accurate trainer
https://www.youtube.com/watch?v=ptdGF-ccIsE (https://www.youtube.com/watch?v=ptdGF-ccIsE)
-
IGN's version is also rife with upscaling and interlacing artifacts. There's a distinct lack of sharp edges anywhere, and mostly-horizontal slopes have obvious combing issues. I would guess by looking at it that this was a 480i video to begin with, and whoever uploaded it to youtube upscaled it (badly) first. Much better results could be achieved from the 60fps version I linked, with proper deinterlacing and a better scaling algorithm. In fact, I might even look into doing that myself; it could be a fun waste of time if I can find the time to do it.
-
I have spent the past few years searching on and off, and was unable to find the PS3 tech demo in better than 480p 4:3 with letterboxing. Certainly nothing HD. The best I found was a 60fps version (http://archive.videogamesdaily.com/viewer/vidview_games.asp?games/e32005/kikizo_ps3_ffvii_demo_hd.wmv) buried in an obscure corner of the internet. I can believe that it was produced in HD, but I'm certain it was never actually released in HD.
I think it was first shown at e3 2005 during the Sony presentation where the PS3s hardware specs were finalized. I can't find anything better either. Really anything on youtube isn't in its original encoding to begin with.
Maybe just concept art: http://finalfantasy.wikia.com/wiki/Final_Fantasy_VII/Concept_Art
I also found the raw Crisis Core cutscenes. The game was originally for the PSP (480 × 272 screen): http://www.silenttweak.net/index.php?file=cc/ccvideos
It seems like the only actual HD content for FFVII was Advent Children which has an official Blu-ray release with extended/updated scenes.
So at best you have 100 minutes of official HD video ([email protected] H264/AVC x264 [10872 kbit/s] was the best encoding I can find), some concept art, and various cutscenes at DVD quality or less from every other source: Original FMVs (320×224@15Hz), Toshiba DVD (640×448@15Hz & 717×456@15Hz), Crisis Core FMVs (480×[email protected]); FFVII Technical Demo (640x480@60Hz)
That's 143,856 HD frames, 23,877 DVD frames that need to be color corrected, 6,180 DVD frames from the Technical Demo, and I'm too lazy to figure out the math on the FMVs which are all antsized.
-
unfortunately AC is a departure stylistically from the original. Although I only need something that is basically devoid of characters and only contains the scenery of FF7. I'm sure some shots from AC would work, I actually have the bluray from years ago lying around somewhere. Funny thing is I don't have any sort of optical drive bay on my computer lol.
-
Funny thing is I don't have any sort of optical drive bay on my computer lol.
Right?
Unfortunately AC is a departure stylistically from the original.
It's really hard to place the steam/cyberpunk aesthetic from any other game also. How do you feel about using assets from later Final Fantasy titles?
-
unfortunately AC is a departure stylistically from the original. Although I only need something that is basically devoid of characters and only contains the scenery of FF7. I'm sure some shots from AC would work, I actually have the bluray from years ago lying around somewhere. Funny thing is I don't have any sort of optical drive bay on my computer lol.
AC and CC you would only need one of the throwback scenes like cloud at the train station even just a few seconds of video is thousands of images. :)
-
Awesome!! I've been testing the sample fields you provided for the bombing mission. Would absolutely play the game again with these if the project is completed :)
-
AC and CC you would only need one of the throwback scenes like cloud at the train station even just a few seconds of video is thousands of images. :)
but finding raw videos in a high resolution is the problem compressed videos have too many artifacts to be useful
-
yarLson,
thanks to DLPB's suggestion, I have discovered this thread of yours and I am wondering whether you are going to go through with a complete upscaling?
In my opinion, it is surely worth the effort. I can but agree with DLPB about this being so far the best result!
-
I think this is an amazing idea.
I was actually thinking of doing something similar myself. But one step further.
Using the Waifu2x upscale as a basis, and then going over each image by hand (In Photoshop or another program with a tablet) to refine it to try and create the higher res backgrounds this game deserves.
Square Enix couldn't be bothered with the PS4 port (Which I could've been happy with, if they did. Instead of being stuck with the same old blurry shit with a bilinear upscale and the same translation) and they clearly don't care about those of us who wanted FFVII:R to be the same game.
-
Well, Bonkers, if you really want to give it a try, I will love to see the result!