Qhimm.com Forums

Project forums => Team Avalanche => Topic started by: Mayo Master on 2013-05-13 04:54:07

Title: [HD Remake] Midgar sewers
Post by: Mayo Master on 2013-05-13 04:54:07
Hi folks,

I've started to work on one scene of the Midgar sewers (colne_be3). One of the big changes for me when it comes to make that scene is that I've decided to switch to Blender's "new" rendering engine called Cycles, after the recommendations from anaho. Cycles has many cool features, most of all to make realistic rendering of the light interactions with objects (indirect lighting and such), which would also make my stuff more consistent with the works made by people who use 3ds, for instance. The downside of it (for the moment) is that materials and texturing are quite different from doing it with Blender internal rendering, so I have to relearn how to texture. For that matter, apologies to anaho about some past mis-communication when we were talking about texturing, as I was unaware about how you do it in Cycles back then.
Also, when you have a crappy computer, it's difficult to make tests with "complicated" lighting because it takes hours to make a "somewhat noise-free" image. Anyways, this is what I have at the moment:
(http://img10.imageshack.us/img10/8162/wipsewers2.png) (http://imageshack.us/photo/my-images/10/wipsewers2.png/)
This render has been resized to 50% before I uploaded the picture. Rendering this took about 5:30 h (which gives you an idea of how crappy the computer is), I had only 100 samples. As far as texturing goes, I'm pretty happy with what I've done so far with the bricks, the red tiles and the concrete parts (though I don't understand why I have this small patch of very clear and shiny tiles- I'll have to correct that). The metal ducts may need some revisions, while the concrete sewer pipes have been textured very hastily (I'll add some muck, moss and stuff later).
So... that's it for the moment. And as a side note, I hope things to improve a lot in the next few weeks because I've just ordered my new desktop (i7 core, 16 Gb ram)  MWAHAHA :evil:
Title: Re: [HD Remake] Midgar sewers
Post by: Ulvenoko on 2013-05-14 16:06:07
Not bad at all. :)
Title: Re: [HD Remake] Midgar sewers
Post by: anaho on 2013-05-14 21:11:01
Like I said much nicer lights and shadows plus better looking bumps. Looks so much better, with so much less effort, does it not ?

Two things:

1.) Use the MingW64-build for twice the render speed ( only in CPU-mode) if you have a 64-bit Windows: http://builder.blender.org/download/

2.) I seriously hope that regardless of your processor choice you settled for an nVidia card in your new rig. Not being a fanboy here, but most GPU-renderers only run on nVidia-hardware (for good reason), and Cycles is no exception here. Get at least something like a GTX670, 4GB version recommended. If you can find a used GTX580 3GB locally for about 200€, that is in working condition, get it instead. It is a steal for the money if you are doing GPGPU-tasks.
Title: Re: [HD Remake] Midgar sewers
Post by: Rundas on 2013-05-14 21:52:54
Titan, Titan, Titan. IF your computer can handle it.
Title: Re: [HD Remake] Midgar sewers
Post by: Mayo Master on 2013-05-14 23:40:02
Like I said much nicer lights and shadows plus better looking bumps. Looks so much better, with so much less effort, does it not ?

Two things:

1.) Use the MingW64-build for twice the render speed ( only in CPU-mode) if you have a 64-bit Windows: http://builder.blender.org/download/

2.) I seriously hope that regardless of your processor choice you settled for an nVidia card in your new rig. Not being a fanboy here, but most GPU-renderers only run on nVidia-hardware (for good reason), and Cycles is no exception here. Get at least something like a GTX670, 4GB version recommended. If you can find a used GTX580 3GB locally for about 200€, that is in working condition, get it instead. It is a steal for the money if you are doing GPGPU-tasks.
Yeah, Cycles looks much better when it comes from lighting and shadows. Soft shadows come out realistically without having to tune parameters, and indirect lighting is so much of an improvement compared to (badly) faking it with ambient lighting. Bumpmaps are also better. Texturing seems a bit difficult for me at this point, but I guess it's because I'm still in the learning curve. Mostly, with blender internal I used to do a lot of texturing with procedurals and preset projections (cubic, tube...) which are not yet supported in Cycles. Thus I've already started to run into the same issues you were having about low-res textures.
Then, for my new rig I'll have a basic nvidia graphic card. I didn't want to get a machine for gaming (I play on PS), so I'll probably settle for CPU rendering (thanks for the tip about that blender build) but the graphic card should be enough to handle the "real-time render preview" thing, which should make the work much easier. I knew I had to settle for nvidia after having read so much about compatibility issues of blender with other graphic cards, plus nvidia was the default choice I had on that computer build. Anyway, I'm looking forward to playing with this new toy.
Title: Re: [HD Remake] Midgar sewers
Post by: Sapphire on 2013-05-15 03:58:33
Sweet stuff dude, I wish I had the time to work on these scenes again :(.
Title: Re: [HD Remake] Midgar sewers
Post by: Mayo Master on 2013-05-15 15:05:32
Hey Sapphire,

Do you have by any chance any remnant of your Midgar model? As I pointed out in the Midgar remake thread, I thought of using renders of your Midgar models along a panoramic view in order to build HDR images for lighting scenes we make in Midgar. That'd be awesome. Would that be possible or is your Midgar model gone to dust?
Title: Re: [HD Remake] Midgar sewers
Post by: Mayo Master on 2013-06-15 04:00:02
Hi folks,

Just making a progress update... I think that I will be sticking to Cycles from now on, it's so much better! Especially since now I have a computer to handle it  :P
(http://imageshack.us/a/img515/5952/5gl.png)
Though at this point, I think there are only the trash bags and the boot that I would consider completely done. I had lots of fun making the trash bags - a real cloth simulation feast! I revised the concrete and the orange tiles textures, but they still lack all the dirt and grime. I made some quick textures on the pipes, but I'll have to revise these. I'm rather ahppy with the bricks, but it isn't necessarily a final version (dirtmap notwithstanding). So, still a lot of work to be done: more trash objects, more texturing, and the lighting to be revised (this image is made with a huge ceiling light, which won't be in the final version - it reduces render times and allow for a better view on the objects). Oh, and I intend to top this with a fluid simulation, though I'll leave that for last  :P
Title: Re: [HD Remake] Midgar sewers
Post by: Cyberman on 2013-06-15 12:23:25
That looks fairly realistic .. grime hmmm it would be nice to create a filth simulation to layer dirt and grime over time :D

Is it possible to fix objects (merge into a new object I guess) once placed in a scene then use a 'spray' approach to add the grime?
I remember someone using the output of a kinect and painting the texture data onto objects in his scene (something similar)..

Bricks did come out pretty well, unfortunately those bags look a bit too 'fresh' that doesn't mean they don't look realistic. Anyhow nice job did you use radiosity to generate the ambient light (or is that what you were refering too in 'cycles'?).

Cyb
Title: Re: [HD Remake] Midgar sewers
Post by: Mayo Master on 2013-06-15 13:20:22
I wish there could be a "dirt & grime simulator"! Making a scene dirty and aged is tedious and a bit difficult. In Blender, you could do some "texture painting" where you directly paint on your object in your 3d viewport, that's a bit similar to what you've mentioned. I've used that quite a bit in the item shop scene (mds5_i), it's particularly useful when you want to add grime on a precise location on your surface (something you can hardly do with procedurals). Another very good technique is to generate a dirt map based on ambient occlusion textures, it's a bit tedious because you need to bake the ambient occlusion of every single object under Blender internal rendering engine.
By the way, in my opinion, it's not really the bags that are too fresh (they're satisfyingly realistic in my view), but rather the concrete underneath that is too clean. The brick texture is actually a bit deformed and low-res, but you need to zoom (beyond the x 5 target resolution) to notice that.
About the lighting: "Cycles" is Blender's alternate (newer and better) rendering engine, based on raytracing, which accounts for all the indirect light coming from bounces on the objects (anaho has been using it for a while - it's fairly close to renderers from other 3d modelling software). Thus no need to have "ambient lighting" as with the Blender internal. You can also (as I did here) create walls which have the light bounce on them, yet make them invisible to the camera. Pretty nifty stuff!
Title: Re: [HD Remake] Midgar sewers
Post by: Template on 2013-06-16 00:51:00
That looks simply amazing... all of this stuff does. Like, trying to touch the screen nice. I really just can't help but wonder who put the trash bags there....
Title: Re: [HD Remake] Midgar sewers
Post by: alloy on 2013-06-16 22:09:56
Grime simulator would be nice xD, try looking for dirt, grime, weathering shader's maybe there are some out there for blender.

Who knows i knows max has plenty. For example (http://www.enrichpro.com/en/richdirt/freeversion.html)
Title: Re: [HD Remake] Midgar sewers
Post by: Micky on 2013-06-18 07:54:02
I wish there could be a "dirt & grime simulator"!
Did you try this? http://quixel.se/ddo/ I downloaded a demo ages ago but never got around to play with it.
Title: Re: [HD Remake] Midgar sewers
Post by: Cyberman on 2013-06-19 01:49:53
Unfortunately any simulation has it's caveats for example dirt and grime would require fluid flow analysis (Yes exciting but think about it dust flows in the atmosphere and muck flows in other fluids like water, water can handle larger amounts of dirt that's all). Each fluid changes how the material deposites out. For example alluvial fans in a river are muck from upstream the same goes with the "fine filth" deposited by water tricking and surging through sewers. This is a water runoff sewer obviously.

Dirt color(s) is dependant on source and volume. Not sure how one would simulate that satisfactorally, it's a non trivial thing.
You have fluid flow (CFD) and you have numerous boundary conditions in the fluid flow. Not only that the fluid is changing (IE as the liquid slows more material is deposited out and it varies depending on the material density thus many color variations) you need a source for the material (pigmentation I suppose) but that will vary depending on flow rate etc. Oddly some material tends to flow on the surface of the water than IN the water and other visa versa (this also leads to stratification in fluids).

That's just an off the top of my head thing too. :D

Their is some on the subject per sea on wikipedia :D


cyb
Title: Re: [HD Remake] Midgar sewers
Post by: Mayo Master on 2013-06-30 20:50:59
Hi guys,

Just a small progress update on this scene, as I'm looking for some advice about texturing with Cycles...
So, below is where I am at. I still need to add a bunch of trash objects (papers, bags, empty containers, debris, small rocks) and add grime and dirt on the bricks and the concrete.
(http://imageshack.us/a/img441/7720/doy.png)

Now, I have the feeling something is off about how the walls and floor interact with the light. I am not sure what, though. 2 things I'm unsure off:
- I have set the green bricks and the concrete with purely diffuse shaders. Should I integrate some kind of glossy shaders as well? I thought diffuse-only made more sense, physically, but the contrast with how the orange tiles interact with the light is odd. Or perhaps should I lower the glossy factor of these orange tiles?
- The light feels too even. I only have 4 lamps based on meshed objects (2 of them being visible), but it gives a lighting very similar to an ambient light setting (while I have no such setting). Should I set the lighting objects with angle-dependent lighting? I thought of using something like this http://blenderartists.org/forum/showthread.php?264052-IES-Like-Lights-in-Cycles-Architectural-Lighting (http://blenderartists.org/forum/showthread.php?264052-IES-Like-Lights-in-Cycles-Architectural-Lighting) but I'm not sure.

I'll appreciate some tips on the subject.
Title: Re: [HD Remake] Midgar sewers
Post by: Cyberman on 2013-06-30 21:33:03
Hi guys,

Just a small progress update on this scene, as I'm looking for some advice about texturing with Cycles...
So, below is where I am at. I still need to add a bunch of trash objects (papers, bags, empty containers, debris, small rocks) and add grime and dirt on the bricks and the concrete.


Now, I have the feeling something is off about how the walls and floor interact with the light. I am not sure what, though. 2 things I'm unsure off:
- I have set the green bricks and the concrete with purely diffuse shaders. Should I integrate some kind of glossy shaders as well? I thought diffuse-only made more sense, physically, but the contrast with how the orange tiles interact with the light is odd. Or perhaps should I lower the glossy factor of these orange tiles?
- The light feels too even. I only have 4 lamps based on meshed objects (2 of them being visible), but it gives a lighting very similar to an ambient light setting (while I have no such setting). Should I set the lighting objects with angle-dependent lighting? I thought of using something like this http://blenderartists.org/forum/showthread.php?264052-IES-Like-Lights-in-Cycles-Architectural-Lighting (http://blenderartists.org/forum/showthread.php?264052-IES-Like-Lights-in-Cycles-Architectural-Lighting) but I'm not sure.

I'll appreciate some tips on the subject.
Well my memory serves hopefully.

Diffuse lighting was a 'short cut' invented by early render designers to simulate radiosity (because they hadn't really thought of it probably). This sets a fixed illuminance of a surface.  However illuminated surfaces radiate light, and do so in an uneven manner. In the first posting it appeared you had saturated both diffuse and radiant lighting. I suggest reducing the diffuse to just visible (IE render it with diffuse first and make the scene JUST visible) then add in the radiant lighting and direct lighting and see how it looks afterward.

I'll spare you a long discussion of lighting models ... hehehe

Cyb
Title: Re: [HD Remake] Midgar sewers
Post by: Mayo Master on 2013-07-01 17:33:04
The lighting difference between the last image and the previous one is that, in the previous (which looks more illuminated), there's an additional big area light near the ceiling (which I've put only for testing purposes). This area light allows me to have a good idea of how the textures look like with very few light samples (i.e. 10 samples are enough). Without it, I would need about 600 samples to get something relatively noise-free (the last image was made with 1500 samples).
Then, what you call "radiant" is physically related to specular reflections. Diffuse reflections are rather even in space, while specular reflections are direction-dependent (they're mirror-like). In Blender's Cycles, specular reflections are mostly driven by setting "Glossy" shaders in the materials properties (there's even an Anisotropy shader for simulating reflections on brushed metal). This Glossy shader has additional settings accounting for the surface roughness. Physically speaking, a very rough surface only yields diffuse reflections, a somewhat rough surface has a bit of glossy reflections, and the more polished a surface, the more it tends to behave like a mirror. I'll try to play around with the inclusion of a glossy shader for the concrete and the bricks and see what happens. However, from "real-life" observations, I don't think I've seen specular reflections from concrete or bricks (at best they would be reflections from gravels showing at the surface, which doesn't happen in most concrete).
   
Title: Re: [HD Remake] Midgar sewers
Post by: Mayo Master on 2013-07-31 04:48:59
Apparently it's become fashion to come out of hibernation, so here's my turn. As a matter of fact, I ditched my side-job, which gives me a bit more free time, meaning I can get back on track with modelling scenes with Blender.
So, here's the update, I've done quite a bit of work on texturing. I had to rework all my UVs to be able to bake ambient occlusion textures and add all the other grime textures.
(http://imageshack.us/a/img843/4343/cniw.png)
This image is done with a sort of floodlight, it's not with the final lighting (the final lighting takes ages to render because I need thousands of samples). What remains to be done is:
- Make a lot of trash objects, and litter the scene with them using particle systems
- Make a fluid simulation
Until then, I hope you enjoy what I've come up with so far.
Title: Re: [HD Remake] Midgar sewers
Post by: Mayo Master on 2013-08-11 23:10:51
Hi folks,

Here's an update on my works on colne_be3, before I'm off for 2 weeks of vacations.
Since last time, progress can be summarized by:
- Addition of populations of fine gravel (particle systems). However most of this detail is barely pixel-sized.
- Texturing of the posters
- Addition of a few ladder bars to make the whole scene a bit more sensible (it matches the walkmesh besides).
- Addition of many trash objects: 3 plastic bags, 1 large folded tarp, 3 soda bottles, one empty gallon of milk (not clearly visible because that's where my render is noisy.
- Fluid simulation: I had some trouble with it initially, because the scene needed to be scaled down for the fluid sim to work, despite the scale of the fluid sim is not supposed to be dependent on the scene scale (normally, a parameter within the Fluid Domain properties should take care of that). Thus my notice on SpooX's stickied thread. I made a long bake (750 frames) that took about 11 hours, so that it'd correspond to 30 seconds of flow.
- I tried to do some texturing, with the idea of modelling a muddy flow with a bit of foam on the surface. I fiddled with this for a couple of hours, I believe it can be improved. If you have ideas about how to improve it, be my guest. At the moment: the mud-flow texture is basically an arrangement of a diffuse shader with a glass shader (with IOR set to water properties). The Diffuse/Glass overlay is controlled by a noise-like (cloud-like) texture and color ramps, so that white areas would show white froth and almost purely diffuse shading, and the dark areas would show glass (water) in majority with a tint of brown mud (via diffuse shader) in approx. 75/25 glass-to-diffuse ratio. 

(http://imageshack.us/a/img203/903/o6jl.png)

Some comments about this render (obviously not final):
- Rendered with a "fill light" with only 50 samples. For final renders, I'll use the yellowish small lamps (missing on this render) but it will take ungodly amounts of samples to make it noise-free (I'm guessing about 5000, at least).
- I will have to re-run my fluid simulation. As it was set up, I originally placed a volume of liquid to make a "pre-filling" of the canal, but this "pre-filling wasn't very useful and built up too much water in the bottom of the canal (as you can see, for one thing, one of the steps of the stairs is submerged, and some debris/trash is submerged as well while I would like them to poke through the water surface).
- I will have to devise of a trick regarding my foam on the mud flow surface. As it is now, the noise-like (cloud-like) texture which controls the presence of the foam is based on overall object generated coordinates. Given this setting, this texture will be static. However, eventually I will want to make several renders to animate the flow (thanks to Palmer, I know the original version had 8 frames to animate the flow). Therefore, I would very much prefer to have a noise texture whose input is based on the object geometry (normals?) which would evolve with each frame. I have pretty much no idea about how to do that (never used the "Geometry" nodes), so again suggestions are welcome. If you wish to tinker with this texture, you can find the node setup here (texture images referenced at cgtextures.com (http://cgtextures.com)): http://imageshack.us/photo/my-images/96/95fv.png/ (http://imageshack.us/photo/my-images/96/95fv.png/).

So, that's it for now, I hope you like it.

[EDIT]: Just before I go (my flight's in a couple of hours), I think I solved the issue about generating foam depending on the geometry. Basically, I placed a Geometry node and connected the "Normal" value to the input of the noise-like (cloud-like) texture. I made a small animated gif that shows how it looks, and I'm quite happy with it. I hope you'll enjoy.

(http://imageshack.us/a/img7/4879/mrbf.gif)

Title: Re: [HD Remake] Midgar sewers
Post by: KnifeTheSky77 on 2013-08-12 08:46:18
To my layperson eyes, absolutely brilliant. You are getting really good at this
Title: Re: [HD Remake] Midgar sewers
Post by: Captain Epic on 2013-08-12 11:00:02
Wow, this looks incredible! :o Keep up the great work.
Title: Re: [HD Remake] Midgar sewers
Post by: LeonhartGR on 2013-08-13 02:01:00
So close to how I imagined what it should look like!!!  :mrgreen:
I wish more scenes were so detailed!
Title: Re: [HD Remake] Midgar sewers
Post by: BloodShot on 2013-08-13 07:34:36
Absolutely gorgeous. I can't wait to see it ingame.
Title: Re: [HD Remake] Midgar sewers
Post by: Killerx20 on 2013-08-26 13:35:03
Thats really awesome looking, high quality, highly detailed stuff you have there. And the renderer's lighting is superb. I only wish I could render Midgar to look that good.
Title: Re: [HD Remake] Midgar sewers
Post by: Mayo Master on 2013-08-30 03:56:37
Hey folks,

Thank you very much for your comments, I'm glad you like that scene. I've yet a couple of things to tweak before I can try to get it in the game.
By the way, I'm having a weird problem, I hope someone can help me with that: I wanted to play the game so that I would get a save file close to that sewer scene. However, for some reason the wall market quest doesn't start and the game is just stuck there. More specifically: when first going to Don Corneo's mansion, and triggering the dialogue with the bouncer, Aerith doesn't appear and there's never the starting dialogue about cross-dressing. Is there any way to solve that problem? I tried to restart the game from earlier save files but the quest repeatedly fails to initiate.
Thanks.
For reference: I'm with the 2012 PC version, converted to old using Kranmers' game converter v0.9, and downloaded Aali's driver v0.7.11b.

Title: Re: [HD Remake] Midgar sewers
Post by: Kaldarasha on 2013-08-30 07:27:48
You need to talk to the old fat guy in front of the honeybee to start the quest.
Title: Re: [HD Remake] Midgar sewers
Post by: Mayo Master on 2013-08-30 15:56:00
You need to talk to the old fat guy in front of the honeybee to start the quest.
Really?  :-[   I simply forgot to do it, how did I miss that? Shame on me for blundering that badly, that was quite the derp moment.
And thank you for the tip, which was unfortunately needed.
Title: Re: [HD Remake] Midgar sewers
Post by: cmh175 on 2013-08-30 17:46:15
Here's the game save compilation. Has more or less every single event bookmarked, it's great for testing stuff like this. You'll need to use the readme to figure out which save and file to use though, the saves are in Spanish I think. 

http://forums.qhimm.com/index.php?topic=8880.msg115218#msg115218 (http://forums.qhimm.com/index.php?topic=8880.msg115218#msg115218)
Title: Re: [HD Remake] Midgar sewers
Post by: Mayo Master on 2013-09-04 02:31:41
Hi folks,

After quite a bit of tinkering, I got that field layered, animated, reloaded into Palmer. I uploaded it in the WIP repo, given that they were made with temporary lighting setup (one that gives me a decent picture after a few dozens of samples). The final version with real lighting will take ages to render, so it will be delivered in approximately 2 weeks (because I have to render each frame of the sewage flow). In the meantime, you're more than welcome to give it a try. It comes in 3 .png files that can be found under /Field Scenes/colne_b3/.
For people who don't have access to the repo, I've uploaded a 50% size version open to everyone, if there are other people willing to try it out.
http://imageshack.us/a/img818/5070/vqum.png (http://imageshack.us/a/img818/5070/vqum.png) has to be renamed colne_b3_00_00.
http://imageshack.us/a/img401/6245/9gf5.png (http://imageshack.us/a/img401/6245/9gf5.png) has to be renamed colne_b3_01_00.
http://imageshack.us/a/img843/9288/8cbh.png (http://imageshack.us/a/img843/9288/8cbh.png) has to be renamed colne_b3_02_00.
These images simply need to be placed in /mods/<modpath>/field/colne_b3/ within your Final Fantasy VII directory.

Some side comments about getting that field to work: for that scene, it was not really straightforward. The big problem here is the absence of control on the overlaying process. Each layer is cut in a pre-established manner, so if there is something you want to overlay but falls outside the boundaries of how the layers are cut, then it cannot be done (unless somebody has a solution for it?). For example, if you compare the image below (50% render size) with what I previously rendered (see page 1 of this thread), I eventually had to mask the outflow of the sewage (bottom right corner of the image) because otherwise this section gets badly cut out. I also wanted to place a wooden plank across the bottom of the canal to "show" the end of the walkmesh (which I did eventually), however my first attempt were about placing that plank slightly more upstream and place it in a layer lying "above" the character. However, the "upper layers" were automatically cut out in such a way that the plank wasn't displayed. So I finally had to place the plank so that it wouldn't look like Cloud would step over it. I also had problems with the layering of the stairs, and it also turned out that the last two steps to the bottom could not be placed in an appropriate layer. I simply left those steps only in the bottom layers. I guess the way it's set up now is pretty much the best that could be done.

(http://imageshack.us/a/img856/9278/od3v.png)

Another note I'd have is about the animation. As it is, there are 8 frames to play the animation of the flowing sewage. It takes about 1.2 seconds for the animation to loop. In my Blender fluid sim, I have 25 fps. Consequently, I chose to build the 8 frames animation by picking 1 Blender frame out of 4. It may be a bit "choppy" (as was the original game), but I think it "loops" better when returning to the first frame of the animation. Something I was wondering: would there be a way to expand the number of frames for such animation? It may be way outside what Palmer can do, but would it be possible at all? At this point, that last question is rather a matter of curiosity. In the same trend, I wonder if there's a way to modify how the overlays are cut out (with respect to my problems to arrange all the layers).

Anyway, I'd be happy if some people could try this WIP and give me some feedback  :)
Title: Re: [HD Remake] Midgar sewers
Post by: anaho on 2013-09-04 15:11:32
1.) Looks good. Ingame layers work and animation is good. I saw no glitches whatsoever.

Edit: After seeing it ingame it is probably the easiest way to just rerender it at higher settings and leave the work to the computer.

(2.) Obsoloete : Why not render the water flow seperately in Internal and just comp it together? I dont think it is worth rendering a real animation of that anyway given the choppy nature of embedded frames in game. I mean I am all for quality but what you try to do seems just not worth the time and energy ( and to think you would have to do this again in case you have to alter something ....))
3.) I dont know how long this took you to render but the "noise-free" renders from Mideel took me 15 minutes and 45minutes in 4x4 resolution on my computer for a single frame with no layers. Doing them on a 2600k would probably have taken 8 times as long.  But they were not really optimised for speed anyway.
4.) IF you decide to do the water in internal, you no seem to no longer have any transparent object in the scene. You can then set transmission and transparency bounces to 0 to begin with. And set the min and max bounes to the same value like 2 and see what happens.
5.) How is this scene lit? I guess it is just two lights. But did you stick them inside the lamp objects? If so, that is likely to be a sure contributor to noise.
Title: Re: [HD Remake] Midgar sewers
Post by: Mayo Master on 2013-09-04 16:30:57
1.) Looks good. Ingame layers work and animation is good. I saw no glitches whatsoever.

Edit: After seeing it ingame it is probably the easiest way to just rerender it at higher settings and leave the work to the computer.

(2.) Obsoloete : Why not render the water flow seperately in Internal and just comp it together? I dont think it is worth rendering a real animation of that anyway given the choppy nature of embedded frames in game. I mean I am all for quality but what you try to do seems just not worth the time and energy ( and to think you would have to do this again in case you have to alter something ....))
3.) I dont know how long this took you to render but the "noise-free" renders from Mideel took me 15 minutes and 45minutes in 4x4 resolution on my computer for a single frame with no layers. Doing them on a 2600k would probably have taken 8 times as long.  But they were not really optimised for speed anyway.
4.) IF you decide to do the water in internal, you no seem to no longer have any transparent object in the scene. You can then set transmission and transparency bounces to 0 to begin with. And set the min and max bounes to the same value like 2 and see what happens.
5.) How is this scene lit? I guess it is just two lights. But did you stick them inside the lamp objects? If so, that is likely to be a sure contributor to noise.

1. When I was working on getting the layers right with Palmer and testing them in game, I was using 5x5 resolution renders (When you tried these files, did you try those from the repo or those open to everyone?). It still needed quite a bit of tayloring with gimp to cut them out properly and have them not glitch, because of the cuts you have no control over.
2. I mean to render the flow with Cycles. I am pretty happy with how I managed to set up the foamy mud flow material, I don't think I can replicate that with Blender's internal, and I don't feel like discarding that.
3. How many samples did you need for a noise-free image on 4x4 resolution?
4 and 5. I tried to launch a render last night using "real lights" but the picture was remaining fairly noisy after 5000 samples. As you guessed, the light sources where stuck inside the lamp objects, which themselves had "glass" surfaces to let the light out. And as you guess, this setup cannot work with a reasonable amount of samples. I switched back to a setup where the light source is the "glass surfaces" of the lamp objects, and it reduced noise drastically (I made tests where the picture was already decently sharp only after 10 samples), so I think I should be good with that later setup. I have a bit of tweaking to do but hopefully I should start the proper renders tonight.
Overall I won't mind if the renders take 10-20 hours each or so, I'll just let the computer to work overnight.
Title: Re: [HD Remake] Midgar sewers
Post by: anaho on 2013-09-04 17:35:28
1.) I was testing the 640 version posted above. Granted, the resolution is not the best to judge and I play the game from 2 meters away from my CRT monitor (not TV), so maybe I should check again the high res version.
2.) I can understand that as it looks great the way it is.
3.) You must know this is dependant on your scene and especially your lighting setup and cannot be generalised. Materials also come into play but I am 99% sure that they wont be a problem in your scene as everything seems more or less noisy to same amount of degree. But Itmin 2 resulted took 750 samples to converge. Itmin 1 lighting was worse and it took 1000 samples and I had to render at twice the res to get rid of persisting noise. This helps if you have bad lighting it it never converges. Render twice res and scale down. But to give a number, most people would argue that even 1000 is overkill and should already be very smooth. 5000 is just killing it - especially for animation.
4.) How many samples did the version posted above version have ? If it is in double digits range then I guess you have optimised it reasonably well.
Title: Re: [HD Remake] Midgar sewers
Post by: Aali on 2013-09-04 17:52:52
It's certainly possible to change up all of the tiling, animations and what-have-you and it would even be possible to create an automatic tool that spits out a suitable set of tiles from a render, its depth buffer and a walkmesh. It is a huge undertaking though and in the end all you get is a tool that modifies a horribly outdated format designed for the PSX and shoe-horned into 90's PCs. It's probably easier to change how the game renders field scenes.
Title: Re: [HD Remake] Midgar sewers
Post by: anaho on 2013-09-04 19:10:36
I would remove that red can in the lower left because it totally looks like a red materia if you dont zoom in.
But the layers work perfectly even in the hires version.
Title: Re: [HD Remake] Midgar sewers
Post by: Mayo Master on 2013-09-04 22:43:00
3.) You must know this is dependant on your scene and especially your lighting setup and cannot be generalised. Materials also come into play but I am 99% sure that they wont be a problem in your scene as everything seems more or less noisy to same amount of degree. But Itmin 2 resulted took 750 samples to converge. Itmin 1 lighting was worse and it took 1000 samples and I had to render at twice the res to get rid of persisting noise. This helps if you have bad lighting it it never converges. Render twice res and scale down. But to give a number, most people would argue that even 1000 is overkill and should already be very smooth. 5000 is just killing it - especially for animation.
4.) How many samples did the version posted above version have ? If it is in double digits range then I guess you have optimised it reasonably well.
In the above version, the big "fill" light makes it very easy to have a noise-free image. It took only 50 samples, so I suppose it's doing pretty good. With dimmer light (as in the final version) I'll really need to ramp up the number of samples.
It's certainly possible to change up all of the tiling, animations and what-have-you and it would even be possible to create an automatic tool that spits out a suitable set of tiles from a render, its depth buffer and a walkmesh. It is a huge undertaking though and in the end all you get is a tool that modifies a horribly outdated format designed for the PSX and shoe-horned into 90's PCs. It's probably easier to change how the game renders field scenes.
Thanks for the info. I'm guessing it would be too much pain to go through such an endeavor given the benefits we would get from it. Anyway, it will take us so long to make all the scenes that maybe somebody will show up with an alternate game engine that could handle these fields as 3d environments :P
I would remove that red can in the lower left because it totally looks like a red materia if you dont zoom in.
But the layers work perfectly even in the hires version.
I didn't think that could be a bother. I'll keep the can, just change the color of its side.

By the way, for the sense of scale of the items in a scene, I recommend the use of Kaldarasha's unshaded models. It's of a greater help than chibis in my opinion.
Title: Re: [HD Remake] Midgar sewers
Post by: Mayo Master on 2013-09-12 06:26:16
While the "real" renders of colne_b3 are being made, I started to work on the other sewer scene. It was a bit difficult to get the camera right (it's one of the few exceptions where SpooX's script doesn't work that well) but I eventually got it (did a bit of match photo with Google Sketchup). I only just started the modelling. 
(http://imageshack.us/a/img6/2069/7ekb.png)
Funny, it's kind of "Back to where it all began" since it's the same scene I tried to model with Sketchup one year ago, as my earliest attempt to contribute.
By the way, I intend to reuse the texture I made for the previous scene. Not only it will make both environments consistent (which is actually not the case of the original images), but that should also speed up my work.
Title: Re: [HD Remake] Midgar sewers
Post by: Mayo Master on 2013-09-17 14:51:21
(http://www.frogforum.net/attachments/tree-frogs/24502d1329586823-got-new-red-eye-intro-tips-farnsworth-good-news.jpg)
All renders for colne_b3 are completed, and layered  :D
I'll import them in Palmer and then upload the moded scene to the repo tonight. This one is well and truly finished (until we decide to aim at 4K resolution  :P ). In the meantime, here's the base layer as a "sample":
(http://imageshack.us/a/img689/9362/6z4f.png)

I'm also making good progress on the other sewer scene (colne_b1), I hope to post an update soon.

[EDIT - UPDATE]: I've uploaded the Full Res (5x5) field images to the Repo of the final versions, under Fields/colne_b3. The images colne_b3_00_00, colne_b3_01_00 and colne_b3_02_00 are to be placed in <modpath>/field/colne_b3/.
I have also made a 50% size (2.5 x 2.5) open to everyone. You'll find the images on the links below (again to be placed in <modpath>/field/colne_b3/):
http://imageshack.us/a/img94/1145/w34r.png (http://imageshack.us/a/img94/1145/w34r.png) has to be renamed colne_b3_00_00.
http://imageshack.us/a/img89/2922/suma.png (http://imageshack.us/a/img89/2922/suma.png) has to be renamed colne_b3_01_00.
http://imageshack.us/a/img38/3136/5j84.png (http://imageshack.us/a/img38/3136/5j84.png) has to be renamed colne_b3_02_00.

Please have a try, and enjoy!
Title: Re: [HD Remake] Midgar sewers
Post by: sl1982 on 2013-09-19 18:13:06
Ooh 4k. Anyone have a 4k monitor yet?

Sent from my SGH-I747M using Tapatalk 4

Title: Re: [HD Remake] Midgar sewers
Post by: Mayo Master on 2013-09-21 16:39:05
Here's a small progress update on colne_b1. In fact, it can show you that the sewer scenes will be 4K ready. By the time we can reach completion of this project, probably 4K or 8K will be the norm :P

(http://imageshack.us/a/img62/5995/4n7c.png)

(The target on the image is just Blender's cursor).
Credits to my girlfriend for the image on the poster, especially since drawing the guy made her shiver with disgust  :P
Title: Re: [HD Remake] Midgar sewers
Post by: LeonhartGR on 2013-09-21 21:38:10
Well done for the GF! I love handmade drawing!
Title: Re: [HD Remake] Midgar sewers
Post by: dkma841 on 2013-09-22 01:31:26
4k, 8k? The file size will be immense :o 1080p is good enough imo
Title: Re: [HD Remake] Midgar sewers
Post by: Cyberman on 2013-09-22 14:58:19
4k, 8k? The file size will be immense :o 1080p is good enough imo
I think you might be a bit confused. The rendering image resolution is different than the background bitmap resolution. None of the backgrounds are 320x216 for example but instead might be 512x272. The background consists of tile sets and the background is generated by using these tiles to make up the background layers (on the Ps1 it was done in real time as  the screen panned) The actual screen was 1 block (16pixels) wider both vertically and horizontally to allow the tiles to be added on the edges I believe (I could be wrong however).

Anyhow the actual resolution they are talking about is the background resolution not the screen resolution. Of course that affects how 'good' it looks on the screen in the end.

Anyhow I could be wrong but I think that is what they are talking about.

Cyb
Title: Re: [HD Remake] Midgar sewers
Post by: dkma841 on 2013-09-22 18:06:39
I see... i think :o ;D
Well if it doesn't affect file size then great :)
Title: Re: [HD Remake] Midgar sewers
Post by: Mayo Master on 2013-09-23 05:53:34
First I would like to clear things up, regarding resolution and image sizes...
Given that we want to play in HD, we do need to deliver images that are (theoretically) 4.5 x 4.5 larger than the originals. To round the figures, we aim at 5 x 5 resolution (some argue that 4 x 4 may be enough at this point, though). As Cyb mention, many images exceed the original 320 x 216 size. Mostly, these correspond to fields where moving characters scrolls through a scene (the background image needs to be larger than the screen). The "field textures" which result from the Palmer process, those which are to be used as mod files, are squares which are 5 x 5 (or 4 x 4) times 216 x 216. If a background is particularly large (or if there are animations), one field is eventually made with a combination of several square field textures.
I was wondering that, by the time we may be remotely near the completion of this project, 4K displays may actually be the norm. In that case, it would still be possible for us to generate 4K compatible backgrounds, provided our current scenes offer enough detail. The only thing to change on our end is the render size (9x9). Hopefully computational power will increase to an extent that would allow us to make these renders in a decent amount of time. Inevitably, if you want to play at 4K resolution, the size of all field textures will have to be 1944 x 1944 (versus 1280 x 1280 that I currently deliver).   
By the way, as an anecdote, I am using very high resolution textures to make the colne_b1 scene. One of them is about 14000x2000.

Anyway, I also wanted to make a progress update on colne_b1, I think I manage some decent progress fairly quickly. I hope you'll enjoy.

(http://imageshack.us/a/img849/1285/b1yh.png) 
Title: Re: [HD Remake] Midgar sewers
Post by: Cyberman on 2013-09-23 21:07:40
About 4K displays, it is extremely unlikely they will be the norm.

I had a long diatribe typed up regarding this but let me clue you in 4K is a lie.
What I mean is it is just yet another thing you don't need. In fact you can't even use 4K 2560 is actually at the resolution limits of the physical eye. You might see a LITTLE bit more on a 4K monitor IF you are younger than 20 years old and you are 1 out of 10 people with REALLY GOOD eyesight.  If you are starting to understand what I am saying, you will also have to realizes that 4K also requires a detailed analyisis and video editing to make the video actually visually discernable at that resolution (meaning they have to play with spectral and lumanance output very carefully to force your eyes to actually see ANY DIFFERENCE).

That's the truth, others will tell you something else however it is only because they want to SELL SOMETHING. Much like the forced move in the US to HD had nothing to do with actually seeing anything better (really the broadcast bands were being adjusted to sell bandwidth of existing TV stations for CELL phone usage MONEY in other words was the motive). Same goes with 4K it's purely someone wanting to make more money off of ... you. :D

If you are truely curious hit wikipedia and look at the information regarding the human eye. In particular look at the eyes response to spectra and intensity. The eye is really an interesting device, it has spectra sensitivity shifts depending on relative brightness, this also affects what resolution you see things at (which also turns into yet another non linearity aspect in your eye). Gamma adjustments on monitors exist for numerous reasons (your eyes non linear response to lumanance and the spectra shift issue I just mentioned for example).

Cyb
Title: Re: [HD Remake] Midgar sewers
Post by: Covarr on 2013-09-23 21:25:32
In fact you can't even use 4K 2560 is actually at the resolution limits of the physical eye.
That really depends on screen size, and how far your eyes are from it. The difference between 4K and 2.5K would absolutely be visible to quite a few people on an Oculus Rift, or on a movie screen large enough you can't see the whole thing at once.

4K for a standard 20" monitor, or even a 60" TV at six feet away, is slightly overkill, but it's better to have too much resolution than not enough if you don't want to be able to tell the difference.
Title: Re: [HD Remake] Midgar sewers
Post by: Mayo Master on 2013-10-04 03:23:44
Time for an update!
BY the way, I would agree more with Covarr, it all depends on the size of the display. For that matter, the original FF7 still looks decent when played on a PSP, however if it's displayed on a 40" HD TV it really sucks. Besides, it doesn't hurt to be ready for greater resolution display. Better be ready for a change in case it happens, than feel sorry for ourselves if we hadn't planned for it while having the capabilities to do so.
Anyway, here's the progress update... (this render was made with 100 samples, that's why it's a tad noisy).
(http://imageshack.us/a/img841/2416/yxlr.png)

So... actually it's very close to completion. In fact most of the remaining job needs to be handled by the computer. It's going to take a while, but it won't need much input on my part and I can start working on another scene. Basically, I'll need to run the fluid simulation and make all the renders (with 7 extra renders for animating the flow). Probably that will take about 3 weeks.
As a side note: I enjoyed using the rigid body simulation for making the pile of junk. Maybe I should make a quick tutorial on it, since it may come in handy for many scenes in the slums (or North Corel).
By the way, has anyone tried to play with my mod of colne_b3 yet?
Title: Re: [HD Remake] Midgar sewers
Post by: Kaldarasha on 2013-10-04 04:41:14
I even have changed the light for the models.
Original light color:
(http://imageshack.com/scaled/medium/198/afwi.jpg)

Altered light color:
(http://imageshack.com/scaled/medium/856/mjcd.jpg)

It is really amazing to see the new scene in action. Great work!
Title: Re: [HD Remake] Midgar sewers
Post by: Mayo Master on 2013-10-04 05:24:58
I'm glad you liked it! Actually I have been using your awesome character models to verify the scale of the objects in the scene, they've been really useful.
In fact, I have been wondering how it could be possible to modify the lighting for the 3d models, given the large variety of lighting exposure across the scenes. How did you do that?
 
By the way, do you have access to the full resolution field scene (in the file repository of Team Avalanche)? I made only the 50% resolution open to everyone. If you don't have them, I can send you the 100% resolution field files by email if you wish.
Title: Re: [HD Remake] Midgar sewers
Post by: Kaldarasha on 2013-10-04 06:15:29
In Makou Reactor you can change the global light for a model in the field:
(http://imageshack.com/scaled/medium/819/lc80.jpg)
The right color of the pallet is the one which is used in a field.
But it's not possible to change the direction from where the light comes from.
Anyway I plan to do it for whole Midgar.

For the moment I'm happy with the 50% resolution. It gives a good preview of what could be done to the backgrounds.
Title: Re: [HD Remake] Midgar sewers
Post by: Mayo Master on 2013-10-17 05:00:20
Small update on my part: after quite a lot of tinkering, my fluid simulation is done. Yay! The damn thing needed max resolution and 1500 frames to calculate!
So I'll start the renders in a bit, I'll update that as they're done. They're going to take ages, since I'll need a huge amount of samples for a 2560 x 2080 picture  :P
Title: Re: [HD Remake] Midgar sewers
Post by: KnifeTheSky77 on 2013-10-17 17:04:40
I can't wait to see the glorious HD render.  ;D I had absolutely no concept of render time or 3d modelling before I started following this thread
so bear with me here. How many samples do you think the scene you are rendering will need and how
long exactly does it take to go from model to render in this case?
Title: Re: [HD Remake] Midgar sewers
Post by: Mayo Master on 2013-10-17 23:49:14
I've started rendering the animation frames of the sewage flow, and so far it's nowhere near as bad as I expected. It takes less than 8 hours to render a frame (but it's just the sewage flow being rendered there). The whole picture (which makes the base layer) will take much longer to render - probably 2 to 3 days. I'm using 3000 samples. Usually that number is overkill, but in my case it is necessary because a/more samples are needed when you have materials using "glass" type of shaders (and the sewage flow is just like that) b/ more samples are needed in scenes where the light is dim. My case is a happy combination of both situations. After 3000 samples, I don't even have a noiseless image of the flow, but in our case it won't be really noticeable because it'll be animated.
Title: Re: [HD Remake] Midgar sewers
Post by: Cyberman on 2013-10-18 23:19:09
Small update on my part: after quite a lot of tinkering, my fluid simulation is done. Yay! The damn thing needed max resolution and 1500 frames to calculate!
So I'll start the renders in a bit, I'll update that as they're done. They're going to take ages, since I'll need a huge amount of samples for a 2560 x 2080 picture  :P
LOL I remember my ray tracing experimentation and waiting a week just to see the output. Resolution was much lower, and these days I have 4-cores so the render happens a LOT faster.  Still a few days sounds about right. OpenCL implementations make it easier to do a pipelined method of tracing and rendering, unfortunately I'm not sure blender has support for OpenCL in windows or linux.

Cyb

Cyb
Title: Re: [HD Remake] Midgar sewers
Post by: Mayo Master on 2013-10-24 14:17:44
Small progress update: I have to redo my renders because the main scene has been eventually rendered with a lot of "fireflies" (the  glass bottles were riddled with them). I found some good setting to eliminate them, however it changes the lighting of the scene a tiny bit, so it's best I re-render the other frames (those of the sewage flow). It shouldn't take too long however. Then I'll have to arrange the layers. It should be all finished in the course of next week.
Title: Re: [HD Remake] Midgar sewers
Post by: Mayo Master on 2013-10-26 17:17:53
Update: HD render of colne_b1. I'm currently re-rendering the animation frames of the sewage flow, I'll also have to render a couple of extra layers and compile them in Palmer before making the scene playable in game. I removed the fireflies by setting a "Clamp" value to 2.
Enjoy!
(http://imageshack.us/a/img59/4560/ti68.png)
Title: Re: [HD Remake] Midgar sewers
Post by: Mayo Master on 2013-11-07 02:04:51
Hi folks,

After some delays (work going insane, Windows 8 making a crash of doom, mostly), I finally managed to complete my works on colne_b1. It's layered and ready to go, ending up being 6 images (each being 1280 x 1280). The files are uploaded in the Final versions repository, under /Field Scenes/colne_b1/.
For people who don't have access to the repo, I've uploaded a 50% size version open to everyone, if there are other people willing to try it out.
http://imageshack.us/a/img10/2781/jedj.png (http://imageshack.us/a/img10/2781/jedj.png) has to be renamed colne_b1_00_00.
http://imageshack.us/a/img6/526/4llx.png (http://imageshack.us/a/img6/526/4llx.png) has to be renamed colne_b1_01_00.
http://imageshack.us/a/img811/9743/2h8s.png (http://imageshack.us/a/img811/9743/2h8s.png) has to be renamed colne_b1_02_00.
http://imageshack.us/a/img24/149/tijq.png (http://imageshack.us/a/img24/149/tijq.png) has to be renamed colne_b1_03_00.
http://imageshack.us/a/img855/656/vkt1.png (http://imageshack.us/a/img855/656/vkt1.png) has to be renamed colne_b1_04_00.
http://imageshack.us/a/img822/7060/axa8.png (http://imageshack.us/a/img822/7060/axa8.png) has to be renamed colne_b1_05_00.
These images simply need to be placed in /mods/<modpath>/field/colne_b1/ within your Final Fantasy VII directory.

Please enjoy.
Title: Re: [HD Remake] Midgar sewers
Post by: Mayo Master on 2013-11-09 00:11:38
sl1982 agreed that I should go ahead and release the HD scenes, so here they are:
http://www.mediafire.com/download/y2l2pe3t827e5yy/Sewer_fields.7z (http://www.mediafire.com/download/y2l2pe3t827e5yy/Sewer_fields.7z)
Enjoy!
Title: Re: [HD Remake] Midgar sewers
Post by: Mayo Master on 2013-11-28 14:25:59
Hi folks,

Very small progress update on my part: my actual work has become completely insane and I have very little time left with enough brain capability to work on Blender these days. I expect the situation to last until Xmas. I should be able to get actively back to 3d modelling during the holidays.
I spent what I could on the main Sector 5 field screen (not much, but that's the best that could be done):
(http://imageshack.us/a/img10/874/5cbg.png)
Title: Re: [HD Remake] Midgar sewers
Post by: LeonhartGR on 2013-11-29 00:13:30
Keep it up mate! Looking forward for X-mas.