3D Engine Options

[Kenyai]

I've been interested in building my own 3D engine. But I think I have a problem. I'm too interested in the math involved in 3D rendering.

I know how matrices work for rotations, I've become best friends with vectors, I learned how to interpolate the colors for gouraud shading, I figured out an algorithm for a ray-triangle intersect (by the way, is there a faster way to do perspective correction for sampling than ray-tracing the pixels?), and so on. I used DirectDraw and directly plotted the pixels I needed to draw my triangles with my own functions. But I realized pretty quickly that my engine will never be very fast if I only use DirectDraw. I could only draw about 100000 solid-color triangles per cycle before my FPS became unacceptable.

I hate Direct3D. I don't know why. Maybe it's the lack of freedom involved when one has to call a particular Microsoft function to do what one wants instead of writing one's own code. I just can't get comfortable with it. Is there a way for me to access hardware acceleration or to draw triangles with lightning speed using my own code? I want all the speed that comes with Direct3D but all the freedom of coding from scratch. Is it a one-or-the-other situation?

[Kislinskiy]

OpenGL

[sfx1999]

(by the way, is there a faster way to do perspective correction for sampling than ray-tracing the pixels?)

Do most software renderers even use perspective correction?

I hate Direct3D.

Me too. No portability.

Is there a way for me to access hardware acceleration or to draw triangles with lightning speed using my own code? I want all the speed that comes with Direct3D but all the freedom of coding from scratch. Is it a one-or-the-other situation?

What's the point? Hardware would be much faster. I mean what are you going to do, feed spans to OpenGL?

Anyway, if you are using a depth buffer, you can get a significant speed boost by using a span buffer. There is also another buffer called a coverage buffer where you can merge spans, but it requires that your data be perfectly sorted from front to back. That can be done with BSPs, but I have no idea how to sort models doing that because they would be difficult to feed to the renderer in a perfect front to back order.

[Micky]

... (by the way, is there a faster way to do perspective correction for sampling than ray-tracing the pixels?)...

See this absolutely invaluable article series by Chris Hecker about perspective texture mapping:
http://www.d6.com/users/checker/misctech.htm
This is the "most correct" way I have seen.

Having said that, you should have a look at OpenGL or Direct3d. There are very few reasons to write a software renderer nowadays. And I found OpenGL relatively easy to get into.

Is there a way for me to access hardware acceleration or to draw triangles with lightning speed using my own code? I want all the speed that comes with Direct3D but all the freedom of coding from scratch. Is it a one-or-the-other situation?

You rarely get the documentation to access the triangle setup hardware, and then your code would depend on the card you wrote the code for. The closest you get to the hardware is on something like a console, see this PSX doc or this GameCube doc.

[Cyberman]

I hate Direct3D. I don't know why. Maybe it's the lack of freedom involved when one has to call a particular Microsoft function to do what one wants instead of writing one's own code. I just can't get comfortable with it. Is there a way for me to access hardware acceleration or to draw triangles with lightning speed using my own code? I want all the speed that comes with Direct3D but all the freedom of coding from scratch. Is it a one-or-the-other situation?

As Kislinskiy suggest OpenGL, it's not D3D and most cards support it. MS badly supports it (no comment on MS ).

A good link to get OpenGL fun from is NeHe's site lots of great tutorials there and a few examples. Ask questions etc.

Cyb

[mirex]

A good link to get OpenGL fun from is NeHe's site lots of great tutorials there and a few examples. Ask questions etc.

I wanted to recommend the same thing. I learned OpenGl from those tutorials, they are quite good.

[sfx1999]

One thing you would do in software is reduce the number of polygons sent to the renderer.

[Cyberman]

One thing you would do in software is reduce the number of polygons sent to the renderer.

You mean backface culling? This is great however OpenGL supports it internally.  It's handy but sometimes you need to render a scene from more than one perspective, an example is if you have mirror objects.  In that case you have to take your scene render it off screen then map the texture on the surface it needs to be visable on.  This means you have to render the scene twice however.  So you get two sets of polygons you need to do this.  Interesting huh?

Mirror Mirror on the wall, dang it you make things complicated!

Cyb

[Kenyai]

Lots of helpful links and advice, thank you guys. If only D3D wasn't such a pain, I wouldn't mind, but it seems Microsoft enjoys torturing developers.

[sfx1999]

You mean backface culling? This is great however OpenGL supports it internally.  It's handy but sometimes you need to render a scene from more than one perspective, an example is if you have mirror objects.  In that case you have to take your scene render it off screen then map the texture on the surface it needs to be visable on.  This means you have to render the scene twice however.  So you get two sets of polygons you need to do this.  Interesting huh?

I mean like using a PVS and/or portals.

Anyway I already guessed how to do mirrors/special portals.

[Cyberman]

I mean like using a PVS and/or portals.

Anyway I already guessed how to do mirrors/special portals.

Hmmm that's a name I'm unfamiliar with, likely it's one of 'those' things a company created instead of using conventional terminology OR I just am clueless

I started my work with Redoing my model system for display.  Using C++ and OGL together makes things interesting.  Think Vertex color and draw thyself!.  So now my rendering code is just going to be a lot of objects.  Still need to add some automation tricks in. (SIGH).

Cyb

[sfx1999]

PVS = potential visibilty set, often lovingly refered to as a VIS. Quake used this.

Portals are what Duke Nukem used to reduce polygons rendered. Unreal uses these and has it's sectors divided into BSPs for sorting.

Also, another optimization would be not even attempting to render polygons that are brhind the camera, because I do not know if OpenGL does this. You can also use lightmaps in order to reduce lighting calculations (in other words, only light models), and you can also only calculate lights for that object if they are in the PVS  for that object.

[Cyberman]

PVS = potential visibilty set, often lovingly refered to as a VIS. Quake used this.

Portals are what Duke Nukem used to reduce polygons rendered. Unreal uses these and has it's sectors divided into BSPs for sorting.

Also, another optimization would be not even attempting to render polygons that are brhind the camera, because I do not know if OpenGL does this. You can also use lightmaps in order to reduce lighting calculations (in other words, only light models), and you can also only calculate lights for that object if they are in the PVS  for that object.

All right you could have used the terms a little differently. BSP's have to be pre calculated, this is a REAL problem if you have moving objects and these days it's unneeded mostly. BSP's are only useful in a very fixed set of circumstances.  Essentially you are talking about methods of handling the structural informtation of static data.

BSP's get problems with non obstructing view objects as well. As they can't be used with them. Ever wonder why most of Doom and Quake are rooms and not 'outdoors' with lots of data?   For outside 'fog' works great.  If you have ever played Medievil you'll notice things fade into darkness as you move around.  This actually is the objects being removed from the rendering queue to be ignored by the engines Z buffer.

If you've played vagrant story they used an 'out of view' and texture resolution tweaking to reduce the amount of data needed.  Each room was a specific size max they did a lot of object obscuration removal I think.

Cyb

[mirex]

I think that portals/bsp tree is great idea for 3d games, because levels usually have lots of indoors.

Dunno if you seen the game Far Cry, it has great engine with lots of outdoors / jungle with visibility that can be counted in kilometers ... very nice engine, with lots of features (water mirroring, shadows, physics) i cannot see way how did they did it.

And fog is needed, because if you would see everything then it would be too time-expensive, none of computers could possibly handle that.

[Cyberman]

I think that portals/bsp tree is great idea for 3d games, because levels usually have lots of indoors.

Dunno if you seen the game Far Cry, it has great engine with lots of outdoors / jungle with visibility that can be counted in kilometers ... very nice engine, with lots of features (water mirroring, shadows, physics) i cannot see way how did they did it.

And fog is needed, because if you would see everything then it would be too time-expensive, none of computers could possibly handle that.

Well if you want 'the perfect' resolution your computer will be grunting along at 1 frame per 5 minutes The hardware has to change some, even though the new units have built in FP processors etc.  no one has direct access to the hardware to abuse it properly.  Sad isn't that?

Cyb

[Micky]

Dunno if you seen the game Far Cry, it has great engine with lots of outdoors / jungle with visibility that can be counted in kilometers ... very nice engine, with lots of features (water mirroring, shadows, physics) i cannot see way how did they did it.

You can find a lot of documentation about this on game programming websites.
Things to look for:
- Terrain - there are algorithms that generate less triangles for landscape that is further away then landscape that is in front of you.
- LOD - replacing models with simpler versions when they are further away. There are algorithms that automatically calculate in-between versions, so you don't see pop-up. Covered in a lot of books and articles.

Links:
http://vterrain.org/
http://www.flipcode.com/
http://www.gamedev.net/
http://www.gamasutra.com/

Mailinglist:
http://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
(Beginners questions are not tolerated there, but they should have a lot of valuable information in their archive)

[sfx1999]

All right you could have used the terms a little differently. BSP's have to be pre calculated, this is a REAL problem if you have moving objects and these days it's unneeded mostly. BSP's are only useful in a very fixed set of circumstances.  Essentially you are talking about methods of handling the structural informtation of static data.

BSP's get problems with non obstructing view objects as well. As they can't be used with them. Ever wonder why most of Doom and Quake are rooms and not 'outdoors' with lots of data?   For outside 'fog' works great.  If you have ever played Medievil you'll notice things fade into darkness as you move around.  This actually is the objects being removed from the rendering queue to be ignored by the engines Z buffer.

If you've played vagrant story they used an 'out of view' and texture resolution tweaking to reduce the amount of data needed.  Each room was a specific size max they did a lot of object obscuration removal I think.

Cyb

Precalculating things allows significant speed boost at run time, which is GOOD. If you haven't noticed, just about all games levels are precalculated. Look at FF7, all of the rooms are pre-rendered. Quake has lightmaps, Unreal has lightmaps, etc.

I know why BSPs do not tend to have large outdoor areas, and you should know that I know that because I mentioned the PVSs and whatnot. The thing is, unless you plan to have huge outdoor scenes, you should be OK.

And for moving geometry, it has been done in BSPs with the use of brush-based entities. Can you name an engine that does not have that problem with moving objects? Portals cannot do it well, either. A pure terrain engine would have even more problems.

If you plan to combine huge outdoor scenes with indoor ones, you would combine a terrain renderer with a portal render. It's that easy.

[Cyberman]

Actually the easiest thing is to use a combination of what the first flight sim used and fog.

You have several radius regions where detail is reduced as needed.
Filght sim (the first) changed rendered information as it became closer.  It computed the distances and worked from there.  It ran in real time on a 4.7 mhz XT too.  (and on a 1mhz 64K color computer and other tiny machines).  What made it work basically was detail changing by radiii.  It would shift the radii regions as you moved a certain amount in a given direction.  One almost never noticed because you only lost a single frame when it did it.  It kept the amount of detail information in memory quite small that way.   This works the same as fog.  The really only difference in the algorythm is just what how the details are rendered now.  You have higher resolutions and a lot more data to show.
Inside buildings I don't believe this concept will be a problem either.  It can change if you are using First person or a chase camera.  I prefer the latter myself.  BSPS are only good in FP. It's more difficult to precalculate something that's not FP since you can't (grin).  NWN for example uses a vertical cut plane (built into objects) and a distance fade, you can add fog (kind of helpful if you want something dreary or dawnish).

There are many ways to do anything, my opinion is do what works best for you.  Ultimately BSP's aren't used much anymore.  Mostly because computing power is so much greater than before, 72fps in a game is rather silly. When things work fine at 30 fps and the game looks much nicer.   Speed is only an issue if you have to manually do everything (IE combat is like a first person shooter).

Cyb

[sfx1999]

I beg to differ about BSPs not being able to third person. They CAN. If you don't believe me, Quake II could, Heavy Metal F.A.K.K. 2 could, Quake III could, and I am pretty sure Quake could too. 3rd person is done by basing the camera matrix off of the player's position.

IIRC, Doom 3 is going to use BSPs.

If you are doing a software renderer, BSPs can help ypu get a perfect front to back order, so you can optimize your renderer using a c-buffer, and when all of the pixels are drawn you can stop.

Also, some terrain renderers may use BSPs. You can convert the height maps into them. You might have some trouble with LOD, though.

But does it really matter? All he wants to do is make a software renderer as a learning experience. It does not necessarily mean he wants to make levels.

Oh and BTW, UT2004 uses BSPs to define its sectors.

[Cyberman]

This reminds me to never write something when half awake. Or was that half asleep

I really don't see the point of using BSP's unless they have another problem.  BSP's seem to be more of a problem then a help.  I suppose if one uses the BSP's to look for areas one needs to apply specialized handling of the view. Then it makes sense. However BSP's had a considerable volume to the size of things.  Disk space isn't the problem, it's actually swaping the data in and out to use.  NWN has this problem in that an entire modules worth of walk meshes is persistantly kept in memory (I can attest to this as my module of the hells made a lot of problems related to this).

Irregardless what you or I think, people will use
What's familiar to them.
What works.



Cyb

[sfx1999]

What I am trying to say is BSPs give you information in perfect order. Since they do that, you can disable depth testing and reenable it when you have to. It was mainly done to speed up software renderers, but you can still use it.

The problem is that you need depth testing in a hardware renderer to keep overdraw to a minimum, and even then I don't know how you tell if all the pixels are filled on the screen to stop rendering.

They work. Tons of games use them. They are very effective.

One of the limitations with BSPs is they can't do mirrors very well, but you CAN do it. You would have to implement a portal entity and open another viewport to another location in the map. You would also have to modify the netcode so that the server will send information about player locations through the portal or tweak the PVS.

But like I said, does it really matter? He is just writing a toy engine for educational purposes.

[Cyberman]

One of the limitations with BSPs is they can't do mirrors very well, but you CAN do it. You would have to implement a portal entity and open another viewport to another location in the map. You would also have to modify the netcode so that the server will send information about player locations through the portal or tweak the PVS.

But like I said, does it really matter? He is just writing a toy engine for educational purposes.

Yep.. well.. maybe a toy, heh, I've seen engines that went from toys to comercial usage.  

Mirrors are one area, water is another area any place where refraction or reflection ocure BSP's will have problems.

Now I need to go work on TV for a little bit.

Cyb

[mirex]

some pics and links about the FarCry cryengine:
http://www.crytek.de/screenshots/index.php?sx=xisle&px=1227.jpg
http://www.crytek.de/screenshots/index.php?sx=xisle&px=1245.jpg
http://www.crytek.de/screenshots/index.php?sx=xisle&px=1248.jpg
http://www.crytek.de/screenshots/index.php?sx=cryengine&px=engine_06.jpg

http://www.crytek.de/technology/index.php?sx=cryengine

its a pure pleasure to play in such a engine (all of that really moves, those are screenshots from game)
And game has indoors too.

[Cyberman]

But Mirex... I can't download the engine

hehehe
Looks nice
I think it might be over kill for an FF7 engine though, don't you think?

Cyb

[Darkdevil]

Playing FF7 in the Far Cry engine would be HEAVEN...Only problem would be the only the top 10% of machines would be able to pla6y it at max graphics  :isee: