ficedula, I agree with you. If you are doing any 3D work you really should just go straight to OpenGL for a couple of reasons.
1) Since SDL is only a wrapper to Direct3D/OpenGL, there is, in theory, a small (really nanoseconds here) amount of time that SDL would add to your execution time. This is because for every high level call you make, SDL has to make equivalent lower level calls.
2) If you are doing 3D work you probably would want to make sure that your code runs at an optimal speed.. which is another reason I guess to work directly with Direct3D or OpenGL.
In my opinion SDL is really just a pretty good wrapper for 2D programming. Especially if your only choices are SDL or going straight to the old DirectDraw interfaces. If you were looking for say.. a great 2D gaming library I'd recommend Allegro.
Anyways, I don't want to get too offtopic.. so i'll stop here.