Qhimm.com Forums
Miscellaneous Forums => Scripting and Reverse Engineering => Topic started by: Borde on 2006-05-23 10:36:03
-
Hello everyone.
I'm trying to make a patch to double the resolution of the backgrounds. I'm not really sure if I'll be able to finish it though. For now it's just an experiment.
In order to do it, I'm trying to get the backgrounds, double their resolution (turning 1 tile into 4), filtering and reinserting them into the field files. Of course I can see this won't look right in game because the image won't fit on the screen. I'd need to halve the size of the drawn tiles.
Thus It would be very helpful for me to know exactly where (offset) is the code that draws the backgrounds in the EXE. Yeah, you can call me lazy, but my hacking skills sucks and It would take me much more than waht you'd expect.
By the way, while I was trying to decode the images I used the information at GEARS.PDF. I found some information there that SEEMS (maybe I just didn't understand it well) incorrect. Particulary, I had to subrtact 50 bytes to the Raw Image data offset and add 2 bytes to find the correct Pallette entries. Am i doing something wrong or the information on the file is outdated?
Thanks for the help.
-
I've tried something similar for something I'm working on and it really doesn't work. I can import new backgrounds which look fine in Cosmo as well as my program, but the FF7 engine won't load them and crashes as soon as the field is loaded. I assume it's most likely because of the 'unknowns' in section 9 as well as the large number of unknowns in the tile structure. Until they're hammered out I don't think it'll be possible to do what you're suggesting though if you do manage to hack it some more, do let us know.
Yeah, I had some issues with offsets given in Gears too, though like you tweaking it somewhat seems to work.
-
Well, that was pretty depressing Sinergy Blades :cry:. Anyway, i'll give it a try. :-D
-
Well, that was pretty depressing Sinergy Blades :cry:. Anyway, i'll give it a try. :-D
At least it was an honest response.
I decoded some of the background information for the PS1 variant. To be certain there is a lot of information in the background.
There appears to be 4 subsections for the background information. One is for the actual background tile information and one is for the animated overlay sections. Another is likely for static overlay sections (IE tables). The remaining one might be for indicating the second image inside the MIM.
Obviously you would want to modify the engine itself to give it a larger region for drawing the background data in. Well good fortune :D
Cyb
-
Nah, not necesarily Cyb. If I haven't been fooled, the background in FF7PC is made of textured QUADS, so I simply need to divide the vertices coordinates by 2 befor they are entered at the Vtables to get smaller tiles. Of course, I'm not an expert and could very well be wrong.
-
It's a real shame. The other day I tried FF7 PSX on the ePSXe emulator. I raised the resolutions to max within the emulator config and the backgrounds looked smooth as silk, though the battle and field models looked sub-par. Ah, just to be able to get the best of both worlds. Well with Q-Gears, the new engine will be written with some more flexibility, so I'm really looking forward to the end product, reguardless of how long it may take. :-D
-
The problem with the FF7 backgrounds are two fold.
1) The backgrounds are constructed of small square sprites. This means that when you use any kind of filtering algo, the filter stops on the sprite edge. This produces "cracks" on every sprite border.
2) The backgrounds have a real bright background color used for layer alignment debugging. This was to show there were no "holes" in the layers. If a bright color poked through, you needed to fix the top layer. However, when you filter the layers, the bright debug colors "show through" the edges. This is becuase the pixel blending moves the pixels and exposes the debug color under it.
The current engine works like this (both PSX and PC)
1- Load the field file with the background broken into liitle square chunks.
2- Get where the camera is.
3- Render the lowest layer.
4- Figure out where the 3d entities are
5- If they are behind a particular Z threshold of the next layer, draw them.
6- Draw the next layer, overwriting any entities that are "behind" it.
7- go to 5 and loop for each closer layer.
8- Draw text boxes if needed.
9- flip the buffer, execute fieldscript, get input, update entity position and goto 2.
The only way to make the graphics look good in higher resolution is to re-do the engine. It will work like this.
1- Render the whole background in low resolution with all the layers applied.
2- Take a "snapshot"
3- Increase the size of the whole background, applying whatever technology is needed to make them still look good.
4- Use the layers as a mask.
5- .....etc
In the pretend world of "Awesome", the layer mask could be vector-ized as opposed to using a bitmap overlay. This makes the "cuts" sharp at any resolution.
This is all fanciful thinking. But this would be the easiest way of doing it.
-
I'm back. I've gone a bit further in my experiment and reinserted a background after filtering it. Here is the result:
(http://i6.photobucket.com/albums/y231/usebio/ResizeTest1.jpg)
Horrible, right? Well, let me explain the image a bit:
-There are some artifact (like the brown block where the stairwais should be) caused by my buggy loading routine.
-Some colors are wrong because of my poor saving routine. Currently it doesn't even add new pallettes. It just fills the empty colors found on the pallete and, when there is no more room for new colors, it reasignates the closest colors.
-There are some black squares. I don't have a clue where they come from, but I suspect I can find about it looking at LGPTools source code (because they are there if I preview this field there).
-The image overflows the screen. Obiously, because I don't know yet how to disable the pixel doubling (if someone told me how to do it, I'd be happy).
Anyway, the point of this image is showing that filtering the backgrounds is possible even without modifying the engine (well, just enough so that it draws the tiles smaller). The image won't look cracked as long as you apply the filter to the whole image.
By the way Synergy Blades, I think the reason why your backgrounds crash the game is not at section 9. I think your problem is at section 4. If you make it too large (adding too many color entries), the game crashes as soon as it tries to load the field. This is the reason why encoding images in the FF7 format is painfully hard.
Oh, one last thing. I found one more error at GEARS (in both pdf and wiki). There is a 4-byte field called nbColors. I don't know what it conatins, but I'm completly sure it's not the number of colors (at least not at that exact offset).
-
The only way to make the graphics look good in higher resolution is to re-do the engine. It will work like this.
1- Render the whole background in low resolution with all the layers applied.
2- Take a "snapshot"
3- Increase the size of the whole background, applying whatever technology is needed to make them still look good.
4- Use the layers as a mask.
5- .....etc
This could be done as a pre-process, if you apply on each layer the layers that are in front of it. Alternatively you could use a dilution process, to fill the pixels that are behind another layer from a border pixel, which should be enough to avoid artefacts when filtering. After filtering you can then fill everything with a mask colour again to improve compression.
-
By the way Synergy Blades, I think the reason why your backgrounds crash the game is not at section 9. I think your problem is at section 4. If you make it too large (adding too many color entries), the game crashes as soon as it tries to load the field. This is the reason why encoding images in the FF7 format is painfully hard.
Thanks for that. Come to think of it my code does generate a hefty number of colours, so I'll take another look when I get more time.
EDIT: Actually it crashed with an image of only 10 colours (and hence only one palette page) but thinking about it, it *may* have been because I replaced uutai1 which has animated tiles (which I believe may be down to the script? - at least there's a script in there with a name sounding like it does animation) and as such the tiles aren't where it expects. Either that or it's because I had an issue with the documentation on the number of palette colours which didn't seem to correlate with the field files I examined.
-
Well it's a start!
I think you will need to find the 'layer'/ 'sprite' info in the image first. if you are doubling your pixels for example you need to double your sprite sizes as well.
I believe that N colors is palette count likely. is the PC variant anything like the MIM files in the PS1? If so it's a TIM with a striped header and a fixed number of palettes. I recomend starting with one that has only a singular palette and a singular sprite entry. You can use the nearest neighbor algo instead of a standard filter program. IF the pallete has dark entries you can use these to add new in between colors are 'close' enough colors. The palettes are fixed with 256 colors. Be careful with black as well since this might be a transparent pixel.
If it's not similar to the PS1 format processing is much the same.
Looks like it's doable. You might consider first enlarging then smoothing and then tweaking it. Run it through a palette matching algo (IE the original palette plus whatever other colors you might need) for each section. I THINK you can use a palette for more than one chunk of tiles as well. As the palettes are determined inside the tile collection data :D Again you need to sure you don't screw up transparent pixels. Masks might be helpful as Micky suggested.
Cyb
-
Well Synergy Blades, maybe you are right. I haven't tried with an animated background yet. Anyway, if you wan't my code, I can send it to you. You may get some idea.
Cyberman, I don't know how this works in the PS1 version, but in the PC version the pallettes and the tiles collection are defined in a separate section. Particulary, the pallete is an enumeration of colors implicitly divided in 256-colors palletes. So yes, two tiles (or N tiles) can share the same pallette.
About the empty colors, yes, I'm keeping at least one empty. And the masked colors (those with the mask bit set to 1) are directly taken from the original tiles.
-
In the PS1 variant it's in a seperate file (tiles and palettes)
They use a modified TIM structure to do so.
The width is in words (multiply by 2 for pixels)
I am not going to post the TIM format :D
anyhow it sets up a palette swatch of tiles in 128x128 chunks in sections of 256 x 256. Sounds complicated because it is :)
Anyhow the go through the tile collection data and render it to the display as needed it appears.
The real pain isn't how the tiles are stored but
palette information
tile location information.
Those two make things challenging :)
-
I'm back. It took me ages to figure out how FF7 draws the tiles (I told you, I'm really bad at hacking), but after long hours of debuging, I was able to render the image as It should be:
(http://i6.photobucket.com/albums/y231/usebio/Hack.jpg)
The first word that comes to my mind after looking at this image is... unimpresive. Well, I supose It's because of my filtering code. I should do something about it. But then again, I should think about those image artifacts that are still there :-P. If someone can provide me with more detailed information about the backgrounds (animations, transparencies...) I'd be glad.
NOTE: Of course, this hack screws the rest of the locations (they are rendered at half their normal size while the characters and walkmesh remain the same).
-
Off note but You cloud model looks a bit.......... Cartoonish in size
-
Of course, I like it that way :-P All the other characters look that way in my version.
-
That looks quite good...
You don't even use the models from the NPC RP? And you are part of it....
-
That looks quite good...
You don't even use the models from the NPC RP? And you are part of it....
Those are NPC RP models; he's just modified the model so that it's smaller. He's said he likes it that way. If you take a look at the guide he wrote, you too can modify the models to your liking!
Sheesh! :-P
-
First off i was kidding. And no technically they arent... because they are modified. He keeps them all SD (which makes them look ugly in my opinion) but w/e.
I like the models in the NPC RC.
Anyways that background looks awesome. So we might be able to see higher res backgrounds?
-
I'm back. It took me ages to figure out how FF7 draws the tiles (I told you, I'm really bad at hacking), but after long hours of debuging, I was able to render the image as It should be:
<Image removed fore simplicity>
The first word that comes to my mind after looking at this image is... unimpresive. Well, I supose It's because of my filtering code. I should do something about it. But then again, I should think about those image artifacts that are still there :-P. If someone can provide me with more detailed information about the backgrounds (animations, transparencies...) I'd be glad.
NOTE: Of course, this hack screws the rest of the locations (they are rendered at half their normal size while the characters and walkmesh remain the same).
I would say that's a marked improvement. That pattern might be an encoding issue (with your data) since it's present in both variants. Who really knows.
After software filtering perhaps you could have the image touched up by a pixel artist? That might give larger improvements in visual quality. You can then reduce colors as needed later. If you can figure out how to safely add more palettes there is that option as well.
Cyb
-
(...)which makes them look ugly in my opinion(...)
Exactly. Your opinion.
In this situation, that's all it boils down to. Discussion over. :lol:
-
ANYWAYS... How's this going? Any improvements?
-
Well Cyb, I finally was able to add new pallettes. As a result, I've been able to improve the colors (althought they are not as good as i'd like yet). However, I found a rellay stupid constraint about the number of colors: you can't have more than 13057. If you try to add one more, the game crashes. Frankly, I don't get it. This number isn't even a multiple of 256.
I was also able to remove some image artifacts (mostly the brown blocks). However, I haven't managed to remove those black squares (that only seem to afect the foreground) nor that strange green cloud. I had a look at LGPTools source code, but couldn't find the code that draws the backgrounds. I found It at the shared code, but it looked horribly outdated (it did nothing about transparencies nor animations). as usual, if someone can point me to a more complet version, I'd be glad.
-
13057 might not be, but 13056 is. I'm not sure where that extra one comes from.
-
Hmm, One extra color? A mystery to me! ( white )
-
Actually, it's more likely black than white. Black is the only color you can get with no color data (hex #000000), whereas white is the opposite, full color data (hex #FFFFFF)
-
Ahh I got mixed up! I was thinking backwards, Im slightly familliar with the digital color wheel, but I was thinking
"White is the absence of color"
-
10357 is 51 palettes + 1 color isn't it? :D
This is a rather suspicious number no?
So your real limit is 51 palettes. I suspect it's hard set for 50 palettes, nothing you can do about that in the engine eh? :D
I suggest using 2 approachs.
1) use fractal resolution enhancement instead of traditional ones. This will make the image more closely resemble the original.
2) palette match. Do not use a unique palette for each section, it's kind of wasteful. Since you are likely using 4 times the number of tiles you should be able to use 4 tile sections to a palette. You can use the 'close enough' method of reducing color count. IE if a color is +-1 count off of an existing RGB value AND you have exhausted the palette then find something closer. This process would take quite a few passes on the image to get an optimal palette. Also you need to compute the colors by popularity. Being that you can mix and match tiles in all sorts of strange ways you should do a 'pallette' pass and sort them by tile location. IE each tile gets a palette sorted by popularity. (16x16 256 pixels etc.) so you end up with a huge number of palettes. You can then collect tiles by matching the palettes. As I said this takes forever but the results are pretty good. since decoding is easy by comparison. It's not a big deal. You just have to be sure to have no more than 50 palettes afterward.
I've used 2 for reducing palettes for GBA tiles myself. It works quite well, inspite of the fact it takes a LONG time. LOL.
Cyb
-
I'm back. I finally found what was causing those stupid black holes: it was simply a pice of code I didn't remove after doing some early tests. Oh well. Anyway, now I've got a new problem (well, not exacly new): some of the foreground tiles are transparent.
(http://i6.photobucket.com/albums/y231/usebio/Test02.jpg)
This is starting to become quiet boring. Is there anyone interested in finishing this? :-P
About the filtering, fractal interpolation would really rock. The problem is that I was trying to create a completly automated patch. So unless someone knows about some DLL that allows me to use that filter...
Oh, about the extra color, I don't think it's black (because every pallete has his black entry). However, I've just realized that GEARS says that the number of colors is stored incremented by one :-P. By the way, the number of colors was actually there, the problem is that it's a 2 byte value instead of a 4 bytes one (as noted in GEARS).
-
Okay, wait, this is a dumb question but:
What is the resolution of the regular FF7 backgrounds?
-
Borde, you are doing very well, ill love that patch!!!
i cant help you here, but i can wish much success!! :lol:
-
This is starting to become quiet boring. Is there anyone interested in finishing this? :-P
Nay, Borde! You should go on! We've already got better models! All we need is higher res backgrounds.
-
Okay, wait, this is a dumb question but:
What is the resolution of the regular FF7 backgrounds?
Well I imagine this answer might be a bit shocking.. but they are all different :D
They range from 320x224 (Some place in the subway) to 1280 x 320 (forest of the ancients).
Cyb
-
Oh, ermm, it should have been easy to notice that.
In the screenshot posted above, is that double the normal resolution?
-
Yes, it is. It's scaled and (not specially well) filtered.
-
Well, I've got some very bad news. FF7 can display a limited amount of tiles. That's because it decides first which tiles must be drawn, calculates their vertex and stores them in a temporal buffer. So basically if there are too many tiles on screen the game just crashes. And that's something that happens very soon (in the second field, for examle).
Is this Game Over then? Well, maybe not yet. I've got an idea: test just one of every 4 tiles then, when drawing them, draw the 3 extra tiles. However, I must find where the tiles are drawn and see if I can fit the extra code there. if this doesn't work... well, then this surpases my limited hacking hablities.
-
You are learning how the background tile system works. Remeber that in science even in failure, you learn something the can benifit the next experiments
-
Well, I've got some very bad news. FF7 can display a limited amount of tiles. That's because it decides first which tiles must be drawn, calculates their vertex and stores them in a temporal buffer. So basically if there are too many tiles on screen the game just crashes. And that's something that happens very soon (in the second field, for examle).
Is this Game Over then? Well, maybe not yet. I've got an idea: test just one of every 4 tiles then, when drawing them, draw the 3 extra tiles. However, I must find where the tiles are drawn and see if I can fit the extra code there. if this doesn't work... well, then this surpases my limited hacking hablities.
Have you acertained the limit? It is likely something to do with the 'maped' area IE the tile sets. I know in my experiementation with decoding the PS1 versions MIM and field information it appeared that they used a 'tile swatch' of 256 x 1280 which is 2x10 tile sets in the video memory. This gives you 1280 tiles max. Technically one could add 2 more tile swatchs BUT, no background uses that many. Since the PC version is based on the PS1 code and the data was recompiled some for the PC. I suspect this is the 'hard' limit for the PC as well.
-
About the filtering, fractal interpolation would really rock. The problem is that I was trying to create a completly automated patch. So unless someone knows about some DLL that allows me to use that filter...
Well I found this fractal interpolation .dll. I haven't experimented with it personally, but it also mentions using "Adaptive Noise Reduction" before or after you use fractal interpolation. I guess that could be something to test out. Also be sure to check out the batch processing plugin right above it. It may make your life a little easier down the road. :-D Hope this helps.
Fractal Interpolation.dll (http://meesoft.com/Analyzer/Plugins/FractInterpolation.dll)
Plugin Info Page (http://meesoft.logicnet.dk/Analyzer/plugins/)
EDIT:
I should mention it is for the Image Analyzer editor. I wasn't sure if you had been using Photoshop or whatever, but at least its freeware.
An alternative would be to download the trial of Genuine Fractals, which comes as a Photoshop plugin.
-
Mmmm... probably you are right about the limit Cyberman. I haven't botehered testing it though. It would take some valuous time.
Otokoshi, thanks for directing me to that link, but I wonder if I'll really be able to use that DLL under Visual Basic or Visual C.
Anyway, I'm going to quit from this project temporaly. I want to work on the next version of Kimera first. I'm trying to implement some interesting features this time :-D.
-
Otokoshi, thanks for directing me to that link, but I wonder if I'll really be able to use that DLL under Visual Basic or Visual C.
Not a problem. I gave it a try and it works as advertised, giving good results when doubling the pixel dimensions. I figured the .dll may need to be tweaked to work with anything but the supported program. If not that, it could serve as a reference to a new .dll. I just figured that could lay the ground work for something greater, such as this patch.
Nonetheless, I'm looking forward to whatever treats you got cooking for your next version of Kimera.
-
::resurrect thread::
Borde, if you're interested in continuing this project, I'm pretty savvy with asm and dll injection, not so much however with graphics programming. Nice backgrounds would be awesome.
Sorry for the thread resurrection, but it didn't really make sense to start a new one.
-
Start a new thread by referencing this one with a link please. Necromancy is still bad.
The reason why Necromancy is bad is not because of some desire for thread cleanliness, it's because you just posted on a thread that dealt with an experiment almost a year and a half old. I can assure you no one is working on it.
What you just did is walk into an old room and started talking to yourself. No one is here to answer you.
You need to start a new conversation, we'll see what you've got.