[PC] External Texture Support - Tonberry: Enhanced (2.04)

[FatedCourage]

That's a classic collision FatedCourage. If you say it worked on 2.01 and it doesn't on 2.02, there's no other explanation. All I changed on 2.02 was the naming for the debug folder and made an enumerated type for the type of mached hash. Nothing to do with that. Also, I've found out that some collisions will happen 100% of the time, and some others are more probabilistic since they depends on where are they placed on memory. Anyway, you all will have to wait until we finish the new hashing to see most collisions solved

I have no problem with this. Didn't want to gloss over anything like I did last time. And if textures that didn't work before are starting to work now, then that's a good sign. It just motivates me more to continue collecting codes/bitmaps for this.

[JeMaCheHi]

I have no problem with this. Didn't want to gloss over anything like I did last time. And if textures that didn't work before are starting to work now, then that's a good sign. It just motivates me more to continue collecting codes/bitmaps for this.

Yeah, it's a good sign indeed . Don't forget to keep all the debug textures and sending them to Mavirick or me!

[Omzy]

Actually, now that I remember, if I detected that there was no match or an unresolvable collision, I ignored it and let d3d keep the original texture there. So if there are now colliding textures in previously pixelated spots, it might mean that 'ignore' was turned off and it is just placing the last thing in memory there. Again, sorry if I overlook stuff, it has been a while since I looked at the code.

[FatedCourage]

Yeah, it's a good sign indeed . Don't forget to keep all the debug textures and sending them to Mavirick or me!

Right now, I'm trying to collect as much as possible to send off to you guys. It also helps me since I'm gathering codes for another release down the road.

[Kazter]

Mavirick, as a continuation from what I have been PMing you and Mcindus, how do I use the Debug Mode in Tonberry so I can find what is randomly crashing my games?  I feel like a proper write up of the Debug Mode should be included in the original post.

[mavirick]

Actually, now that I remember, if I detected that there was no match or an unresolvable collision, I ignored it and let d3d keep the original texture there. So if there are now colliding textures in previously pixelated spots, it might mean that 'ignore' was turned off and it is just placing the last thing in memory there. Again, sorry if I overlook stuff, it has been a while since I looked at the code.

As far as I know this wasn't changed, but my work was really focused on the cache. I know the logic for replacing textures had to be changed to some degree but Jema is dealing with some personal business that is keeping him away; I will take a look when I get the chance.

Mavirick, as a continuation from what I have been PMing you and Mcindus, how do I use the Debug Mode in Tonberry so I can find what is randomly crashing my games?  I feel like a proper write up of the Debug Mode should be included in the original post.

I don't own the thread so I can't modify the first post, but to turn debug mode on, simply set debug_mode=yes in FFVIII\tonberry\prefs.txt. All debug folders then need to have the zero removed, so \tonberry\debug0 becomes \tonberry\debug, \tonberry\debug\replaced0 becomes \tonberry\debug\replaced, etc.

Edit: You guys were right--it looks like our new cache broke collision resolution.  I believe I've found the problem and I've made a quick fix but it will need more testing before I release it. If you are experiencing a crash or texture problem that you can easily replicate and would like to help, send me a PM and I can provide the new d3d9Callback.dll for you to try. Otherwise hold tight and I'll get it released as soon as I can.

[theelderscrolls]

Edit: You guys were right--it looks like our new cache broke collision resolution.  I believe I've found the problem and I've made a quick fix but it will need more testing before I release it. If you are experiencing a crash or texture problem that you can easily replicate and would like to help, send me a PM and I can provide the new d3d9Callback.dll for you to try. Otherwise hold tight and I'll get it released as soon as I can.

Hi sorry total modding newbie here,

my game keeps crashing right as you reach the dollet solder and face the anacondor. Ive pretty much eliminated all my mods but tonberry. is this something that d3d9Callback.dll would fix once the details are worked out?

[mavirick]

Hi sorry total modding newbie here,

my game keeps crashing right as you reach the dollet solder and face the anacondor. Ive pretty much eliminated all my mods but tonberry. is this something that d3d9Callback.dll would fix once the details are worked out?

It's hard to say--I certainly hope so. I haven't heard of anyone else having this glitch and this next fix should restore Tonberry to 1.61 efficacy, so it's very possible that you're experiencing a collision issue that will be resolved by the new version. The other possibility is that you're getting an achievement or Steam notification--folks, the overlay is still not supported by Tonberry and should be either disabled or circumvented using Mcindus' GeDoSaTo configuration.

I have the new d3d9Callback.dll all packed up and ready to go, but am waiting on some feedback and a bit more testing before I release it, hopefully tomorrow. I hate the fact that we've released so many broken versions, but we're close!

[Kazter]

TES, it's probably the Steam Overlay, it seems that a lot of people still miss this one, but it MUST be turned off unless you are using GeDoSaTo and Mcindus' FF8 loadout.  If that still doesn't fix the problem it may be the way Project Eden (if you have it installed) and Tonberry v2.02 interact w/ each other.  I am one of the people that Mav is referring to on feedback, I will try out his new Callback.dll and see if it fixes my random crashing w/ Project Eden and Tonberry v2.02.

[Umbran0x]

Hey guys,

First off, I just want to say thank you for this excellent project.  I only recently got FF8 on PC and decided to try modding it.  It's so cool to be able to play this old game "remastered" by the community.

My only problem is crashing every so often when I hit a load screen (screen going black when entering a new area/screen).  All steam overlays are off.  The annoying thing is I can't reproduce the crash.  It will crash on a loading screen and then when I restart, that loading screen is fine, no crash.  It seems to be completely random.  In the error report it shows d3d9Callback.dll as the problem so maybe your next release will fix it.  I've just been pushing through and saving a million times to try to counter it I may try removing Project Eden and seeing if that's the problem.

Also, when I add those 2 collision files, Collision and Hash2Map, this happens: http://i.imgur.com/LXHo2QY.jpg.    Maybe I'm doing something wrong.  It's no big deal, I just removed them for now.

Thanks again for this excellent tool, looking forward to more in the future!

[Kazter]

Umbra, sounds like you are having the same issue I do w/ Project Eden.  I will have you know, Mavirick's new Callback file DOES fix all the conflicts w/ collisions.csv (including the one in your link, I had the exact same issue).  However, it does NOT cure the random crashing while running Project Eden.  I used to run a modified collisions.csv file, but am now able to run on the stock one that is linked in the original posting.

I currently have every one of the latest mods installed from this website, EXCEPT for Project Eden (which is a drag).

Tonberry
SeeD Reborn
Horizon
X360 Button Config
Lunar Cry
Tripod
Rebirth Flame
RAW4

[mavirick]

Download Tonberry Enhanced v2.03 Here

Okay, I've decided to go ahead and release v2.03. I don't own the thread so I can't update the main post, but you can get it using the link above. This includes all necessary .dll's, an empty debug directory structure, and the minimum necessary hashmap .csv's. To find hashmaps for individual mods, see Mcindus' post on the first page (first comment).

What's new?:
2.03:

• 2.02 broke collision resolution--textures from the collisions.csv hashmap were not properly stored in the cache, which caused bad replacements and glitching textures especially on loading screens and in menus; this issue is resolved

Known issues:

• Some users are experiencing random crashes when using Tonberry with Project Eden. If you are having this or any other issue please post here with details so I can look into it. Until more is known about the crash, my best advice would be to save frequently.
• The overlay is still not supported by Tonberry and should be either disabled or circumvented using Mcindus' GeDoSaTo configuration.

[KJT]

Hey pals,

I've installed tonberry and I'm using SeedReborn (german), Tripod (german), Horizon, project eden, lunar cry and the Controller Buttons Pack (X360)
Since i've installed the mods i get the "Unknown exceotion occured" message irregulary and my game crashes. And yes, I already have disabled the Steam overlay.

Do you have an idea, what could be wrong?

Thanks for your Help and keep up the good qork 

[mavirick]

Hey pals,

I've installed tonberry and I'm using SeedReborn (german), Tripod (german), Horizon, project eden, lunar cry and the Controller Buttons Pack (X360)
Since i've installed the mods i get the "Unknown exceotion occured" message irregulary and my game crashes. And yes, I already have disabled the Steam overlay.

Do you have an idea, what could be wrong?

Thanks for your Help and keep up the good qork 

I have not experienced this crash myself, so it is proving difficult to debug. It sounds on the surface like a caching problem (trying to access NULL textures) but it seems others have isolated it to Project Eden which is full of colliding textures, so I'm not so sure. I will be targeting this issue and hopefully will have resolved it soon. In the meantime, disabling Project Eden may help reduce the frequency of crashes.

[KJT]

I have not experienced this crash myself, so it is proving difficult to debug. It sounds on the surface like a caching problem (trying to access NULL textures) but it seems others have isolated it to Project Eden which is full of colliding textures, so I'm not so sure. I will be targeting this issue and hopefully will have resolved it soon. In the meantime, disabling Project Eden may help reduce the frequency of crashes.

At this point i can't tell if it has worked, but since I have disabled Project Eden i haven't had any new crashes. Seems like Eden was the Problem. Thank you very much

[theelderscrolls]

It's hard to say--I certainly hope so. I haven't heard of anyone else having this glitch and this next fix should restore Tonberry to 1.61 efficacy, so it's very possible that you're experiencing a collision issue that will be resolved by the new version. The other possibility is that you're getting an achievement or Steam notification--folks, the overlay is still not supported by Tonberry and should be either disabled or circumvented using Mcindus' GeDoSaTo configuration.

I have the new d3d9Callback.dll all packed up and ready to go, but am waiting on some feedback and a bit more testing before I release it, hopefully tomorrow. I hate the fact that we've released so many broken versions, but we're close!

negative i had the overlay disabled and was still receiving that error. though i am going to try this new release and try it without project eden. will update with how well it works

Update: did not work, still crashed.... basically it let me play long enough to do all the chocobo forests, then on my way out the sanctuary it crashed with the "Unknown exception" error

[mavirick]

Update: did not work, still crashed.... basically it let me play long enough to do all the chocobo forests, then on my way out the sanctuary it crashed with the "Unknown exception" error

It crashed even with Project Eden disabled? What is your cache_size set to in \tonberry\prefs.txt?

[mavirick]

To those who are having trouble with Tonberry 2.03:

I have been unsuccessful in replicating the crash you guys are experiencing so I am releasing this ALPHA version of 2.04 with verbose debug logging enabled. This is not an official, supported release of Tonberry and should only be used by those who are experiencing the crash and would like to help out with the debugging process. I found a few possible sources of unhandled exceptions that have now been cleared up and also made what I believe are some pretty substantial speed improvements, especially in the realm of area transitions and black loading screens. It's possible that these were causing the random crash but I have no way to be sure without your help.

Instructions:

• Download TonberryEnhanced2.04_ALPHA here
• Extract the new d3d9Callback.dll and copy/paste it into your \FFVIII root folder.
• There must be a valid \FFVIII\tonberry\debug directory; if your debug folder is named "debug0" you must rename it to "debug" before proceeding.
• Enable any mods you like. For those of you who have had success disabling ProjectEden, re-enable it. If there are specific textures within PE that are causing this exception, I want to know.
• This is optional, but I suggest you set debug_mode=no in \FFVIII\tonberry\prefs.txt; the logging I've enabled will already slow things down and if it takes a long time playing before the crash the texture writing will only make things even slower.
• Play. Save often. Wait for the crash.
• After the game crashes, navigate to \FFVIII\tonberry\debug. There should be two new files there: "global_context.log" and "texture_cache.log". Send these two files to me, either via PM or to mavirickstone_at_gmail_dot_com.
  • Note that, depending on how long you've played before the crash, these files could be quite large (although nothing like the debug texture folders). They're just text but they're logging every single texture modification and every operation in the texture cache. You are welcome to take a look, but without a thorough understanding of how Tonberry works it likely won't make much sense.

If this is successful, I will start releasing unsupported alpha versions for those of you who are generous enough to help out. Hopefully that will speed up development as we approach the new hashing system and an even more beautiful FFVIII.

Again, this is not an official, supported release of Tonberry and should be ignored by anyone who is not experiencing the crash.

[Kazter]

Looks good Mav, I will test it first thing when I get home from work.  At this point I have the Project Eden folders memorized so it's a pretty easy process for me to add/remove it from Tonberry.

[Mcindus]

Looks good Mav, I will test it first thing when I get home from work.  At this point I have the Project Eden folders memorized so it's a pretty easy process for me to add/remove it from Tonberry.

In the future, in order to disable a mod, you just need to put its _hm.csv file and _obj.csv (if it has one) files into the 'disabled' folders

[KJT]

Hey pals,

I've installed tonberry and I'm using SeedReborn (german), Tripod (german), Horizon, project eden, lunar cry and the Controller Buttons Pack (X360)
Since i've installed the mods i get the "Unknown exceotion occured" message irregulary and my game crashes. And yes, I already have disabled the Steam overlay.

Do you have an idea, what could be wrong?

Thanks for your Help and keep up the good qork 

I have not experienced this crash myself, so it is proving difficult to debug. It sounds on the surface like a caching problem (trying to access NULL textures) but it seems others have isolated it to Project Eden which is full of colliding textures, so I'm not so sure. I will be targeting this issue and hopefully will have resolved it soon. In the meantime, disabling Project Eden may help reduce the frequency of crashes.

At this point i can't tell if it has worked, but since I have disabled Project Eden i haven't had any new crashes. Seems like Eden was the Problem. Thank you very much

Guess at which times i now get the "unknown exception"-Error  nearly every time when i do the "fated circle"-Limit Break i crashes while doing the explosion animations. Any Idea what it could be?

[mavirick]

To those who are experiencing the crash:

Set your cache_size=250 or less in \tonberry\prefs.txt. I believe the problem is that the cache is growing too large.

I managed to reproduce the crash today, and the debugging I've been able to do is making it look like the FF8_EN.exe process is trying to use greater than the ~1.75GB of memory allowed to 32-bit processes. It seems the simulated PSX GPU is using more non-GPU memory than I realized... and I may have added a zero when I was doing my memory calculations to determine optimal cache size. 

But on the bright side, debugging this thing has led to all sorts of optimizations that should make things faster once we get the crash sorted out. I'll know more in the next few days, but our solution may be as simple as keeping the cache size between 100-250 (which seems to be about 20 gameplay minutes worth of textures).

[Kazter]

Thanks for the tip Mcindus.  I am going to revert back to the v2.03 Callback and drop my cache size down in Prefs tonight to see if that works.  To me it sounds like a logical problem why only Project Eden appears to crash the game w/ Tonberry.  The texture sizes of the pre-rendered backgrounds are probably the largest sized (memory-wise) textures in the game.

[mavirick]

To me it sounds like a logical problem why only Project Eden appears to crash the game w/ Tonberry.  The texture sizes of the pre-rendered backgrounds are probably the largest sized (memory-wise) textures in the game.

It's not so much that the backgrounds are larger as much as that there are so many of them. There are more than 6,500 replacement textures from Project Eden, and this means more textures in the cache at a faster rate. The game itself doesn't use that many unique texture objects--it recycles them frequently--so we're often removing items from the cache, but the more textures we replace, the faster it fills up. Disabling Project Eden has not worked for everyone and I suspect it ultimately only slows down the time it takes before the crash rather than fixing it.

I'm only sorry I didn't realize this sooner. JeMa's and my debug efforts were focused on small caches because we assumed the problems would arise when the cache filled up, so I was often debugging with a cache_size of 100. It wasn't until I opened up the cache_size and played for 20+ minutes in a transition and texture-intensive area that I experienced the crash myself.

If anyone has used a cache_size of 250 or less and experienced problems, please let me know.

[Kaldarasha]

Do the backgrounds use the same cache as the rest of the textures? If so would be there any benefits if they have their own cache?