Author Topic: Bootleg bugs and feedback  (Read 4432 times)

Xiaopang

  • *
  • Posts: 5
    • View Profile
Bootleg bugs and feedback
« on: 2013-03-21 17:29:11 »
I recently started playing FF7 again and found out about those awesome mods you guys created here. Seeing that there was also supposed to be a tool that helps with setting everything up, this looked like a piece of cake. So I decided to delay my gaming hour for the day just to grab some mods and install them quickly and start playing in an hour...or so I thought. 24 hours later, dozens of installation attempts on 4 different operating systems, error tracing, batch file debugging and a good amount of manual polish finally got me the game to the point I expected it to have 24 hours earlier.

While I'm glad that I finally got what I wanted, I have to say that bootleg is one of the most frustatingly and annoyingly written software I have ever seen and I don't say that lightly. Bootleg shows a blatant disregard for programming standards and thus creates trouble on all fronts. I'm not here to bash the program though. It's kind of obvious that this is the work of a beginner and we all started at some point. So I'm here to  reveal some of the many bugs I encountered and give some feedback what should definitely be improved to avoid trouble and so that lesser capable users in future will not have to pull their hair out over the shortcomings of the current release.

1. Bootleg doesn't clean up after itself. A dozen failed attempts left my temp-folder overflowing with *.tmp-folders and the Bootleg_Setup folder taking up gigabytes of space.

2. No file verification BEFORE installation. There are some rudimentary checks to see if the program has everything it needs to start, but it leaves out its most important features: The mods themselves. Having a mod missing will result in a botched installation. The user will have to sit through the lengthy installation procedure again. This is especially frustrating if the failure occurs right at the end.

Now before you blame the user for not checking the presence of the mods in the frontend, that doesn't help much. In my case bootleg reported three mods missing, the frontend told me they were present. The batch file contains the names of all mods and there are typos in them resulting in the described problem. Example:

The frontend checks for "Trish(aka Mike)s custom model pack.7z" and that's also the file offered for download. Bootleg checks for: "Trish(aka Mike)'s custom model pack.7z" and errors out just minutes before finishing....great.

3. More user control: The aforementioned problem wouldn't be much of a biggie if bootleg would actually WAIT upon encountering a missing file and would give the user the chance to name it properly or download it. Even better would be a combined data source for the batch and the frontend. Why not have the gui extract the proper mod-names and put them into a text file and read that back into bootleg to actually get up to date filenames? You can parse it with a for-loop. Right now two sources have to be updated which is not just inefficient, but also error prone as you can see from the current bootleg. Furthermore, this would make your code easier to maintain.

4. Just because you can write spaghetti code doesn't mean you should. Structure the source by using external batches and scripts.
There's also a lot of headroom for clean up. 600kb for a batch file is overkill. Many years ago I wrote a batch installer that would actually make use of functions stored in other batch files. This batch is screaming for a similar system. Extracting an lgp, inserting new content and packing it back up is a prime example of such a possible function. This batch could very well be smaller than 100kb. Moving the variable definitions and text messages into separate files could clean up the installation code tremendously and would probably limit it to a couple of kb. Now which code do you think is easier to maintain and update, your 600kb monster or a streamlined 20-50kb batch?

5. Introduce a debug mode and learn how to pipe stderr and stdout to a txt-file for logging purposes. It's nice that you try to manually generate a log, but it fails on all accounts because it only is updated if the function between those "echo"-updates are actually accessed. I had loops not finishing and other parts failing...never got an error message. In case of the "missing" mods your log gave me the following info:

Searching for "Trish(aka Mike)'s custom model pack.7z" in "F:\Bootleg040\"
Extracting "Trish(aka Mike)'s custom model pack.7z"

How can you extract something that you reported missing earlier? While your generated log messages are better than nothing, they do not actually show what is really going on and are misleading to the unsuspecting user who encountered errors and doesn't have a clue where to look. stderr and stdout give you the actual output that is dynamical and thus accurate. Refrain from writing your own interpretations. That doesn't work out as you'll see in a later example.

In the above case, I had to run the bat manually without echo off to see what was going on. A smarter way of implementing error tracking would be NOT to use "echo off", but a switch like %1. Add a debug option to the frontend and let it decide whether to send echo off as a switch or not. It's also possible to pipe the raw output to a txt-file and show the filtered result to the user.

6. ulgp.exe not found! I encountered this several times. Workaround: Extract sprinkles.7z manually and place it in the proper subfolder. For some reason I didn't care to explore why bootleg extracted sprinkles.7z AFTER needing to access ulgp.exe. It was able to access it a couple of times and then threw the error after/before determining the game language. Perhaps a misplaced/ambivalent del-command?

7. Refrain from using external tools and use API as much as possible!!! The command line syntax has been changed massively over the years. Some changes are small and some big, but there are always some. XP behaves differently in this regard than Win7 or Vista does. Besides, it's also good practice and saves space.

Bootleg relies on way too many external tools to do a simple job. Why do I have to bloat up my clean XP installation of 1.5GB to 3.5GB just by having me install Net Framework 2? For a frontend that could have been created in every other self-reliant IDE and that does nothing more than start a batch file? If a program demands to install an extension that doubles the size of an operating system it better be the holy grail of coding. Well, this is neither a John Carmack game nor a h265 encoder, so why the hell the added bloat and complexity? And yeah I know that Framework is included in Vista and up, but there are plenty of people who streamline their windows installations or run an unspoiled XP. Some users will probably even have tried to run this from a WinPE in lack of another alternative after encountering errors.

Why does bootleg use xcopy instead of copy? Not every Windows has xcopy! Copy is part of the command kernel and thus a native command. As far as I can tell, Bootleg also doesn't utilize features that would demand xcopy over copy, so why the added complexity? Just makes it more error prone. If you really must use external tools, then make sure there are no native alternatives and then supply the third party alternative (robocopy would be one in this case)...Also avoids syntax differences of the same command among the operating systems (check out choice.exe syntax for win98 xp and win7 for example).

Bootleg doesn't wait or give the user the choice to fix things before it plasters over a failed installation attempt, so it should better make sure that the installation is as fail-safe as possible. In my case BootlegSFX.exe was stuck for 2 freaking hours. On this forum someone already complained about the error, but received nothing but off-topic remarks. Killing the exe allowed the script to move on, but what if this had been a mod that needed to be incorporated into an lgp-file? In case of BootlegSFX you should rely on a proven tool like xdelta instead of using a program that fails without giving an error code! Same goes for the other automated installers. Who knows whether they actually do finish successfully? Does bootleg check whether their installation conditions are fulfilled prior to starting them and checks the success afterwards? No, it doesn't, so it's no wonder I ended up with a 7GB installation that looked like the unmodded game the first time around, because the files were probably dumped at the wrong location. Sprinkles.7z contains inno unpack, yet you don't seem to use it to install the automated installers manually. Why?

8. "Access Denied" - And here's why you should not rely on your own error messages. They are inaccurate regarding the way you test for their conditions. The fact that I didn't have xcopy on my system gave me that very error sending me off the track when looking for a solution. Of course being an admin on an open system without UAC with an installation directory on a non-system drive didn't help at all, if this error is the conclusion of a failed file copy check...

Not being able to copy bootlog_readme.txt could result in a plethora of errors. Missing admin privileges is just one of many: What about intercepting AV-Software? What about xcopy erroring out for some reason or the txt not being accessible for other reasons? What about a full hard drive or xcopy choking on the path name? There's a plethora of reasons why a file can't be copied.  You want to see if you have access, then access the folder using the API: type nul >admincheck.txt
If EXIST admincheck.txt... see, this cuts out the whole can of worms that's opened with using external tools...

9. Why is the batch-file not in the bootleg-folder, but stashed away in the gui's executable? As backwards and ineffecient batch-files are, they at least offer the possibility to be modified quickly and thus allow the user to correct errors himself. But this advantage is killed by making the batch unavailable unless the user intercepts the command line from the frontend that is passed to it and feeds it to a modified version of the batch he has to grab from the temp folder...

10. And what the hell is up with "The game files might be damaged." They might? Are you kidding me? Are you really telling me that after downloading 10GB of mods and Bootleg wasting an additional 17GB of temp space for its installation procedure, it isn't capable of copying those couple of hundred megabytes of original files into a safe location just in case something goes wrong?? And especially if you actually expect corruption of a proper installation in case of an error?? Seriously, wtf?? This is probably the worst design flaw of all, especially since there is no warning. In this day and age, people expect that files are backed up before being overwritten. This behaviour comes straight out of the 80s and should stay there.

Compared to this behavior, the oversight to inform the user to start the game with a batch after the installation instead of the normal game exe is almost forgettable if it wasn't such a nuisance to actually pester Google once more to find out why the game wouldn't play any music. It's still a nuisance and could be remedied with a simple message after the installer finishes. Even better would be a link in the start menu that replaces the original one which results in the musicless gameplay.

11. Use an intelligent file and execution structure:

What's the point of supplying sprinkles.7z as self-extracting executable and having it extracted  to a new archive each and every time the user starts bootleg? Supply sprinkles.7z without sfx-code and extract from it directly. Saves the user 300MB per (crashed) installation and also cuts down installation time.

Supply the batch directly!

Check if all files are available and perform ALL other checks including the right of elevated access BEFORE starting to extract a gigabyte of data that is left behind if the installer errors out early on.

Make Backups of ALL files that are being modded and automatically create a cleanup script in case the user wants to revert changes. This is dead simple. Even if the installation has been botched. The user might want to salvage some modified files. Then he can run a cleanup batch that copies the backed up files over the modded ones --> no messed up game directory in case something goes wrong! Personally, I'd never access the game directory, but create a hardlinked clone and work on this one. If anything goes wrong, the user still would have a vanilla version to fall back on.

Try to implement some kind of progress bar, preferably numbers based. This makes it easier for people to see how far into the installation they are. Also, use a more compact way of displaying data. People can't scroll endlessly in the window, so less is more:

Installing modname 1/22... completed
Installing modname 2/22... completed
Installing modname 3/22... error: File not found

Once the installation is done, remove your own junk and check for it. It's so bloody easy to do and yet you don't do it:

IF EXIST %temp%\Bootleg_Setup rd %temp%\Bootleg_Setup

Same goes for the temp-dir the batch is currently run from.

BTW, consider the code snippets as being pseudo code.

Anyway, kudos for at least trying to automate this painstaking process. Trying to install these mods manually yields hardly any result for people having never modded the game before, especially since hardly any mod comes with a readme to explain how it's installed - something that has been standard ever since Doom started the whole modding craze. Instead it looks like modders rely on Bootleg to do the job for them which makes it so much more crucial to get the source sophisticated enough to make errors the exception, not the norm. Not every mod installer has to be so close to perfection as The DarkMod, but at least the rudimentary functions should work well without leaving the user frustrated and head scratching. I hope this feedback helps to improve bootleg to this stage.
« Last Edit: 2013-03-21 17:35:31 by Xiaopang »

FF7Remastered

  • *
  • Posts: 11
    • View Profile
Re: Bootleg bugs and feedback
« Reply #1 on: 2013-03-21 20:00:48 »
Hi Xiaopang,

I understand your frustration with Bootleg, however it is still the best all in one installer out there and is already a huge improvement compared to previous all in one FF7 mod installers. But you feedbacks will definitely improve Bootleg in the next version.

Anyway, I suggest that you can also considering manually install the mods. It sounds as if it would be take more time than using Bootleg, but it is actually quite simple. Most of the FF7 mods come with its own installer and can create backup or uninstall on its own. You can find all the mods from the Big list of mods thread. If you just wish to have all the mods that upgrades the original gameplay, then you can follow my instruction:

1. Install Aali graphics driver
2. Install Menu Overhaul
3. Install Unified Model Installer (choose DLPB suggestion)
4. Now you should have a folder named Mods\DK in your FF7 folder. From now on you can download every mods that are textured related such as Felix Battle scenes, Felix World textures, Grimmy Magic Textures, Omzy Field Packs and simply just paste them into the DK folder
5. Install FMV restoration patch or any FMV improvement patch
6. Install FF7music

Hope this helps

Xiaopang

  • *
  • Posts: 5
    • View Profile
Re: Bootleg bugs and feedback
« Reply #2 on: 2013-03-21 20:35:05 »
I understand your frustration with Bootleg, however it is still the best all in one installer out there and is already a huge improvement compared to previous all in one FF7 mod installers.

Yes that's true. The only reason I stuck with it was because the mods I downloaded through the links given in the frontend didn't seem to include text-files on how to install them. I take it that those are actually repacks that had them removed so that they could be installed into a single location in the game directory. But for them to show up the installer has to finish successfully.


But you feedbacks will definitely improve Bootleg in the next version.

I sure hope so :)


Anyway, I suggest that you can also considering manually install the mods. It sounds as if it would be take more time than using Bootleg, but it is actually quite simple. Most of the FF7 mods come with its own installer and can create backup or uninstall on its own.

Interesting! I take it then that the mods Bootleg uses are indeed repacks of self-installing mods. If I had known that earlier I wouldn't have tried so desperately to get Bootleg to work. Thank you also for your short summary of how to install these mods. By now, I have everything running and couldn't be happier :)

PitBrat

  • *
  • Posts: 1376
  • Change to feed the machine.
    • View Profile
Re: Bootleg bugs and feedback
« Reply #3 on: 2013-03-21 22:34:26 »
Thank you for the detailed critique.  Those are some great ideas.
You're right, this started out as a simple first time learning exercise and grew into a monster.
There was really no plan or outline for Bootleg, so it's just cobbled together.
The mess is compounded by mods that come and go and change frequently.
Shucks, I wish mod makers created the mods with Bootleg in mind.
I have to accommodate all of the funky filenames and varied installers.

I'll definitely use your critique as a guide for improving the next version.
Your suggestion for maintaining one list of mods and checking for the files before installation is a priority.
I have a lot of work to do.

Xiaopang

  • *
  • Posts: 5
    • View Profile
Re: Bootleg bugs and feedback
« Reply #4 on: 2013-03-22 00:41:36 »
Thank you for your kind reply :)

Seeing that I wrote a mod manager/installer myself a while ago I know the problem of dealing with filenames. There are actually a few ways to deal with them:

Have the mod makers include a filename that describes the mod in a specifically formated way similar to the formerly popular file_id.diz-files. This way the archive can have any name it wants as long as file_id.diz is properly formated.

You can also use a tag that can be applied either to the name of the archive or a file inside it like a readme or just a zero byte file carrying it for the sake of information, e.g. "[200310].txt" or "[200310] - Field - Cloud with Sword.zip". Then all you need is a catalog to pull the name from and you can ignore the archive's name.

This of course requires wide support from the mod makers and a general naming convention, but as soon as such a standard is established, people will follow. To help people migrate you could offer a tool that automatically packages the mod and automatically generates a description file. This way you have total control over what you actually need to keep it hassle free :)

Most of these suggestions are actually being used with great success by the Thief mod community.

The frontend apparently is capable of identifying the mods. Probably due to the fact that the modlist is fed manually into it. It would be easy to have the list exported and used by the batch directly. Another idea would be to allow the mod makers to add their mods to the installer themselves, for example by using an ini that lists them all. This saves you work and should make it easier to maintain the overall project.

Anyway, using a description file is probably the easiest and safest way to go. I can recommend sfk (swiss file knife) for processing text files via batch file/command line. It offers extensive text search and manipulation options, so you're not stuck with tracing text with for-loops. I relied heavily on it when I wrote a batch installer years ago.

Of course batch files offer only so much possibilities. This whole task would be easier to tackle with a high-level programming or scripting language. May be the guy who wrote the frontend could employ the installation routines :)

Kaldarasha

  • *
  • Posts: 2449
  • Prince of Model Editing
    • View Profile
Re: Bootleg bugs and feedback
« Reply #5 on: 2013-03-23 15:40:06 »
I'm looking forward to it.  ;D
A standard is a great idea, this way we could provide models to bootleg directly and the users don't have to wait for the next release of bootleg to have the latest mods.
This could make bootleg to a more dynamic installer.

Xiaopang

  • *
  • Posts: 5
    • View Profile
Re: Bootleg bugs and feedback
« Reply #6 on: 2013-03-24 12:10:15 »
For a second I thought about giving it a shot and writing an installer myself, because I'd like to implement said ideas in a very automated fashion. Programmatically that is not much of a problem, but I lack the insight into the FF-mod community to actually get going. If there was some support by the community, I could probably have a rudimentary beta in a couple of days.

Kompass63

  • *
  • Posts: 695
    • View Profile
Re: Bootleg bugs and feedback
« Reply #7 on: 2013-03-24 16:22:45 »
Hi Xiaopang,

You can also take a look at my FF7_GameConverter.
It contains a few tricks that may be of interest to the game.


Xiaopang

  • *
  • Posts: 5
    • View Profile
Re: Bootleg bugs and feedback
« Reply #8 on: 2013-03-24 16:51:27 »
Thank you for the hint :)

From what I can see from  the thread FF7 Game Converter is a language converter? If so pretty cool. Looks like there's more to mod about the game than textures, music and video.