Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Messages - Chthon

Pages: [1]
I think you should know that FFNx will soon be able to automatically adjust colors to approximate the brighter and more vibrant color gamut of the 1990s Japanese television sets that FF7 and FF8 were designed for. (Preview here.) This will probably accomplish the same goals as your manual color corrections.

I wish I had known about your mod earlier. Since this might save you a lot of work, I feel badly for not telling you sooner.

Graphical / Preview of NTSC-J Mode for FFNx
« on: 2023-06-14 03:14:25 »
"NTSC-J mode" is a forthcoming feature for FFNx. If NTSC-J mode is enabled, FFNx performs a conversion to simulate the color gamut of the 1990s Japanese television sets that FF7 and FF8 were originally designed for. The resulting colors are more faithful to the original artistic intent, and generally brighter and more vivid.
This mode is appropriate for:
  • Vanilla FF7/FF8.
  • Mod assets that are upscales of the original assets.
  • Mod assets made from scratch where the modder used the original assets as a color reference without doing a gamut conversion.
This mode is not appropriate for mod assets made from scratch that truly use the sRGB color gamut. Such assets can be made suitable for this mode using this converter program.

If you would like to try out a preview of NTSC-J mode:
  • Make a backup of your FFNx files!
  • Download this preview build and extract to your FF7/FF8 directory, overwriting files.
  • Make sure that FFNx.toml includes enable_ntscj_gamut_mode = true

Some notes on mod compatibility:
  • As expected, SYW's upscales look great in NTSC-J mode.
  • Ninostyle models also look good. These hew closely enough to the original palette that I'd say NTSC-J mode is more "right" for them than sRGB mode.
  • ESUI and Finishing Touch don't look good. These were made from scratch in the sRGB gamut. So I've made some unofficial versions using inverse conversions that should look like the sRGB originals when used in NTSC-J mode.
  • (That link also has a "compatibility pack" mod that forces some mods to work together in the way I want them to, includes color range and banding fixes for SYW's videos, a new "buster" screen, SNES-style button icons, and misc stuff.)
  • Avalanche Arisen is a mixed bag. Uprisen increased brightness and saturation in many places, which is sometimes too much for NTSC-J mode, and sometimes too little for sRGB mode. I'm preparing an inverse conversion to see what that looks like.
Github-inclined people can review the pull request including NTSC-J mode here.

Still to-do

8. Look into a way of dealing with the save location and sharing of save files between 1998 version/The Reunion and Steam.

I have some thoughts on this. Though first a couple disclaimers:

  • I'm not sure I fully understand the problem. From your post here, I gather that Steam is wantonly deleting files if it doesn't like their checksum. I'm assuming that you don't mean the checksum internal to the save file itself but rather the checksum for the save file noted in Steam's metadata.xml file. If I'm incorrect about that, then everything I'm about to say should probably be ignored.
  • I'm not in a position to test any of this because I don't have the Steam version (still have my discs from 1998) or Windows 10 (I use Linux).

Anywho, I think the following would work:
  • Add two fields to Reunion's config file:
    • LinkSavesWithSteam (bool, default: false)
    • SteamUserID (string, default: this_is_not_a_valid_steam_id)
  • Add the following behavior to the installer:
    • Check if %userprofile%\My Documents\Square Enix\FINAL FANTASY VII Steam\ exists. If not, stop.
    • Ask the user if they want to link their Reunion saves with their Steam saves. If so, set the LinkSavesWithSteam option in Reunion's config file. If not, stop.
    • Enumerate the immediate child subdirectories of %userprofile%\My Documents\Square Enix\FINAL FANTASY VII Steam\. If none, stop. If one, select that. If more than one, ask the user to select one.
    • Save the name of selected subdirectory to the SteamUserID field of Reunion's config file.
    • Check if the selected Steam save directory contains any save files. Also check if the '98-edition save directory contains any conflicting save files. As appropriate, ask if the user wants to keep all, none, or only the non-conflicting Steam save files.
    • Move the indicated Steam save files to the '98-edition save directory.
    • Perform the same steps as the modified launcher behavior, described below.
  • Add the following behavior to the launcher, to be performed AFTER ff7.exe returns:
    • Check LinkSavesWithSteam in the Reunion config file. If false, stop.
    • Check if %userprofile%\My Documents\Square Enix\FINAL FANTASY VII Steam\ exists and contains an immediate child directory matching the SteamUserID stored in the Reunion config file. If not, stop.
    • Iterate over the save files in the '98-edition save directory. For each file, create a symlink in the Steam save directory to that file, overwriting if necessary.
      • NOTE: It's important to symlink each individual file rather than to symlink the containing directory in order to make sure Steam can't muck with the real save files. A process with access to a directory symlink can delete files within the target directory, but a process with access to a file symlink can't delete the target file. (At least that's the way things work in operating systems that properly implement symlinks. I've heard rumors that in certain circumstances Win10 will allow you to delete a target from the link, but I haven't cared enough to look into them.)
    • Rewrite Steam's metadata.xml file. How? It looks like this source file from Black Chocobo contains most of the answers we need:
      • Crib the overall xml structure from an existing file. Looks simple.
      • Timestamps are actually quite a pain. We're going to have to keep a persistent memo file on the side to figure out which slots have changed.
        • Ignoring the first 9 bytes, chop the save file up into 15 chunks of 4,340 bytes representing the individual slots. (Those numbers came from here.) For each chunk:
          • Check if the first two bytes equal 0x4D1D. If so, the slot is empty, so the timestamp should be blank.
          • Otherwise, hash the chunk (using whatever hash algorithm you like) and compare against the prior hash from the memo file.
          • If the hash is unchanged, use the prior timestamp from the memo file.
          • Otherwise, the new timestamp is the file's last modified date (accessible via system call) expressed as milliseconds since the Unix epoch.
        • Now go rewrite the memo file with the updated hashes and timestamps for next time.
        • (I'm not really sure how to deal with the initial case when we don't have a memo file yet. I guess for files moved from the Steam save directory, we'll need to preserve whatever the original metadata.xml said; and for files originating in the '98-edition save directory, just timestamp all the nonblank slots with the file's last modified date and live with whatever errors that creates. (I guess you could get fancy and parse the game time out of each slot to invent believable in-order timestamps...))
        • (Since this is such a pain, I'd be tempted to experiment with what Steam does when the timestamps are wrong. If the consequences aren't that bad, I'd put less effort into getting them right.)
      • To calculate the "signature," start with the file content, append the Steam user ID encoded as ASCII (or ISO 8859-1, same difference for numerals), take the md5 hash of that, and express the hash output as lowercase hex.

To partially answer my own question:

The problem with the conversion software is caused by the fact that these files have multiple palettes. So I need to find (or write) conversion software that can handle multi-palette .tex files. Can anyone point me to something?

I was wrong about Reunion R06 ignoring some .png textures. What's really going on is that the Enhanced Stock UI mod I was borrowing from doesn't have .png textures for the menu font because it uses hext files to make the menu use the field font. Unfortunately, that hext isn't fully compatible with Reunion R06, so I'm going to have to make new font textures. But at least it can be done with .png and I don't need to convert back to .tex.

Which is great because, as per this thread, it's not possible to upscale the font .tex files because sometimes the .exe grabs rectangles from them using fixed integer offsets and dimensions (rather than offsets and dimensions expressed as fractions of the texture dimensions).

Is there a tool for converting the menu .tex files for FF7 PC to a readily-editable format, then back again? I've tried the following:
  • TexTool -- produces png output with wrong colors.
  • FF7 Tex Image Tool -- crashes
  • img2tex -- crashes

(This issue seems to be limited to the menu textures. I tried TexTool to extract seffect1 and it seems to have worked perfectly.)

Some context:
My ultimate goal is to get a smooth font into Reunion R06+. I started out by plopping in the .png textures from Enhanced Stock UI, and quickly ran into two problems:
  • The text in the dialogs looks like it was kerned by a madman. Probably because FF7 has no kerning functionality and the widths of the replacement font's glyphs aren't in proportion with the widths on the original glyphs.
    • The solution for this is obvious, if tedious: Extract and convert the original textures, vector trace them in fontforge, make up a .png using the traced glyphs. The problem is that I don't have a tool that can properly convert the original textures.
  • It looks like Reunion R06 is ignoring the supplied .png textures in the \CUSTOM\ folder in favor of some .tex files in \BASE\New-Translation\. I'm guessing everything run through FF7's menu module is doing this.
    • It seems that the solution would be to convert Reunion's .tex files to see what changes were made, incorporate those changes into my .png files, then convert my .png files into .tex and replace Reunion's files. The problem, again, is that I don't have a tool for converting to .tex. (At least, if it can't get the conversion right the other way, I don't trust it to get this way right.)

Pages: [1]