Author Topic: [FF7] command line tools for LGP archives - LGP/UnLGP (v.5)  (Read 93332 times)

Fleet Command

  • *
  • Posts: 135
    • View Profile
lgp/unlgp - command line tools for LGP archives
« Reply #75 on: 2009-08-15 04:49:50 »
I don't know C neither C++.
Some C# i may understand as it's quite like vb.net.
If i can understand lgp tools code? well, some parts yes... some dont. I'll try to figure now..
Then your job is much tougher than that of someone like Kramner but still possible. If you are persistent, then let's get started.

First and foremost, you must start learning a bit of C/C++ so that you at least have a grasp of struct data types and of reading and writing from a structured file the way LGP Tools does. (Otherwise, the only way I can help you will be to embed me into your brain.) Why don't you start with Microsoft Express training material on C++? There are quite a few guides there.

After that, I'll lay out the details of encapsulating the file I/O into appropriate classes (which is the tough part of the work) and help you binding them into a GUI (which should be easy, given the fact you already know VB.NET.)
Quote
If u r willing to help me add @ msn.  :-)
I'm afraid that's not an option. I cannot be disrupted during the work hours. We should keep the discussion in this thread (which is relevant) as much as possible. I'll check it everyday after returning from work.

LeeHiOoO

  • *
  • Posts: 128
    • View Profile
lgp/unlgp - command line tools for LGP archives
« Reply #76 on: 2009-08-15 15:04:43 »
@ Fleet Command :

Understanding of structured data types also manipulating structures data : DONE!


Also... I've just realized I can use that for the issue I'm having in my prog in this thread.  :-o
« Last Edit: 2009-08-15 15:06:15 by LeeHiOoO »

Fleet Command

  • *
  • Posts: 135
    • View Profile
lgp/unlgp - command line tools for LGP archives
« Reply #77 on: 2009-08-16 07:06:22 »
OUTSTANDING! I didn't expect you to do it in such a short time.

So, now you have complete understanding of this part of code, right?
Code: [Select]
struct toc_entry
{
char name[20];
unsigned int offset;
unsigned char unknown1;
unsigned short conflict;
__attribute__((__packed__));

struct file_header
{
char name[20];
unsigned int size;
} __attribute__((__packed__));

struct lookup_table_entry
{
unsigned short toc_offset;
unsigned short num_files;
} __attribute__((__packed__));

struct conflict_entry
{
char name[128];
unsigned short toc_index;
} __attribute__((__packed__));

Especially, you do know what does __attribute__((__packed__)) actually do, right?

Good, now let's start with building classes. At first, I advise you to initially build an application that only extracts files from LGP, then expand it to one that also archives files into LGP. But a precaution: I know C# and Delphi very well but I don't remember anything about VB.NET anymore. But I'm going to assume VB.NET is only grammatically different from C#. (At least for our purpose.) If it isn't, let me know and we'll try to resolve any conflict.

(I don't know why my eyes hurts...)

Start by building a test program that opens LGP archive, counts the number of files in it and announces the count in a message box. Then, you'll expand it into a full-fledged LGP extractor. For now, download lgp.c, unlgp.c, lgp.h and MinGW. (You'll need some of MinGW source codes, such as stat.h.) Also, start a Visual Basic project (Windows Form) and put a Menu Bar on it with just a File menu and an Open command. Put an Open Dialog Box into the form and set its properties, so it can open "LGP Archive (*.LGP)" and "All files (*.*)".

Create a DeLGP class. (Are classes case-sensitive in VB.NET too? In C#, they are. In Delphi, they aren't.) Class constructor gets one parameter: the whole path to the LGP file. The constructor checks whether the file exists, whether NTFS permissions on it allows read-only access and opens the file as read-only. (Don't implement the opening of the file yet.)

Then, the main job: Build a LGPStream class as a Structured Storage class and add a File Stream plus the structs in LGP.h to it. You are not going to have the benefit of using packed structs, so prepare for extra job when reading and writing structs. There is a reason for that: The precious Unicode! You are going to read non-unicode strings from archive but write Unicode strings to disk, and vice versa. (Aali's LGP Tools read non-unicode and write non-unicode. You don't do this.)

Once you are done, let me see how you've done this critical part. Meanwhile, I'll be downloading a Visual Basic 2008 Express, just in case.

LeeHiOoO

  • *
  • Posts: 128
    • View Profile
lgp/unlgp - command line tools for LGP archives
« Reply #78 on: 2009-08-16 14:09:35 »
I'm quite sure I'm not doing it right at classes code, but whatever.. that's what I've got so far

LGP_VBNET_TEST
« Last Edit: 2009-08-16 15:26:12 by LeeHiOoO »

Fleet Command

  • *
  • Posts: 135
    • View Profile
lgp/unlgp - command line tools for LGP archives
« Reply #79 on: 2009-08-17 04:53:08 »
Man, you are better than me.

Just started to verify it. I'm not yet through, but I've spotted the first flaw: You defined whatever data that was defined as "char" and "unsigned char" in Aali's code as "char" in your own code, which is wrong. In C and C++ "char", "signed char" and "unsigned char" are only 8-bits long while in .NET, they are 16-bits long and store Unicode characters. Consider using "System.SByte" (or its VB.NET equivalent) instead of "char" and "System.Byte" (or its VB.NET equivalent) instead of "unsigned char".

I'm getting busy right now, but don't worry, if I ever died, you're better than me.

EDIT: Oh, forgot to mention: You'll be using either a bit of type-casting (Byte -> Char or vice versa) or unpacked data read/write of bytes in 16-bit aligned manner. I prefer type casting.
« Last Edit: 2009-08-17 04:55:51 by Fleet Command »

Fleet Command

  • *
  • Posts: 135
    • View Profile
lgp/unlgp - command line tools for LGP archives
« Reply #80 on: 2009-08-17 18:47:23 »
Well, I'm almost through and there are a few issues to resolve.

First and foremost, reading the count of objects in an LGP archive. Your code reads:
Code: [Select]
        Dim fs As FileStream = New FileStream(OpenedFile, FileMode.Open, FileAccess.Read)
        Dim stream As New StreamReader(fs)
        Dim b As Integer
        stream.BaseStream.Seek(12, SeekOrigin.Begin)
        For i = 0 To 3
            Select Case i
                Case 0
                    b += stream.Read()
                Case 1
                    b += (stream.Read() * 256)
                Case 2
                    b += (stream.Read() * 65536)
                Case 3
                    b += (stream.Read() * 16777216)
            End Select
        Next
        stream.Close()
        fs.Close()
        MsgBox("This file Contains " & b & " objects!")
Are you kidding, LeeHiOoO? Why have you added these redundant For...Next and Select...Case statements? There will be absolutely no difference if you delete them, like this:
Code: [Select]
        Dim fs As FileStream = New FileStream(OpenedFile, FileMode.Open, FileAccess.Read)
        Dim stream As New StreamReader(fs)
        Dim b As Integer
        stream.BaseStream.Seek(12, SeekOrigin.Begin)
                    b += stream.Read()
                    b += (stream.Read() * 256)
                    b += (stream.Read() * 65536)
                    b += (stream.Read() * 16777216)
        stream.Close()
        fs.Close()
        MsgBox("This file Contains " & b & " objects!")
However, both codes are inherently erroneous. Check this out: http://rapidshare.com/files/268451194/Counts_of_objects_in_LGP_archives__as_reported_by_various_LGP_Tools.xlsx

This is a report that compares the count of objects inside various FF7 LGP Archives as reported by Aali/Kranmer UnLGP.exe v0.5 and your LGP_VBNET. As you can see, there are inconsistencies. Your LGP_VBNET reports wrong number of items for magic.lgp, flevel.lgp, chocobo.lgp, condor.lgp and world_us.lgp.

Basically, you are calculating the number of items in an unaligned self-composed Little-Endian manner. What's more is that you use a Text File Stream Reader to read bytes from a binary file, while you should have used a Binary Stream Reader to do so. Meanwhile, be careful not to read "Char" in UTF-8 formats.

The correct code to read the number of items is:
Code: [Select]
        Dim fs As FileStream = New FileStream(OpenedFile, FileMode.Open, FileAccess.Read)
        Dim TheLGPFile As BinaryReader = New BinaryReader(fs, Encoding.GetEncoding(1252))
        Dim tmp(12) As Byte
        TheLGPFile.Read(tmp, 0, 12)
        Dim LGPObjectsNum As Integer = TheLGPFile.ReadInt32()
        TheLGPFile.Close()
        fs.Close()
        Return LGPObjectsNum
Here, try this update. I've implemented a debug routine to quickly read and compare the number of items in standard LGP archives of FF7 v1.02: http://rapidshare.com/files/268455952/LGP_VBNET_update_2_Fleet_Command.zip

There is also issues with user-interface too. Consider setting Tab Order properties appropriately. Also add shortcut for the most frequently used menu items (the way I did in case of Open...) but not all menu items need shortcuts. Try to use shortcuts which are familiar for Windows users. (Notepad is a good example that you can use)

And now:
Now, fix the structs and being implementing LGP_Stream class, DeLGP class, their file I/O methods and exception-catching code. Remember: LGP_Stream will have two descendants: DeLGP and EnLGP classes. While LGP_Stream performs generic functions, DeLGP and EnLGP classes will be variants of this class that perform specialized I/O, i.e. DeLGP class opens LGP files that already exist as read only. Meanwhile EnLGP class creates nonexistent empty LGP files with write access.

Currently, focus on DeLGP class and leave implementing EnLGP class for a later time. (You don't want to be buried under a thick layer of unused codes, do you?) For now, focus on creating an LGP extractor and focus more on creating underlying codes than User Interface. (But don't forget about it either.) Start by moving this count-reader code from your form code into LGP_Stream and DeLGP classes.

Remember: These I/O classes should have no User Interface items like message boxes. Have the methods of these classes to return error codes. Your form, which is currently in charge of user interface, and later, maybe other classes, are the only elements which should generate message boxes.

Let me see the results. If they were done properly, the most time-taking part is finished. (You'll probably need no help building the rest of the program on your own but I'll stick around.)

Oh, and whenever you feel you don't need to experiment with the debug code that I added, you can safely remove it and its Listbox1 and Menu item.

Good luck

LeeHiOoO

  • *
  • Posts: 128
    • View Profile
lgp/unlgp - command line tools for LGP archives
« Reply #81 on: 2009-08-18 00:04:13 »
Haha.. I didn't even notice the redundancy... so many codes in my head  :-P

Ok, the thing is... I'm just an amateur dealing with files. The only successful prog I've ever did was a software for Jukebox Machines, which I work with, and I didn't had to handle file streams issues... just wmp ocx stuff.
Also... I may say... I'm an amateur in general programming.... I didn't knew we had classes and functions to deal with binary stream. As I go learning stuff in here... I'm starting to think I'll need to start over the other prog I'm trying to do. So many stuff I didn't even knew about.

Another issue: I don't know nothing about encoding. UTF-8 , ANSI, Unicode... I can't tell the difference.

Can u take a look at my other prog?  :roll:
Of course it'll be a disaster when u open up the code... but it's just to show you how far I got without knowing a lot of stuff...


say there... can we continue this topic on this thread? (new update there)
I think our topic has nothing to do with this thread(until release) and we may be annoying ppl here.

« Last Edit: 2009-08-18 16:41:52 by LeeHiOoO »

Fleet Command

  • *
  • Posts: 135
    • View Profile
lgp/unlgp - command line tools for LGP archives
« Reply #82 on: 2009-08-20 07:17:19 »
^ I see. So, you'll have to boost your knowledge about Unicode. Let's see if I can find anything...

We can continue our dicsussion wherever you wish, even in that thread but I don't think halkun is going to like us doing this. You see, we're not off-topic here. And besides, we're not going to receive much feedback in there. But if you wish so... so be it.

willis936

  • *
  • Posts: 370
    • View Profile
lgp/unlgp - command line tools for LGP archives
« Reply #83 on: 2009-08-24 06:46:18 »
Hi all.
From top to bottom what am I looking at here?
Can anyone give a straight answer on what exactly the binaries do and how to use them?
:3

Fleet Command

  • *
  • Posts: 135
    • View Profile
lgp/unlgp - command line tools for LGP archives
« Reply #84 on: 2009-08-24 07:23:36 »
"Binaries" here means "computer software" or "computer program".

The discussion here is about LGP Tools called "LGP.exe" and "UnLGP.exe" that Aali and Kranmer have created. Now, Aali didn't give us something that can be called a "computer program". Aali just give us their "source codes", instructions for the computer that if undergo a process called "compile", they will become a program, but are otherwise don't do anything. Kranmer finished Aali's work by compiling those source codes and producing "binaries" or the computer programs that you can run.

"UnLGP.exe" and "LGP.exe" (that Kranmer have produced from Aali's source codes) are "command-line" programs, meaning that they can only be run from within Command Prompt (a Windows program) or other such programs, by typing text commands. They are not like normal Windows programs (such as WinZip) that have graphic windows and menus and you can use with mouse.

So, LeeHiOoO decided to change Aali's source code into a normal Windows program. Its a tough job but it is almost finished.

willis936

  • *
  • Posts: 370
    • View Profile
lgp/unlgp - command line tools for LGP archives
« Reply #85 on: 2009-08-24 07:45:01 »
What specifically do the exes do?

Unlgp extracts file x to folder y.  How do we know which file/folder?

Fleet Command

  • *
  • Posts: 135
    • View Profile
lgp/unlgp - command line tools for LGP archives
« Reply #86 on: 2009-08-24 08:33:10 »
"Exe" is the short form of "executable". Exe is another name for computer program.

You specify the file you'd like to extract by giving its full address to UnLGP.exe. That is, by typing it in front of the UnLGP.EXE in Command Prompt. UnLGP.exe does not accept any other parameter. It extracts the LGP archive, which you specified, into the "Current Folder", also known as "Working Folder".

willis936

  • *
  • Posts: 370
    • View Profile
lgp/unlgp - command line tools for LGP archives
« Reply #87 on: 2009-08-24 16:24:43 »
Ah, well there's my hiccup.

I thought there was a step I was missing somewhere.
When I run the exes the command prompt flashes up then terminates.

I figured the exe itself was designed to extract/pack files in the same folder.

I'm running Windows 7x64, admin account UAC disabled.
I'll try different compatibilities.

Hellbringer616

  • *
  • Posts: 1913
    • View Profile
lgp/unlgp - command line tools for LGP archives
« Reply #88 on: 2009-08-24 16:31:28 »
put an LGP in the same folder as unlgp.exe then drag and drop it onto unlgp.

For lgp.exe you need to run it in command line something like this "C:\lgp.exe c:\new c:\battle.lgp"

LeeHiOoO

  • *
  • Posts: 128
    • View Profile
lgp/unlgp - command line tools for LGP archives
« Reply #89 on: 2009-08-25 00:13:31 »
@willis:

If you just open the binaries you won't have time to read the instructions because the command line shows it too fast and close.
If you want to read it, you need to open your command line(Start>RUN>"cmd") and navigate through your directories and open the exe from there...
(ie. type "C:\LGP Folder\lgp.exe" or wherever is your .exe at)

willis936

  • *
  • Posts: 370
    • View Profile
lgp/unlgp - command line tools for LGP archives
« Reply #90 on: 2009-08-25 02:34:02 »
Aha, I got it working now.

Thanks guys!  :-D

Fleet Command

  • *
  • Posts: 135
    • View Profile
lgp/unlgp - command line tools for LGP archives
« Reply #91 on: 2009-08-25 12:28:49 »
As a side note, UAC and admin privileges have nothing to do with UnLGP.exe. UnLGP.exe only needs read permission for the LGP file which it opens and write permission for the folder into which it extracts. (which means almost any folder on your computer except Program Files or Windows is OK for extraction.)

However, if you wish to reduce the amount of alerts that UAC triggers when copying into your Final Fantasy VII folder, I advise you to edit file permissions on "data" folder and give Full Access to Authenticated Users. Now, you can use your Standard (non-admin) User Account to manipulate your LGP files without UAC ever bothering you. No need to worry about security either because viruses never infect data files.

EDIT: The pronoun "you" in the above paragraph refers to general public.

Fleet Command

  • *
  • Posts: 135
    • View Profile
lgp/unlgp - command line tools for LGP archives
« Reply #92 on: 2009-08-27 08:41:03 »
I have created some documentation for LGP Tools v0.5. I hope you guys find it useful.

Here it is:

You are going to need Microsoft Word or Microsoft Word Viewer (free) to view the documentation. But I guess 90% of people should have no problem opening it. But for the other 10%, I've prepared the documentation in alternative formats:

LeeHiOoO

  • *
  • Posts: 128
    • View Profile
lgp/unlgp - command line tools for LGP archives
« Reply #93 on: 2009-08-28 12:09:51 »
Hmm.. I though "LGP Tools" name was already taken by Ficedula a long time ago. I don't think Aali came up to a name for his version. *confused*
Maybe he should, some people may get confused thinking is the same program. Unless it's the generic name "lpg/unlpg".

Nice documentation though! Gratz
« Last Edit: 2009-08-28 12:17:32 by LeeHiOoO »

Shuffle

  • *
  • Posts: 38
    • View Profile
lgp/unlgp - command line tools for LGP archives
« Reply #94 on: 2009-08-28 15:08:50 »
I have a problem with unlgp:
1.lgp and unlgp.exe in C:\LGP\ folder
i was write a .bat file
Code: [Select]
CD /d "C:\lgp\1"
UnLGP.exe "C:\lgp\1.lgp"
pause
when I was open this .bat file command line say
Code: [Select]
C:\lgp>CD /d "C:\lgp\1"

C:\lgp\1>UnLGP.exe "C:\lgp\1.lgp"
'UnLGP.exe' is not recognized as an internal or external command,
operable program or batch file.

C:\lgp\1>pause
Press any key to continue . . .
help
« Last Edit: 2009-08-28 15:11:46 by Shuffle »

sl1982

  • Administrator
  • *
  • Posts: 3764
  • GUI Master :P
    • View Profile
lgp/unlgp - command line tools for LGP archives
« Reply #95 on: 2009-08-28 15:44:42 »
I have a problem with unlgp:
1.lgp and unlgp.exe in C:\LGP\ folder
i was write a .bat file
Code: [Select]
CD /d "C:\lgp\1"
UnLGP.exe "C:\lgp\1.lgp"
pause
when I was open this .bat file command line say
Code: [Select]
C:\lgp>CD /d "C:\lgp\1"

C:\lgp\1>UnLGP.exe "C:\lgp\1.lgp"
'UnLGP.exe' is not recognized as an internal or external command,
operable program or batch file.

C:\lgp\1>pause
Press any key to continue . . .
help



do this instead
CD /d "C:\lgp\"
UnLGP.exe %cd%\1\1.lgp
pause
« Last Edit: 2009-08-28 15:51:39 by sl1982 »

Shuffle

  • *
  • Posts: 38
    • View Profile
lgp/unlgp - command line tools for LGP archives
« Reply #96 on: 2009-08-28 16:22:03 »
i was wrong
CD /d "C:\lgp\1"
UnLGP.exe C:\lgp\1.lgp
pause

sl1982

  • Administrator
  • *
  • Posts: 3764
  • GUI Master :P
    • View Profile
lgp/unlgp - command line tools for LGP archives
« Reply #97 on: 2009-08-28 16:24:56 »
i was wrong
CD /d "C:\lgp\1"
UnLGP.exe C:\lgp\1.lgp
pause

that wont work. change the first line to this if you want it to work
CD /d "C:\lgp"

or you can put the unlgp.exe into the C:\lgp\1\ folder

Shuffle

  • *
  • Posts: 38
    • View Profile
lgp/unlgp - command line tools for LGP archives
« Reply #98 on: 2009-08-28 16:28:44 »
i want to the unlgp extract the lgp archive into "C:\lgp\1" folder, its possible?

sl1982

  • Administrator
  • *
  • Posts: 3764
  • GUI Master :P
    • View Profile
lgp/unlgp - command line tools for LGP archives
« Reply #99 on: 2009-08-28 16:34:03 »
i want to the unlgp extract the lgp archive into "C:\lgp\1" folder, its possible?

unlgp will extract the files into whatever folder unlgp is in
so put unlgp and the lgp into C:\lgp\1