Wow! I wrote about this soooo ling ago. (Like when I had 5 or 6 posts to my name. I was trying to make a quake movie and was working on ripping the characters from FF8. Actually I was attempting to decode the .x files for the battles, as there seemed to be a 3d version of every scene in FF8.
The nutshell most if it is pattern finding. Also remeber that programmers and inharently lazy and much that you can find about a file format is common scense. Now I usally use a data visualizer to see "chunks" in a file, find a particular chunk, change a varible, and run. Also if you have a hex editor, try different witdths and data groupings and see if you find any patterens. I don't do memory hacking unless it's on a PSX. Look for my originak post and you'll kind of get it.