STR (Movie) file format?

[halkun]

Could some willing young chap (or chap-ette) write me a specification on the STR movie format. If you want, you can use my PSX doc as a base for it, it's "sorta" in there The format will be helpful in gears....

I DON'T WANT A CUT AND PASTE OF SONY'S DOCUMENTATION!

That bit is reeeeeally important. How you get the information is how you get it, but I can't use Sony's docs. I can use a user generated specifacation based on whatever information that user finds available to him/her, like the source code to an STR player or someone else's specs they can rewrite.

The movie files are encoded via mode2(XA) , this means every sector has an extra 256 bytes that can't be copied normally (I.E. drag and drop from the cd-rom to the hard drive)

That will start you off I think...

[Micky]

I've been looking at that a few weeks ago.

FF7 has two formats: "STR" and "MOV". Both are stored in XA-format, so you have to get the raw data from the CD. I have a simple (Mac) program that goes through the ISO9660 structure and dumps all the raw sectors for movie files.

STR is just data sectors, so if you have raw CD-rom sectors you have to skip 16 byte for the sector header and 8 byte for the XA-data header.

MOV have other sectors interleaved (XA-Audio?). You still have to skip 16+8 byte in each sector, but you additionally have to filter out some sectors that don't contain movie data.

Each sector contains a 32 byte header, where you can easily identify stuff like resolution (320x224), frame and sector number.
But so far (even with your docs) I was not able to decode the mdec data. I tried it with ffmpeg, which includes some STR and MDEC code, but that didn't work.

I've got some ideas of making a higher-quality decoding from the MDEC macroblocks, like b-spline interpolation of the Y/Cb/Cr blocks separately and some post-processing. But so far I've got no useable data.
----
ffmpeg is available here:
http://ffmpeg.sourceforge.net/
The headers claim that they have the mdec/str information from psx emulators.

[halkun]

Thanks that was really helpful....

However it's more complex than I thought. Anything that uses more than 10 letters to discribe a common fuction is a bit too advanced for me.

For example MP3 -> "Psycoacustics"
Mpeg compression -> "Quantization factor + Direct Current refrence"

I'm guessing it's not unlike the data format that it taken by the MDEC. (From my PSX doc)

Code: [Select]


header
macroblock
...
macroblock
footer


The header is a 32 byte word.
0x3800 |    size

The Macrobocks are further broken up as follows
Cb block
Cr block
Y0 block
Y1 block
Y2 block
Y3 block
Cb,Cr The color difference blocks
Y0,Y1,Y2,Y3 The Luminescence blocks

Within each block the DCT informaton and RLE compressed is is stored.

DCT
RLE
...
RLE
EOD

DCT data has the quantization factor and the Direct Current (DC) reference

Q Quantization factor (6 bits, unsigned)
DC Direct Current reference (10 bits, signed)


RLE Run length data

LENGTH The number if zeros between data (6 bits, unsigned)
DATA The data (10 bits, signed)

EOD End Of Data(Footer)
0xfe00
Lets the MDEC know a block is done. The footer is also the same thing.


I was wondering if anyone could expand on what I have and put it into some comprehensable terms.

Also a quick glance at the code shows that an STR file can have more than two channels (Audio, video), and I fear that there might be camera movements encoded in a 3rd channel. If I wanted my camera to sync with the video, that's where I'd put it.

Also are the STR videos 24 bit all the way through? 15 but all the way through? I ask because the video in the PSX version switches color depths on the fly, but my keen eye can't tell from what to what, only that it goes from a higher color format to a lower one.

I don't know if it's the MDEC that's able to take in 24 bit video and convert it on the fly, or if the movie actually changes format halfway through.

The file seem to be in RIFF format, with the data block being called "CDXA" (I knida wish I can see this for myself, but I can't mount a PSX disk under linux because it's the wrong format.)

Can you hit me with a clue stick?

[Micky]

Thanks that was really helpful....

Let's assume this was not meant sarcastic...

I was wondering if anyone could expand on what I have and put it into some comprehensable terms.

From what I understand, the mdec data you describe can be directly decompressed and converted into rgb in the MDEC decoder. But it looks like there is an additional step required for the CPU to convert the data stream into a format the MDEC can use. The ffmpeg code seems to do some Huffman decompression, but I didn't find many resources on this.

I assumed, as you wrote the documentation, that you're familiar with the terms. So a few explanations:
Y/Cb/Cr colourspace is a colour representation used for video encoding. Y is a grayscale picture, Cb and Cr are used to "colour" this grayscale picture. This conversion is done by multiplying this vector of 3 values with  a 3x3 matrix, which results in an RGB vector.
The advantage of this transformation is that you can store colour at a lower resolution than the grayscale data, because the human eye is more sensitive to changes in brightness than in changes to colour.

The DCT (Discrete Cosine Transform) transform separates high frequency from low frequency content, and then you can store the low frequency content with fewer bits. Human eye again: You are sensitive to high frequency changes in your view, because that is how to separate different objects.

Also a quick glance at the code shows that an STR file can have more than two channels (Audio, video), and I fear that there might be camera movements encoded in a 3rd channel. If I wanted my camera to sync with the video, that's where I'd put it.

That seems to be used in MOV files. The frames in the STR files all begin with the 0x3800 magic number for MDEC data.

Also are the STR videos 24 bit all the way through?

MDEC data is not stored in a specific bit depth, as mentioned above it is not even in RGB.

The file seem to be in RIFF format, with the data block being called "CDXA" (I knida wish I can see this for myself, but I can't mount a PSX disk under linux because it's the wrong format.)

That seems to be an artefact of the extraction. The STR files on the CD are just a sequence of XA-format sectors, each sector with a STR header.

You will probably have to write some advanced software on Linux to access the movies. If you just open the /dev/cdrom device you will just have access to the normal data sectors. I'd have to look up how to configure the driver to return raw sectors. But have a look on http://freshmeat.net , maybe there is already software available for that.

[halkun]

No, it was *NOT* sarcastic... I really thank everyone effort in helping me with Gears. It was the internet making words "mean" again.

The MDEC documentation was one of the last things to go into my doc. I was being helped by PSX programmers who posted on a psxdev mailing list, (That sony shut down after my doc became public) The told me long ago how it worked, but keep in mind it's been over 4 years

[Cyberman]

Thanks that was really helpful....

However it's more complex than I thought. Anything that uses more than 10 letters to discribe a common fuction is a bit too advanced for me.

For example MP3 -> "Psycoacustics"
Mpeg compression -> "Quantization factor + Direct Current refrence"

I'm guessing it's not unlike the data format that it taken by the MDEC. (From my PSX doc)

I was wondering if anyone could expand on what I have and put it into some comprehensable terms.

Also a quick glance at the code shows that an STR file can have more than two channels (Audio, video), and I fear that there might be camera movements encoded in a 3rd channel. If I wanted my camera to sync with the video, that's where I'd put it.

Also are the STR videos 24 bit all the way through? 15 but all the way through? I ask because the video in the PSX version switches color depths on the fly, but my keen eye can't tell from what to what, only that it goes from a higher color format to a lower one.

I don't know if it's the MDEC that's able to take in 24 bit video and convert it on the fly, or if the movie actually changes format halfway through.

The file seem to be in RIFF format, with the data block being called "CDXA" (I knida wish I can see this for myself, but I can't mount a PSX disk under linux because it's the wrong format.)

Can you hit me with a clue stick?

My guess (this is a guess yes) the STR files do not have AUDIO included.  MDEC have Video whcih is in 2048 byte sectors and Audio which is actually in 2304 byte sectors.  So imagine the format of the disk like this 320x240 15fps 3 video 1 audio 3 video 1 audio. That's how the movies are laid out. Each on a sector boundry. The reason for this is the MDEC data is read in directly and DMA'd through the MDEC. The resulting block of data is then sent to the display.  The AUDIO is queued into the SPU for playing. STR likely do not have audio at all in them.

MDEC data is stored as motion JPEG, so the data is in Y U V format because your eyes are more sensitive to luma than the two chromo factors.  The ratio is 4Y U V .  The MDEC accepts a stream of hoffman compressed data decodes it in real time then it performs a DCT (descret cosine transform) to get the original Y UV matrix. It then converst the Y U V data into RGB data which is streamed to the Display in a LOT of 8x8 chunks

Cyb

[Micky]

The MDEC documentation was one of the last things to go into my doc.

And if you look for MDEC information on google most of it points to your doc...
Anyway, if you need more information: DCT and IDCT (Inverse DCT) are standard algorithms in video processing, so you should find a lot of documentation. The same with YCbCr, that is for example explained in the Colour FAQ. And Huffman is a textbook algorithm so you should find it in any textbook, and lots of online documentation. And if you don't know RLE, here is some documentation for it as well.