Need help with C file streams

[sfx1999]

I have a problem. Let's say I have a string. Is there any way to make it into a FILE pointer without actually saving it?

[bulk_4me]

What's the point of doing that?

[Cyberman]

sfx1999:
Perhaps you ought to start with..  what are you trying to do?

I'm not sure why you would want to make a string into a file pointer, in fact it really makes no sense. to me (likely the reason for the prior reply).

Cyb

[L. Spiro]

I am not sure what you want to do, but my guess is that you want to save a string into a FILE pointer, without actually saving that file to the hard drive.


You can specify the “T” and/or “D” flags when you call fopen() in the mode.


So, if your string is szBuffer[MAX_PATH] and you want to save the path of your executable file into a file, but without saving it to hard drive,

Code: [Select]

char szBuffer[MAX_PATH];
GetModuleFileName( NULL, szBuffer, MAX_PATH );

FILE * pFile = fopen( "C:\\Temp.txt", "wtT" );
fprintf( pFile, "The path to my executable is %s.\n", szBuffer );
fclose( pFile );


Here we filled a string with whatever value we wanted, and then made a temporary file in which to store it.
The file was opened in wtT mode, which means it was created for writing, overwritten if existing, formatted for text, and temporary—not flushed to disk if possible.


The reason you would want to use this is if you are extracting a small file out of a large archive.
You could store the small file into a FILE stream but without actually writing it to disk.
Then you can read from the file as if it was a normal FILE, close it, and be done with it.



Specifying D in the mode means the file is temporary, and will be written to disk, but will be deleted after you close it.


L. Spiro

[DragonZap]

If you want to generate a temporary filename, you're probably better off using tmpnam(). Can you make a FILE* without a temporary file? Other than with fdopen() and a socket or pipe, not really. If you're using Windows, the socket part won't work, but the pipe part should. Create a pipe with pipe() (or _pipe() for Windows) and then use fdopen() (_fdopen() for Windows) on the read file descriptor. Anything you write to the write file descriptor will then come out on the read end. This is not a great solution, but it's the only one I can think of off the top of my head that works.

If you have control over the function that needs to use this FILE*, I'd suggest abstracting the streams instead; you'll get a lot more flexibility that way.

[mirex]

The file was opened in wtT mode, which means it was created for writing, overwritten if existing, formatted for text, and temporary—not flushed to disk if possible.

Huh, where is the 'T' from ? I haven't seen it anywhere in documentation ... is it compiler independent ?

Hm generally I dont get what is the point but I'd like to show you a file class that I made some time ago, it could be used for temporary things like this. It can work with memory just like with file. Usage would be:

Code: [Select]

CConvFile  tempfile;

tempfile.OpenMemory( NULL, 0, CConvFile::fm_write );
tempfile.Print( "Temporary number is %i\n", number );
tempfile.Write( 10, ten_bytes_of_something );

// and later on when you want to read the text;
tempfile.Seek( 0, SEEK_SET );
tempfile.gets( text_buffer, text_buffer_size );
tempfile.Read( 10, buffer_for_ten_bytes_of_something );


Its here: http://mirex.mypage.sk/FILES/ConvFile.rar although this is an older version without some bugfixes  :erm:

[Cyberman]

That looks like a MemoryStream object Mirex

It's essentially a buffer with stream IO access. Seems to be used a lot of these days with 'large tracks of memory' available in computers.  I use them for redirecting the output of a named pipe into a Strings list and stuffing it into a Rich text box.  I suppose I should use those for My FF7 file toy for LSZ decoding huh?

Cyb

[Qhimm]

She's got huuuuge ...tracts of RAM!

[sfx1999]

OK let me explain what I am trying to do. I have an image library that accepts either a file name or a FILE pointer. The problem is, I will be loading some images from ZIP files, so it would need decompressed first. Is there anyway to cast a buffer to a FILE?

[mirex]

Ehm I dont think so. I'm using a LibJpeg library for loading jpeg's, and it has the same problem ... I have to save data to temporary disk file ... i haven't find a way to make it handle memory buffer... but i would be glad if you would find a way

Cyberman: its something similair to MFC's CMemFile and CFile and other file classes ... but I just wanted to make one on my own

[Cyberman]

OK let me explain what I am trying to do. I have an image library that accepts either a file name or a FILE pointer. The problem is, I will be loading some images from ZIP files, so it would need decompressed first. Is there anyway to cast a buffer to a FILE?

Hmmmm
You have several options.  I have to decompress FF7 files (nothing like a 97K file growing to 470k all the time) but with computers these days that's not a problem keeping it in memory.  I would create a Memory stream object and write the decompressed data to it then, read it like a file.  Of course if your image is large then you will be sucking down 10 or more megs of RAM per image buffer.  It adds up quickly.  I have a news reader that store entire groups headers in memory.  Sure no problem I have 1/2 gig of RAM! NOT.  300,000 headers at 512 bytes per header. 150M of RAM just for one group. Now lets take a big group that might have 800000 or 1.2 million.  It's not crashed my machine but I can't browse 2 large groups at the same time without it draging for a while when I page down through the news groups.

Anyhow, I suggest using a Memory stream, it works like a regular IO stream, and the nice thing is it doesn't leave a temp file on your disk.  You can also read it 'quickly' over and over again.  If the Image size is bigger than 1280x1024 I suggest creating a temporary file instead.

Cyb

[mirex]

Cyb: could you post some example how to use those streams this way ? And how to cast them to FILE* ?

[sfx1999]

The library I use doesn't support a steam, only a C style file.

[DragonZap]

I would suggest using another library, then. I'm sure there are plenty of free ones out there. Any other solution is most likely going to be a hack; FILE*s are supposed to be an opaque type (that is, you're never supposed to access any of their fields directly, because they can be completely different on different platforms) and thus you can't really make any custom ones without digging into undocumented and totally unportable stuff.

What platform are you trying to make this for, anyway?

[mirex]

And what kinds of images are you trying to load ?

[sfx1999]

I am developing on MinGW and I am intending to load PNGs, TGAs, and JPEGs. At some point, I'd like to load DDS files, but that isn't going to happen for a while (or at least until Corona supports them).

Anyway, I could probably just use the tmpfile() function.

[DragonZap]

DevIL has worked as a pretty good image library for me, and I've used it to load from BLOBs in memory without having to have a separate file on disk. On the other hand, I suppose it's been a while since the last release. It does work with all of those file types you listed, however.

[sfx1999]

I've tried to use DevIL, and it didn't work. It crashed whenever I told it to load a file.

I have no idea if it works with strings, anyway. Corona works really well. I like it. I am hoping it gets DDS support.

[DragonZap]

If by "strings" you mean binary data in memory, then yes, it does -- that's what I meant by BLOBs (Binary Long OBjects). As for it crashing, my first guess would be that maybe you forgot to initialize the library at the start of your program? It's an easy thing to forget when you're concentrating on everything else, and that's the only reason I've ever had it crash when loading anything.

[mirex]

I've made one image library too, maybe it will suit your needs. It can be found here: http://mirex.mypage.sk/index.php?selected=3 , its called BiturnBitmapLibrary (BBL) ; its just a 2nd version so its still missing some features, but it can load and save Bmp, Tga, Dds and Jpg (jpg is loaded through a libJpeg library, which is hidden inside BBL library)

Though has some constraints, Bmp Load/Save does not support RLE compressed BMP's, TGA does not support interleaved palettes, JPG needs to create a temporary file (libJpeg has same problem as you have) and DDS output is saved only in uncompressed ... but other than that its quite usable library

[sfx1999]

Yes DragonZap, I did intialize it, but it still crashed.

Anyway, look what I found while looking through stdio.h:

Code: [Select]

/*
 * The structure underlying the FILE type.
 *
 * I still believe that nobody in their right mind should make use of the
 * internals of this structure. Provided by Pedro A. Aranda Gutiirrez
 * <[email protected]>.
 */

[DragonZap]

Anyway, look what I found while looking through stdio.h:

Code: [Select]

/*
 * The structure underlying the FILE type.
 *
 * I still believe that nobody in their right mind should make use of the
 * internals of this structure. Provided by Pedro A. Aranda Gutiirrez
 * <[email protected]>.
 */

You see what I mean, then. Any library that doesn't allow any flexibility/abstraction in where it gets its data from is probably one you want to avoid.

[Cyberman]

Cyb: could you post some example how to use those streams this way ? And how to cast them to FILE* ?

Short answer NO

Hehehe you can't cast a stream object to a FILE structure or visa versa.  It is a nonsequiter thing to do.  For sfx1999 he should have no problem making a simple stream memory object he should be able to use streams as well.  MingW is not just C it's C++ as well.

Are you trying to do image manipulation under Linux? You might be better off using the PBM library functions for that, and use pipe redirection to dump your internal memory data in and out for handling the image. Essentially you end up with two applications your main program feeding the data to the filter and the filter processing the data and returning it on stdout.  It's much easier to do under Linux than windows, and more importantly under Linux they can operate simultaneously.

Cyb

[sfx1999]

Mwahaha! I've just located a feature in the new version of Corona that allows me to create a fake file. Score!

[Cyberman]

Mwahaha! I've just located a feature in the new version of Corona that allows me to create a fake file. Score!

Gouda.. err good Well glad you found a way to get from point A to point B without going through a bunch of pointless points!

Cyb