Home > DeveloperSection > Forums > How to write super fast file streaming code in C#?
Ankita Pandey
Ankita Pandey

Total Post:183

Points:1285
Posted on    February-09-2014 11:09 PM

 C# C# 
Ratings:


 1 Reply(s)
 1486  View(s)
Rate this:

I have to split huge file into many smaller files. each of the destination file is defined by offset and length as number of bytes. I'm using the following code:

 

private void copy(string srcFile, string dstFile, int offset, int length)

{

    BinaryReader reader = new BinaryReader(File.OpenRead(srcFile));

    reader.BaseStream.Seek(offset, SeekOrigin.Begin);

    byte[] buffer = reader.ReadBytes(length);

    BinaryWriter writer = new BinaryWriter(File.OpenWrite(dstFile));

    writer.Write(buffer);

}

Considering that I have to call this function about 100,000 times, it is remarkably slow.

 

Is there a way to make the Writer connected directly to the Reader? 



Pravesh Singh

Total Post:411

Points:2881
Posted on    February-09-2014 11:31 PM

Hi Ankita,

How large is length? You may do better to re-use a fixed sized (moderately large, but not obscene) buffer, and forget BinaryReader... just use Stream.Read and Stream.Write.

private static void copy(string srcFile, string dstFile, int offset,

     int length, byte[] buffer)

{

    using(Stream inStream = File.OpenRead(srcFile))

    using (Stream outStream = File.OpenWrite(dstFile))

    {

        inStream.Seek(offset, SeekOrigin.Begin);

        int bufferLength = buffer.Length, bytesRead;

        while (length > bufferLength &&

            (bytesRead = inStream.Read(buffer, 0, bufferLength)) > 0)

        {

            outStream.Write(buffer, 0, bytesRead);

            length -= bytesRead;

        }

        while (length > 0 &&

            (bytesRead = inStream.Read(buffer, 0, length)) > 0)

        {

            outStream.Write(buffer, 0, bytesRead);

            length -= bytesRead;

        }

    }       

}


Don't want to miss updates? Please click the below button!

Follow MindStick