HOW TO WRITE SUPER FAST FILE STREAMING CODE IN C#?

Ankita Pandey

Total Post:183

Points:1285
Posted by  Ankita Pandey
C# 
C#
 1890  View(s)
Ratings:
Rate this:

I have to split huge file into many smaller files. each of the destination file is defined by offset and length as number of bytes. I'm using the following code:

 

private void copy(string srcFile, string dstFile, int offset, int length)

{

    BinaryReader reader = new BinaryReader(File.OpenRead(srcFile));

    reader.BaseStream.Seek(offset, SeekOrigin.Begin);

    byte[] buffer = reader.ReadBytes(length);

    BinaryWriter writer = new BinaryWriter(File.OpenWrite(dstFile));

    writer.Write(buffer);

}

Considering that I have to call this function about 100,000 times, it is remarkably slow.

 

Is there a way to make the Writer connected directly to the Reader? 

  1. Pravesh Singh

    Post:412

    Points:2888
    Re: How to write super fast file streaming code in C#?

    Hi Ankita,

    How large is length? You may do better to re-use a fixed sized (moderately large, but not obscene) buffer, and forget BinaryReader... just use Stream.Read and Stream.Write.

    private static void copy(string srcFile, string dstFile, int offset,

         int length, byte[] buffer)

    {

        using(Stream inStream = File.OpenRead(srcFile))

        using (Stream outStream = File.OpenWrite(dstFile))

        {

            inStream.Seek(offset, SeekOrigin.Begin);

            int bufferLength = buffer.Length, bytesRead;

            while (length > bufferLength &&

                (bytesRead = inStream.Read(buffer, 0, bufferLength)) > 0)

            {

                outStream.Write(buffer, 0, bytesRead);

                length -= bytesRead;

            }

            while (length > 0 &&

                (bytesRead = inStream.Read(buffer, 0, length)) > 0)

            {

                outStream.Write(buffer, 0, bytesRead);

                length -= bytesRead;

            }

        }       

    }

Answer

NEWSLETTER

Enter your email address here always to be updated. We promise not to spam!