How do you implement file chunking for large file uploads?

ICSM Computer 838 16 May 2025

Ask by ICSM Computer

Updated 30 May 2025

1 Answers

ICSM Computer 16 May 2025

Implementing file chunking for large file uploads involves breaking the file into smaller chunks on the client side and sending them to the server, where the chunks are reassembled. Here's how to do this in C# for the server-side logic:

Concept Overview

Client:

Splits large file into chunks (e.g., 5 MB each).
Sends each chunk via HTTP POST/PUT.
Includes metadata: fileId, chunkIndex, totalChunks, etc.

Server:

Receives and stores each chunk temporarily.
Once all chunks are received, merges them into a final file.

1. API Endpoint to Receive a Chunk

[HttpPost("upload-chunk")]
public async Task<IActionResult> UploadChunk(
    [FromForm] IFormFile chunk,
    [FromForm] string fileId,
    [FromForm] int chunkIndex,
    [FromForm] int totalChunks)
{
    string tempDir = Path.Combine("TempUploads", fileId);
    Directory.CreateDirectory(tempDir);

    string chunkPath = Path.Combine(tempDir, $"{chunkIndex}.chunk");

    using var stream = new FileStream(chunkPath, FileMode.Create);
    await chunk.CopyToAsync(stream);

    return Ok(new { message = "Chunk uploaded", chunkIndex });
}

2. Endpoint to Merge All Chunks

[HttpPost("merge-chunks")]
public IActionResult MergeChunks([FromForm] string fileId, [FromForm] string finalFileName)
{
    string tempDir = Path.Combine("TempUploads", fileId);
    string outputPath = Path.Combine("FinalUploads", finalFileName);

    Directory.CreateDirectory("FinalUploads");

    var chunkFiles = Directory.GetFiles(tempDir)
                              .OrderBy(f => int.Parse(Path.GetFileNameWithoutExtension(f)))
                              .ToList();

    using var output = new FileStream(outputPath, FileMode.Create);

    foreach (var chunkFile in chunkFiles)
    {
        using var input = new FileStream(chunkFile, FileMode.Open);
        input.CopyTo(output);
    }

    // Optionally clean up
    Directory.Delete(tempDir, recursive: true);

    return Ok(new { message = "File merged", outputPath });
}

3. Example Front-End (JavaScript File Chunking)

async function uploadFileInChunks(file) {
  const chunkSize = 5 * 1024 * 1024; // 5MB
  const totalChunks = Math.ceil(file.size / chunkSize);
  const fileId = crypto.randomUUID(); // unique ID

  for (let i = 0; i < totalChunks; i++) {
    const start = i * chunkSize;
    const end = Math.min(file.size, start + chunkSize);
    const chunk = file.slice(start, end);

    const formData = new FormData();
    formData.append("chunk", chunk);
    formData.append("fileId", fileId);
    formData.append("chunkIndex", i);
    formData.append("totalChunks", totalChunks);

    await fetch("/upload-chunk", {
      method: "POST",
      body: formData
    });
  }

  // Trigger merge
  await fetch("/merge-chunks", {
    method: "POST",
    body: new URLSearchParams({
      fileId,
      finalFileName: file.name
    })
  });
}

Benefits

Handles unstable connections (retry per chunk).
Works well for large files (even GBs).
Enables upload resumption and parallelism (with tracking).

Markdown for AI

A clean, structured version of this page for AI assistants and LLMs.

Open .md

interview