---
title: "How to remove duplicate files in Python?"  
description: "How to remove duplicate files in Python?"  
author: "Revati S Misra"  
published: 2023-07-24  
updated: 2023-07-25  
canonical: https://www.mindstick.com/forum/159262/how-to-remove-duplicate-files-in-python  
category: "python"  
tags: ["file", "python"]  
reading_time: 2 minutes  

---

# How to remove duplicate files in Python?

How to [remove duplicate](https://www.mindstick.com/forum/157765/how-to-remove-duplicate-rows-in-sql) [files](https://www.mindstick.com/articles/23302/the-importance-and-advantage-of-keeping-your-important-files-on-the-cloud) in [Python](https://www.mindstick.com/articles/75378/simple-yet-useful-tips-when-using-python)?

## Replies

### Reply by Aryan Kumar

Sure, here is a Python code that you can use to [remove](https://yourviews.mindstick.com/story/4554/8-harmful-weeds-to-remove-from-garden) [duplicate](https://www.mindstick.com/forum/160911/sql-query-to-find-duplicate-records-in-a-table-in-sql-server) files:

Python

```plaintext
import os
import hashlib

def remove_duplicates(directory):
  """Removes duplicate files in the specified directory."""

  # Create a dictionary to store the hashes of all the files in the directory.
  file_hashes = {}

  for file in os.listdir(directory):
    file_path = os.path.join(directory, file)

    # Get the hash of the file.
    file_hash = hashlib.sha1(open(file_path, "rb").read()).hexdigest()

    # If the file hash is already in the dictionary, remove the file.
    if file_hash in file_hashes:
      os.remove(file_path)

    else:
      file_hashes[file_hash] = file_path

if __name__ == "__main__":
  directory = "/path/to/directory"

  remove_duplicates(directory)
```

This code first creates a dictionary to store the hashes of all the files in the directory. Then, it iterates over all the files in the directory. For each file, it gets the hash of the file and checks if the hash is already in the dictionary. If the hash is already in the dictionary, the file is removed. Otherwise, the hash of the file is added to the dictionary.

To use this code, you need to pass the path to the directory that you want to remove duplicate files from as the argument to the `remove_duplicates()` function. For example, to remove duplicate files from the directory `/path/to/directory`, you would run the following code:

Python

```plaintext
remove_duplicates("/path/to/directory")
```

This code will remove all duplicate files from the directory `/path/to/directory`.


---

Original Source: https://www.mindstick.com/forum/159262/how-to-remove-duplicate-files-in-python

Copyright © MindStick Software Pvt. Ltd. This Markdown version is provided for developers, AI systems, and offline reading.