Services
Technology Web App Development Mobile App Development Windows App Development Database Development UI/UX Development Cloud Development SEO Digital Marketing Guest Posting Content Writing Artificial Intelligence Data Science ERP Development CRM Development CMS Development LMS Development Business Listing Blockchain Technology Startup Promotion AI Tools Integration
Products
Data Converter Cleaner TUC - Unit Converter Import Export Survey Manager Tweet Controller
Developer
Article Blog Forum Interview Beginner News Startup Quiz
Users
Career
Apply Job Internship
Pricing
Login Sign Up Get a Quote

forum

Home Forums How to remove duplicate files in Python? – MindStick
Ask Question

1 Answers

Aryan Kumar
Aryan Kumar 25 Jul 2023
Report Abuse

Sure, here is a Python code that you can use to remove duplicate files:

Python

import os
import hashlib

def remove_duplicates(directory):
  """Removes duplicate files in the specified directory."""

  # Create a dictionary to store the hashes of all the files in the directory.
  file_hashes = {}

  for file in os.listdir(directory):
    file_path = os.path.join(directory, file)

    # Get the hash of the file.
    file_hash = hashlib.sha1(open(file_path, "rb").read()).hexdigest()

    # If the file hash is already in the dictionary, remove the file.
    if file_hash in file_hashes:
      os.remove(file_path)

    else:
      file_hashes[file_hash] = file_path


if __name__ == "__main__":
  directory = "/path/to/directory"

  remove_duplicates(directory)

This code first creates a dictionary to store the hashes of all the files in the directory. Then, it iterates over all the files in the directory. For each file, it gets the hash of the file and checks if the hash is already in the dictionary. If the hash is already in the dictionary, the file is removed. Otherwise, the hash of the file is added to the dictionary.

To use this code, you need to pass the path to the directory that you want to remove duplicate files from as the argument to the remove_duplicates() function. For example, to remove duplicate files from the directory /path/to/directory, you would run the following code:

Python

remove_duplicates("/path/to/directory")

This code will remove all duplicate files from the directory /path/to/directory.

MindStick Training
Markdown for AI

A clean, structured version of this page for AI assistants and LLMs.

Open .md