User:Ahazred8/dupelist

COMPARISON of duplicate file utilities. This is a list of duplicate file management utilities.

Draft List Of Utils

• clink - replaces duplicates with symlinks
• duff - uses fast O(n log n) comparison
• dupedit - Compares many files at once without checksumming. Avoids comparing files against themselves when multiple paths point to the same file.
• dupefinder -
• duper -
• dupmerge - runs across most platforms (Win32/64 with Cygwin, *nix, Linux etc.)
• dupseek - Perl with algorithm optimized to reduce reads
• epac - Erik's Partial Archive Collator - replaces duplicates with hardlinks
• fdf - Perl/c based and runs across most platforms (Win32, *nix and probably others). Uses MD5, SHA1 and other checksum algorithms
• fdupe - a small script written in Perl. Doing its job fast and efficiently.
• fdupes -
• findrepe - Java-based command-line tool; can search within zips and jars. (Linux, Mac OS X, *nix, Windows)
• freedup - POSIX C compliant and runs across platforms (Windows with Cygwin, Linux, AIX, etc.)
• freedups - shell script
• fslint - has command line interface and GUI.
• ftwin -
• liten - Pure Python deduplication command line tool, and library, using md5 checksums and a novel byte comparison algorithm. (Linux, Mac OS X, *nix, Windows)
• liten2 - A rewrite of the original Liten, still a command line tool but with a faster interactive mode using SHA-1 checksums (Linux, Mac OS X, *nix)
• rdfind - One of the few which rank duplicates based on the order of input parameters (directories to scan) in order not to delete in "original/well known" sources (if multiple directories are given). Uses MD5 or SHA1. uses fast O(n log n) comparison
• rmlint - Fast finder with command line interface and many options to find other lint too (uses MD5)
• ssdeep - identify almost identical files using Context Triggered Piecewise Hashing
• ua - Unix/Linux command line tool, designed to work with find (and the like).

Related: duplicate image match finders Related: data deduplication

DummySortableList

FIXME*FIXME misc duplicate file utilities

Name	FIXMECategory	Description	First appeared
admin	SCCS	Create and administer SCCS files	PWB UNIX
alias	Misc	Define or display aliases
ar	Misc	Create and maintain library archives	Version 1 AT&T UNIX
what	SCCS	Identify SCCS files	PWB UNIX
xargs	Shell Programming	Construct argument lists and invoke utility	PWB UNIX
yacc	C Programming	Yet another compiler compiler	PWB UNIX
zcat	Text Processing	Expand and concatenate data	4.3BSD

See also

• List of GNOME applications
• List of GNU packages
• List of KDE applications
• List of Unix daemons
• List of web browsers for Unix/Linux
• Unix philosophy

External links

{X{wikibooks|Guide to UNIX|Commands}} {X{ComXmons category|Unix cheatsheets}}

• IEEE Std 1003.1,2004 specifications
• IEEE Std 1003.1,2008 specifications
• Rosetta Stone For *Nix – configurable list of equivalent programs for *nix systems.
• The Unix Acronym List – explains the names of many Unix commands.
• The UNIX System Homepage

{X{UnXix commands}}

[X[CaXtegory:Lists of software|Unix programs]] [X[CaXtegory:System administration]] [X[CaXtegory:Unix SUS2008 utilities| ]]