Duplicates creep into lists in ways that are hard to see. Two keyword exports get merged. A subscriber file is appended to itself after a failed import. Someone pastes the same block twice and does not notice because the list is 3,000 lines long and no screen shows more than fifty at a time.
The consequences are rarely dramatic but always annoying: inflated counts that make a report wrong, a subscriber emailed twice, a bid placed twice on the same keyword, an import that fails on a unique constraint at row 2,847.
This tool removes the repeats and keeps the first occurrence of each line. Order is preserved unless you ask for sorting, which matters more often than people expect - a list of steps, a ranked set of keywords, or a chronological log all lose meaning if they are alphabetised.
Case sensitivity is the setting to think about
By default the comparison ignores case, so Newsletter@example.com and newsletter@example.com are treated as the same line and only the first survives. This is almost always what you want for email addresses, since the domain half is case-insensitive and virtually every mail provider treats the local half that way too.
Turn case sensitivity on and those two become separate entries. You want this when the case carries meaning: variable names, file paths on a Linux server, API keys, product codes where AB-100 and ab-100 are genuinely different items.
Getting this wrong in either direction is the most common way to lose data with a tool like this, which is why the counters underneath tell you how many lines went in, how many unique ones came out, and how many were removed. If that last number is far larger than you expected, undo and check the case setting before you overwrite your source file.
Trimming and blanks
Trim whitespace strips leading and trailing spaces before comparing. Without it, apple and apple are different lines and both survive, which defeats the purpose. Copy-pasted lists are full of trailing spaces, so this is on by default.
Remove empty lines clears the blanks that accumulate between pasted blocks. Leave it off if the blank lines are structural - separating paragraphs, or marking record boundaries in a fixed-format file.
Finding duplicates rather than removing them
The show only duplicates option inverts the tool. Instead of a cleaned list, you get just the lines that appeared more than once. This turns it into an auditing tool rather than a cleaning one.
It is genuinely useful for diagnosis. Which email addresses are in both of these two exports? Which SKUs did the supplier list twice? Which URLs appear in more than one sitemap? Run the combined file through in duplicates-only mode and the overlap is the answer.
Everything runs in your browser, so a subscriber list or customer export is never uploaded anywhere. That is the point of doing this on your own machine rather than pasting personal data into a service you have not vetted.