Free · No sign-up · Runs in your browser
Remove Duplicate Lines
Paste a list and strip out every repeated line, keeping the first occurrence of each. The original order is preserved, which matters when your list is ranked or chronological.
How to use it
- Paste your list with one item per line.
- Choose whether matching should be case sensitive — leave it off to treat "Apple" and "apple" as the same entry.
- Click Remove duplicates. The first occurrence of each line is kept in its original position.
The two settings that matter most
- Case sensitivity. Email addresses, tags, and usernames are usually case-insensitive in practice, so leaving this off catches duplicates that differ only in capitalisation. Turn it on for passwords, IDs, or anything where case is meaningful.
- Ignore surrounding spaces. A trailing space makes two visually identical lines different. With this on, "london" and "london " are treated as the same entry — which is nearly always what you want in real, messy data.
Order is preserved, unlike a sort-based approach
The common shortcut for deduplication is to sort a list and remove neighbours that match, which is what sort | uniq does on a command line. That works, but it destroys your original ordering. This tool keeps the first occurrence of each line exactly where it was, so a priority-ordered or date-ordered list survives intact. If you do want the output alphabetised, run the result through the line sorter afterwards.