How to use
01
Paste your list
Put one item per line in the Your list box, or press Try sample to load a short keyword list that has repeats, mixed capitals and a trailing space.
02
Choose what to keep
In What to do, keep one copy, keep only lines that appear once, or show only duplicates, and pick the first or last copy. Tick Ignore case if Apple and apple should match.
03
Check the counts and copy
Read the lines in, out and removed counter, and check the Lines that repeated table and the Removed lines box below. Then press Copy, or Download .txt to save the clean list.
Why Remove Duplicate Lines
- Keeps the first or last copy of each line where it was, so your order stays intact.
- Can keep only unique lines, or list only the duplicated ones.
- Works on lines or on comma, tab or custom separated items.
- Lists removed lines and repeat counts, and runs in your browser.
Before and after: cleaning a keyword list
Here is the ten-line sample list with default settings: case-sensitive matching, spaces at the ends ignored, blank lines kept. The line "running shoes " has a trailing space, so it matches "running shoes" and is removed. "Running Shoes" stays because its capitals differ.
Tick Ignore case and Remove empty lines and the output shrinks to four lines: running shoes, trail running shoes, running shoes for flat feet and waterproof trail shoes. The repeated lines table then reports running shoes as found 4 times.
| Input line | Default result | With Ignore case and Remove empty lines |
|---|---|---|
| running shoes | Kept (first copy) | Kept (first copy) |
| trail running shoes | Kept | Kept |
| Running Shoes | Kept (capitals differ) | Removed |
| running shoes for flat feet | Kept | Kept |
| (blank line) | Kept | Removed |
| trail running shoes | Removed | Removed |
| running shoes (with a trailing space) | Removed | Removed |
| (blank line) | Kept | Removed |
| waterproof trail shoes | Kept | Kept |
| running shoes | Removed | Removed |
How to remove duplicate lines in Excel, Notepad++ and the command line
If the text already lives in another program, these built-in options do the same job. Note which ones keep your original order, because several of them sort the list as a side effect.
- Excel: select the column, then Data, Remove Duplicates. It keeps the first copy of each value. In Excel 365, =UNIQUE(A2:A500) returns the list without repeats in a new range.
- Google Sheets: Data, Data cleanup, Remove duplicates, or the =UNIQUE() formula.
- Notepad++: Edit, Line Operations, Remove Duplicate Lines. There is also Remove Consecutive Duplicate Lines, which only removes repeats that sit next to each other.
- Linux and macOS: sort -u list.txt removes repeats but sorts the list. uniq on its own only removes adjacent repeats, so it is normally used after sort.
- To keep the original order on the command line, use awk '!seen[$0]++' list.txt, which prints each line the first time it appears.
Common questions
Does removing duplicate lines change the order of my list?
No. The tool reads your list from top to bottom and keeps the first copy of each line in the spot where it appeared. Later copies are dropped and every other line stays where it was. If you want the result sorted, pick A to Z or Z to A in Sort the result.
Can I remove duplicate lines without caring about upper and lower case?
Yes. Tick Ignore case and lines like Running Shoes and running shoes are treated as the same entry. The copy that is kept is the first or last one in your list, as set in Which copy to keep, with its original capitals. Set Letter case in the result to lowercase if you want every line in the same case.
Why are two lines that look the same not removed as duplicates?
Usually one has a different capital letter or a hidden character. Turn on Ignore case, and keep the option to ignore spaces at the start and end of lines switched on. Invisible characters such as a zero-width space or a non-breaking space inside the line still make lines differ, so clean those first with the AI Watermark Remover.
Can I see which lines were duplicated and how many times?
Yes. Below the panes, the Lines that repeated table lists each line that appeared more than once and how many times it was found, most repeated first. The Removed lines box lists every dropped line with its original line number, and you can copy it or download it as .txt.
What happens to blank lines when I remove duplicate lines?
When removing duplicates, blank lines are kept where they are and are never treated as duplicates of each other, so paragraph gaps survive. Tick Remove empty lines to drop them in the same pass. The unique-only and duplicates-only modes always leave blank lines out.
How do I keep only the lines that appear once in a list?
Choose Keep only lines that appear once in What to do. Every line that occurs two or more times is dropped completely, including its first copy, so only truly unique entries remain. To see the opposite, choose Show only the duplicated lines, which gives one copy of each repeated line.
Does the duplicate check ignore spaces at the start and end of a line?
Yes, when Ignore spaces at the start and end of lines is ticked, which is the default. The kept line is output exactly as you pasted it, spaces included. Untick the option if leading spaces matter, for example in indented code.
How do I remove duplicate rows from an Excel or Google Sheets column?
Copy the column, paste it here and copy the result back, since each cell arrives as one line. Inside Excel you can also use Data, then Remove Duplicates, and Google Sheets has Data, then Data cleanup, then Remove duplicates. This tool works on one column of text, not on whole rows with several columns.
How many lines can the duplicate line remover handle?
Tens of thousands of lines work without trouble, because each line is checked against a lookup set in one pass. The limit is your browser's memory, not a quota. Very large files may make the text box itself slow to scroll.
Is my list uploaded when I remove duplicate lines online?
No. The page does all the work with JavaScript in your browser tab and does not send your text to a server. That makes it fine for client keyword lists, internal URLs or email addresses.