Remove duplicate lines
Remove duplicate lines from a list, keep the first or last copy, ignore case or spaces, or drop every line that repeats.
Code examples
Copy and paste these into your own project. Each one uses only the language's standard library, so there is nothing to install.
const lines = ['b', 'a', 'b', 'c', 'a']
console.log([...new Set(lines)].join('\n'))What it does
Removes repeated lines from a list. Keep one copy of each line, drop every line that appears more than once, or show only the lines that repeat. The lines you keep stay in their original order and exactly as you typed them.
How to use it
- Type or paste one item per line, or drop a text file.
- Pick a mode and whether to keep the First copy or the Last copy. Under More, turn off Case sensitive so
Appleandapplematch, or turn on Ignore surrounding whitespace so a trailing space does not make a line different. - Copy or download the result. Remove blank lines drops empty lines first.
Examples
Keeping one copy of each line:
b b
a a
b → c
c
a
With Remove every repeated line, the same list leaves only c. With Show only the repeats it leaves b and a.
Good to know
- Keeping the last copy puts each line where it last appeared:
b, a, b, c, abecomesb, c, a. - Without case sensitivity the first copy you see is the one kept, in the case you typed it.
- Blank lines count as lines too. Two empty lines are duplicates unless you remove blank lines.
- A single line break at the end of the text ends the last line rather than adding an empty one.
How are duplicate lines found?
Two lines are duplicates when their text is identical. With case sensitivity off, Apple and apple count as the same line, and with surrounding whitespace ignored, a line with a trailing space matches the one without. The lines you keep are left exactly as you typed them, and they stay in their original order.
Frequently asked questions
What is the difference between keeping one copy and removing every repeated line?
Keeping one copy leaves a single instance of each line, so a, b, a becomes a, b. Removing every repeated line drops all copies of anything that appears more than once, so a, b, a becomes b. Use it to find the values that only ever appear once.
Does keeping the last copy change the order?
The kept lines stay in the order of the copy you keep. With the last copy kept, a, b, a becomes b, a, because the final a comes after b.
Does this tool send my lines anywhere?
No. Everything happens in your browser. What you type is never sent to our servers.