Remove Duplicate Lines

Choose what counts as the same line

The default comparison uses the exact line text, including case and leading or trailing spaces. Alpha and alpha are different entries; so are a line with spaces around it and the same word without those spaces. The tool keeps the first occurrence of each comparison key and removes later matches. It does not sort or rewrite the retained lines.

Ignore case compares JavaScript-lowercased text. It is not full linguistic case folding: Straße and STRASSE, accented letters and unaccented letters, and composed or decomposed spellings may remain different. Ignore leading/trailing spaces trims only the comparison key, so the first retained line keeps its original spaces and capitalization. Neither option changes text in the middle of a line.

Remove blank lines drops empty or whitespace-only entries before checking for duplicates. Their removals are counted separately from repeated lines. When this option is off, blank lines follow the same comparison rules as other entries: repeated exact empty lines are reduced to one, while different space-only lines can remain distinct until you also ignore leading/trailing spaces.

Review the counts and the downloaded text

The result reports input lines, lines kept, duplicate lines removed and blank lines removed. Input equals the other three counts added together. A final newline closes the last line and does not add a phantom extra item: a list ending with Alpha followed by a newline contains one line. Two newline characters alone represent two empty lines before deduplication.

CRLF, LF, CR, NEXT LINE and Unicode line or paragraph separators are recognized as boundaries. The output uses LF separators and keeps a final newline when the input had one and at least one line remains. An ending blank line is explicitly terminated so it remains a line in the saved result. The retained line contents stay unchanged. A result with zero retained lines can still be copied or downloaded as an empty TXT file; the interface states that nothing remains.

Processing accepts up to 500,000 UTF-16 units and 50,000 lines. Some emoji use more than one UTF-16 unit. Oversized input is rejected without truncation. The text stays in this tab and is not saved to browser storage, put into a URL or sent to a processing service. Copy or download the UTF-8 TXT before closing. For URLs and CSV-like rows, this is literal whole-line comparison: it does not normalize addresses, parse columns or check whether two differently written entries mean the same thing.

How to use it

1. Paste one item per line — Paste a plain-text list or fill the example. The example does not run automatically.
2. Choose comparison rules — Keep exact matching or choose case, edge-space and blank-line options. Press Remove duplicate lines.
3. Review and save — Check retained text and separate removal counts, then copy or download the TXT. Clear removes the current input and result.