Guides / CSV encoding

Encoding and parsing

Why does my CSV look weird?

Garbled accents usually mean an encoding mismatch. One giant column means the delimiter was missed. Changed IDs and dates mean the viewer inferred a type and rewrote the display.

Match the symptom

SymptomLikely cause
café instead of caféUTF-8 decoded as Windows-1252
All values in one columnWrong delimiter: comma vs semicolon, tab or pipe
Columns shift mid-fileBroken quoting, embedded newline, or ragged row
00123 becomes 123Automatic numeric conversion
Long ID becomes scientific notationSpreadsheet type inference

What UTF-8 actually changes

CSV defines rows, delimiters and quoting; it does not reliably announce its character encoding. UTF-8 can represent the full Unicode range, but a viewer must choose it. A UTF-8 byte-order mark sometimes helps spreadsheet importers, though it is not part of the data itself.

Diagnose before saving

  1. Keep a copy of the original bytes.
  2. Try UTF-8 first, then the known source encoding.
  3. Confirm the delimiter using several rows, not only the header.
  4. Inspect quoted commas and embedded newlines.
  5. Check identifiers, dates and long numbers before exporting.

With csvlite, cell display text is preserved while inferred types are used only for sorting, filtering and profiling. It supports a manual encoding override and delimiter detection for common separated formats.