I spent the past nine months building a data importer. It is the screen where a person brings records they already have into a web app, such as a customer list or a price list exported from another system, as a CSV or Excel file. The app reads the file and turns every line into a row it can store.
Reading a CSV looks like a small job, and a first version can fit in six lines.
const text = await file.text();
const { data } = Papa.parse(text, {
header: true,
dynamicTyping: true,
});
setRows(data);
This is a reasonable first CSV import. PapaParse reads the text, header: true turns every line into an object keyed by column name, and dynamicTyping: true turns numeric strings into numbers. A test file goes through it cleanly, because the developer wrote that file to fit the code.
When the file comes from a system you control and already matches the shape your app expects, this code can be enough. A file from someone else follows the conventions of the software that produced it, such as how it writes dates, decimals, and IDs, and each of those conventions can break this code in its own way.
Type inference strips leading zeros
Some values look like numbers and are not. Customer IDs, ZIP codes, and product codes are made of digits, and the zeros at the front belong to the value. A parser that converts everything numeric into a number throws those zeros away.
A customer export holds a customer_id column with the values 00123, 00401, and 07030. dynamicTyping tests each value against a number pattern, and on PapaParse 5.6.0 that pattern accepts leading zeros, so "00123" becomes 123.
The row still looks valid. It stops matching customer 00123 the moment the app uses the value as an identifier.
The import keeps the raw cell text and lets the schema decide the type. A field declared as an ID stays text. A field declared as a number gets checked as a number, and 00123 in that field becomes a value to flag. The parser sees one value at a time and has no way to tell an ID from an amount. The schema knows which field is which.
In an Excel file, a cell formatted as 00000 stores 401 and shows 00401, so the importer reads the formatted text.
Some zeros are gone before the file reaches you. Microsoft documents that Excel "automatically removes leading zeros", so a CSV that passed through Excel can arrive holding 401. Nothing in that value says the ID had five digits. Your own code pads it back, because only your app knows the width.
The whole column reveals the date format
A date written as 03/04/2025 means 3 April in the UK and 4 March in the US. The file does not say which convention it follows, and JavaScript picks one on its own.
An export from a UK system writes dates day first. The obvious conversion hands each value to the Date constructor.
new Date("03/04/2025"); // Tue Mar 04 2025
new Date("28/11/2025"); // Invalid Date
new Date("15/01/2026"); // Invalid Date
In V8, the engine behind Chrome and Node, the first value parses without an error and lands on 4 March. The file meant 3 April. The other two fail, so one column produces a silent wrong date and two errors.
One value cannot settle the format, because 03/04/2025 is a valid date read either way. A sample from the whole column can. The importer takes a sample of values from the column and lets every candidate format try to read it, day first, month first, year first, and the rest. The format that reads the most values wins. 28/11/2025 has no 28th month, so month first fails on it and loses the count. Day first reads all three, wins the column, and 03/04/2025 becomes 3 April for every row.
A column holding only 03/04/2025 and 05/06/2025 reads fully both ways, and the data cannot break that tie. A convention from outside the data has to, such as the locale of the browser. The importer should report that it guessed, because a wrong guess produces dates that look valid.
The whole column reveals the separators
Countries write numbers differently. One thousand two hundred thirty-four and fifty-six hundredths is 1,234.56 in the US and 1.234,56 in Germany. The dot and the comma swap roles, and code written for one convention misreads the other.
Price columns from different systems hold 1.234,56, 1 234,56 with a non-breaking space, €1,234.56, and (250,00), which accounting software uses for a negative amount. parseFloat reads each of them from the left and stops at the first character it does not expect.
parseFloat("1.234,56"); // 1.234
parseFloat("1 234,56"); // 1
parseFloat("€1,234.56"); // NaN
A validator catches NaN. 1.234 passes the same check, because it is a valid number, and it is a thousand times smaller than the amount in the file.
The column votes again, and this time the candidates are pairs of marks, one that groups thousands and one that starts the decimals. Dot and comma is one pair, comma and dot another, space and comma a third. A pair reads a value only when the digit groups sit where that pair expects them, so 1.234,56 fits dot and comma and fails comma and dot. Before the vote, the importer strips the currency symbol, normalizes the non-breaking space, and turns (250,00) into -250.
The column settles values that look identical on their own. 1,234 means 1234 in a column that also holds 1,234.56, and 1.234 in a column that also holds 1.234,56. A column of nothing but 1,234 ties the way dates do.
Value mapping works on distinct values
Some fields accept only a fixed list of values, such as a department, a status, or a country. People type those values in their own words, with different spelling, case, and abbreviations. The importer turns every variant into one of the values the app accepts.
The file's Department column already matched the department field by its header, and a lookup table converts the words.
const DEPARTMENTS = { Engineering: "eng", Sales: "sales" };
row.department = DEPARTMENTS[row.department];
The column holds Engineering, engineering, Eng, R&D, and Enginering. The table finds the first and returns undefined for the other four. The header matched, and the values inside the column still use the words of whoever typed them.
The importer collects the unique values of the column, resolves each one once, and applies the finished mapping to every row as a lookup. A million rows with five spellings of a department need five decisions. Fuzzy string matching that ignores case pairs engineering and Enginering with Engineering.
String similarity alone does not tell you that R&D means Engineering, or that Eng is short for it. That knowledge belongs to your domain. It reaches the mapping through a synonym list you keep, through your own logic, such as the mappings the same customer confirmed last time, or through the person importing the file, who picks the value by hand.
The browser decodes every file as UTF-8
A file on disk is a sequence of bytes, and an encoding is the rule that turns those bytes into letters. Older systems and different countries use different rules. When the app reads a file with the wrong rule, letters such as ü and é come out broken.
The import code at the start of this article reads the file with file.text(), which always decodes it as UTF-8. A CSV written by an older system in Windows-1252 stores ü as the single byte FC, which is invalid in UTF-8. The decoder swaps it for a replacement character, and Müller arrives as M�ller. The opposite mistake, UTF-8 bytes read as Windows-1252, turns Müller into Müller.
The replacement character is final. Once � sits in the string, nothing recovers the letter it replaced, so the decision has to happen while the bytes are still in hand. The importer checks for a byte order mark first, because it can identify several Unicode encodings before any guessing is needed. Then it tries UTF-8 in strict mode, where an invalid byte makes the decoder throw.
const decoder = new TextDecoder("utf-8", { fatal: true });
const text = decoder.decode(await file.arrayBuffer());
If strict decoding throws, the bytes are not valid UTF-8. Only then does the importer infer a legacy encoding from the bytes with a detector such as chardet, and decode again with the label it returns.
Large files break both parsing and rendering
Every fix so far works on a file of a few hundred rows. A file with a million rows asks the browser to read, process, and show all of it while the page stays responsive.
The import code at the start of this article runs on the main thread, which also paints the page and answers clicks. As the importer gains decoding, parsing, format detection, and value mapping, doing that work on the main thread turns it into one long task, and the tab stops responding until it ends.
The importer hands the File object to a Web Worker and decodes and parses there, so the page keeps painting. Some cost stays on the main thread. On 4 September 2026 I ran a 108 MB CSV with one million rows and eleven columns through a worker in Chrome 151. The main thread stayed free while the worker parsed, and still hit one 389 ms task when the parsed rows arrived back. A worker that reads the file in one call also holds the whole file in memory.
The rows then reach the second limit. A normal DOM table does not scale to a million rows, because every cell it renders is an element the browser lays out and paints. The grid has to limit rendering to a small window around the visible rows and work out the rest from the scroll position. A virtualized list does that with elements, and a canvas grid does it with pixels.
An importer answers what the data means
Other files break a simple CSV import in other ways. A report can place its header row several lines down, under a title and a blank line. Two columns can share the header id, and PapaParse renames the second to id_1, which no field in the app expects. A short row can shift its values into the wrong columns. An Excel workbook can hold several sheets, and only one of them holds the data.
A CSV parser answers whether a file can be read. An importer has to answer what the data means, what is wrong with it, what the person can fix, and what the app should receive.
I learned most of this while building Updog, a client-side importer for CSV and Excel files.
I wrote this article with the help of AI.
