pola-rs/polars
View on GitHubConfusing error message when reading CSV with duplicate column name
Open
#28,310 opened on Jul 9, 2026
A-io-csvP-mediumacceptedbuggood first issuepython
Repository metrics
- Stars
- (38,496 stars)
- PR merge metrics
- (Avg merge 8d 3h) (156 merged PRs in 30d)
Description
Checks
- I have checked that this issue has not already been reported.
- I have confirmed this bug exists on the latest version of Polars.
Reproducible example
import io
import polars as pl
data = "a,a_duplicated_0,a\n1,2,3"
print(pl.read_csv(io.StringIO(data)))
Log output
Traceback (most recent call last):
File "/home/elt/code/polars-test/csv_dup.py", line 5, in <module>
print(pl.read_csv(io.StringIO(data)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/elt/code/polars-test/.venv/lib/python3.12/site-packages/polars/_utils/deprecation.py", line 128, in wrapper
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/elt/code/polars-test/.venv/lib/python3.12/site-packages/polars/_utils/deprecation.py", line 128, in wrapper
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/elt/code/polars-test/.venv/lib/python3.12/site-packages/polars/_utils/deprecation.py", line 128, in wrapper
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/elt/code/polars-test/.venv/lib/python3.12/site-packages/polars/io/csv/functions.py", line 581, in read_csv
df = _read_csv_impl(
^^^^^^^^^^^^^^^
File "/home/elt/code/polars-test/.venv/lib/python3.12/site-packages/polars/io/csv/functions.py", line 733, in _read_csv_impl
pydf = PyDataFrame.read_csv(
^^^^^^^^^^^^^^^^^^^^^
polars.exceptions.ComputeError: found more fields than defined in 'Schema'
Consider setting 'truncate_ragged_lines=True'.
Issue description
Duplicate column names in the CSV file are renamed into name_duplicated_N, however a column with this name might already exist in the file. (This might actually happen if someone writes back this pl.DataFrame as a csv file.) When this happens, the previous column in the schema is overwritten, so the schema ends up with fewer fields than the dataframe, leading to an obscure error. It would be nice if this was caught and renamed until all names are unique.
Expected behavior
Raise a DuplicateError instead, printing the duplicated name.
Installed versions
--------Version info---------
Polars: 1.42.1
Index type: UInt32
Platform: Linux-6.18.33.2-microsoft-standard-WSL2-x86_64-with-glibc2.39
Python: 3.12.3 (main, Mar 23 2026, 19:04:32) [GCC 13.3.0]
Runtime: rt32
----Optional dependencies----
Azure CLI <not installed>
adbc_driver_manager <not installed>
altair <not installed>
azure.identity <not installed>
boto3 <not installed>
cloudpickle <not installed>
connectorx <not installed>
deltalake <not installed>
fastexcel <not installed>
fsspec <not installed>
gevent <not installed>
google.auth <not installed>
great_tables <not installed>
matplotlib <not installed>
numpy <not installed>
openpyxl <not installed>
pandas <not installed>
polars_cloud <not installed>
pyarrow <not installed>
pydantic <not installed>
pyiceberg <not installed>
sqlalchemy <not installed>
torch <not installed>
xlsx2csv <not installed>
xlsxwriter <not installed>