You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, pandas has separate IO methods for each file format (to_csv, read_parquet, etc.). This requires users to:
Remember multiple method names
Change code when switching formats
Feature Description
A unified save/read API would simplify common IO operations while maintaining explicit control when needed:
File type is inferred from the filepath extension, but a file_type arg can be passed to be explicit, raising an error in some cases where the inferred file type disagrees with passed file type.
Both methods accept **kwargs and pass them along to the underlying file-type-specific pandas IO methods.
Optionally, support some basic translation across discrepancies in arg names in existing IO methods (i.e. "usecols" in read_csv vs "columns" in read_parquet).
# Simplest happy path:
df.save('data.csv') # Uses to_csv
df = pd.read('data.parquet') # Uses read_parquet
# Optionally, be explicit about expected file type
df.save('data.csv', file_type="csv") # Uses to_csv
df = pd.read('data.parquet', file_type="parquet") # Uses read_parquet
# Raises ValueError for conflicting file_type info:
df.save('data.csv', file_type='parquet') # Conflicting types
df.save('data.txt', file_type='csv') # .txt implies text format
# Reading allows overrides for misnamed files (or should we require users to rename their files properly first?)
df = pd.read('mislabeled.txt', file_type='parquet')
# Not sure if we should allow save when inferred file type is not a standard type:
df.save('data', file_type='csv') # No extension, needs type
df.save('mydata.unknown', file_type='csv') # Unclear extension
Alternative Solutions
Existing functionality is OK, just not the simplest to use.
Additional Context
No response
The text was updated successfully, but these errors were encountered:
Feature Type
Adding new functionality to pandas
Changing existing functionality in pandas
Removing existing functionality in pandas
Problem Description
Currently, pandas has separate IO methods for each file format (to_csv, read_parquet, etc.). This requires users to:
Feature Description
A unified
save
/read
API would simplify common IO operations while maintaining explicit control when needed:**kwargs
and pass them along to the underlying file-type-specific pandas IO methods.read_csv
vs "columns" inread_parquet
).Alternative Solutions
Existing functionality is OK, just not the simplest to use.
Additional Context
No response
The text was updated successfully, but these errors were encountered: