All 15 File I/O & Context Managers questions
More in File I/O & Context Managers
How would you implement a context manager that temporarily changes the current working directory and restores it afterward? How do you implement contextlib.ExitStack to dynamically manage a VARIABLE number of context managers determined at runtime? How would you write a reusable context manager decorator using contextlib.contextmanager that ALSO works when applied directly to a function, like @contextmanager-decorated functions support? How would you implement a context manager for managing a lock (like threading.Lock) while ALSO adding a timeout, raising an error if the lock can't be acquired quickly enough? How would you implement asynchronous file I/O using aiofiles alongside an async context manager, to avoid blocking the event loop during file operations? How would you write a context manager that atomically writes to a file, avoiding leaving a corrupted or partially-written file if an exception occurs mid-write?
Advertisement
Python › File I/O & Context Managers

How do you correctly handle character encoding when reading text files that might not be UTF-8, and what error occurs if you guess wrong?

Advanced

Answer

Explicitly pass the 'encoding' parameter to open() (e.g., encoding='latin-1' or 'utf-8') rather than relying on the platform-dependent default — reading a file with the WRONG encoding raises a UnicodeDecodeError (for incompatible byte sequences) or silently produces GARBLED text (for compatible-but-wrong encodings that don't error).
try:
    with open("legacy_data.txt", encoding="utf-8") as f:
        content = f.read()
except UnicodeDecodeError:
    with open("legacy_data.txt", encoding="latin-1") as f:  # fallback encoding
        content = f.read()
Real-world example Reading a legacy data file of unknown or non-UTF-8 encoding without crashing or silently corrupting non-ASCII characters.

Common follow-ups: Why does Python's open() NOT default to UTF-8 consistently across all platforms in older versions, and how does PEP 686 address this?

Debugging & Profiling
All 15 File I/O & Context Managers questions