Skip to content

Python Text and Binary Files and Resource Cleanup

Step 36 of 41

Python Text Files: open Modes, UTF-8, Reading, and Cleanup

Introduction

Python text files are accessed through stream objects that encode and decode between strings and bytes. Choosing w erases an existing file at open time; choosing a preserves content and appends. Reading a line also advances the position for the next operation.

This lesson covers objectives 5.4 and 5.5 in PCAP-31-03 Comprehensions, Lambdas, Closures, and I/O (22% of the score, 9 of 40 items). You will trace reads and writes, control encoding and newlines, and use a file context manager for cleanup.

Learning objectives

After this lesson you can:

  1. Name the three standard streams and explain which functions use them.
  2. Choose the correct mode for reading, writing, appending, exclusive creation and update.
  3. Explain the difference between text mode and binary mode, and the role of the encoding.
  4. Use read(), readline(), readlines(), iteration over a file, write() and writelines() and predict their results.
  5. Follow the file position with tell() and seek().
  6. Use with to guarantee that a file is closed, even when an exception occurs.
  7. Handle FileNotFoundError and other OSError subclasses for a missing file.

Standard streams

When a Python program starts, three streams are already open and available in the sys module:

Stream Object Default use
Standard input sys.stdin Keyboard input; used by input()
Standard output sys.stdout Normal output; used by print()
Standard error sys.stderr Error messages and diagnostics

print() writes to sys.stdout by default, and its file argument lets you redirect the output to another stream:

Python
import sys
print("a", "b", sep="-", end="!\n", file=sys.stdout)
print("problem", file=sys.stderr)
print(sys.stdin.name, sys.stdout.name, sys.stderr.name)

In a typical command-line interpreter with its original standard streams, standard output is:

Text
a-b!
<stdin> <stdout> <stderr>

The word problem goes to standard error and is not part of the standard output. In a terminal both are displayed, but they can be separated with shell redirection. You do not close these three streams in normal code.

Opening a file

open() returns a stream object. The most important parameters are the file name, the mode and, for text, the encoding:

Python
f = open("report.txt", "w", encoding="utf-8")
print(f.name, f.mode, f.closed)
f.close()
print(f.closed)

Output:

Text
report.txt w False
True

The mode is a short string. Memorise this table:

Mode Meaning If the file exists If the file does not exist
"r" (default) Read Opens at the start FileNotFoundError
"w" Write Truncated to zero length Created
"a" Append Keeps content; writes go to the end Created
"x" Exclusive creation FileExistsError Created
"r+" Read and write Keeps content; position at the start FileNotFoundError
"w+" Read and write Truncated Created
"a+" Read and append Keeps content; writes go to the end Created

Add "t" for text (the default) or "b" for binary: "rb", "wb", "ab". Opening a file with "w" destroys its previous content immediately, even if you write nothing:

Python
with open("stock.txt", "w") as f:
    pass
import os
print(os.path.getsize("stock.txt"))

Output:

Text
0

This is a classic exam trap. Opening the wrong existing file in write mode erases its contents without removing the file’s directory entry.

Text mode and encoding

In text mode, the stream works with str objects. Python converts them to bytes with an encoding when writing and back when reading. In Python 3.14, the default text encoding depends on locale and UTF-8 mode configuration, so it need not be the same in every runtime. Always specify encoding="utf-8" for portable text.

Python
with open("stock.txt", "w", encoding="utf-8", newline="\n") as f:
    print(f.write("café,12\n"), f.write("tea,5\n"))

with open("stock.txt", "rb") as f:
    raw = f.read()
print(raw, len(raw))

Output:

Text
8 6
b'caf\xc3\xa9,12\ntea,5\n' 15

write() returns the number of characters written: 8 for café,12\n (the accent counts as one character) and 6 for tea,5\n. The explicit newline="\n" prevents platform newline translation, making this byte count reproducible on Windows as well as Linux. The binary read shows the encoded content: the letter é takes two bytes in UTF-8 (\xc3\xa9), so the file contains 15 bytes in total, not 14. The distinction between characters and bytes is central to this chapter: text mode counts characters, binary mode counts bytes.

Invalid byte sequences for the selected decoder raise UnicodeDecodeError (a subclass of ValueError, not of OSError). This is a decoding problem rather than an operating-system error. A wrong decoder can also silently produce wrong text when the bytes are valid in it. Review Unicode code points and UTF-8 bytes to distinguish those cases.

Reading text

Four ways to read from a text stream:

Call or form Returns Position after
f.read() The rest of the file as one string End of file
f.read(n) At most n characters After them
f.readline() Next line, retaining a terminator if present; "" at EOF After that line
f.readlines() A list of all remaining lines End of file
for line in f Lines, one at a time, lazily Advances line by line

Examples on the file above:

Python
with open("stock.txt", encoding="utf-8") as f:
    print(f.readline(), f.readline(), f.readline() == "")

with open("stock.txt", encoding="utf-8") as f:
    print(f.readlines())

with open("stock.txt", encoding="utf-8") as f:
    for line in f:
        print(line.rstrip("\n").split(","))

Output:

Text
café,12
 tea,5
 True
['café,12\n', 'tea,5\n']
['café', '12']
['tea', '5']

In the first output, each readline() keeps its newline, and print adds a space between the arguments and its own newline, which explains the layout. For a normal read at EOF, readline() returns "" rather than raising StopIteration. A closed stream, decoding failure, or I/O failure can still raise an exception. Iterating directly over the file is the most memory-efficient way to process a large file, because only one line is held at a time.

read(n) advances the position; a later call continues where the previous one stopped:

Python
with open("a.txt", "w") as f:
    f.write("hello world")
with open("a.txt") as f:
    print(f.read(5), f.read(3), repr(f.read()), repr(f.read()))

Output:

Text
hello  wo 'rld' ''

The first read gives hello, the second gives a space followed by wo (three characters), the third gives what is left, rld, and the last read finds the end of the file and returns an empty string.

Writing text

write(s) writes a string and returns its length; it does not add a newline. writelines(list) writes each string of an iterable, also without adding separators.

Python
with open("log.txt", "w") as f:
    f.writelines(["a", "b"])
with open("log.txt") as f:
    print(f.read())

Output:

Text
ab

If you want lines, include "\n" yourself. Appending uses mode "a":

Python
with open("stock.txt", "a", encoding="utf-8") as f:
    f.write("milk,7\n")
with open("stock.txt", encoding="utf-8") as f:
    print(len(f.readlines()))

Output:

Text
3

Wrong operations raise io.UnsupportedOperation, which is both an OSError and a ValueError:

Python
with open("a.txt") as f:
    f.write("z")        # not writable

The message is not writable. Reading from a file opened with "w" gives not readable. Using a closed stream raises ValueError: I/O operation on closed file.

The file position: tell() and seek()

On a seekable stream, tell() reports a position and seek changes it. For text streams that position is an opaque cookie, not generally a character or byte count. Use a previous cookie with the default origin or seek(0) to rewind. Text streams also permit zero-offset seeks to the current position or end; nonzero relative seeks are unsupported.

Python
with open("a.txt") as f:
    print(f.tell(), f.read(2), f.tell())
with open("a.txt", "r+") as f:
    f.write("J")
    f.seek(0)
    print(f.read())

Output:

Text
0 he 2
Jello world

In "r+" mode the position starts at 0, so write("J") overwrites the first character. After writing, the position is 1; seek(0) returns to the start before reading everything.

Closing a file and the with statement

An open stream can retain unwritten buffered data and an operating-system resource. Closing flushes Python’s buffer; this is separate from guaranteeing storage durability after power loss. You must close it. Calling f.close() works only if no exception interrupts the code before it. The with statement calls the closing method automatically when the block ends, whether it ends normally or because of an exception:

Python
try:
    with open("a.txt") as f:
        raise RuntimeError("x")
except RuntimeError:
    print(f.closed)

Output:

Text
True

The name f still exists after the block, but the file is closed. The context manager runs cleanup during normal Python unwinding; see exception cleanup with finally and with for how exceptions leave a protected block. Several files can be managed in one statement: with open(a) as s, open(b, "w") as t:.

Handling a missing file

When the file does not exist, open(..., "r") raises FileNotFoundError, a subclass of OSError with useful attributes:

Python
try:
    with open("nope.txt") as f:
        pass
except FileNotFoundError as e:
    print(e.errno, e.strerror, e.filename)

Assuming nope.txt is missing, one common English-language system reports the following; message wording is platform-dependent:

Text
2 No such file or directory nope.txt

Mode "x" raises FileExistsError if the file is already there. Catching the specific subclass is better than catching all OSError, and a try block should surround only the code that can fail with that error. The OSError hierarchy is the subject of the next lesson.

Text files hold strings, not numbers

A text file is stored as bytes interpreted through an encoding. The text stream exposes characters. If you write the number 12 with f.write(str(12)), the file holds the two characters 1 and 2. When you read it back you get the string "12", and you must convert it with int() or float() yourself. Passing a number directly to write() raises TypeError: write() argument must be str, not int. This matters in practice: a program that reads quantities and adds them without converting concatenates strings ("12" + "5" gives "125") instead of computing a sum. Conversions can fail with ValueError when a line is damaged, so a loader that reads user-supplied files should decide whether to skip such a line, report it, or stop.

Another practical consequence concerns blank lines. A file that ends with a final newline produces no extra element with readlines(), but read().split("\n") produces a last empty string. Prefer iterating over the file or using splitlines(), which does not create that trailing empty item.

Worked example: write and reload a stock report

Statement. A shop saves its stock in a UTF-8 file, then reloads the quantities. Predict the output of:

Python
rows = [("café", 12), ("tea", 5), ("milk", 7)]

with open("stock.csv", "w", encoding="utf-8") as f:
    for name, qty in rows:
        f.write(f"{name},{qty}\n")

total = 0
with open("stock.csv", encoding="utf-8") as f:
    first = f.readline()
    for line in f:
        total += int(line.split(",")[1])

print(repr(first), total)

Step 1: the content written. Three lines, each ending with \n: café,12, tea,5, milk,7.

Step 2: the first read. readline() returns the first line with its newline: 'café,12\n'. The position is now at the start of the second line.

Step 3: the loop. The for loop continues from the current position, so it sees only the second and third lines, not the first. It adds 5 and 7.

Step 4: the result. total is 12, not 24, because the first line was consumed by readline() and never reached the loop.

Text
'café,12\n' 12

Why other answers are wrong. 24 assumes the loop restarts from the beginning of the file. An output of 'café,12' without \n forgets that readline() keeps the newline. Assuming the successful writes shown and no intervening external changes, the later read finds the file created by the first block.

Common pitfalls and exam tips

Pitfall What happens Remedy
Opening with "w" to read the file Content is erased Use "r", or "a" to add
Forgetting \n in write() All text on one line Add "\n" explicitly
Expecting readline() to strip the newline The \n is kept Use rstrip("\n")
Calling read() twice Second call returns "" Use seek(0) or reopen
Mixing readline() and for on the same file The loop continues from the current position Trace the position
Not giving an encoding Platform-dependent result Use encoding="utf-8"
Reading or writing after the with block ValueError on a closed file Perform I/O inside the block; metadata such as f.closed can still be inspected afterward
Catching Exception for a missing file Hides other errors Catch FileNotFoundError

Exam tip. readlines() retains each line terminator that is present; a final unterminated line has no newline to retain. Count characters carefully in the options.

Exam tip. "w" truncates at open time. If a question opens a file with "w" and then reads the old content, the answer is that the content is gone.

Exam tip. write() returns the number of characters in text mode and the number of bytes in binary mode.

Exam tip. Text-mode reads produce Unicode strings; len of the same data read in "rb" can be larger when non-ASCII characters are present.

End-of-lesson case study

Situation. A script must append the daily total to a report and then show the last line. A colleague wrote:

Python
with open("report.txt", "w", encoding="utf-8") as f:
    f.write("mon,10\ntue,20\n")

with open("report.txt", "w", encoding="utf-8") as f:
    f.write("wed,30\n")

with open("report.txt", encoding="utf-8") as f:
    lines = f.readlines()

print(len(lines), lines[-1].strip())

Predict the output and correct the problem.

Solution, commented. The first block creates the file with two lines. The second block uses mode "w", so opening the file truncates it: the two earlier lines are lost and only wed,30\n is written. The last block reads one line. The output is:

Text
1 wed,30

The developer wanted to keep the existing lines, so the second block should use "a". With "a", the writes go to the end of the existing content, and the program prints 3 wed,30. Mode "a" also creates a missing file when its parent directory exists. If a separate workflow starts by reading an optional report, handle FileNotFoundError at that read; appending does not create missing directories.

Key terms

  • Stream: an object that provides sequential access to data, such as a file or a standard stream.
  • Mode: the string passed to open() that defines reading, writing, appending, creation and text or binary processing.
  • Text mode: a mode in which data are str objects converted with an encoding.
  • Encoding: the rule that maps characters to bytes, for example UTF-8.
  • File position: the offset where the next read or write occurs; read with tell(), changed with seek().
  • Context manager (with): a construct that guarantees cleanup, here closing the file.
  • Truncate: reduce a file to zero length, which "w" does when the file is opened.

Summary checklist

  • I know sys.stdin, sys.stdout and sys.stderr, and where print() writes.
  • I can choose between "r", "w", "a", "x" and "r+".
  • I know "w" truncates the file as soon as it is opened.
  • I can explain the difference between characters and bytes, and why I give an encoding.
  • I can predict the results of read, readline, readlines and iteration.
  • I know write() adds no newline and returns a count.
  • I can follow the position with tell() and seek().
  • I use with to close files, and catch FileNotFoundError precisely.

Official sources