Python Binary Files: bytearray, readinto, errno, and Safe Replacement
Introduction
Python binary files preserve bytes without text decoding or newline translation. A bytearray provides mutable storage, and readinto can reuse that storage across reads. The returned count identifies which bytes are new; the rest of a buffer can contain an earlier result.
This lesson continues PCAP-31-03 objectives 5.4 and 5.5, with exception handling from 2.1, in Comprehensions, Lambdas, Closures, and I/O (22% of the score, 9 of 40 items). You will trace byte counts, diagnose filesystem failures, and replace an output only after its new content has been written.
Learning objectives
After this lesson you can:
- Create a
bytearrayfrom a size, a list of integers or a bytes object, and index, slice and modify it. - Explain the differences between
bytesandbytearray, and between text and binary files. - Read and write binary files, including in chunks, and use
readinto()with a buffer. - Predict the count returned by
write(), the bytes returned byread(), and the count returned byreadinto(). - Use the
OSErrorattributeserrno,strerrorandfilename, and theerrnomodule constants. - Select the correct exception subclass and order
exceptclauses correctly. - Distinguish decoding errors from operating-system errors and avoid partial updates of a file.
bytes and bytearray
A byte is an integer from 0 to 255. Python has two sequence types for bytes:
| Type | Mutable? | Literal or constructor | Item type |
|---|---|---|---|
bytes |
No | b"abc", bytes([65, 66]) |
int |
bytearray |
Yes | bytearray(b"abc"), bytearray(5) |
int |
Creating a bytearray:
ba = bytearray(5)
print(ba, len(ba))
ba = bytearray(b"abc")
ba[0] = 65
ba.append(100)
print(ba, ba[1], list(ba))
b2 = bytearray([72, 105])
print(b2, b2.decode())Output:
bytearray(b'\x00\x00\x00\x00\x00') 5
bytearray(b'Abcd') 98 [65, 98, 99, 100]
bytearray(b'Hi') Hibytearray(5) makes five bytes with value zero: an integer argument is a size, not a content. A list of integers creates one byte per value. Indexing returns an integer, so ba[1] is 98 (the code of "b"), not b"b" or "b". Slicing returns a new bytearray.
Values outside 0 to 255 are rejected, and bytes objects cannot be modified:
try:
ba[0] = 256
except ValueError as e:
print(e)
try:
bytes(b"ab")[0] = 1
except TypeError as e:
print(e)Output:
byte must be in range(0, 256)
'bytes' object does not support item assignmentOther operations behave like a list of small integers: append, extend, pop, del ba[i], slice assignment, reverse(), concatenation with +, comparison with bytes:
b = bytearray(b"hello")
b.extend(b"!!")
b.reverse()
print(b)
print(bytearray(b"abc") == b"abc", bytes(bytearray(b"ab")))Output:
bytearray(b'!!olleh')
True b'ab'To move between text and bytes, use str.encode() and bytes.decode() (or bytearray.decode()). The length in bytes may differ from the length in characters:
print(len(bytearray("é", "utf-8")), len("é"))Output:
2 1Binary files
Add b to the mode: "rb", "wb", "ab". In binary mode the stream deals only with bytes-like objects. There is no encoding, no newline translation and no str.
data = bytearray(range(250, 256))
with open("p.bin", "wb") as f:
print(f.write(data))
with open("p.bin", "rb") as f:
print(list(f.read(2)), f.read(100) == bytes([252, 253, 254, 255]), f.read())Output:
6
[250, 251] True b''write() returns the number of bytes written (6). read(2) returns a bytes object with two bytes; list() shows the integers. read(100) asks for more than what is left and simply returns the four remaining bytes. A read at the end of the file returns an empty bytes object, b'', which is falsy. This is the standard way to detect the end in a loop.
Writing a str to a binary file is a TypeError, and writing bytes to a text file is also a TypeError. Opening the same file with the wrong kind of mode is a frequent mistake.
Reading into a buffer with readinto()
readinto(buffer) fills an existing mutable buffer, such as a bytearray, and returns the number of bytes actually read. It reuses the caller’s data buffer rather than returning a fresh bytes payload on each read. These examples use blocking regular files; raw non-blocking stream behavior requires separate handling.
with open("p.bin", "rb") as f:
buf = bytearray(4)
k = f.readinto(buf)
print(k, buf)
print(f.readinto(buf), buf)Output:
4 bytearray(b'\xfa\xfb\xfc\xfd')
2 bytearray(b'\xfe\xff\xfc\xfd')The first call fills all four positions. The second finds only two bytes left, returns 2, and overwrites only the first two positions of the buffer. The last two bytes of the buffer, \xfc\xfd, are left over from the previous read. Always use the returned count (for example buf[:k]) and never assume the whole buffer is new.
Copying in chunks
For distinct source and destination files, a bounded-memory copy uses chunks. Opening the destination with wb replaces its prior contents. This example uses ordinary blocking, buffered files:
def copy(src, dst, size=4):
if type(size) is not int or size <= 0:
raise ValueError("chunk size must be a positive integer")
total = 0
with open(src, "rb") as s, open(dst, "wb") as t:
while True:
chunk = s.read(size)
if not chunk:
break
total += t.write(chunk)
return total
print(copy("p.bin", "q.bin"))Output:
6The loop reads blocks of 4 bytes, then 2 bytes, then b'', which ends it. The sum of the values returned by write is the size of the file.
OSError and errno
Errors from the operating system are reported by OSError or by one of its subclasses. An OSError from a system call commonly carries the following information. Some attributes can be None, for example on an exception created manually. See the Python exception hierarchy for matching its subclasses:
| Attribute | Meaning | Example |
|---|---|---|
errno |
Numeric error code | 2 on common platforms for a missing file |
strerror |
Message from the system | 'No such file or directory' |
filename |
Path involved, if any | 'missing/x.bin' |
The module errno defines symbolic names such as errno.ENOENT (no such file or directory), errno.EEXIST (file exists), errno.EACCES (permission denied) and errno.EISDIR (is a directory). Comparing with the name is more readable and more portable than comparing with a number. os.strerror(code) converts a code to a message.
import errno
import os
import tempfile
with tempfile.TemporaryDirectory() as directory:
try:
with open(os.path.join(directory, "missing", "x.bin"), "wb"):
pass
except OSError as e:
print(type(e).__name__, e.errno == errno.ENOENT)
try:
with open(directory, "rb"):
pass
except OSError as e:
print(type(e).__name__, e.errno == errno.EISDIR)A typical POSIX result is:
FileNotFoundError True
IsADirectoryError TrueOn Windows, opening a directory commonly raises PermissionError with EACCES instead. Match the actual exception and symbolic code rather than assuming this second line is portable.
The first attempt fails because the directory missing does not exist. Python maps common error codes to subclasses:
| Exception | Typical errno |
Typical cause |
|---|---|---|
FileNotFoundError |
ENOENT |
Missing file or directory in the path |
FileExistsError |
EEXIST |
Mode "x" or os.mkdir on an existing path |
PermissionError |
EACCES, EPERM |
No right to read or write |
IsADirectoryError |
EISDIR |
Opening a directory as a file |
NotADirectoryError |
ENOTDIR |
A path component is a file |
All of them inherit from OSError. In Python 3, IOError and EnvironmentError are only other names for OSError: IOError is OSError is True.
Order of except clauses
Python tests the clauses from top to bottom and runs the first one that matches. A handler for the base class placed first would catch everything and make the specific one unreachable:
try:
open("missing/x.bin", "wb")
except OSError:
print("os first")
except FileNotFoundError:
print("fnf second")Output:
os firstPut the specific subclasses first and the general OSError last. Remember also that error numbers differ between operating systems, so the portable approach is to catch the subclass (or compare with errno constants), and not to hard-code a number in a condition.
Decoding failures are not OSError
Reading bytes that are not valid for the chosen encoding raises UnicodeDecodeError. It is a subclass of ValueError, not of OSError:
with open("t.txt", "wb") as f:
f.write(b"\xff\xfe")
try:
with open("t.txt", encoding="utf-8") as f:
f.read()
except UnicodeDecodeError as e:
print(type(e).__name__, issubclass(UnicodeDecodeError, ValueError), issubclass(UnicodeDecodeError, OSError))Output:
UnicodeDecodeError True FalseThe file was found and opened correctly; its bytes are invalid for the selected decoder. Compare Python text streams and open modes before treating every read failure as an OSError. A handler that catches only OSError would not catch it. You can decide how to react: open in binary mode and decode explicitly, pass errors="replace" to replace invalid bytes with the character �, or reject the file with a clear message.
Avoiding partial updates
Opening a file with "w" truncates it before anything is written. If the program fails halfway, the old content is gone and the new content is incomplete. A safer pattern writes a temporary file first and replaces the target only after the write succeeds:
import os
import tempfile
from pathlib import Path
def safe_write(path, text):
target = Path(path)
temporary = None
try:
with tempfile.NamedTemporaryFile(
mode="w", encoding="utf-8", newline="\n",
dir=target.parent, prefix="." + target.name + ".",
suffix=".tmp", delete=False
) as stream:
temporary = stream.name
stream.write(text)
os.replace(temporary, target)
finally:
if temporary is not None:
try:
os.unlink(temporary)
except FileNotFoundError:
pass
safe_write("r.txt", "v1")
safe_write("r.txt", "v2")
with open("r.txt", encoding="utf-8") as stream:
print(stream.read())Output:
v2The temporary file has a unique name in the destination directory, keeping replacement on the same filesystem. It is closed before os.replace, which also matters on Windows. A failure before replacement preserves the old target, and the finally block attempts to remove the temporary file. Successful replacement removes the temporary name, so that cleanup tolerates FileNotFoundError. Other cleanup failures propagate.
Successful replacement is atomic; that visibility guarantee is distinct from durable storage after a crash and does not coordinate competing writers.
A permission problem shows up when you open a read-only file for writing as an ordinary user: PermissionError with errno.EACCES. Be aware that a privileged account can bypass permissions, so a test that works for you may behave differently in production; that is why you should handle the exception rather than predict it.
Worked example: copy a payload and diagnose a failing destination
Statement. Predict what the following program prints.
import errno
payload = bytearray(b"ID") + bytearray([0, 255, 10])
with open("in.bin", "wb") as f:
written = f.write(payload)
def copy_to(dst):
try:
with open("in.bin", "rb") as s, open(dst, "wb") as t:
return t.write(s.read())
except FileNotFoundError:
return "no folder"
except OSError as e:
return f"os error {e.errno == errno.EACCES}"
print(written, copy_to("out.bin"), copy_to("nofolder/out.bin"))Step 1: build the payload. bytearray(b"ID") has 2 bytes. bytearray([0, 255, 10]) has 3 bytes. The + operator concatenates them into a bytearray of 5 bytes.
Step 2: first write. write(payload) accepts a bytes-like object and returns 5. The variable written is 5.
Step 3: first copy. in.bin opens, out.bin is created in the current directory, read() returns 5 bytes, and write returns 5. The return inside the with block ends the function, and both files are closed on the way out.
Step 4: second copy. The destination path points to a folder that does not exist. The open(dst, "wb") raises FileNotFoundError. The first except matches, so the function returns "no folder". The second handler is not reached.
5 5 no folderWhy other readings fail. An answer of 3 for the first copy forgets the two bytes of ID. An answer that prints os error False for the last call puts the OSError handler first in mind; but FileNotFoundError is a subclass handled by the earlier clause. Expecting an exception to escape from copy_to ignores that all OSError cases are caught.
Common pitfalls and exam tips
| Pitfall | What happens | Remedy |
|---|---|---|
bytearray(5) read as the content 5 |
Five zero bytes | An integer is a size |
Indexing a bytearray and expecting b"a" |
You get an integer | Use ba[0:1] for a one-byte bytearray, or bytes(ba[0:1]) for bytes |
Writing str to a binary file |
TypeError |
Encode first with .encode() |
Assuming readinto() fills the whole buffer |
Old bytes remain after a short read | Use the returned count |
except OSError before except FileNotFoundError |
Specific handler is unreachable | Put subclasses first |
Catching OSError for a decoding problem |
UnicodeDecodeError escapes |
Handle it separately (ValueError) |
Comparing errno with a literal number |
Not portable | Use errno.ENOENT and similar |
| Writing directly to the final file | Partial update on failure | Write a temporary file, then os.replace |
Exam tip. Check the first argument of bytearray(...): an int means a size, a list means contents, a bytes object means a copy of that content.
Exam tip. The end of a binary file is signalled by an empty bytes object. In loops, while chunk: is the normal idiom.
Exam tip. In a question about exceptions with several except lines, find the first clause that matches the raised class or one of its parents. The remaining clauses are skipped.
Exam tip. Do not confuse length in bytes with length in characters when non-ASCII characters are written.
End-of-lesson case study
Situation. A tool stores a small header and a name, then reads into an eight-byte buffer. A tester reports a surprising result.
name = "Zoë"
raw = bytearray(b"H1") + bytearray(name.encode("utf-8"))
with open("h.bin", "wb") as f:
n = f.write(raw)
buf = bytearray(b"????????")
with open("h.bin", "rb") as f:
k = f.readinto(buf)
print(n, k, buf)Predict the output and explain the surprise.
Solution, commented. The text "Zoë" has 3 characters but 4 bytes in UTF-8, because ë takes two bytes (\xc3\xab). The header b"H1" adds 2 bytes, so raw has 6 bytes and write returns 6. The buffer is 8 bytes long and was filled with question marks. readinto finds only 6 bytes in the file, fills the first six positions and returns 6; the last two question marks remain. The printed result is:
6 6 bytearray(b'H1Zo\xc3\xab??')The surprise is the two question marks at the end: they are not part of the data but leftovers of the previous buffer content. The correct way to use the buffer is buf[:k], which has exactly six bytes. A second remark: if the developer had computed the payload size with len(name) + 2, the value would have been 5, one byte too small, because len(name) counts characters and not bytes.
Key terms
bytes: an immutable sequence of integers from 0 to 255.bytearray: a mutable sequence of integers from 0 to 255.- Binary mode: opening a file with
bso that data are bytes, with no encoding. - Chunk: a block of bytes read or written in one operation.
readinto(): reads into an existing mutable buffer and returns the number of bytes read.OSError: the base class for errors reported by the operating system.errno: the numeric error code of anOSError, with symbolic names in theerrnomodule.- Atomic replacement: replacing a file in one step with
os.replaceso that readers never see a half-written file.
Summary checklist
- I can create and modify a
bytearray, and I know that indexing returns an integer. - I know that
bytesis immutable andbytearrayis mutable. - I can read and write binary files and detect the end with an empty result.
- I can predict the byte counts of
write(), including with non-ASCII text. - I know that
readinto()can leave old bytes in a buffer after a short read. - I can use
errno,strerrorandfilename, and theerrnoconstants. - I can order
exceptclauses from the specific to the general. - I know
UnicodeDecodeErroris aValueError, not anOSError, and how to avoid partial updates.
Official sources
-
Python Institute, PCAP-31-03 exam syllabus (objectives 5.4, 5.5 and 2.1): https://pythoninstitute.org/pcap-31-03-exam-syllabus (opens in a new tab)
-
Python documentation, binary sequence types
bytesandbytearray: https://docs.python.org/3.14/library/stdtypes.html#binary-sequence-types-bytes-bytearray-memoryview (opens in a new tab) -
Python documentation, built-in exceptions (
OSError): https://docs.python.org/3.14/library/exceptions.html#OSError (opens in a new tab) -
Python documentation, the
errnomodule: https://docs.python.org/3.14/library/errno.html (opens in a new tab) -
Python documentation,
os.replace: https://docs.python.org/3.14/library/os.html#os.replace (opens in a new tab) -
Temporary files and cross-platform closing: https://docs.python.org/3.14/library/tempfile.html (opens in a new tab)
-
Windows directory-open error mapping: https://github.com/python/cpython/issues/87261 (opens in a new tab)
