Python Module Search Path and Import Errors for PCAP
Introduction
The Python module search path helps explain why an import can succeed yet load an unexpected module. A local random.py can hide the standard library’s random, causing a missing-attribute error only after the import runs. Inspecting the loaded module’s origin tells you which file Python actually used.
This PCAP-31-03 lesson continues objective 1.1 of Modules and Packages (12% of the score, 6 of 40 items). You will trace module discovery, distinguish the module cache from the path search, and diagnose import failures. Review Python imports, aliases, and namespaces if you need to check which names an import binds.
Learning objectives
After this lesson you can:
- Explain what happens the first time and each later time a module is imported.
- Describe how
sys.pathcontrols the search and what its first entry is. - Use dotted names to import a module that lives inside another one (for example
os.path). - Explain why a local file named like a standard-library module changes program behaviour.
- Read a traceback top to bottom and identify the failing import line and the exception type.
- Apply safe habits that avoid shadowing without editing any global Python installation paths.
What import really does
When Python executes import tools, it checks sys.modules for the full module name. A cached module can be reused. A cache entry explicitly set to None instead blocks that import with ModuleNotFoundError.
On a cache miss, finders on sys.meta_path look for the module. The default finders handle built-in and frozen modules as well as the path-based search. For a top-level module the path-based finder uses sys.path. For a submodule it uses its parent package’s __path__.
For an ordinary source module, Python creates a module object and puts it in sys.modules before the loader executes its code. That order allows another module to find the same object during a circular import, though some attributes may not exist yet. If execution fails, the failing module’s cache entry is removed. After a successful import, the statement binds the requested name in the importing scope.
Suppose greeting.py contains:
print("greeting is being loaded")
def hello():
return "hello"and the main script does:
import greeting
import greeting
print(greeting.hello())The output is:
greeting is being loaded
helloThe message appears once. The second import greeting finds the cached module in sys.modules and only re-binds the name. This is the reason you can safely import the same module from many different files in a project: in one interpreter, ordinary imports reuse the same module object while its cache entry remains. Explicit reload and import under a different name can change that behavior.
You can look at the cache yourself:
import sys
import math
print('math' in sys.modules) # True
print(sys.modules['math'] is math) # TrueThe search path: sys.path
sys.path is an ordinary Python list of strings. Entries normally name directories or archives for the path-based finder. For a regular file-backed module, the first matching entry normally wins within that search. Built-in modules, frozen modules, and cached modules can be resolved before the path is used.
A typical content is:
- In a normal invocation, the script’s directory for
python script.py, the initial working directory forpython -m package, or an empty string representing the working directory forpython -cand an interactive prompt. - Directories listed in the
PYTHONPATHenvironment variable, if it is set. - The standard-library directories of the installation.
- The
site-packagesdirectory, where third-party packages installed withpiplive.
import sys
for entry in sys.path:
print(entry)The entries depend on the interpreter, environment, and invocation. For example, -P skips the initial potentially unsafe path entry and -I enables isolated mode. The usual script directory entry explains why a local file can shadow a standard-library module.
Because sys.path is a list, a script can modify it:
import sys
sys.path.append("/opt/company/shared") # searched last
sys.path.insert(0, "/home/anna/experiments") # searched firstAppending adds a directory at the end. Inserting at index 0 puts it first within the path-based search. Neither action replaces a cached module. Use this sparingly. Editing the search list inside a program is a fallback for special situations; it makes behaviour depend on the file system layout and is harder to reproduce than a normal project structure.
Python also keeps compiled bytecode in
__pycache__folders to load modules faster. A normal source-backed__pycache__entry is not a substitute for a deleted source file.
Qualified names for nested modules
Modules can be grouped inside other modules or packages. A dotted name can name a package and its submodule. The standard library also offers the special case os.path, which points to the path implementation for the current operating system:
import os.path
print(os.path.basename("/var/log/app.log")) # app.logTwo points often tested:
- After
import os.path, the name bound in your namespace isos(the top-level one), and you reach the inner module asos.path. There is no separate namepath. from os import pathbindspathdirectly, andosis not bound.
import os.path
print(os.getcwd() is not None) # True: os is bound
print(path) # NameError: name 'path' is not definedfrom os import path
print(path.basename("/tmp/x.txt")) # x.txt
print(os.getcwd()) # NameError: name 'os' is not definedYou can also give the whole chain an alias: import os.path as osp binds only osp.
For an ordinary package, import urllib.parse binds urllib and exposes the submodule as urllib.parse. See Python package structure and imports for examples with packages you write yourself.
When the wrong file wins: shadowing by local modules
A file in the script directory can shadow a standard-library module during the usual path-based search if that module is not already cached or resolved by an earlier finder. Check the origin when a module behaves unexpectedly.
Worked example: a project that contains random.py
A developer creates a folder dice_game with two files.
random.py (written as an experiment):
print("loading local random")
def choice(items):
return items[0]main.py:
import random
print(random.choice([5, 6, 7]))
print(random.randint(1, 3))The developer runs python main.py from inside dice_game and gets:
loading local random
5
Traceback (most recent call last):
File "main.py", line 4, in <module>
print(random.randint(1, 3))
AttributeError: module 'random' has no attribute 'randint'Step 1: read the traceback from the bottom. The final line gives the exception type and message: AttributeError: module 'random' has no attribute 'randint'. The type is not ImportError; the import succeeded, so the failure happened later, when the attribute was accessed.
Step 2: ask why a module lacks randint. The real random module defines randint, so the loaded module cannot be the real one. The message says "module 'random'", so the name is a module, just not the expected one.
Step 3: notice the unexpected output. The text loading local random was printed, and choice returned 5, the first item, instead of a random one. That is the developer's own random.py running.
Step 4: connect the dots with sys.path. In this fresh interpreter, random has not been loaded and the path-based search finds the script directory’s random.py first.
Step 5: confirm. Printing random.__file__ shows the path of the loaded file (for example /home/anna/dice_game/random.py), which proves which module was used.
Fix. Rename the local file to dice_helpers.py. Then start a new Python process so the old module object is no longer cached.
Why other fixes are worse. Adding sys.path.append for the standard library would not help: the local directory is still first. Reinstalling Python does nothing, because the interpreter is not broken. Editing the installation's search paths changes every project on the machine for a problem that belongs to a single folder.
A subtle variant
The shadowing can also occur indirectly. If a library imports random and the path-based search selects your project’s random.py, the library receives your module too. You may see a traceback pointing inside the library, with the root cause in your folder. Always look at the file names in the traceback: a path that mixes library files and your own project files is a strong hint.
Another variant: a script named like the module it imports
Suppose a script named random.py contains import random followed by random.sample(...). Running it as a script executes it as __main__. The import can load the same file again under the name random. When that second execution imports random, it sees the partially initialized module in the cache. Rename the script and start a new interpreter to avoid the cycle.
Reading a traceback for import errors
A traceback lists the call chain from the oldest call at the top to the failing line at the bottom. For imports, three exception types matter:
| Message | Exception | Meaning |
|---|---|---|
No module named 'inventory' |
ModuleNotFoundError |
The requested module or one of its imports could not be found |
cannot import name 'load' from 'inventory' |
ImportError |
The import could not resolve the name load |
module 'inventory' has no attribute 'load' |
AttributeError |
Module imported with import, attribute missing |
Catch ModuleNotFoundError only for the optional module you expected to be missing. A broad except ImportError can hide a failure inside a module that was found.
try:
import fastjson as json
except ModuleNotFoundError as error:
if error.name != "fastjson":
raise
import json
print(json.dumps({"a": 1}))If fastjson itself is absent, the example falls back to the standard json module. If a dependency of fastjson is missing, the original error is raised so the broken installation stays visible. The alias lets the remaining code use the same name in either case.
Inspecting where a module comes from
A few attributes help you diagnose a problem without altering anything:
module.__name__: the module's name, for example'random'.module.__file__: the module’s file path, when available. Built-in and namespace modules may lack it.module.__spec__.origincan provide more detail.sys.path: where Python searched.sys.modules: what has already been loaded.
import json
print(json.__name__) # json
print(json.__file__) # e.g. /usr/lib/python3.11/json/__init__.pyA short diagnostic routine for any suspicious import:
import random, sys
print(random.__file__)
print(sys.path[0])Compare the resolved paths to see whether the module file is inside the project. A shared text prefix alone is not enough to establish that.
Comparing the failure modes
| Situation | Symptom | Typical cause | Remedy |
|---|---|---|---|
Local random.py |
AttributeError on library functions, odd output |
Script folder searched first | Rename the local file |
| Package not installed | ModuleNotFoundError |
Not on sys.path |
Install it, or fix the project layout |
| Misspelt name inside module | ImportError: cannot import name |
Name not defined in that module | Fix the spelling or the module |
| Name rebound after import | AttributeError or TypeError |
Shadowing in your own code | Rename the variable |
| Module imported twice | Nothing (print runs once) | Cache in sys.modules |
None needed |
Common pitfalls and exam tips
- Exam tip: "Which statement is executed when a module is imported twice?" The top-level code runs only the first time; later imports use
sys.modules. - Exam tip:
sys.pathis a list, so list methods such asappend,insertand indexing work on it. The first match wins. - Pitfall: Believing that running
python main.pysearches the current working directory. For a script, Python puts the script's own directory first; forpython -cor an interactive session it puts an empty entry that means the current directory. - Pitfall: Assuming that an
AttributeErroron a module means the library is broken. First printmodule.__file__. - Pitfall: Naming your own files
test.py,random.py,math.py,string.pyorjson.py. - Pitfall: Believing
import os.pathcreates a namepath. It bindsos. - Exam tip: When two answers differ only by exception type, ask whether the import failed (
ImportError,ModuleNotFoundError) or a later attribute access failed (AttributeError). - Pitfall: Changing the global installation (copying files into the standard-library folder or editing site configuration) to "fix" one project. Fix the project instead.
Mini case study: the report that always gives the same answer
A data analyst writes report.py that picks 3 random customers for a satisfaction call. After a refactoring, the script raises an error before it selects any customers. The folder now looks like this:
calls/
report.py
random.py # created last week to test a fixed ordering
customers.pyreport.py:
import random
from customers import ALL
print(random.sample(ALL, 3))The run prints:
Traceback (most recent call last):
File "report.py", line 4, in <module>
print(random.sample(ALL, 3))
AttributeError: module 'random' has no attribute 'sample'Questions. Why does the error appear, which line causes it, and what is the minimal fix? Would a try/except around the import help?
Solution.
- The error appears because
import randomon line 1 loadscalls/random.py, which lackssample. The script's folder is first onsys.path. - The failing line is line 4, where the attribute is used, but the cause is the file name. The traceback shows the symptom.
- The minimal fix is to rename
random.py(for example tofixed_order.py) and re-run. No code inreport.pyhas to change. - A
try/except ImportErrorwould not help: the import succeeds, since a module namedrandomis found. The exception type isAttributeError, raised later. CatchingAttributeErrorwould only hide the problem.
An extra check proves the diagnosis: python -c "import random; print(random.__file__)" run from a different directory prints the standard-library path, while the same command from calls/ prints the local file.
Key terms
sys.path: ordered search locations used by the path-based finder for top-level modules.sys.modules: the cache of modules already imported.- Module search: the process of resolving a module name through Python’s import machinery.
- Shadowing (of a module): a local file hiding a library module of the same name.
- Dotted name: a path to a nested module, such as
os.path. ModuleNotFoundError: raised when no module of that name can be found.__file__: attribute holding the path of the file a module was loaded from.- Traceback: the report of the call chain that ended in an exception.
Summary checklist
- I can explain the cache, finders, loader execution, and name binding.
- I know when an ordinary repeated import reuses a module without rerunning its code.
- I can identify the normal first path entry for a script and explain when the path search is skipped.
- I can predict what
import a.bbinds and whatfrom a import bbinds. - I can tell
ModuleNotFoundError,ImportErrorandAttributeErrorapart in a traceback. - I can diagnose a local file that shadows a library module and fix it by renaming.
- I can use
__file__,sys.pathandsys.modulesto inspect an import.
Official sources
- Python Institute, PCAP-31-03 syllabus, objective 1.1: https://pythoninstitute.org/pcap-31-03-exam-syllabus (opens in a new tab)
- Python documentation, the import system: https://docs.python.org/3.14/reference/import.html (opens in a new tab)
- Python documentation, search path initialization: https://docs.python.org/3.14/library/sys_path_init.html (opens in a new tab)
- Python documentation, the
sysmodule: https://docs.python.org/3/library/sys.html (opens in a new tab) - Python tutorial, modules: https://docs.python.org/3/tutorial/modules.html (opens in a new tab)
