Skip to content

Python Modules, Packages, and Entry Points

Step 8 of 41

Python Package Structure, Relative Imports, and Module Execution

Introduction

A Python package structure groups related modules under qualified names such as shop.billing.invoice. Understanding that structure helps you choose imports and run a module with the package context it needs.

This PCAP-31-03 lesson covers regular packages in objective 1.5 and revisits import syntax from objective 1.1. You will trace __init__.py, compare absolute and relative imports, and diagnose why a command works in one environment but fails in another.

Learning objectives

After this lesson you can:

  1. Map a dotted name such as shop.billing.invoice to a folder and file structure.
  2. Explain the role of __init__.py and predict when it runs.
  3. Use absolute and explicit relative imports inside a package.
  4. Refer to nested modules with the correct qualified name.
  5. Explain the difference between running a file inside a package as a script and running it with python -m.
  6. Use sys.path, dir() and __path__ to diagnose why an import fails.

From module to package

A package is a module with a __path__ attribute. A regular file-system package has an __init__.py file and can contain submodules. Namespace packages are another supported form. Take this project:

Text
project/
    main.py
    shop/
        __init__.py
        catalog.py
        billing/
            __init__.py
            invoice.py
            tax.py

The packages are shop and shop.billing. The modules are shop.catalog, shop.billing.invoice and shop.billing.tax. In this file-system example, the dots correspond to package boundaries. In main.py you can write:

Python
import shop.billing.invoice

shop.billing.invoice.make(42)

With this form the full qualified name must be used every time. You can shorten it with an alias or with from:

Python
import shop.billing.invoice as inv
from shop.billing import invoice
from shop.billing.invoice import make

All three forms load the same module object once. They differ only in the name that is created in main.py: inv, invoice or make.

Where Python looks

When main.py is started, Python puts the folder of that script at the front of sys.path. Because project/ is on the path, the name shop is found as a folder inside it. Python then looks for billing inside shop, and for invoice inside billing. The parent directory of shop must be discoverable. A normal python main.py command uses the script’s directory, even if your shell is in another directory:

Python
import sys
print(sys.path[0])     # folder of the script that was started

sys.path is a normal list of strings. You can append to it, but a cleaner solution is to arrange your files so that the top-level package is in the folder of the script (or installed in the environment).

The role of __init__.py

The file __init__.py marks a folder as a regular package and holds the package initialisation code. It runs automatically the first time anything in the package is imported, and before the submodule that you asked for.

Python
# shop/__init__.py
print("shop init")
from . import catalog
Python
# shop/catalog.py
print("catalog loaded")
ITEMS = ["pen", "book"]
Python
# shop/billing/__init__.py
print("billing init")
Python
# shop/billing/invoice.py
from ..catalog import ITEMS

def count():
    return len(ITEMS)

def make(number):
    return {"number": number, "item_count": count()}

print("invoice", __name__)

If main.py contains only import shop.billing.invoice, the output is:

Text
shop init
catalog loaded
billing init
invoice shop.billing.invoice

The order follows the dotted path from left to right: first the package shop is initialised (and its own from . import catalog executes), then the sub-package shop.billing, and finally the module invoice. Each initialisation happens once, thanks to sys.modules.

Several points are tested again and again:

  • __init__.py can be empty. It is still the file that makes the intention explicit, and an empty one prints nothing.
  • Importing a package binds only the package: import shop gives you shop, with the attributes that __init__.py created. Submodules are not loaded automatically. After a bare import shop.billing the name shop.billing.invoice fails with AttributeError unless something imported invoice somewhere.
  • The __init__.py file may import submodules, define variables, and define __all__, which controls from shop import *.
  • A package has a __path__ iterable used to discover its submodules. A normal file-system regular package typically has one directory there.

Re-exporting names for a cleaner interface

A package can expose a public entry point by importing useful names in __init__.py. For this separate example, add tax.py with def vat(amount): return amount * 0.20, and replace the original billing initializer with:

Python
# shop/billing/__init__.py
from .invoice import make
from .tax import vat

__all__ = ["make", "vat"]

Callers can now write from shop.billing import make without knowing that the function lives in invoice.py. The dot in .invoice is an explicit relative import, explained next.

Absolute and relative imports

Inside a package, a module can refer to its siblings in two ways.

Kind Example in shop/billing/invoice.py Meaning
Absolute from shop.catalog import ITEMS Starts from a top-level package found through sys.path
Relative, same package from . import tax or from .tax import vat . = the package that contains this module (shop.billing)
Relative, parent package from ..catalog import ITEMS .. = the parent of that package (shop)
Relative, grandparent from ...x import y each extra dot goes one level higher

Rules to remember:

  • Relative imports always use the from ... import ... form. import .tax is a syntax error.
  • They require a known package context. In Python 3.14, module metadata such as __spec__.parent supplies that context.
  • Going above the top-level package raises an error ("attempted relative import beyond top-level package").
  • Absolute imports are easier to read and are recommended for most projects; relative imports are convenient when a package may be renamed.

Running a file that lives inside a package

Review module lifecycle and entry points for how the same source can be executed under a different module name.

This is the most common source of confusing errors. Suppose invoice.py contains from ..catalog import ITEMS.

Text
$ python shop/billing/invoice.py
ImportError: attempted relative import with no known parent package

Started directly, the file runs as __main__ without a known parent package, so the relative import cannot be resolved. Also, sys.path[0] becomes shop/billing, not project, which would break the absolute form import shop... too.

The remedy is to run it as a module from the folder that contains shop:

Text
$ cd project
$ python -m shop.billing.invoice
shop init
catalog loaded
billing init
invoice __main__

With -m, Python finds and imports the package chain, then executes the target as __main__ with a package context. Relative imports work. For this output, use the original billing/__init__.py containing only print("billing init"). If it re-exports from invoice, that file is imported before execution as __main__, which can also produce a warning.

Worked example: diagnosing a failing import

Problem. A developer has this layout and runs python app/main.py from the folder work/:

Text
work/
    app/
        main.py
        helpers/
            __init__.py
            text.py
    lib/
        __init__.py
        report.py

main.py contains:

Python
from helpers.text import clean
from lib.report import build

The first line works, the second one raises ModuleNotFoundError: No module named 'lib'. Why, and how can it be fixed?

Step 1: find sys.path[0]. For python app/main.py the first entry is the folder of the script, work/app.

Step 2: check line 1. helpers is a folder directly inside work/app, so it is found. Python imports helpers/__init__.py, then text.py. It works.

Step 3: check line 2. Python looks for lib inside work/app (the first entry) and then in the other path entries, but lib is in work/, which is not on sys.path. The error is raised.

Step 4: pick a fix. There are three reasonable options:

  1. Move lib inside app/ so that it is next to main.py. Simple, and it makes app self-contained.
  2. Run the program as a module from work/: python -m app.main. The current folder (work/) is then the first path entry, so lib is found. For this to work, app should be a regular package with an __init__.py file, and the import in main.py becomes from app.helpers.text import clean.
  3. Add work to sys.path (with sys.path.append) or set the environment variable PYTHONPATH. This works but couples the code to the environment, so it is the least elegant.

Solution. The best answer is option 2 or option 1: change the layout or the command, not the interpreter configuration.

Why the others are weaker. Option 3 hides the real problem, which is the location of the project root, and breaks again on another machine.

Qualifying nested modules

The name you use after the import depends on the statement. Using the project from the beginning:

Statement Name you must use
import shop shop.catalog works only if __init__.py imported catalog
import shop.billing.tax shop.billing.tax.vat(...)
import shop.billing.tax as t t.vat(...)
from shop.billing import tax tax.vat(...)
from shop.billing.tax import vat vat(...)
from shop.billing import * the names in __all__ (or the public names if __all__ is missing; submodules not imported earlier are not included)

A frequent trap: import shop.billing.tax binds only shop. The names billing and tax are reached as attributes of shop, so tax.vat(...) without the prefix raises NameError.

Looking inside: dir(), __file__ and __path__

The built-in dir() lists the names in a namespace. For an ordinary module or package, it is a useful way to inspect available names. A custom __dir__ can customize the list:

Python
import shop.billing
print([n for n in dir(shop.billing) if not n.startswith("_")])
# ['invoice', 'make', 'tax', 'vat']

It shows the submodules that have been imported (here, because __init__.py loaded them) and the names that __init__.py defined. shop.billing.__path__ gives the folder list of a package, while shop.billing.invoice.__file__ gives the path of a module file. Both are useful when you suspect that Python is importing a different copy than the one you edit.

A local random.py can shadow the standard library during a normal path-based search if the module has not already been loaded. See Python module discovery. The standard library module is hidden and the program fails with strange errors ("module 'random' has no attribute 'randint'"). Never name your modules like standard modules.

Comparing ways to organise code

Approach Good when Weakness
One module Small scripts Grows unmanageable
Several flat modules next to the script Medium programs Name collisions, no grouping
A package with sub-packages Reusable libraries, large programs Needs correct start location and imports
Package + __main__.py A package that can run with python -m pkg One more file to maintain

A package can contain a file named __main__.py. When you run python -m shop, Python executes shop/__main__.py as the main program. This is the standard way to give a package a command-line entry point.

Quick tracing drill

Use this drill to check yourself. Given import shop.billing only (with the files above), which of these statements work afterwards?

  1. shop.catalog.ITEMS: works, because shop/__init__.py ran from . import catalog, which stored catalog as an attribute of shop.
  2. shop.billing.make: works, because shop/billing/__init__.py imported make from .invoice.
  3. shop.billing.invoice.count(): works, because importing .invoice inside __init__.py also made invoice an attribute of shop.billing.
  4. billing.make: fails with NameError, because import shop.billing binds only the name shop.

Notice that every answer is explained by what the __init__.py files actually executed, not by the folder layout alone. If you delete the line from .invoice import make from shop/billing/__init__.py, statements 2 and 3 stop working until some code imports shop.billing.invoice explicitly.

Common pitfalls and exam tips

  • Bare import pkg and missing submodules. After import shop, shop.billing exists only if code has imported it.
  • Order of initialisation. Parent first, then child. A message in __init__.py appears before the message of the module you named.
  • Running a package file as a script. Relative imports fail and the first sys.path entry is the wrong folder. Use python -m.
  • import .x. This is a syntax error; relative imports need from.
  • Circular imports. If a imports b and b imports a at the top level, the second import finds a half-initialised module and a name may be missing. Fix by moving the import into a function or restructuring.
  • Shadowing standard modules with your own file names.
  • Assuming an empty folder is a package. A folder without __init__.py can still be imported as a namespace package in modern Python, but the exam expects you to create __init__.py for regular packages.

Exam tip. Draw the tree first, then list the search roots. The parent of a top-level file-system package must be on the search path, or another finder must provide it. Then write the output in the order of initialisation: parent __init__, child __init__, module.

Mini case study: the converter package that fails in CI

A team builds this package:

Text
convert/
    __init__.py       (empty)
    length.py         contains: from . import units
    units.py          contains: FEET = 0.3048
tests/
    run_length.py     contains: import convert.length ; print(convert.length.units.FEET)

On a laptop where everyone runs python tests/run_length.py from the project root, the script fails with ModuleNotFoundError: No module named 'convert'. One developer states that it works on his machine because he runs it inside the tests folder.

Task. Explain the failure and fix it.

Solution.

  1. For python tests/run_length.py, sys.path[0] is tests/, not the project root, so convert (a sibling of tests) is not found.
  2. The developer for whom it "works" most likely has the project root on the path through another mechanism, such as a PYTHONPATH setting or an installed copy of convert; the difference is the environment of that machine, not the code.
  3. The import itself is correct: length.py uses from . import units, which works because the file is imported as part of the convert package, and then convert.length.units is available because length imported it.
  4. Fix: run the test as python -m tests.run_length from the project root after adding an empty tests/__init__.py, or move the executable test code to the root. Do not rely on PYTHONPATH that exists only on one machine.

Key terms

  • Package: a module with a __path__ used to find submodules. A regular file-system package has an __init__.py.
  • Sub-package: a package inside another package.
  • __init__.py: file executed when the package is first imported.
  • Qualified name: the dotted path from the top-level package to the module.
  • Relative import: from . import x or from ..y import z, resolved using the module’s package context.
  • python -m: runs a module by name, with the package chain imported.
  • __main__.py: file run by python -m package.
  • sys.path: list of folders that Python searches for top-level modules.

Summary checklist

  • I can map dotted names to regular file-system packages and modules.
  • __init__.py runs once, parent before child, before the submodule.
  • import pkg does not import its submodules automatically.
  • Relative imports use from and dots and need a package context.
  • Running a package file as a script breaks relative imports; use python -m.
  • I can identify the normal path root for direct execution and for python -m.
  • import a.b.c binds only a; import a.b.c as x binds c as x.
  • Never name your own file like a standard module.

Official sources