Understanding Python’s __all__ Variable

Python’s import system works very intuitively by design. Importing entire modules or single variables can be done with single-line syntax. For cases where the entirety of a package is being imported, a custom implementation of the __all__ variable can help restrict just what “all” really means.

Python’s import system, in cadence with many other programming languages import systems, allows for the use of the wildcard operator *. This instructs Python to import “all” of a module’s contents. By default, this includes functions, Classes, variables, and everything else without a name prefixed with an underscore.

Custom __all__ Implementation

The __all__ variable in Python is very intuitive to use but can be difficult to express in words. Let’s consider the following code:

examples/custom.py

# Import a library
import random

# Define constants
__COLORS__ = ("red", "blue", "yellow", "green")
__SHAPES__ = ("round", "square", "cylindrical")

# Define a custom class
class Candy:
    """
    An object model for a piece of candy expressing color and shape
    """
    def __init__(self, color: str, shape: str):
        self.color = color
        self.shape = shape

    def __str__(self):
        return f"A tasty piece of {self.shape} {self.color} candy"

# Define a custom function
def get_random_candy():
    """
    Creates a random Candy object
    Returns:
        Candy object
    """
    return Candy(random.choice(__COLORS__), random.choice(__SHAPES__))

# Create a random piece of candy
random_candy = get_random_candy()

This contrived selection of code defines a wide range of objects including variables, Classes, functions, and even imports another package. Now, let’s consider the use of this file within another Python file:

examples/custom_two.py

# Import everything from custom file
from custom import *

# Access variable
candy_1 = random_candy

# Create new object using function
candy_2 = get_random_candy()

# Create new object using Class
candy_3 = Candy('cyan', 'triangular')

# Create new object using Class using protected constants
candy_4 = Candy(__COLORS__[0], __SHAPES__[1])

# Access imported library
random_number = random.choice([0, 1, 2, 3, 4])

This file accesses everything we created in our examples.py file initially. When we run this code, we encounter a snag when creating the candy_4 object, which accesses the double-underscore prefixed variables:

NameError: name '__COLORS__' is not defined

This happens because Python does not implicitly import anything prefixed with a underscore (including double underscores) by default. To ensure these objects are imported during wildcard import statements, modules must explicitly define them. To achieve this, we would add the following code to the examples/custom.py file:

__all__ = ['__COLORS__', '__SHAPES__', 'Candy', 'get_random_candy', 'random_candy', 'random']

Now, when we run the code from our examples/custom_two.py file, there are no errors. This is because the __COLORS__ and __SHAPES__ constants were explicitly added to the module exports via the __all__ variable.

Using __all__ to Explicitly Remove Import/Exports

Python’s __all__ variable is useful to include underscore prefixed variables that would otherwise not be imported. However, this brings about a certain added level of responsibility for developers: everything intended for export must be explicitly added to the __all__ variable. That means leaving off a variable, Class, function, or other form of object that would have been exported by default—now results in that object not being exported. Consider the following code:

__all__ = ['__COLORS__', '__SHAPES__', 'Candy', 'random_candy', 'random']

This now leads to a NameError exception where we’re calling the get_random_candy() function. Normally, during an import * statement, function such as this one would be included by default. Since we’ve overwritten the default __all__ variable however—anything not included won’t be imported!

Final Thoughts

I am of the school that using from package import * is rarely a good idea. As such, I find myself with very little need for use of this approach at customizing the __all__ variable. However, knowing what it is and how it can be used is helpful—if for no other reason than to be able to digest Python projects which make heavier usage of this approach.