generators in __init__.py file — Python

Gateway to home © Diane Khambu

If you have come across application directories in python, you must have seen directories converted to packages with __init__.py file. The basic function of the file is to initialize a directory just like in class where we use __init__ method to initialize a class.

That was my understanding, until recently where I could not make code run that was in __init__.py just by re-importing a package inside a function.

This article is to find a cause and bring a solution to it. Let’s dive in!

tl;dr

control-flow diagram of how to structure code in __init__.py file

This is a file structure:

directory structure for generator example in __init__.py file

In package_1 ‘s __init__.py let’s have a simple for loop:

In package_3 ‘s __init__.py file, we have a generator:

In package_2 ‘s computation.py file, let’s use the PRODUCT_SO_FAR variable from package_1 ‘s __init__.py file.

Now, let’s run the computation.py . For example depending on where I am in file path, I ran like this: python package_2/computation.py 8 .

The output is like this:

In continue_cube_calculation method
Hello World from package_1
PRODUCT_SO_FAR: 120
PRODUCT_SO_FAR in package_2: 40320

See, the package_1 ‘s __init__.py file ran before we used PRODUCT_SO_FAR variable in the function continue_cube_calculation. Note that we imported the package inside a function.

Also important to know that __init__.py file will run whether you use the imported package or not.

For example, this will run __init__.py file of package_1 even when we did not use anything from the package.

On running python package_2/combination.py 8, this gives:

Hello World from package_1
PRODUCT_SO_FAR: 120
In continue_cube_calculation method
PRODUCT_SO_FAR in package_2: 336

See! __init__.py file do the initialization of a directory regardless of whether you use the directory or not. So not lazy…

Things get bit tricky if we want to load a package twice! Let’s dissect it.

On running the file , we get:

Hello World from package_1
PRODUCT_SO_FAR: 120
Inside continue_cube_calculation
PRODUCT_SO_FAR in package_2: 40320

Well, the generator in package_1 ‘s __init__.py ran only while loading the package. When we called continue_cube_calculation function, the import of the package_1 statement inside the function did not generate the values. Well, __init__.py files are supposed to run when we import packages right?

Yes, you are right. For that we have to delete the package from the python path so that we can reload the package and have __init__.py file run. Let’s illustrate it with an example:

On running the file, we get:

Hello World from package_1
PRODUCT_SO_FAR: 120
Inside continue_cube_calculation
Hello World from package_1
PRODUCT_SO_FAR: 120
PRODUCT_SO_FAR in package_2: 40320

See, the __init__.py file from the package_1 ran again inside the function’s import statement. Be careful, we need to delete the package from python path so that we can do load the package which runs the __init__.py file. If we didn’t, well Python will say, nope it’s already present. So there will not be reload of __init__.py file.

What is the case for generators in __init__.py? Looks like it is the same.

On running the file, we get:

Hello World from package_3
1
8
27
64
125
216
In the yield_values function
Hello World from package_3
1
8
27
64
125
216

There were also instances where printing of cubes was done twice both during importing in a module and inside a function. My educated guess is one is triggered from the __init__.py file and another from a module that imported the package. 🤨

Let’s put the generator call in a function so that we can call the function instead of having it loaded implicitly in a caller module. That way we don’t have to remove the package from the sys module but call the function again.

This is how package_3 ‘s __init__.py file looks like now:

Now call the package in package_2 ‘s computation.py file.

The output of the file is:

1
8
27
64
125
216
In the yield_values function
1
8
27
64
125
216

In summary, what I found is if you want to call some computation in __init__.py file have it wrapped in a function so they can be called with just a function call instead of having to delete the package from sys.modules .

Congratulations on the completion! Hope it was useful.

See you soon in my next article.✨

You can support me in my Patreon!