Django’s setup method in the primary init.py file consists of a whopping nine lines of code; five if we don’t count the import lines.

def setup(set_prefix=True):
    from django.apps import apps
         from django.conf import settings
         from django.urls import set_script_prefix
         from django.utils.log import configure_logging

    configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
    if set_prefix:
        set_script_prefix(
            '/' if settings.FORCE_SCRIPT_NAME is None else settings.FORCE_SCRIPT_NAME
        )
    apps.populate(settings.INSTALLED_APPS)

Hidden behind these few lines of code is a surprising amount of complexity, and by studying them we can gain a better understanding of how Django’s developers approach design and architecture, as well as a deeper understanding of Python’s built-in functionality.

Today’s post is the first of a four-part series on how Django’s setup process works.
1. How Django accesses project settings.
2. How Django and Python manage logging.
3. Why Django allows the script_prefix to be overridden and how this is used.
4. How Django populates and stores applications.

You may notice that each post in this series corresponds to an import line in the setup function. This is because all four of these pieces rely on quite a bit of plumbing behind the scenes, and it’s exactly that plumbing that makes them so interesting.

When setup gets called
If you read the last post on how Django uses WSGI in its communication with the web server, you would have seen the following code from the wsgi.py file.

import django
from django.core.handlers.wsgi import WSGIHandler

def get_wsgi_application():
   django.setup(set_prefix=False)
   return WSGIHandler()

We focused on the line return WSGIHandler() and the ensuing functionality. This series is focused on the preceding line, django.setup(set_prefix=False) meaning that everything that happens in this series takes place prior to the functionality described in the previous post.

Settings
Within the init.py file, there are two lines of code that we care about for the current topic.

from django.conf import settings
...
configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)

The first line imports settings from Django’s conf package (i.e. config), and the second passes two of that object’s attributes to the configure_logging function.

At first this looks simple; surely settings is just some simple object pulled from a settings or config file, and now Django is looking up some pre-existing attributes.

Not quite.

By navigating to the /django/django/conf/__init__.py file, we see that this file consists of three separate class definitions: LazySettings, Settings, and UserSettingsHolder. The actual settings variable isn’t defined until the very last line of the file, where we find the following.

settings = LazySettings()

I encourage you to take a moment and look at the code briefly, as it’s surprisingly involved.

This a pattern we will see often in Django, in which an import object is actually just a variable set to the value of some object from the relevant package, and not necessarily the name of an actual class.

Lazy Settings
The LazySettings class has the following profile (I’ve removed comments and replaced some areas we won’t be focusing on with ‘...‘).

ENVIRONMENT_VARIABLE = "DJANGO_SETTINGS_MODULE"

class LazySettings(LazyObject):

    def _setup(self, name=None):
        settings_module = os.environ.get(ENVIRONMENT_VARIABLE)
        ...
        self._wrapped = Settings(settings_module)

    def __repr__(self):
        ...

    def __getattr__(self, name):
        if self._wrapped is empty:
            self._setup(name)
        val = getattr(self._wrapped, name)
        self.__dict__[name] = val
        return val

    def __setattr__(self, name, value):
        ...

    def __delattr__(self, name):
        ...

    def configure(self, default_settings=global_settings, **options):
        if self._wrapped is not empty:
            raise RuntimeError('Settings already configured.')
        holder = UserSettingsHolder(default_settings)
        for name, value in options.items():
            setattr(holder, name, value)
        self._wrapped = holder

    @property
    def configured(self):
        return self._wrapped is not empty

High level observations:
1. This class inherits from LazyObject.
2. There’s no __init__ function, as it’s getting inherited from LazyObject .
3. Of the seven methods, four are magic methods, one is considered private (via the preceding underscore in _setup), one has the @property decorator, and only one is a non-magic, public method.

What we don’t see are any attributes getting set. That makes sense if it’s inheriting the __init__ function from LazyObject, so let’s take a look there and see if it’s setting the settings.LOGGING_CONFIG and settings.LOGGING attributes that get addressed in init.py.

_wrapped = None

def __init__(self):
    self._wrapped = empty

Nothing, except for the _wrapped private attribute.

What’s going on?

In short, LazySettings isn’t actually the main settings object. It’s just a wrapper around another object that holds the actual settings. If that’s confusing, we’ll first look at how this interaction works, and then discuss why it’s designed this way.

Get Attribute

The key method for understanding how setup works within Django is LazySettings.__getattr__.

def __getattr__(self, name):
    if self._wrapped is empty:
        self._setup(name)
    val = getattr(self._wrapped, name)
    self.__dict__[name] = val
    return val

This method is called when an attribute that has not yet been defined gets accessed. This is exactly what happens with the settings.LOGGING_CONFIG attribute. When accessed within init.py, it hasn’t yet been set Accessing it results in a call to LazySettings.__getattr__, which checks if LazySettings._wrapped is empty. If so, it calls LazySettings._setup, causing all settings to get initialized.

Once LazySettings._wrapped has been defined, the value is looked up from the LazySettings._wrapped object, assigned to the internal LazySettings.__dict__ attribute, and returned to the client. The __dict__ attribute is the dictionary that holds object-level attributes, which makes the following lines of code functionally equivalent.

self.x = 5
self.__dict__[x] = 5

Django is looking up an attribute from whatever object is stored at self._wrapped, then saving that value as an attribute of the LazySettings object. Future lookups will then be able to access the LazySettings attribute directly, and won’t go through the LazySettings.__getattr__ method.

LazySettings._setup
The LazySettings._setup method does the following: It looks up the DJANGO_SETTINGS_MODULE environment variable, then passes this path into the __init__ method of the new Settings object. This object is then assigned to the LazySettings._wrapped attribute referenced earlier.

def _setup(self, name=None):
    settings_module = os.environ.get(ENVIRONMENT_VARIABLE)
    ...
    self._wrapped = Settings(settings_module)

This whole flow is a bit confusing, so let’s take a look at it visually.

1. After the initial import, we have an empty LazySettings object.
   
2. During the first lookup, after self._setup(name) has been run, but before self.__dict__[name] = val is executed.
   
3. After self.__dict__[name] = val has been executed.
   

Non-lazy settings
We now have a LazySettings object that acts as a wrapper around a Settings object, which isn’t instantiated until one of the settings is actually accessed. Let’s take a look at the inner Settings object, to complete the picture of how settings get stored.

As shown in the (abridged) snippet of code below, the actual Settings object is quite simple. There are two primary parts of the __init__ method.

1. Iteration over the global_settings, defined in the /django/django/conf/__init__.py file. This is the default Django settings module. Individual settings are saved as attributes of the Settings object.
2. Import the user-defined settings_module path that was received as an argument, and iterate over these values. These may override settings previously set via the global_settings module, and each one is added to the Settings._explicit_settings set, to keep a record of all user-defined settings.

   from django.conf import global_settings
   ...
   class Settings:
       def __init__(self, settings_module):
           for setting in dir(global_settings):
               if setting.isupper():
                   setattr(self, setting, getattr(global_settings, setting))
   
           # store the settings module in case someone later cares
           self.SETTINGS_MODULE = settings_module
   
           mod = importlib.import_module(self.SETTINGS_MODULE)
   
           ...
   
           self._explicit_settings = set()
           for setting in dir(mod):
               if setting.isupper():
                   setting_value = getattr(mod, setting)
                   ...
                   setattr(self, setting, setting_value)
                   self._explicit_settings.add(setting)
   
           ...
   
       def is_overridden(self, setting):
           return setting in self._explicit_settings
   
   ...

Accessible Settings
Now that Settings.__init__ has run, the individual settings are accessible within the rest of the project and framework. In fact, if we return to the initial two lines of code where we first encountered settings, this entire process takes place while settings.LOGGING_CONFIG is first being looked up. When settings.LOGGING is accessed, the LazySettings object simply grabs the attribute from an already-populated Settings object, stores it as an attribute of its own, and returns the value to the user.

If LOGGING is accessed a second time, the __getattr__ method will be skipped entirely, and the attribute will be pulled from the LazySettings object just like any other attribute.

configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)

Wait, but why?
We’ve covered how the settings are accessed, but we haven’t taken a look at why they work this way. It isn’t the most simple method for accessing settings. Why not just create a Settings object without a wrapper, which can be accessed directly?

Additionally, why was the UserSettingsHolder class defined in this file if it wasn’t used for anything thus far? The answers are related.

Django supports the ability of different modules to configure settings manually when they are run separately from the rest of the Django app. The documentation gives the example of using Django’s template system by itself. In this case, the module could include from django.conf import settings and access settings.{attribute} directly, thereby initiating the same configuration process described above in the __init__.py file. The problem is that the separate module might not want this group of settings, and it may not want to rely on DJANGO_SETTINGS_MODULE environment variable.

Enter LazySettings.configure. This method allows custom, manual configuration of the settings. As shown below, the configure method receives the settings as parameters and passes them to the UserSettingsHolder object. In this case the self._wrapped attribute is stored as a UserSettingsHolder object rather than an instance of Settings, simply because the methods of retrieving and accessing settings are different in the two cases, and are best served by distinct classes.

def configure(self, default_settings=global_settings, **options):
    if self._wrapped is not empty:
        raise RuntimeError('Settings already configured.')
    holder = UserSettingsHolder(default_settings)
    for name, value in options.items():
        setattr(holder, name, value)
    self._wrapped = holder

This capability is why the LazySettings class is necessary (or at least a useful solution). Django only allows the settings to be configured one time. If the settings were configured by default every time, then specific modules would be unable to custom-configure their settings. The existing solution may seem overly-complicated, but it allows a somewhat unique set of rules to be enforced with minimal overhead for the user.

1. Settings are guaranteed to only be configured one time.
2. Settings can be set manually if the developer would like.
3. Otherwise, the default configuration process takes place, and the developer doesn’t need to manually configure anything.

Conclusion
Settings may seem like an unlikely candidate for a (relatively) deep dive, but taking a closer look gives us a more nuanced understanding of how Django actually works, how it utilizes Python’s built in functionality, and perhaps more importantly, why Django is built in this manner. In building a fully featured framework, Django’s core developers clearly took great time and care to design solutions that make usage as simple as possible for developers and users. But simple and flexible end-products sometimes necessitate fairly complex architectures, and we see that here.

Thanks for reading, and tune in next time when we dive into the (actually) exciting world of logging, as we continue this tour through Django’s setup process.

Takeaway Questions: Does this solution seem like it adequately fits the problem, or does it seem like over-engineering? Assuming the purpose was for the end product to be simple and flexible, does it succeed in this job?

Feel free to leave your thoughts in the comments below.