Spawning Django Subprocesses

We have some maintenance tasks that require some run time, that we’d like to launch from the web browser. Programmatically, the most natural thing is to spawn a process that performs the task and completes asynchronously. The results are recorded in the database for later harvesting.

As far as I can tell, the same general rules for forking apply when forking from within Django: close database connections, close open file handles, and release other resources that cannot be shared across process boundaries.

Django, apparently, will automatically re-connect to the database if the connection is closed. This makes the job much simpler. Different web sites say that the parent process should close its database connection, and others say that the child process should close its database  connection.

In the face of this conflicting information, I chose to close the parent process’ database connection before calling os.fork(). Reöpening database connections incurs a small penalty that are not a concern as this is done once.

Before forking:

    from django.db import connection

    # Don't fork with a database connection open.

    new_pid = os.fork()
    if not new_pid:
        # Child process

Thus far there seem to be no side effects from taking this approach. As always, additional information is welcomed.

This entry was posted in Programming and tagged , , , . Bookmark the permalink.

Leave a Reply