Add Django integration

ryanhiebert · ryanhiebert · commit 1dcb9ba3a888 · 2026-02-16T22:21:14.000-06:00
Override the runserver and runserver_plus management commands to wrap
inner_run in activate(), providing automatic broker lifecycle management
during development. Support configurable base class resolution via
tool.queueio.django.runserver in pyproject.toml to allow chaining with
providers like daphne or django.contrib.staticfiles.

Integrate with django-cmd for settings discovery from
tool.django.settings in pyproject.toml, and with djp for automatic
INSTALLED_APPS registration. Fall back to DJANGO_SETTINGS_MODULE
environment variable when django-cmd is not installed.

Include a sample Django app in queueio/samples/django/ with an
end-to-end integration test that spins up a real Django server and
queueio worker, submits a job via HTTP, and verifies ORM updates.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -8,8 +8,14 @@ The format is loosely based on [Keep a Changelog](https://keepachangelog.com).
 [Unreleased]
 ------------
 
+[0.6.0] - 2026-02-16
+--------------------
+
 ### Added
 
+- Django integration (`queueio.django`) with `runserver` lifecycle management.
+- [djp](https://djp.readthedocs.io/) support for automatic `INSTALLED_APPS` registration.
+- [django-cmd](https://pypi.org/project/django-cmd/) integration for settings discovery.
 - Documentation site.
 
 [0.5.0] - 2026-02-08
@@ -100,7 +106,8 @@ Thank you to Nick Anderegg for allowing me to use the queueio name for this proj
 - The queuespec syntax to `queue run` to consume multiple queues with shared capacity.
 - `queueio monitor` command to monitor activity in the queueio system.
 
-[Unreleased]: https://github.com/ryanhiebert/queueio/compare/tag/0.5.0...HEAD
+[Unreleased]: https://github.com/ryanhiebert/queueio/compare/tag/0.6.0...HEAD
+[0.6.0]: https://github.com/ryanhiebert/queueio/compare/tag/0.5.0...tag/0.6.0
 [0.5.0]: https://github.com/ryanhiebert/queueio/compare/tag/0.4.0...tag/0.5.0
 [0.4.0]: https://github.com/ryanhiebert/queueio/compare/tag/0.3.0...tag/0.4.0
 [0.3.0]: https://github.com/ryanhiebert/queueio/compare/tag/0.2.2...tag/0.3.0
diff --git a/mkdocs.yml b/mkdocs.yml
@@ -20,6 +20,7 @@ theme:
       primary: black
 
 markdown_extensions:
+  - admonition
   - pymdownx.highlight
   - pymdownx.superfences
 
@@ -36,4 +37,5 @@ nav:
   - Invocation: queueio/invocation.md
   - QueueSpec: queueio/queuespec.md
   - QueueVar: queueio/queuevar.md
+  - Django: queueio/django.md
   - Changelog: CHANGELOG.md
diff --git a/pyproject.toml b/pyproject.toml
@@ -1,6 +1,6 @@
 [project]
 name = "queueio"
-version = "0.5.0"
+version = "0.6.0"
 description = "Python background queues with an async twist"
 readme = "README.md"
 license = "MIT"
@@ -27,12 +27,21 @@ dependencies = [
     "typer>=0.15.4",
 ]
 
+[project.optional-dependencies]
+django = ["django>=6.0", "django-cmd>=3.0"]
+
+[project.entry-points.djp]
+queueio = "queueio.django.djp"
+
 [project.scripts]
 queueio = "queueio.__main__:app"
 
 [dependency-groups]
 dev = [
     "basedpyright>=1.32.1",
+    "django-extensions",
+    "djp",
+    "queueio[django]",
     "mkdocs-material>=9.7.1",
     "mkdocs-same-dir>=0.1.3",
     "pytest>=8.4.1",
@@ -60,6 +69,7 @@ pika = "amqp://localhost:5672"
 
 [tool.pytest.ini_options]
 timeout = 30
+norecursedirs = ["site"]
 
 [tool.coverage.run]
 branch = true
diff --git a/queueio/django.md b/queueio/django.md
@@ -0,0 +1,82 @@
+# Django Integration
+
+## Settings
+
+When running `queueio run`,
+Django isn't started automatically.
+If your routines use the Django ORM or other Django features,
+queueio needs to call `django.setup()` before importing routine modules.
+
+The `queueio[django]` extra includes
+[django-cmd](https://pypi.org/project/django-cmd/),
+which reads `tool.django.settings` from `pyproject.toml`
+and sets `DJANGO_SETTINGS_MODULE` automatically:
+
+```toml
+[tool.django]
+settings = "myproject.settings"
+```
+
+If `DJANGO_SETTINGS_MODULE` is set in the environment
+(whether by `django-cmd` or directly),
+queueio calls `django.setup()` before registering routines.
+
+## Runserver
+
+!!! warning "Development server only"
+
+    The runserver integration only affects the `runserver` management command.
+    Production servers like gunicorn need their own lifecycle hooks.
+
+queueio ships a Django app
+that overrides the `runserver` management command
+to wrap the server in `with activate():`,
+giving you automatic lifecycle management
+of the broker connection and background threads
+during development.
+
+### Setup
+
+If you use [djp](https://djp.readthedocs.io/),
+`queueio.django` is added to `INSTALLED_APPS` automatically.
+
+Otherwise, add it manually:
+
+```python
+INSTALLED_APPS = [
+    "queueio.django",
+    # ...
+]
+```
+
+This must come **after** whichever app provides the `runserver` command
+you want to chain with
+(e.g. `daphne`, `django.contrib.staticfiles`),
+because Django resolves management commands by last-app-wins order.
+
+### Custom provider
+
+If you use a custom `runserver` provider
+like `django.contrib.staticfiles` or `daphne`,
+tell queueio which app to chain with:
+
+```toml
+[tool.queueio.django]
+runserver = "daphne"
+```
+
+The value is the app label (or dotted module path).
+queueio resolves it to `{value}.management.commands.runserver`
+and subclasses that command.
+If omitted, it defaults to Django's built-in `runserver`.
+
+### `runserver_plus`
+
+There is also a built-in `runserver_plus` command
+that wraps [django-extensions](https://django-extensions.readthedocs.io/)' `runserver_plus`
+with `activate()`.
+This is available automatically
+when `django-extensions` is installed
+and `queueio.django` is in `INSTALLED_APPS`.
+
+A complete working example is in `queueio/samples/django/`.
diff --git a/queueio/django/__init__.py b/queueio/django/__init__.py
@@ -0,0 +1,17 @@
+import os
+
+
+def setup():
+    try:
+        from django_cmd import configure
+
+        configure()
+    except ImportError:
+        pass
+
+    if "DJANGO_SETTINGS_MODULE" not in os.environ:
+        return
+
+    import django
+
+    django.setup()
diff --git a/queueio/django/apps.py b/queueio/django/apps.py
@@ -0,0 +1,6 @@
+from django.apps import AppConfig
+
+
+class QueueIOConfig(AppConfig):
+    name = "queueio.django"
+    label = "queueio"
diff --git a/queueio/django/djp.py b/queueio/django/djp.py
@@ -0,0 +1,6 @@
+import djp
+
+
+@djp.hookimpl
+def installed_apps():
+    return ["queueio.django"]
diff --git a/queueio/django/management/__init__.py b/queueio/django/management/__init__.py
diff --git a/queueio/django/management/commands/__init__.py b/queueio/django/management/commands/__init__.py
diff --git a/queueio/django/management/commands/runserver.py b/queueio/django/management/commands/runserver.py
@@ -0,0 +1,41 @@
+import tomllib
+from importlib import import_module
+from pathlib import Path
+
+
+def _find_pyproject() -> Path | None:
+    for path in [cwd := Path.cwd(), *cwd.parents]:
+        candidate = path / "pyproject.toml"
+        if candidate.is_file():
+            return candidate
+    return None
+
+
+def _queueio_django_config() -> dict:
+    if pyproject := _find_pyproject():
+        with pyproject.open("rb") as f:
+            config = tomllib.load(f)
+        return config.get("tool", {}).get("queueio", {}).get("django", {})
+    return {}
+
+
+def _resolve_base_command(config: dict):
+    runserver_app = config.get("runserver", "django.core")
+    module = import_module(f"{runserver_app}.management.commands.runserver")
+    return module.Command
+
+
+def _build_command():
+    base = _resolve_base_command(_queueio_django_config())
+
+    class Command(base):
+        def inner_run(self, *args, **options):
+            from queueio import activate
+
+            with activate():
+                super().inner_run(*args, **options)
+
+    return Command
+
+
+Command = _build_command()
diff --git a/queueio/django/management/commands/runserver_plus.py b/queueio/django/management/commands/runserver_plus.py
@@ -0,0 +1,9 @@
+from django_extensions.management.commands.runserver_plus import Command as BaseCommand
+
+from queueio import activate
+
+
+class Command(BaseCommand):
+    def inner_run(self, options):
+        with activate():
+            super().inner_run(options)
diff --git a/queueio/django/runserver_test.py b/queueio/django/runserver_test.py
@@ -0,0 +1,63 @@
+import os
+import sysconfig
+
+import pytest
+
+from .management.commands.runserver import _build_command
+
+
+@pytest.mark.parametrize(
+    ("runserver_app", "expected_module"),
+    [
+        pytest.param(
+            None,
+            "django.core.management.commands.runserver",
+            id="default",
+        ),
+        pytest.param(
+            "django.contrib.staticfiles",
+            "django.contrib.staticfiles.management.commands.runserver",
+            id="staticfiles",
+        ),
+        pytest.param(
+            "daphne",
+            "daphne.management.commands.runserver",
+            id="daphne",
+            marks=pytest.mark.skipif(
+                bool(sysconfig.get_config_var("Py_GIL_DISABLED")),
+                reason="zope.interface re-enables the GIL on free-threaded builds",
+            ),
+        ),
+    ],
+)
+def test_runserver(tmp_path, runserver_app, expected_module):
+    config = "[tool.queueio]\n"
+    if runserver_app is not None:
+        config = f'[tool.queueio.django]\nrunserver = "{runserver_app}"\n'
+
+    pyproject = tmp_path / "pyproject.toml"
+    pyproject.write_text(config)
+
+    original_cwd = os.getcwd()
+    os.chdir(tmp_path)
+    try:
+        Cmd = _build_command()
+        from importlib import import_module
+
+        expected = import_module(expected_module)
+        assert issubclass(Cmd, expected.Command)
+    finally:
+        os.chdir(original_cwd)
+
+
+def test_runserver_plus():
+    from django.conf import settings
+
+    if not settings.configured:
+        settings.configure()
+
+    from django_extensions.management.commands import runserver_plus as expected
+
+    from .management.commands.runserver_plus import Command
+
+    assert issubclass(Command, expected.Command)
diff --git a/queueio/freethreading_test.py b/queueio/freethreading_test.py
@@ -22,7 +22,7 @@ def test_all_queueio_modules_preserve_freethreading():
     for _, modname, _ in pkgutil.walk_packages(
         queueio.__path__, f"{queueio.__name__}."
     ):
-        if not modname.endswith("_test"):
+        if not modname.endswith("_test") and ".django" not in modname:
             importlib.import_module(modname)
 
     assert sys._is_gil_enabled() is False
diff --git a/queueio/queueio.py b/queueio/queueio.py
@@ -11,6 +11,7 @@
 
 from .broker import Broker
 from .consumer import Consumer
+from .django import setup as _django_setup
 from .invocation import Invocation
 from .journal import Journal
 from .message import Message
@@ -105,6 +106,9 @@ def __get_pika_uri(self) -> str:
 
     def __register_routines(self):
         """Load routine modules from pyproject.toml."""
+        for hook in [_django_setup]:
+            hook()
+
         config = self.__config()
         modules = config.get("register", [])
 
diff --git a/queueio/samples/django/__init__.py b/queueio/samples/django/__init__.py
diff --git a/queueio/samples/django/apps.py b/queueio/samples/django/apps.py
@@ -0,0 +1,6 @@
+from django.apps import AppConfig
+
+
+class QueueIOSampleConfig(AppConfig):
+    name = "queueio.samples.django"
+    label = "queueio_sample"
diff --git a/queueio/samples/django/job.py b/queueio/samples/django/job.py
@@ -0,0 +1,29 @@
+from django.db import models
+from django.http import JsonResponse
+
+from queueio import routine
+
+
+class Job(models.Model):
+    status = models.CharField(max_length=20, default="pending")
+    objects = models.Manager()  # pyright: ignore[reportAssignmentType]
+
+    class Meta:
+        app_label = "queueio_sample"
+
+
+@routine(name="update_job", queue="django")
+def update_job(job_id: int):
+    job = Job.objects.get(pk=job_id)
+    job.status = "complete"
+    job.save()
+
+
+def index(request):
+    if request.method == "POST":
+        job = Job.objects.create()
+        update_job(job.pk).submit()
+        return JsonResponse({"job_id": job.pk, "status": job.status})
+
+    job = Job.objects.latest("pk")
+    return JsonResponse({"job_id": job.pk, "status": job.status})
diff --git a/queueio/samples/django/models.py b/queueio/samples/django/models.py
@@ -0,0 +1,3 @@
+from .job import Job
+
+__all__ = ["Job"]
diff --git a/queueio/samples/django/settings.py b/queueio/samples/django/settings.py
@@ -0,0 +1,19 @@
+DEBUG = True
+
+SECRET_KEY = "insecure-example-key"
+
+INSTALLED_APPS = [
+    "queueio.django",
+    "queueio.samples.django",
+]
+
+DATABASES = {
+    "default": {
+        "ENGINE": "django.db.backends.sqlite3",
+        "NAME": "db.sqlite3",
+    }
+}
+
+DEFAULT_AUTO_FIELD = "django.db.models.BigAutoField"
+
+ROOT_URLCONF = "queueio.samples.django.urls"
diff --git a/queueio/samples/django/settings_djp.py b/queueio/samples/django/settings_djp.py
diff --git a/queueio/samples/django/urls.py b/queueio/samples/django/urls.py
diff --git a/queueio/samples/django_test.py b/queueio/samples/django_test.py
