Django¶
Serving a project of Django with the servers of Pyjoyment.
This document describes how a project of Django, as it is, runs on the
servers of Pyjoyment in the place of gunicorn or uvicorn: its WSGI
or its ASGI application mounted on / of an application of
Pyjoyment.Lite with Pyjoyment.Plugin.Mount, with the static
files, the probes and the access log of Pyjoyment around it.
From gunicorn translates the options of gunicorn into the ones of
Pyjoyment.Command.prefork.
The project¶
The project of this guide has the layout of django-admin startproject,
with a model of notes in SQLite and a /healthz of its own, and one file
more, config/server.py next to config/wsgi.py, the script of the
servers of Pyjoyment.
import Pyjo.IOLoop
from Pyjoyment.Lite import *
import os
from django.conf import settings
from django.core.signals import request_finished, request_started
from django.db import connection
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'config.settings')
# The static files of collectstatic, served before the routes, with the hashed names of ManifestStaticFilesStorage
# cached for a year
app.static.paths = [str(settings.STATIC_ROOT.parent)]
app.static.immutable = r'^static/.+\.[0-9a-f]{12}\.[^/.]+$'
app.static.cache_control = 'public, max-age=60'
# The database, in a request of Django, whose handlers close a connection that the database has dropped
def database():
request_started.send(sender=__name__)
try:
with connection.cursor() as cursor:
cursor.execute('SELECT 1')
finally:
request_finished.send(sender=__name__)
# /livez and /readyz, with the check of the database as blocking code, and the /healthz of the project, which stays its
# own
plugin('Health', checks={'database': lambda c: Pyjo.IOLoop.blocking(database)}, healthz=None)
# The access log of gunicorn, without the probes
plugin('AccessLog')
# The project, which gets every request that the routes before do not match
plugin('Mount', {'/': 'config.wsgi:application'})
if __name__ == '__main__':
app.start()
import os
from pathlib import Path
BASE_DIR = Path(__file__).resolve().parent.parent
SECRET_KEY = os.environ.get('DJANGO_SECRET_KEY', 'insecure')
DEBUG = os.environ.get('DJANGO_DEBUG') == '1'
# The kubelet sends the address of the pod as Host, which the probes of Pyjoyment take before Django
ALLOWED_HOSTS = os.environ.get('DJANGO_ALLOWED_HOSTS', 'localhost,127.0.0.1').split(',')
INSTALLED_APPS = ['django.contrib.staticfiles', 'notes']
MIDDLEWARE = ['django.middleware.security.SecurityMiddleware', 'django.middleware.common.CommonMiddleware']
ROOT_URLCONF = 'config.urls'
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.environ.get('DJANGO_DATABASE', BASE_DIR / 'db.sqlite3'),
},
}
DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField'
# The time zone of the process, which Django sets in TZ, so the logs of Pyjoyment have it too
TIME_ZONE = 'UTC'
USE_TZ = True
# The output of collectstatic, which Pyjoyment serves from the parent directory, with favicon.ico and robots.txt
STATIC_URL = '/static/'
STATIC_ROOT = BASE_DIR / 'public' / 'static'
The script is a module of the project, run from the directory of
manage.py, as manage.py imports the packages of the project from
there, and it starts the server under if __name__ == '__main__', so that
a test can import it.
$ python manage.py migrate
$ python manage.py collectstatic
$ python -m config.server prefork -w 8 -c 1 -r 1
The event loop¶
A WSGI application blocks, and runs in the event loop of its worker,
which waits for it. Pyjo.App.WSGI calls it with
Pyjo.IOLoop.blocking(), so that it sees no running loop of
asyncio while it runs, as on a WSGI server, where Django would
refuse its database otherwise, as every query would raise
SynchronousOnlyOperation: You cannot call this from an async context - use
a thread or sync_to_async., and a blocking request of Pyjo.UserAgent
would raise too. The code of Django that runs outside the application, as the
check of the database of the probes, is blocking code too, which
Pyjo.IOLoop.blocking() runs.
The project runs on Pyjo.Reactor.Asyncio, the default reactor, next to
what needs asyncio, as an ASGI application mounted on another
path, and on Pyjo.Reactor.Poll as well, which PYJO_REACTOR
selects, except that an async def view, which Django runs in a thread
with an event loop of its own, awaits the promises of Pyjo.UserAgent
only with the reactor of asyncio.
What passes through¶
The request comes to Django as a server of WSGI gives it:
SCRIPT_NAMEis empty on/, so the URLs that Django builds need noFORCE_SCRIPT_NAME;the headers are the
HTTP_variables of the environment, asX-Forwarded-ProtoisHTTP_X_FORWARDED_PROTO, whichSECURE_PROXY_SSL_HEADERreads;REMOTE_ADDRis the address of the connection, and with-p, the reverse proxy support of the servers, the address ofX-Forwarded-For, with the scheme ofX-Forwarded-Protoaswsgi.url_scheme, so thatrequest.is_secure()is true withoutSECURE_PROXY_SSL_HEADER.
Django sets TZ of the process to its TIME_ZONE when it loads its
settings, America/Chicago unless the project sets another one, so the
time of the log and of the access log of Pyjoyment is in it too, as of
the rest of the process; tz of Pyjoyment.Plugin.AccessLog gives
the access log another one.
Static files¶
Pyjoyment.Static serves the files of collectstatic before the
routes, in the place of whitenoise, whose middleware leaves MIDDLEWARE:
the parent of STATIC_ROOT is the directory of the static files, which
has the files of / too, as robots.txt, and the hashed names of
ManifestStaticFilesStorage are cached for a year, as Mounting
describes it. The directory of WHITENOISE_ROOT, of the files of / as
favicon.ico, is another directory of app.static.paths, and
app.static.extra = {} leaves out the favicon and the images of
Pyjoyment for a project that has its own.
With DEBUG and before collectstatic, staticfiles_urlpatterns() of
Django serves the static files of the apps, as config/urls.py does, after
the files that Pyjoyment has, as python -m config.server daemon
of a checkout has none.
Probes¶
Pyjoyment.Plugin.Health answers /livez and /readyz, which
Kubernetes requests, before the routes, so healthz=None leaves the
/healthz of the project to it, which its monitors may request with other
expectations. Django does not see the probes, so ALLOWED_HOSTS does not
reject the address of the pod, which the kubelet sends as Host, and which
a view of Django answers with 400 Bad Request: /readyz is the
readiness probe without a Host header of its own.
With -c 1, a probe of HTTP is a connection as any other, which waits in
the backlog while every worker is busy, so a liveness probe of HTTP restarts
the pod in a slow spell of a busy application. A liveness probe on
tcpSocket, which the kernel accepts into the backlog, does not.
readinessProbe:
httpGet:
path: /readyz
port: http
livenessProbe:
tcpSocket:
port: http
Outside a request of Django, nothing closes a connection of Django to the
database that the database has dropped, as after its restart, so a check
that keeps using it fails every probe until the worker restarts. The check
of config/server.py is framed as a request of Django, whose handlers of
request_started and request_finished close a connection that is
broken or older than CONN_MAX_AGE, as for a view, so that the next probe
connects again.
The timeout of a check of Pyjoyment.Plugin.Health is the one of a
check that returns a promise or a coroutine. A check that blocks runs to its
end first, so the timeouts of its driver bound it, as connect_timeout in
the OPTIONS of the database.
Access log¶
Pyjoyment.Plugin.AccessLog writes a line of each response to the
standard output, in the combined format of the access log of gunicorn,
without the probes, as Cookbook describes it. Behind a gateway that
gives the address of the client in a header of its own, client takes it,
as plugin('AccessLog', client='X-Envoy-External-Address'), as the first
address of X-Forwarded-For is whatever the client wrote there.
Requests to other services¶
Pyjo.UserAgent is the HTTP client of a view, in the place of
requests or httpx. In a view of WSGI, a blocking request runs
an event loop of its own, while the worker waits for it as for a query of the
database, so a request to the application itself needs another worker that is
free.
import Pyjo.UserAgent
from Pyjoyment.Lite import *
import os
from django.conf import settings
from django.core.wsgi import get_wsgi_application
from django.http import HttpResponse, JsonResponse
from django.urls import path
# A project of Django in one file
settings.configure(ROOT_URLCONF=__name__, ALLOWED_HOSTS=['*'], SECRET_KEY='insecure', TIME_ZONE='UTC')
# The user agent of the views, in the place of requests, and the API of GitHub, or the one of GITHUB_API_URL
ua = Pyjo.UserAgent.new()
api = os.environ.get('GITHUB_API_URL', 'https://api.github.com')
def index(request):
return HttpResponse(f'Hello from Django at {request.path}!')
# A blocking request, which runs an event loop of its own, while the worker waits for it as for a query of the ORM, as
# the application sees no running event loop
def release(request):
res = ua.get(f'{api}/repos/pyjoyment/Pyjoyment/releases/latest').result
return JsonResponse({'status': res.code, 'release': res.json('/tag_name')})
urlpatterns = [path('', index), path('release', release)]
plugin('Mount', {'/': get_wsgi_application()})
app.start()
ASGI¶
The ASGI application of the project runs on the default reactor, of
asyncio, with the same script, which mounts it in the place of the
WSGI one.
plugin('Mount', {'/': 'config.asgi:application'})
Its async def views run in the event loop, where they await anything of
asyncio and the promises of Pyjo.UserAgent, while the event
loop serves the other requests. Its synchronous views and middleware run in a
thread, which sync_to_async of Django gives them, one thread for all of
them, and so do the queries of its async ORM, as
await Note.objects.acount(), as the backends of databases of Django are
synchronous.
A project of synchronous views and queries serves one request at a time in
each worker, and runs better on WSGI.
import Pyjo.UserAgent
from Pyjoyment.Lite import *
import os
from django.conf import settings
from django.core.asgi import get_asgi_application
from django.http import HttpResponse, JsonResponse
from django.urls import path
# A project of Django in one file
settings.configure(ROOT_URLCONF=__name__, ALLOWED_HOSTS=['*'], SECRET_KEY='insecure', TIME_ZONE='UTC')
# The user agent of the views, in the place of httpx, and the API of GitHub, or the one of GITHUB_API_URL
ua = Pyjo.UserAgent.new()
api = os.environ.get('GITHUB_API_URL', 'https://api.github.com')
def index(request):
return HttpResponse(f'Hello from Django at {request.path}!')
# A request awaited in the event loop of asyncio, which serves the other requests meanwhile
async def release(request):
res = (await ua.get_p(f'{api}/repos/pyjoyment/Pyjoyment/releases/latest')).result
return JsonResponse({'status': res.code, 'release': res.json('/tag_name')})
urlpatterns = [path('', index), path('release', release)]
# An ASGI application, on the reactor of asyncio, the default one
plugin('Mount', {'/': get_asgi_application()})
app.start()
The ASGI application of Django has no lifespan, which
Pyjo.App.ASGI notes in the log on debug and serves it without.
Tests¶
The tests of the project, which manage.py test runs, test the static
files, the probes and the routes of Pyjoyment in front of it too, with
Pyjo.Test.Pyjo of the application of config/server.py in a
TestCase of Django, as Pyjo.Test describes the tests of
unittest.
import Pyjo.Test
import Pyjo.Test.Pyjo
from notes.models import Note
import unittest
from config.server import app
from django.core.signals import request_finished, request_started
from django.db import close_old_connections
from django.test import TestCase
def setUpModule():
unittest.enterModuleContext(Pyjo.Test.raise_failures())
# The requests of Pyjoyment reach the WSGIHandler of Django in the transaction of the test, whose handlers of
# request_started and request_finished would close its connection, so they are detached, as the test client of Django
# detaches them for its own requests
class ServerTests(TestCase):
@classmethod
def setUpClass(cls):
super().setUpClass()
cls.t = cls.enterClassContext(Pyjo.Test.Pyjo.new(app))
for signal in (request_started, request_finished):
signal.disconnect(close_old_connections)
cls.addClassCleanup(signal.connect, close_old_connections)
def test_notes(self):
Note.objects.create(text='first')
self.t.get_ok('/').status_is(200).content_like(r'^1 notes')
self.t.post_ok('/notes', form={'text': 'second'}).status_is(200).json_is('/notes', ['first', 'second'])
def test_probes(self):
self.t.get_ok('/livez').status_is(200)
self.t.get_ok('/readyz').status_is(200)
self.t.get_ok('/static/notes/notes.css').status_is(404)
self.assertEqual(Note.objects.count(), 0)
A request of the test reaches the WSGIHandler of Django, as one of a
server does, which sends request_started and request_finished, and so
does the check of the database. Their handler close_old_connections
closes a connection that is not in the autocommit of the settings, as the one
of the transaction of a TestCase, so a query of the view and of the check
fails after it. The test client of Django detaches the handler for its own
requests, and the test detaches it for its class. The test database of SQLite
in memory, the default, hides it, as Django does not close its connection,
and one of PostgreSQL, or of SQLite in a file, shows it.
More¶
The page of Examples has both examples, which request the API of
GitHub, or the one of GITHUB_API_URL.
You can continue with Guides now.