How to Scale Django Admin

Django provides a powerful and flexible admin interface out of the box. It comes in handy while browsing data, doing straightforward daily tasks or performing CRUD operations.

At Sufle, we use Django heavily and Django Admin is part of our daily operations. For one of our projects, it was fast and reliable in the beginning. However, when our tables started hitting millions of rows, we experienced slow search times and timeouts most of the time. While our data continues to grow, we have learned a lot about what works well and what does not with Django Admin. With some simple adjustments, it is possible to continue to use admin even with large datasets. Here are the things we found, which might work for you.

Override get_queryset

First things first, for foreign keys and calculated fields there is no need to run extra queries per row. Performing multi queries for each object can lead to significantly slower response times.

What you can do here is to override the get_queryset method in your model admin class. You can use select_releated on your foreign keys.

class MyModelAdmin(ModelAdmin):
    def get_queryset(self, request):
    	queryset = super().get_queryset(request)
    	return queryset.select_related('foreign_key_1', 'foreign_key_2')

Or you can do annotations to save queries.

class MyModelAdmin(ModelAdmin):
    def get_queryset(self, request):
    	queryset = super().get_queryset(request)
    	return queryset.annotate(field_count=Count('field'))

Update Search with DjangoQL

Standart search works fine but after getting into a point of millions of rows, searching becomes a major issue. When there is more than one field in the search_fields, the SQL query has multiple JOIN statements chained one after the other. Our solution was introducing DjangoQL package here.

DjangoQLSearchMixin replaces the standard Django search with the DjangoQL search. DjangoQL helps us search through only specified fields, related fields included. All you need to do is adding DjangoQLSearchMixin to your model admin:

from djangoql.admin import DjangoQLSearchMixin

class MyModelAdmin(DjangoQLSearchMixin, ModelAdmin):
    pass

DjangoQL autocompletes model fields, supports logical operators, parenthesis and table joins. You can get rid of multiple JOIN statements with just a few lines of code and focus on what you want to search without limitations rather than hard coding search_fields.

Eliminate COUNT queries

After reaching a certain size, expensive COUNT(*) queries become a burden on change_list and cause timeout errors.

MySQL is our go-to solution and naturally, we use Django-MySQL to extend Django’s built-in MySQL support. The package comes with the handy count_tries_approx method which returns the approximate count instead of the exact count for the table. In this way, we eliminate a full table scan on InnoDB and achieve a significantly faster result.

class MyModelAdmin(ModelAdmin):

    def get_queryset(self, request):
        qs = super().get_queryset(request)
        return qs.count_tries_approx()

Before:

SELECT COUNT(*) AS `__count` FROM `mytable`

After count_tries_approx:

SELECT TABLE_ROWS FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'mytable'

Hope these useful tricks help you run Django Admin while you scale and grow.

Got any questions or ready to optimize your application? Book an Appointment now!