Django’s Test Case Classes and a Three Times Speed-Up
This is a story about how I sped up a client’s Django test suite to be three times faster, through swapping the test case class in use.
Speeding up test runs is rarely a bad thing. Even small teams can repeat their test run hundreds of times per week, so time saved there is time won back. This keeps developers fast, productive, and happy.
Django’s Test Case Classes
A quick refresher on how the Django’s three basic test case classes affect the database:
SimpleTestCase
is the simplest one.- It provides the basic features of
unittest.TestCase
plus some Django extras. It blocks database access by default, because it doesn’t do anything to isolate changes you would make there. You should use it for testing components that don’t need the database.
TransactionTestCase
extendsSimpleTestCase
to allow database modifications.- It resets the database at the end by removing all rows from all database tables. This is slow, but reliable. You should use it when you need the database, but you can’t use
TestCase
…
TestCase
is the class you should normally use.- It extends
TransactionTestCase
and replaces its database reset process with one that uses transactions. This is much faster as the database only undoes the changes made during the test, rather than checking each table individually. It means your tests can’t commit, but in practice most tests don’t need that.
The distinction between TransactionTestCase
and TestCase
can be confusing. Here’s my attempt to summarize it in one sentence:
TransactionTestCase
that allows your code to use transactions, whileTestCase
uses transactions itself.
The Speed-Up Story
Recently I was helping my client ev.energy improve their Django project. A full test run took about six minutes on my laptop when using the test command’s --parallel
option . This isn’t particularly long - I’ve worked on projects where it took up to 30 minutes! But it did give me a little time during runs to look for easy speed-ups.
Their project uses a custom test case class for all their tests, to add extra helper methods. It originally extended TransactionTestCase
, with its slower but more complete database reset procedure. I wondered why this had been done.
I searched the Git history for the first use of TransactionTestCase
with git log -S TransactionTestCase
(a very useful Git option!). I found a developer had first used it in tests for their custom background task class called Task
.
Task
closed the database connection at the end of its process with connection.close()
. This helped isolate the tasks. Since they’re run in a long running background process, using a fresh database connection for each task helped prevent a failure in one from affecting the others.
Unfortunately the call to connection.close()
prevented use of TestCase
when testing Task
classes. Closing the database connection also ends any transactions. So when TestCase
ran its teardown process, it errored when trying to roll back the transactions it started in its setup process.
Because of this, the developers used TransactionTestCase
for their custom test case class. And they stuck with it as the project grew.
This was all fair, and the speed difference would not have been noticeable when there were fewer tests. Fixing it then allowed them to focus on feature development.
But as with test time things like this, the seconds added up over time. Much like the metaphorical frog in a slowly boiling pot of water.
Once I’d discovered this piece of history, I guessed most of the tests that didn’t run Task
classes would work with TestCase
. I swapped the base of the custom test class to TestCase
, reran, and only the Task
tests failed!
After changing only broken test classes back to TransactionTestCase
, I reran the suite and everything passed. The run time went down from 375 seconds to 120 seconds. A three times speed-up!
Fin
I hope this post helps you find the right test case class in your Django project. If you want help with this, email me - I’m happy to answer any questions, and am available for contracts. See my front page for details.
Thanks for reading,
—Adam
Learn how to make your tests run quickly in my book Speed Up Your Django Tests.
One summary email a week, no spam, I pinky promise.
Related posts:
- Introducing django-perf-rec, our Django performance testing tool
- Getting a Django Application to 100% Test Coverage
Tags: django