Django unique settings for each test case

I am trying to write some tests for a django plugin. I want each test case to run with it's own settings but django seems to load the settings once and use them for all the test cases.

Here is what I have:
I have created a runtests.py that loads the test runner and runs the tests.

#!/usr/bin/env python
import sys
import django

from django.conf import settings
from django.apps import apps

APP_NAME = 'name'

settings.configure(
    ...
    # common settings here
    ...
)

if hasattr(django, 'setup'):
    django.setup()

from django.test.utils import get_runner
TestRunner = get_runner(settings)
test_runner = TestRunner()
failures = test_runner.run_tests([APP_NAME])
if failures:
    sys.exit(failures)

Then I created a tests.py with some tests.

from django.test import TestCase, override_settings

class MyTests(TestCase):

    @override_settings(SOME_KEY=FIRST_VALUE)
    def test_first_test(self):
        ...

    @override_settings(SOME_KEY=SECOND_VALUE)
    def test_second_test(self):
        ...

    @override_settings(SOME_KEY=THIRD_VALUE)
    def test_third_test(self):
        ...

I know that Django does not guarantee test execution order, I am not worried about that. The problem I have is that if test_first_test is executed first then all the test cases will run with SOME_KEY = FIRST_VALUE.

Is there a way to ensure that django re-initializes the settings for each test case so that the value of SOME_KEY changes appropriately? Or am I doing something wrong?