Add new check for indexing with atomic=True

Briefly, this check helps prevent a long-running index creation from
preventing all reads and writes to a table.

This supplements an existing check
==================================
There is an existing check, `CREATE_INDEX` which warns about creating
indices without the `CONCURRENTLY` argument. It's a helpful check!

However, there are some valid reasons to prefer non-concurrent index
creation. For example, concurrent index creation may struggle to
complete when applied to write-heavy tables (since Postgres will have to
make repeated passes on the table to accommodate the new reads since the
last round).

The danger of `EXCLUSIVE` locks
===============================
If you *do* want to create indices nonconcurrently, there is still a
**huge footgun** to be aware of -- locks which are held in a transaction
are not released until the transaction closes.

Dangerous Django defaults
=========================
By default, any time you add a new Django field with `db_index=True` or
`unique=True`, the automatically generated `Migration` has:

- implicit setting of `atomic=True` (transactions are on)
- an `ALTER TABLE...ADD COLUMN` call (exclusively locks the table)
- a `CREATE INDEX` invocation (*hopefully* quick, due to nullable
  columns)

It's even more dangerous if you combine an `ALTER TABLE` call with
`CREATE INDEX` of an existing, non-nullable column.

Favor `atomic=False` if you must index
======================================
Index-creating migrations should have few `operations`. Since index
creation does not modify row-level data (and only requires locking
against writes), there's no reason to use transactions.

It's prudent to bypass the default `atomic=True` for index creation.
This commit is contained in:
David Cain authored and David Wobrock committed 2023-07-09 14:42:35 +02:00
1 parent 9fdfa06845
commit 0c5519d9dc
11 files changed
+119

No files matched your search

+17
View File
@@ -315,6 +315,23 @@ class PostgresqlAnalyserTestCase(SqlAnalyserTestCase):
sql = "REINDEX TABLE my_table;"
self.assertWarningSql(sql)
def test_create_index_exclusive(self):
sql = [
"BEGIN;",
'ALTER TABLE "users" ADD COLUMN "email" varchar(254) NULL;',
'CREATE INDEX "user_email" ON "users" ("email");',
"COMMIT;",
]
self.assertBackwardIncompatibleSql(sql, code="CREATE_INDEX_EXCLUSIVE")
def test_create_index_exclusive_no_lock(self):
sql = [
'ALTER TABLE "users" ADD COLUMN "email" varchar(254) NULL;',
'CREATE INDEX "user_email" ON "users" ("email");',
]
# Will warn about `CREATE_INDEX`, but *not* `CREATE_INDEX_EXCLUSIVE`
self.assertValidSql(sql, allow_warnings=True)
class SqlUtilsTestCase(unittest.TestCase):
def test_unknown_analyser_string(self):