Commit Graph
107 Commits
Author SHA1 Message Date
David Cain 0c5519d9dc Add new check for indexing with atomic=True
Briefly, this check helps prevent a long-running index creation from
preventing all reads and writes to a table.

This supplements an existing check
==================================
There is an existing check, `CREATE_INDEX` which warns about creating
indices without the `CONCURRENTLY` argument. It's a helpful check!

However, there are some valid reasons to prefer non-concurrent index
creation. For example, concurrent index creation may struggle to
complete when applied to write-heavy tables (since Postgres will have to
make repeated passes on the table to accommodate the new reads since the
last round).

The danger of `EXCLUSIVE` locks
===============================
If you *do* want to create indices nonconcurrently, there is still a
**huge footgun** to be aware of -- locks which are held in a transaction
are not released until the transaction closes.

Dangerous Django defaults
=========================
By default, any time you add a new Django field with `db_index=True` or
`unique=True`, the automatically generated `Migration` has:

- implicit setting of `atomic=True` (transactions are on)
- an `ALTER TABLE...ADD COLUMN` call (exclusively locks the table)
- a `CREATE INDEX` invocation (*hopefully* quick, due to nullable
  columns)

It's even more dangerous if you combine an `ALTER TABLE` call with
`CREATE INDEX` of an existing, non-nullable column.

Favor `atomic=False` if you must index
======================================
Index-creating migrations should have few `operations`. Since index
creation does not modify row-level data (and only requires locking
against writes), there's no reason to use transactions.

It's prudent to bypass the default `atomic=True` for index creation.
2023-07-09 14:42:35 +02:00
David Cain 9fdfa06845 Fix error output when warnings are found
We should log the *warnings* found, not errors!
2023-07-09 14:42:35 +02:00
Caspar Wylie 2d15ea84f9 Support partial function inspection in data migrations 2023-07-03 21:02:14 +02:00
David Wobrock 97a1f5c6c9 Fix #250 -- Don't consider partial index creation as making a column not nullable. 2023-07-03 20:41:03 +02:00
David Wobrock 366d16b01a Add Django 4.2 compat' and update test dependencies. 2023-07-02 20:36:03 +02:00
David Wobrock f511d51d9a Apply pre-commit hooks on all files. 2023-07-02 20:36:03 +02:00
David Wobrock 2ee7dac3b3 Add internal classes and add type hints + mypy. 2022-12-26 16:26:24 +01:00
David Wobrock e8f83708b0 Run black. 2022-11-03 22:00:00 +01:00
Sondre Lillebø Gundersen bd4eb89b8c chore: Upgrade Python syntax with pyupgrade
Run with the following pre-commit configuration:

```
repos:
  - repo: https://github.com/asottile/pyupgrade
    rev: v3.1.0
    hooks:
      - id: pyupgrade
        args: [ "--py3-plus", "--py36-plus", "--py37-plus" ]
```

The rationale for every change can be found at https://github.com/asottile/pyupgrade
2022-11-03 22:00:00 +01:00
Sören Wegener a3fc5f1a50 Improve error message when an unsupported database vendor/SQL analyser was selected. 2022-10-03 15:48:11 +02:00
David Wobrock bef47906d1 Fix #209 - Raise when getting SQL statements fail. 2022-07-28 23:45:14 +02:00
David Wobrock 74a7ee9239 Remove remaining __future__ imports. 2022-07-23 11:59:50 +02:00
David Wobrock f67f27354d Fix #218 - Handle model 'objects' call with other parts in the name. 2022-07-09 15:37:57 +02:00
David Wobrock c7d9129db9 Adapt 'warnings-as-errors' option to allow taking migration test codes as input 2022-03-13 17:26:40 +01:00
David Wobrock 4fa6a33153 Fix #198 -- Allow using Django settings for linter configuration 2022-02-15 19:21:58 +01:00
David Wobrock 76a55dbb12 Raise when unsupported db vendor, and add option to specify SQL analyser 2022-01-30 15:20:18 +01:00
David Wobrock 71ddadda05 Add funtional tests for lintmigrations command 2022-01-28 21:48:05 +01:00
David Wobrock 3219f9cf8a Handle creating a unique index as adding a unique constraint 2022-01-28 21:23:44 +01:00
David Wobrock 3fbc720650 Add Django 4.0 support 2022-01-28 21:03:12 +01:00
David Wobrock 7674a51bc9 Fix #178 -- Don't detect an index creation when the table is being created 2021-12-20 22:38:54 +01:00
David Wobrock 085048d22b Drop Py2.7, 3.5 and Django 1.11, 2.0, 2.1, 3.0 2021-11-12 17:26:38 +01:00
Christian Bundy 397c1f0e16 Default ends_with_default to False
Problem: Previously in gh-117 we aimed to solve gh-115, which described a problem where we used `SET DEFAULT` and then `DROP DEFAULT` immediately afterward. The solution was to track whether any defaults are retained, but the `ends_with_default` variable is initialized as `None` rather than `False`. At the end of the function we check whether `ends_with_default is False`, never occurs unless a default is set.

Solution: Assume that there is no default unless proven otherwise, and iniitializie `ends_with_default` to `False` instead of `None`.

Fixes: gh-158
2021-08-14 15:22:24 +02:00
David Wobrock fa56196f04 Add PG reindex linting warning and improve some smaller stuff 2021-06-19 18:15:40 +02:00
David Wobrock 27e492996c Apply black 2021-06-19 18:15:40 +02:00
David Wobrock 2bf87c0659 Add Postgresql warning when using DROP INDEX without CONCURRENTLY keyword 2021-06-19 18:15:40 +02:00
David Wobrock 343d163c88 Add Postgresql warning when using CREATE INDEX without CONCURRENTLY keyword 2021-06-19 18:15:40 +02:00
David Wobrock 6d595f1555 Add support for positional app_label and migration_name arguments 2021-05-26 18:53:19 +02:00
David Wobrock 062c60cd31 Add DEFAULT_AUTO_FIELD to settings for explicit PK field 2021-05-02 15:07:16 +02:00
David Wobrock c988ee3d51 Delete apps.py files for Django 3.2 automatic discovery 2021-05-02 15:07:16 +02:00
David-Wobrock 137089573c Implement include migration name options 2021-01-31 17:44:58 +01:00
David-Wobrock 6ab1d58c47 Apply isort to the project 2020-11-30 21:46:40 +01:00
David-Wobrock 4b15e11faa Rename data migration lint checks 2020-11-30 21:40:57 +01:00
David-Wobrock f795fa07d2 Add tests for RunSQL tests 2020-11-26 19:02:57 +01:00
David-Wobrock 780630131f Add warning if RunPython model variable name not the same as model class 2020-11-26 19:02:57 +01:00
David-Wobrock 601e02e024 Rename method name and add comments 2020-11-26 19:02:57 +01:00
David-Wobrock a68ff7c306 Make data migration model import check a bit less strict
It now allows:
* get_model(...) calls with one parameter only
* allows the variable name to be not the same as the model name itself (this will become a warning)
2020-11-26 19:02:57 +01:00
David-Wobrock 321b41ddd0 Start using GitHub Actions and remove Travis 2020-11-26 18:43:34 +01:00
David-Wobrock 50294a213b Add Dj3.1 support and update flake8 and black versions 2020-10-10 18:14:56 +02:00
David Wobrock ae84a5ffad Fix most tests for previous Django and Python versions 2020-07-02 19:15:30 +02:00
David Wobrock c2c26ae106 Add tests for the makemigrations override 2020-07-02 19:15:30 +02:00
David Wobrock f21ff2c278 Add (failing) tests cases for makeing a column not null with a Django default value 2020-06-20 18:08:16 +02:00
David Wobrock d411ea2f6b Unrelated: remove unused Django boilerplate files 2020-06-20 18:08:16 +02:00
David Wobrock 31338714dd Fix #112 -- make the behaviour of --include-migrations-from more obvious
* Misspelled filename => fail (instead of linting all migrations and ignoring the error)
* An empty file/a file with no valid migration paths => lint no migrations (instead oflinting all migrations)
* If the option is not specified, keep the behaviour of linting all migrations
2020-06-17 22:32:42 +02:00
David-Wobrock a5e17d0df1 Bump to 2.2.1 and black some files 2020-05-16 21:16:36 +02:00
David Wobrock cb6961539c Handle for forward and backward data migration imports 2020-02-08 22:38:05 +01:00
David Wobrock e49f6dea9c Document, add to changelog the data migrations and rename to 'warnings-as-errors' (plural) 2020-02-07 23:29:15 +01:00
David Wobrock 71614bd377 Handle warning as error and exclude warning tests 2020-02-07 23:29:15 +01:00
David Wobrock d01875b3be add warnings and RunPython basic linting 2020-02-07 23:29:15 +01:00
David Wobrock 691b1b82cb Add app_data_migration used for tests 2020-02-07 23:29:15 +01:00
David Wobrock 813c5c8690 Remove apache license headers in each file. The one full copy of the license at the root of the repo shall define the license for all source files. 2020-01-11 22:08:33 +01:00