What was reported:
Issue grafana/grafana#109101 (closed, 29 comments): ### What happened?
We are running Grafana in Kubernetes with a PostgreSQL 14.5 database since Grafana 10.4.
While attempting to upgrade from **Grafana 12.0 to 12.1,** the database migration gets stuck indefinitely on the following step:
```
logger=sqlstore t=2025-08-03T11:51:43.548045485Z level=info msg="Connecting to DB" dbtype=postgres
logger=migrator t=2025-08-03T11:51:43.564028186Z level=info msg="Locking database"
logger=migrator t=2025-08-03T11:51:43.565415943Z level=info msg="Starting DB migrations"
logger=migrator t=2025-08-03T11:51:43.583652895Z level=info msg="Executing...
What works:
If the Grafana 12.0 to 12.1 upgrade gets stuck on the annotation migration with the pod crashlooping (Kubernetes restarts the pod mid-migration, leaving several copies of the slow query running), the fix shipped in v12.2.3 and v12.3.1 via PR #113589, so upgrade past that. If you are mid-upgrade: scale Grafana down to one replica and run the backfill manually (UPDATE annotation SET dashboard_uid = dashboard.uid FROM dashboard WHERE annotation.dashboard_id = dashboard.id AND annotation.dashboard_id != 0 AND annotation.dashboard_uid IS NULL), then let the migration finish. Also raise readiness and liveness probes to 10 minutes or more so Kubernetes does not kill the pod mid-migration, and trim the annotations table first with the max_age and max_annotations_to_keep settings.