## TL;DR
Two goroutines wrote to the same Go map at the same time, which is a fatal runtime error, not a recoverable panic. It flakes in CI because higher parallelism changes goroutine interleaving. Run the suite with -race: the race detector prints both goroutine stacks with file and line numbers. Then serialize the access with a mutex, switch to sync.Map, or give one goroutine ownership of the map.

## The query
```text
go test fatal error: concurrent map writes flake in CI  -  how to debug
```

## Use this when
- go test dies intermittently with fatal error: concurrent map writes
- the failure moves between runs and never reproduces single-threaded
- the stack mentions runtime.mapassign or concurrent map

## Not for
- a data race on a slice, struct field, or counter (use -race the same way, but the fix differs)
- a deadlock where nothing progresses (no fatal error is printed; use the timeout dump instead)
- the race is in test helper code rather than production code (still fix it; flaky helpers poison the suite)

## Steps

### 1. Rerun with the race detector
The race detector turns the fatal error into a detailed report naming both parties. Run the suspect package with it.
```bash
go test -race -count=1 ./internal/cache/
```
Expected output: the output contains WARNING: DATA RACE with two goroutine stacks.

### 2. Read both stacks to find the shared map
The report shows the write that raced and the other goroutine's access. The map variable and both call sites are named.
```text
WARNING: DATA RACE
Write at 0x... by goroutine 12:
  internal/cache.(*Store).Set(...)
      /app/internal/cache/store.go:44
Previous write at 0x... by goroutine 9:
  internal/cache.(*Store).Set(...)
      /app/internal/cache/store.go:44
```
Expected output: both racing call sites are identified with file and line.

### 3. Serialize the access
Pick the cheapest correct fix: a mutex around the critical section for general use, sync.Map for mostly-read workloads, or channel-based ownership.
```go
type Store struct {
    mu sync.RWMutex
    m  map[string]Item
}
func (s *Store) Set(k string, v Item) {
    s.mu.Lock()
    defer s.mu.Unlock()
    s.m[k] = v
}
```
Expected output: the code compiles and the critical section is covered.

### 4. Verify with -race and add it to CI
Run the full package with the detector and make it part of the CI job so the next race is caught at introduction time.
```bash
go test -race -count=1 ./...
```
Expected output: -race is clean across the module and the CI job runs the detector on every push.

## Tool compatibility
- Go 1.21 - 1.24
- race detector requires cgo (a C toolchain in the CI image)

## Variant phrasings
- fatal error: concurrent map writes flake in CI
- go test WARNING: DATA RACE on map
- concurrent map iteration and map write in CI

## Why it happens
Go maps are not safe for concurrent use by design: the runtime deliberately crashes instead of corrupting memory silently. Local runs often pass because fewer CPUs and less parallelism produce a lucky interleaving every time. CI runners with more cores and parallel packages reshuffle the interleaving until the race fires.

## Edge cases / pitfalls
- the race detector needs gcc in the CI image; a missing toolchain fails the build, not the test
- sync.Map is only faster for append-mostly or read-mostly patterns with stable keys; a mutex wins for general use
- a race that only appears with -count=2 or higher is still real; loop the suite a few times before declaring victory
- maps guarded by a mutex must never escape the guard: returning the map itself reintroduces the race

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_92_qiuxBKY-W_fzTuGLKwA