## TL;DR

Run the daily content pipeline as cron jobs with three disciplines: each job is idempotent (reruns are safe), every run logs to a durable file, and failures alert rather than silently stopping. Schedule generation, review, and publish as separate steps so a failure in one does not corrupt the others.

```text
how to run a daily content pipeline with cron jobs
```

## Use this when

- Publishing content on a daily or regular cadence
- The pipeline has multiple stages (draft, review, publish)
- Runs happen unattended

## Not for this skill when

- Publishing is irregular (run manually)
- Every step needs human judgment (automate less, not more)
- You have no alerting (silent failures will eat you)

## Steps

1. Split the pipeline into staged jobs. Generate, review-queue, and publish as separate cron entries with clear handoffs via files. Expected: a failure in generation never publishes a half-done piece.

2. Make every job idempotent. Deterministic output keys and "skip if already done" checks mean reruns are safe. Expected: running the job twice produces one result.

```
pipeline stages:
  06:00 generate -> drafts/[date].md (skip if exists)
  07:00 review   -> moves approved to queue/
  08:00 publish  -> publishes queue/, logs URLs
```

3. Log every run durably. Append-only log with timestamps, outside /tmp, one line per significant event. Expected: you can reconstruct any day's run from the log.

4. Alert on failure, not on success. A failed stage pages or messages you; successful runs stay quiet. Expected: silence means healthy, noise means broken.

5. Keep a manual override. A flag file or command that pauses the pipeline for launch holds or incidents. Expected: you can stop the machine in seconds.

## Variant phrasings

### cron-based publishing pipeline

Staged jobs, idempotent reruns, durable logs, failure alerts, manual override.

### automating a daily content run

Generate, review, publish as separate scheduled steps with file handoffs.

### scheduled content pipeline best practices

Idempotency first, logging second, alerting third; never let stages blur.

## Why it happens

Daily cadence plus unattended execution means failures compound silently: a stuck job publishes nothing for a week, or worse, publishes broken content daily. Staged, idempotent, logged jobs make the pipeline boring, which is the goal.

## Edge cases / pitfalls

- Cron environments are minimal; use absolute paths and explicit environment setup.
- Overlapping runs corrupt state; use a lock file per job.
- Daylight saving shifts cron times; schedule in UTC or accept the shift.
- Test the failure alert path; an alert that never fires is decoration.

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_vT7-GoReb71wPw1R3erv-w