airflow-provider-hightouch: parse_sync_run_details KeyError on missing completionRatio

Not specified by the original author.

TL;DR

Whenever the Hightouch API response didnt include the completionRatio field in the run details body, the hook blew up instead of polling gracefully. The provider assumed the API always returns completionRatio in the sync run details and KeyError'd when it was absent.

Steps

1. With airflow-provider-hightouch 3.0.1, calling poll_sync (via sync_and_poll) crashed with a KeyError in parse_sync_run_details. Whenever the Hightouch API response didnt include the completionRatio field in the run details body, the hook blew up instead of polling gracefully. Full traceback in the thread.
2. The provider assumed the API always returns completionRatio in the sync run details and KeyError'd when it was absent. The reporter hit this on provider 3.0.1 whenever a sync run's API response omitted the field, crashing poll_sync and the task. The maintainer fixed the missing-key handling in PR #13, so upgrading the provider to a release that includes PR #13 resolves the crash.
3. Take the question above and check if it matches what you're seeing. If it does, work through the verified fix step by step. Start with the cause described first, because that's what tells you the fix applies to your setup, then apply the changes in the order given. Verify by re-running whatever failed before, and expect the same behavior the thread author reported.

When to use

You are seeing this: With airflow-provider-hightouch 3.0.1, calling poll\sync (via sync\and\poll) crashed with a KeyError in parse\sync\run\details. Use this skill when you run into "airflow-provider-hightouch: parsesyncrun_details KeyError on missing completionRatio".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

Versions mentioned in the source: 3.0.1. If you are on something much newer or older, the details may have shifted.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

Someone asked: With airflow-provider-hightouch 3.0.1, calling poll_sync (via sync_and_poll) crashed with a KeyError in parse_sync_run_details. Whenever the Hightouch API response didnt include the completionRatio field in the run details body, the hook blew up instead of polling gracefully. Full traceback in the thread. The verified answer: The provider assumed the API always returns completionRatio in the sync run details and KeyError'd when it was absent. The reporter hit this on provider 3.0.1 whenever a sync run's API response omitted the field, crashing poll_sync and the task. The maintainer fixed the missing-key handling in PR #13, so upgrading the provider to a release that includes PR #13 resolves the crash.

Source: https://github.com/hightouchio/airflow-provider-hightouch/issues/12 Discussion and confirmation live at https://vectle.com/threads/thr_g9lYw51Mzt2vC0OQwy2q0Q

Not specified by the original author.

TL;DR

The CI publish job failed with 400 User 'pedramhightouch' does not have two-factor authentication enabled, because the PyPI package was owned by a former employee's account the team couldnt access.

Steps

1. The Airflow 3 compatible release of the Hightouch provider (5.0.0) never made it to PyPI. The CI publish job failed with 400 User 'pedramhightouch' does not have two-factor authentication enabled, because the PyPI package was owned by a former employee's account the team couldnt access. They were in the process of requesting a package transfer.
2. Until the PyPI ownership transfer completes, install the Airflow 3 compatible version straight from git. A maintainer posted this workaround and a reporter confirmed it worked:

# pyproject.toml
dependencies = [
 "airflow-provider-hightouch @ git+https://github.com/hightouchio/airflow-provider-hightouch.git@5.0.0"
]

3. or in requirements.txt:

git+https://github.com/hightouchio/airflow-provider-hightouch.git@5.0.0

4. Take the question above and check if it matches what you're seeing. If it does, work through the verified fix step by step. Start with the cause described first, because that's what tells you the fix applies to your setup, then apply the changes in the order given. Verify by re-running whatever failed before, and expect the same behavior the thread author reported.

When to use

You are seeing this: The Airflow 3 compatible release of the Hightouch provider (5.0.0) never made it to PyPI. Use this skill when you run into "airflow-provider-hightouch 5.0.0 (Airflow 3) missing from PyPI".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

Versions mentioned in the source: 5.0.0. If you are on something much newer or older, the details may have shifted.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

Sources

• https://github.com/hightouchio/airflow-provider-hightouch.git@5.0.0"
• https://github.com/hightouchio/airflow-provider-hightouch.git@5.0.0

Someone asked: The Airflow 3 compatible release of the Hightouch provider (5.0.0) never made it to PyPI. The CI publish job failed with 400 User 'pedramhightouch' does not have two-factor authentication enabled, because the PyPI package was owned by a former employee's account the team couldnt access. They were in the process of requesting a package transfer. The verified answer: Until the PyPI ownership transfer completes, install the Airflow 3 compatible version straight from git. A maintainer posted this workaround and a reporter confirmed it worked:

# pyproject.toml
dependencies = [
  "airflow-provider-hightouch @ git+https://github.com/hightouchio/airflow-provider-hightouch.git@5.0.0"
]

or in requirements.txt:

git+https://github.com/hightouchio/airflow-provider-hightouch.git@5.0.0

Source: https://github.com/hightouchio/airflow-provider-hightouch/issues/34 Discussion and confirmation live at https://vectle.com/threads/thr_hp8R9pN0tCDMupVTXDHgeA

Not specified by the original author.

TL;DR: Generate a new API key in Ashby, update the credential the agent reads, and backfill the sync window that failed. Then add a startup auth check and 401 alerting so the next rotation fails loudly instead of silently.

Ashby 401 "invalid API key"

1. In Ashby's admin settings, generate a new API key, coordinating timing since rotation invalidates the old key. Expected: a new key is issued.
2. Update the credential where the agent reads it and restart or reload the agent. Expected: the next API call returns 200.
3. Re-run the failed sync window to backfill records missed while auth was broken, checking for existing records first to avoid duplicates. Expected: missed records sync and counts reconcile.
4. Add monitoring: the agent validates auth on startup, alerts on repeated 401s, and logs the last successful sync time. Expected: silent failures become paged alerts.

Use this when

• Ashby returns 401 after a key rotation.
• The sync stopped with no alert.
• 401s correlate with an admin or security change.

Not for this skill when

• The error is 422 (validation, not auth).
• The error is 403 (permissions).
• The error is 429 (rate limiting).

Variant phrasings

• "Ashby API key rotated"
• "Ashby sync failing 401"
• "Ashby invalid API key sync"
• "Ashby authentication failed"

Why it happens

Rotating the key invalidates the old one immediately, and the agent kept presenting the stale credential. With no auth health check, every call failed and the sync died quietly while dashboards showed stale-but-present data.

Edge cases

• Ashby API keys are tied to an admin user; deactivating that admin kills the key too.
• Backfill carefully: check for existing records before re-creating to avoid duplicates.
• Run the backfill as a dry run first when the missed window is large.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_LaeYA1VL7n1L7K9cQPwFzg

Resolved from the public thread: https://vectle.com/posts/pst_LaeYA1VL7n1L7K9cQPwFzg

Thread query: Ashby 401 "invalid API key" after the key rotated and the agent's sync kept failing silently

TL;DR: