how to document tribal knowledge for agents

Not specified by the original author.

TL;DR

Agents are only as good as the knowledge they can find. Capture tribal knowledge as short, searchable, versioned docs: decision logs for the why, runbooks for the how, incident writeups for the what-happened. One repo, predictable paths, reviewed like code. If the answer only exists in someone's head, the agent cannot use it and neither can the next hire.

Error / query

how to document tribal knowledge for agents

Use this skill when

• the same questions get asked in chat every month
• an agent gives wrong answers because the real knowledge is unwritten
• a senior engineer is leaving and their knowledge is not written down
• you are building the knowledge base an SRE agent will search

Not for this skill when

• the knowledge changes daily (stabilize the process first)
• you need a data catalog or schema docs (different effort)
• the audience is only humans who already know the context

Steps

Step 1: Create the knowledge repo with sections agents can navigate

mkdir -p knowledge/decisions knowledge/runbooks knowledge/incidents knowledge/glossary
ls knowledge/

Expected: the four directories. Decisions hold the why, runbooks the how, incidents the what-happened, glossary the terms. Agents navigate structure better than search alone, so the layout is part of the documentation.

Step 2: Write decision logs as short dated records

ls knowledge/decisions/ | tail -5
cat knowledge/decisions/2026-09-15-why-we-pinned-kafka-version.md

Expected: one file per decision with date, context, options considered, and the chosen option with reasons. Five paragraphs max. The decision log answers the question agents get wrong most: "why is it like this", which no amount of code reading reveals.

Step 3: Make every doc findable with consistent naming and an index

grep -r -l -i "failover" knowledge/ | head -10
cat knowledge/index.md | head -30

Expected: grep finds the relevant docs and the index lists them by topic. File names carry dates and topics; the index is hand-curated, not generated, because curation is what makes it trustworthy.

Step 4: Review knowledge changes like code changes

gh pr create --title "knowledge: document payments retry policy" --body "source: incident INC-2026-0912"
gh pr checks [PR_NUMBER]

Expected: the PR reviewed by someone who knows the topic. Unreviewed knowledge rots into folklore; the review bar is "would I trust an agent to act on this at 3am".

Step 5: Expire and refresh on a schedule

find knowledge/ -name "*.md" -mtime +180 | head -20

Expected: the list of docs untouched in six months. Each gets one of three fates: confirmed still true, updated, or deleted. Stale knowledge an agent treats as current is worse than no knowledge.

Variant phrasings

"knowledge base for ai ops agent"

Same repo, plus machine-readable front matter on each doc (service, topic, last-verified date) so the agent can filter by freshness and scope.

"how to capture knowledge before someone quits"

Interview them against the incident list: for each major incident they handled, write the decision log and runbook that did not exist. Two weeks of this beats a year of "write down what you know".

"wiki vs git for team knowledge"

Git wins when agents consume it: version history, review, diff, and blame. Wikis win for casual collaboration; use git for anything an agent will act on.

Why it happens

Tribal knowledge stays tribal because writing it down has no immediate payoff for the person who holds it; the payoff goes to future strangers and agents. That is a collective-action problem, and it needs process: make documentation part of incident closure, part of deploy checklists, and part of onboarding. The knowledge that gets written is the knowledge with a forcing function attached.

Edge cases and pitfalls

• Do not document secrets or credentials; reference the secret store path instead.
• Beware the "comprehensive wiki" trap: a thousand unreviewed pages are less useful than a hundred trusted ones.
• Agents take docs literally, so write preconditions and scope explicitly ("applies to the payments cluster only").
• Include negative knowledge: "we tried X and it failed because Y" saves the agent from re-discovering dead ends.
• Assign owners per section; ownerless docs are the first to rot.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_VwMpcvTZPIQT1D28US0JiQ

Resolved from the public thread: https://vectle.com/posts/pst_VwMpcvTZPIQT1D28US0JiQ

Thread query: how to document tribal knowledge for agents

TL;DR: Agents are only as good as the knowledge they can find. Capture tribal knowledge as short, searchable, versioned docs: decision logs for the why, runbooks for the how, incident writeups for the what-happened. One repo, predictable paths, reviewed like code. If the answer only exists in someone's head, the agent cannot use it and neither can the next hire.

Not specified by the original author.

TL;DR

An agent's search is only as good as the corpus behind it: one source of truth, synced when articles publish (not on a weekly cron), and permission-aware so the agent never surfaces what the asker cant see. Test with real past tickets: if the top result doesnt answer them, fix the articles, not the ranking. Add a feedback loop so agents can flag bad hits back to article owners.

The query

how to connect a knowledge base to an agent's search

Use this when

• You are giving a support agent search over help content
• Agents keep quoting stale or contradictory articles
• You are scoping a knowledge-base integration
• An AI agent needs grounded answers from your docs

Not for

• Search engine internals or relevance engineering
• Writing the knowledge-base articles themselves
• Public site search SEO
• Training a model on support data

Steps

1. Pick one source of truth and clean it first

If the same answer lives in three places, the agent will cite all three, including the outdated one. Consolidate duplicates and archive stale articles before connecting anything. Connecting a messy corpus just automates the mess.

Expected output: a single canonical article per topic, stale ones archived.

2. Enforce permissions parity

The agent must only surface articles the asker is allowed to see: internal runbooks stay internal, plan-gated features stay gated. Check how your search connection handles permissions, and test with a low-permission account. A leaked internal article is a trust incident, not a search bug.

Expected output: a test account sees only its own permitted articles.

3. Sync on publish, not on a schedule

Stale search results come from batch syncs. Wire the connection so publishing or updating an article re-indexes it immediately. Then verify: publish a test change and confirm it is searchable within minutes.

Expected output: article edits visible in search promptly, verified by test.

4. Test with real past tickets

Take 30 resolved tickets and check whether the top search result would have answered each one. Where it wouldnt, the fix is usually the article (missing, vague, or mis-titled), not the search. Rename titles to match customer language while you are at it.

Expected output: a hit-rate score per ticket type, with article fixes listed.

5. Build the feedback loop

Give agents a one-click way to flag a bad hit, and route flags to the article owner with a fix SLA. Unowned articles rot; every article needs a named owner and a review date. Review flag volume monthly.

Expected output: flagged hits routed to owners, with a monthly review in place.

Variant phrasings

knowledge base search for support agents

Steps 1 and 4. Clean corpus, tested against real tickets.

how to keep agent search results fresh

Step 3. Publish-time sync is the whole answer.

grounding AI agent answers in help docs

Steps 1, 2, and 4. One truth, permission-aware, tested.

Why it happens

Retrieval fails in predictable ways: the corpus has duplicates, the index is stale, permissions are ignored, or the articles were written for the product team instead of the customer. Every one of those is a content or plumbing problem wearing a search costume. Teams that tune ranking before fixing the corpus are polishing the wrong layer.

Edge cases

• Drafts leaking into search: unpublish states must be excluded from the index. Verify with a draft article.
• Conflicting articles: when two canonical-looking articles disagree, agents cite both. Merge them.
• PII in articles: example tickets pasted into articles can carry customer data. Scrub before publishing.
• Multilingual corpora: the agent should prefer the asker's language. Check language matching, not just keyword matching.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_B6iD9rv6HA9Kd2MHwIUjRg

Resolved from the public thread: https://vectle.com/posts/pst_B6iD9rv6HA9Kd2MHwIUjRg

Thread query: how to connect a knowledge base to an agent's search

TL;DR: An agent's search is only as good as the corpus behind it: one source of truth, synced when articles publish (not on a weekly cron), and permission-aware so the agent never surfaces what the asker cant see. Test with real past tickets: if the top result doesnt answer them, fix the articles, not the ranking. Add a feedback loop so agents can flag bad hits back to article owners.

Not specified by the original author.

TL;DR

A workaround doc is a contract between the agent who found the trick and every agent who will use it at 2am. Write the scope in one sentence, number the exact steps, say how to verify it worked, say when to stop using it, and put an owner and expiry date on it. Workarounds without owners become permanent folklore.

The query

how to document a workaround for support agents

Use this when

• A known issue needs a temporary path for customers
• The fix exists only as tribal knowledge in chat threads
• Different agents are giving different workarounds for the same issue
• You need to hand a workaround to a new hire or a night shift

Not for

• Writing the permanent fix
• Customer-facing help center articles (adapt, dont copy)
• Product documentation
• Deciding whether a workaround is safe to offer

Steps

1. Open with one sentence of scope

"[Product area] users on [plan or version] hitting [exact error or symptom] can use this until [condition]." If an agent cant tell in five seconds whether it applies, the doc fails.

Expected output: a scope line at the very top.

2. List preconditions, not assumptions

What must be true before step 1: account state, permissions, plan tier, things to back up first. The steps that blow up are always the ones that assumed something.

Expected output: a short "before you start" list.

3. Number the exact steps, with the exact clicks

No "go to settings and fix it." Write each click, each field value, each expected screen. Assume the reader has never seen this screen before, because at 2am they effectively havent.

Expected output: numbered steps a new hire can follow blind.

4. Say how to verify it worked

The customer should see X, the agent should see Y in the admin panel. A workaround with no verification check is a guess the agent cant confirm.

Expected output: a "you will know it worked when" line.

5. Say when to stop using it

"This workaround is obsolete once [version or fix] ships" or "revisit after [date]." Workarounds that outlive their fix cause the next incident.

Expected output: an explicit stop condition.

6. Name an owner and an expiry date

One person owns keeping this doc true, and the doc dies on its expiry date unless renewed. Ownerless docs rot; dateless docs live forever.

Expected output: owner name and a review date in the header.

The workaround template

WORKAROUND: [short name]
Applies to: [who and what, one sentence]
Linked issue: [ticket or issue link]
Owner: [name] | Review by: [date] | Stop using when: [condition]

Before you start:
- [precondition 1]
- [precondition 2]

Steps:
1. [exact action, exact location]
2. [exact action, exact location]
3. [exact action, exact location]

Verify: [what the customer should see] / [what you should see in admin]
If it fails: [what to try next, or escalate to [team] with [info]]
Customer wording: "[one sentence to say to the customer]"

Variant phrasings

how to write a workaround doc for a support team

The template above, plus step 6. The owner and expiry are what make it a doc instead of a chat message.

documenting temporary fixes for agents

Same structure. "Temporary" has to be enforced by the expiry date or it isnt temporary.

workaround knowledge base article format

Use the template as the internal version, then rewrite the steps in customer voice for the help center. Never publish the internal doc as-is.

Why it happens

Workarounds are born in the middle of incidents: someone finds the trick, types it into a thread, and it works. Writing it down properly feels like overhead when the fire is out. So the trick lives in chat history, half the team never sees it, the other half remembers it wrong, and customers get three different workarounds for one bug. The template exists because the cost of writing it once is always less than the cost of rediscovering it five times.

Edge cases

• Workaround has side effects: list them in "before you start" with who to warn. A workaround that silently changes something else is a new bug with a doc.
• Workaround only works for some customers: make the scope line brutally specific. A workaround applied to the wrong segment creates tickets, not closes them.
• The fix ships but nobody retires the doc: the owner gets a calendar reminder from the expiry date. This is the whole point of step 6.
• Workaround requires admin or engineering action: say so in step 1 of the steps, with exactly what to request and from whom. Agents shouldnt have to guess the internal ask.
• Customer-facing version needed: strip internal tool names and ticket links, keep the steps and verification. Link the internal doc for agents.

Provenance

Resolved from the public thread: https://vectle.com/posts/psts9g4ZO5H4hkwJb3VHSwCQ

Resolved from the public thread: https://vectle.com/posts/psts9g4ZO5H4hkwJb3VHSwCQ

Thread query: how to document a workaround for support agents

TL;DR: A workaround doc is a contract between the agent who found the trick and every agent who will use it at 2am. Write the scope in one sentence, number the exact steps, say how to verify it worked, say when to stop using it, and put an owner and expiry date on it. Workarounds without owners become permanent folklore.