An AI knowledge base stays accurate when every authoritative source has an owner, a freshness rule, and a controlled path for changes. New information is captured, conflicts are resolved, approved updates are published, retrieval is tested, and obsolete rules are retired.
Building the initial knowledge base is only the beginning. A service changes its price. A contract adds an exception. A team adopts a new handoff. A policy moves from draft to approved. If those changes do not reach the knowledge layer, people and agents continue applying yesterday's business.
Maintenance is therefore an operating function, not a cleanup project. It makes the difference between a source of truth and a source of increasingly confident old answers.
1. An AI knowledge base is not finished at launch
The company knowledge-base guide explains what belongs in an AI-ready source of truth: facts, rules, context, examples, decisions, provenance, access boundaries, and retrieval structure. Once that foundation exists, maintenance has a narrower job—keep the approved truth aligned with the business.
The maintenance function should be able to answer:
- Which business changes occurred?
- Which knowledge objects may be affected?
- Which source has authority?
- Who must review the change?
- When did the new rule become effective?
- Which agents, workflows, and people depend on it?
- Did retrieval produce the expected answer after publication?
- Which old version must remain available for history, and which must no longer be used?
Corey Ganim's June 25 Build With AI episode discusses ingest and curation routines for a Markdown knowledge system. That provides useful implementation context, but the durable buyer requirement is tool-independent: company knowledge needs a repeatable path from source change to trusted use.
2. Assign an owner and authority to every source
A source owner is accountable for the truth of the information, not merely for the folder where it lives.
The owner should know:
- what the source governs;
- which roles and systems rely on it;
- how a change is approved;
- how quickly a material update must be reviewed;
- which source wins when information conflicts;
- who acts as backup.
Ownership should be assigned at a useful level. “Operations owns the knowledge base” is too broad if pricing belongs to finance, delivery policy belongs to service leadership, and customer commitments belong to an account owner.
Use authority classes to make precedence visible:
| Authority class | Example | Maintenance treatment |
|---|---|---|
| Binding | Signed agreement, approved policy, active price book | Requires controlled approval and effective date |
| Operational | Current SOP, routing map, service checklist | Reviewed by the function owner |
| Contextual | Meeting summary, project history, customer preference | Retained with source and date; does not override binding rules |
| Example | Approved sample output or past proposal | Teaches format or judgment; never silently becomes policy |
| Draft | Proposed change or working note | Excluded from production retrieval until approved |
One fact should have one authoritative home. Other pages can reference it, but duplicated policy creates multiple versions that can drift independently.
3. Set freshness rules by business risk
Not every fact needs the same review cadence.
Pricing, eligibility, safety, access, legal commitments, and customer-facing promises can create immediate harm when stale. Brand background, historical context, and stable internal definitions may change less often.
Define freshness using triggers as well as calendars:
- Event-driven: update when a price, policy, service, integration, role, or agreement changes.
- Scheduled: review at an interval appropriate to volatility and consequence.
- Usage-driven: review after an exception, override, failed retrieval, or repeated question.
- Dependency-driven: review when a connected system, schema, or workflow changes.
Avoid labeling an item current merely because someone opened it recently. Freshness means the authoritative owner has confirmed that the content still describes the business.
Each knowledge object should carry enough maintenance metadata to support action:
- owner and backup;
- source location;
- authority class;
- effective date;
- last reviewed date;
- next review or trigger;
- dependent workflows or agents;
- sensitivity and access;
- superseded version or retirement state.
The review interval is a risk decision. A high-consequence rule that changes unpredictably needs faster detection than a low-risk reference page.
4. Capture and review changes
The change path should fit the way the company already works. Requiring every employee to edit a specialized knowledge system will fail if changes actually originate in contract approvals, CRM updates, team meetings, ticket resolutions, and management decisions.
Create capture points around those events. A change request should include:
- what changed;
- the source evidence;
- the proposed effective date;
- the knowledge objects and workflows likely affected;
- the requester;
- the required approver;
- urgency and consequence if delayed.
Automated detection can help find candidates: a changed source file, a new approved product, a CRM field update, or repeated user correction. Detection is not approval. The maintenance function may prepare a proposed update and impact summary while the business owner decides whether it represents policy.
Use a visible review queue rather than private messages. Each item needs a state such as captured, verifying, awaiting approval, scheduled, published, rejected, or retired. The queue should expose age and owner so unresolved changes do not disappear.
Emergency changes need a fast path, but not an invisible one. Record who authorized the update, why normal review was shortened, what was affected, and when a full review will occur.
5. Resolve conflicting sources
Conflicts should stop confident retrieval and start a resolution workflow.
Common conflicts include:
- two price sheets with different effective dates;
- a general policy and a customer-specific agreement;
- an SOP that contradicts the actual approved workflow;
- a meeting note that appears to change policy without approval;
- an example containing obsolete language;
- two systems claiming ownership of the same status.
The system should show both claims, their provenance, dates, owners, and authority. It should not choose the wording that appears most often or was edited most recently unless that is the published precedence rule.
Use a resolution record:
| Field | Purpose |
|---|---|
| Conflicting statements | Preserve what each source says |
| Source and authority | Show why each claim exists |
| Affected work | Identify people, agents, and decisions at risk |
| Decision owner | Name the person authorized to resolve it |
| Resolution | State the approved truth and effective date |
| Remediation | Update references, tests, and affected records |
For customer-specific exceptions, preserve the relationship: the general rule remains valid, and the agreement overrides it only in the defined account and period.
6. Test retrieval after every material change
Publishing a file does not prove that people or agents will retrieve and apply it correctly.
Maintain a set of representative questions and workflow cases for important knowledge. After a material update, test:
- the normal question that should use the new rule;
- a neighboring question that should remain unchanged;
- an older phrasing or synonym;
- a customer-specific exception;
- a request with insufficient context;
- a prohibited or out-of-scope request;
- a citation or provenance check;
- an access-boundary check.
The expected result should include more than text. It may specify the source, action, escalation, approval, or refusal.
If retrieval uses multiple indexes, caches, summaries, or agent skills, verify each affected layer. A source can be current while a downstream cache or derived instruction remains stale.
Record the version tested, the expected behavior, the actual behavior, and the reviewer. Failed retrieval should block or limit the affected workflow until the source, index, instruction, or test is corrected.
7. Archive and delete without leaving ghost rules
Retirement is part of maintenance.
Deleting the visible page may not remove an old rule from exports, caches, embeddings, prompt files, local copies, or connected agent workspaces. Conversely, erasing all history can make it impossible to explain an earlier decision.
Define three states:
- Active: approved for current use.
- Superseded: retained for audit or historical interpretation but excluded from current retrieval.
- Deleted: removed according to retention, privacy, or legal requirements and propagated to dependent systems.
When retiring knowledge:
- publish the replacement or state that no replacement exists;
- mark the effective end date;
- remove active links and retrieval eligibility;
- update derived summaries, instructions, and tests;
- invalidate affected caches or indexes;
- verify current questions no longer return the retired rule;
- preserve or delete history according to policy.
An archive label without retrieval controls is not enough. The operational test is whether the old rule can still influence current work.
8. Use a maintenance scorecard
A useful scorecard shows whether the knowledge system is governed and usable. It does not need an invented “AI readiness percentage.”
Track:
| Measure | What it reveals |
|---|---|
| Sources with named owners | Whether accountability exists |
| Items past their review rule | Where staleness risk is accumulating |
| Open conflicts by age and consequence | Whether contradictory truth remains unresolved |
| Changes awaiting approval | Where governance is blocking publication |
| Retrieval tests passing after change | Whether updates reach actual use |
| Corrections and overrides by source | Which knowledge needs repair |
| Retired items still retrievable | Whether deletion and archival controls work |
| Dependent workflows not retested | Where change impact remains unknown |
Review patterns, not only totals. A small number of unresolved pricing conflicts may matter more than hundreds of overdue low-risk reference pages.
9. Knowledge-maintenance checklist
- Every authoritative source has a primary owner and backup.
- Authority classes and precedence rules are documented.
- High-consequence knowledge has event-driven change detection.
- Every item carries effective, review, and retirement information.
- Change requests include evidence, impact, owner, and approval.
- A visible queue exposes unresolved changes and conflicts.
- Draft material is excluded from production retrieval.
- Material changes trigger retrieval, exception, and access tests.
- Derived summaries, instructions, caches, and indexes are included.
- Superseded and deleted rules cannot influence current work.
- The scorecard emphasizes risk, age, and affected functions.
Knowledge Maintenance is one of Return My Time's named managed-agent starting points, but no measured client engagement currently supports an outcome claim. The operating model above is a buyer standard: who owns the truth, how changes become approved knowledge, and how the company proves that current work uses it.
If your knowledge base exists but nobody can show how it stays current, book a discovery call. The call qualifies fit and scopes the paid assessment required before managed implementation.
