What Belongs in a Company Knowledge Base for AI?

A company knowledge base for AI should be one structured source of truth for the facts, rules, context, and decisions that people and software need to do reliable work.

It is not a folder dump. Moving every document into one drive does not resolve conflicting policies, stale pricing, unclear ownership, or sensitive information. Connecting a chatbot to that folder only makes those gaps easier to retrieve.

A useful first version is narrower. It identifies the knowledge required for a specific set of business decisions, preserves the source behind each important fact, limits access, and gives every durable record an owner and a maintenance rule.

If you have not yet chosen the operating problem this foundation should support, begin with the AI project readiness scorecard. The knowledge system should follow the business need—not become an expensive filing project without a decision to improve.

1. Build a source of truth, not a folder dump

The goal is not to collect the most files. The goal is to help an authorized person or system find the current answer, understand why it is authoritative, and know what to do when the answer is missing or disputed.

For each important piece of knowledge, the system should answer:

• What business question does this record support?
• Where did the information come from?
• Is it a governing source, supporting context, an example, or an unverified note?
• Who owns its accuracy and access?
• When did it become effective?
• What event or date triggers review?
• Which earlier record does it replace?
• Who may retrieve it, and for which purpose?

A folder structure can help people browse, but folders alone rarely express authority or conflict rules. Two documents can both look current while prescribing different prices or handoffs. The knowledge base needs metadata and operating rules that distinguish the approved policy from the old proposal that should remain only as history.

The first version can be simple. For a small, well-bounded corpus, structured files with a clear index and ownership fields may be enough. Larger collections or more complex retrieval needs may require search, a vector index, record-level attributes, or connections to existing systems. The architecture should fit the corpus and the decisions it supports.

Anthropic’s guidance on building effective agents recommends starting with the simplest solution that meets the need and adding complexity only when it produces a measurable improvement. That principle applies to the knowledge layer too. A sophisticated retrieval system cannot compensate for sources nobody has approved.

2. Include the knowledge behind recurring decisions

Start with the decisions and workflows that matter to the business, then inventory the knowledge required to support them.

Not every source belongs in the authoritative layer. Call transcripts, emails, meeting notes, and draft proposals can provide evidence or history, but they may contain mistakes, unapproved ideas, or information that should not be broadly available. Label them as source material until a responsible owner promotes the relevant knowledge into an approved record.

Examples also need boundaries. An approved proposal can demonstrate structure or tone without making its customer facts, pricing, or commitments reusable elsewhere.

This is why company knowledge should be organized before an agent is asked to apply it. The system needs to know which record governs the decision, not merely which passage is semantically similar to the question.

3. Assign authority, provenance, ownership, and freshness

Every durable record needs four controls:

1. Authority: its place in the decision hierarchy.
2. Provenance: the original source and the path from source to structured record.
3. Ownership: the person accountable for accuracy, access, and corrections.
4. Freshness: the condition that keeps the record current or sends it for review.

Authority rules should be specific enough to resolve predictable conflicts. A signed customer agreement may outrank the standard delivery policy for that customer. The current service catalog may outrank an old proposal for general sales questions. A finance-system record may govern payment status while a CRM note remains supporting context.

Provenance makes review possible. Preserve the source link or record identifier, effective date, last review, and the person or process that approved the structured version. When an answer affects a customer, price, policy, or material business decision, the reader should be able to trace it back to the governing source.

Ownership belongs with the party able to decide what is true. A provider can organize sources, build retrieval, monitor freshness signals, and route conflicts. It should not silently decide company policy. The client still needs named owners for pricing, customer commitments, process rules, security, and access authorization.

Freshness is more than a recurring calendar reminder. Some records should be reviewed every quarter. Others change when a service launches, a contract is signed, a regulation changes, a system field is renamed, or an owner leaves. Record the relevant trigger and define what happens when review is overdue. For high-consequence knowledge, the safe response may be to stop, show the stale-source warning, and escalate rather than answer confidently.

4. Structure retrieval for people and agents

People and agents need the same truth, but they may navigate it differently. A person benefits from a readable index, clear headings, linked source material, and enough history to understand a decision. An agent needs reliable retrieval fields, narrow scopes, explicit authority, and instructions for uncertainty.

A practical record may include:

• title and knowledge type;
• concise approved statement;
• governing source and supporting sources;
• owner and approver;
• effective, reviewed, and review-due dates;
• customer, service, team, or workflow scope;
• authority level and conflict rule;
• sensitivity and allowed audiences;
• related records and superseded records;
• keywords or attributes used for retrieval;
• required behavior when the record is missing, stale, or disputed.

Use plain language in the record itself. Metadata should make the knowledge filterable without hiding meaning in an implementation only a developer can inspect.

OpenAI’s retrieval documentation describes files, vector stores, attributes, filtering, and file update or removal operations. Those capabilities can help locate relevant material, but the company still has to decide which files belong in the index, which attributes control retrieval, and how changed or retired sources are handled.

Microsoft’s Copilot Studio knowledge-source documentation similarly distinguishes knowledge sources and documents behavior around authentication, permissions, grounding, and citations. Product features vary by source and configuration, so evaluate them against the access and evidence requirements of the workflow rather than assuming that a connector creates governance.

Test retrieval with real business questions:

• Does the result use the governing policy instead of the most recent upload?
• Can it distinguish a customer exception from a company-wide rule?
• Does it cite enough source context for a reviewer to verify the answer?
• Does it refuse or escalate when the permitted corpus does not contain an answer?
• Can an authorized person find the same governing record without using the agent?

The knowledge base should improve human operations before autonomy is considered. If only the agent can interpret the system, the company has created a new dependency rather than a durable source of truth.

5. Define access, retention, and sensitive-information boundaries

Centralizing knowledge increases its usefulness and its risk. Do not treat “the company already has this information” as permission to put every record into one searchable corpus.

Classify sources before ingestion:

• public or approved for broad internal use;
• internal operational information;
• customer-confidential or contract-restricted information;
• employee, financial, legal, security, or regulated information;
• prohibited material that should not enter the proposed system.

Then apply least privilege. A sales-support workflow may need current service policies and assigned-account context, not every customer agreement. A delivery agent may need an approved statement of work, not access to company banking or unrelated employee records.

Document:

• permitted users, roles, agents, and service accounts;
• purpose and scope of access;
• authentication and integration permissions;
• whether data may be copied into an index or must remain in the source system;
• retention periods for source material, indexed content, outputs, and logs;
• deletion and offboarding procedures;
• rules for exports, backups, and third-party processors;
• incident contacts and access-review cadence.

Sensitive information should not rely on a warning inside a prompt. Use source separation, permissions, filtered retrieval, limited credentials, approval steps, and logging appropriate to the consequence of exposure. Test access boundaries directly, including attempts to retrieve data across customers, roles, or workspaces.

6. Operate an ingest-and-curate loop

A knowledge base is an operating responsibility, not a one-time import.

New information enters through controlled sources: approved forms, meeting decisions, policy updates, contracts, CRM records, or a reviewed inbox. Ingestion should preserve the original source and mark incoming material as unverified until the appropriate owner approves its durable meaning.

A workable lifecycle is:

1. Capture: retain the source, date, scope, and submitting owner.
2. Classify: identify knowledge type, sensitivity, authority candidate, and affected workflows.
3. Review: confirm accuracy, resolve ambiguity, and obtain approval from the responsible owner.
4. Publish: update the structured record, index, relationships, and retrieval attributes.
5. Test: run representative questions and affected workflow checks.
6. Monitor: surface stale records, retrieval failures, conflicts, and missing ownership.
7. Retire: mark superseded knowledge, remove it from active retrieval, and preserve history only where required.

Conflict resolution needs a visible queue. When two credible sources disagree, do not let upload order decide. Pause affected answers or actions, show both sources to the named owner, record the decision, and update the authority rule if the conflict can recur.

Use a review cadence based on change rate and consequence. Frequently changing pricing or active customer commitments may need event-driven updates and tight service levels. Durable company history may need review only when a linked decision changes. Disposable working context should expire rather than accumulate indefinitely.

The source Build With AI episode demonstrates a simple file-based starting point, a structured index, and ingest-and-curate routines. It is useful implementation context, not proof that one taxonomy or plain-text architecture fits every organization. You can find the episode and related conversations in the Build With AI podcast archive.

7. Use this readiness checklist

Before connecting a company knowledge base to an AI workflow, confirm:

• The first business decisions and workflows are named.
• Governing sources are distinguished from supporting material and examples.
• Policies, pricing, process, customer context, history, decisions, responsibilities, and open work have explicit scope.
• Each durable record has a source, authority level, owner, and freshness rule.
• Predictable source conflicts have resolution rules.
• People can browse and verify the governing knowledge without an agent.
• Retrieval metadata supports the required customer, service, role, and workflow boundaries.
• Missing, stale, or disputed knowledge triggers a defined refusal or escalation.
• Sensitive information is classified before ingestion.
• Access, retention, deletion, offboarding, and incident responsibilities are documented.
• New material follows capture, review, approval, publication, and testing steps.
• Superseded and disposable context leaves active retrieval.
• Named client owners and provider maintenance duties are written down.

If several boxes remain open, the next step is discovery and source cleanup—not a wider integration. The workflow-first implementation guide can help sequence a contained first release after the operating foundation is clear.

8. Keep the evidence boundary visible

This article describes an intended architecture and buyer standard. “One source of truth,” AI-ready structure, and ongoing maintenance are Return My Time service-design goals. Return My Time does not currently have a measured knowledge-base client case study proving revenue gained, time saved, conversion improved, token costs reduced, or another business outcome.

The source episode includes commercial and financial claims from a guest. Those claims are context for the conversation, not validated buyer outcomes, and they are not used as benchmarks here. The episode’s fictional demonstration also does not establish that the same taxonomy, technology, or result will fit another company.

A credible provider should be able to explain:

1. Which business decisions the first knowledge base will support.
2. Which sources will be authoritative and who approves them.
3. How permissions, sensitive information, and customer boundaries will work.
4. How people and agents will retrieve and verify the same governing knowledge.
5. Who handles ingestion, conflicts, stale records, incidents, and retirement.
6. Which assumptions will be tested before the system supports consequential actions.

Return My Time’s services place the knowledge foundation between assessment and managed implementation. If your company’s operating knowledge is scattered across people, inboxes, calls, systems, and documents, book a discovery call. The call begins qualification and scopes the knowledge sources a paid assessment would need to examine. It is not a guarantee that a knowledge-base build—or any AI system—is the right next step.