Skip to content
Plain Help Center home
InboxAsk a human

Knowledge Sources

k1mJAYZeSldtPbjVODni3hAYzb4.jpg

Knowledge sources are the pages and docs you feed to Ari and Help Center AI. The more Plain knows about your product, the better its answers.

This guide covers adding sources, how indexing works, keeping them fresh, and doing it all from the API and CLI.

Overview

By default, Plain AI already reads everything in your customer-facing Help Centers, so publishing clear Help Center articles feeds Ari for free.

Knowledge sources let you add content that isn't in Plain. You manage them under Settings → Plain AI → Knowledge Sources.

Plain AI must be enabled on your workspace to add or index sources.

Source types

When you add a source, you pick one of two types:

  1. Sitemap indexes a whole site. You point Plain at your sitemap.xml and it crawls every URL listed, one indexed page each. Pages that later drop out of the sitemap are removed on the next reindex. Prefer this whenever you can, it keeps Plain in sync as pages come and go.

  2. URL indexes a single page. Use it for a standalone guide or anything not covered by a sitemap.

Your sitemap usually lives at https://yourdomain.com/sitemap.xml. Only normal HTML or Markdown pages can be indexed, files like .pdf are skipped, and pages behind a login can't be crawled unless they support token access.

Very large pages (over 1MB of raw content) are also skipped.

Adding a source

  1. Go to Settings → Plain AI → Knowledge Sources

  2. Choose Sitemap or URL and paste the address

  3. Save, the source indexes automatically

A large sitemap can take a while to finish, since each page is fetched and indexed on its own. You can keep working while it runs.

Statuses

Each source, and each page within a sitemap, shows one of three statuses:

  • Pending → Plain is fetching and indexing the content

  • Indexed → Ready, and Plain AI can use it in replies

  • Failed → Something went wrong, with a reason; fix the cause and reindex

What knowledge was used?

You don't have to guess which of your sources Ari drew on. On every reply, Ari cites the knowledge it used, shown as a collapsible list on the thread timeline and, where useful, as inline links in the reply itself.

See Ari → Citations for the full behaviour.

One thing to know when curating sources: Ari only ever links publicly accessible pages in a reply. A private or internal Help Center can still power Ari's answers, but its articles are never shown to a customer as a clickable link.

Keeping content fresh

Content is reindexed automatically every week, on Monday mornings (Europe/London), so ordinary doc changes get picked up on their own.

To refresh sooner, you have three options:

  1. Reindex a single source on demand with the reindexKnowledgeSource API call

  2. Use the @team-plain/cli to reindex

  3. Manually trigger a reindex for a source in the Plain UI

Reindexing only ingests content that actually changed, so running it often is cheap and safe.

Using the GraphQL API

Everything in the UI is available over Plain's GraphQL API.

Reindex a source

The precise way to refresh one source in place, requires the knowledgeSource:create permission.

mutation ReindexKnowledgeSource($input: ReindexKnowledgeSourceInput!) {
  reindexKnowledgeSource(input: $input) {
    knowledgeSource { __typename }
    error { message code }
  }
}

# variables
{ "input": { "knowledgeSourceId": "..." } }

Add a source

Adds a source and queues it for indexing, requires the knowledgeSource:create permission.

mutation CreateKnowledgeSource($input: CreateKnowledgeSourceInput!) {
  createKnowledgeSource(input: $input) {
    knowledgeSource { __typename }
    error { message code }
  }
}

# variables — type is "SITEMAP" or "URL"
{ "input": { "url": "https://docs.yourcompany.com/sitemap.xml", "type": "SITEMAP" } }

List sources and status

Find source IDs and read the failure reason on any that failed, requires the knowledgeSource:read permission.

query KnowledgeSources {
  knowledgeSources(first: 50) {
    edges { node {
      __typename
      ... on KnowledgeSourceSitemap {
        id url
        status { __typename ... on IndexingStatusFailed { reason } }
      }
      ... on KnowledgeSourceUrl { id url status { __typename } }
    } }
  }
}

Delete a source

Removes the source and its content from Plain AI's index, requires the knowledgeSource:delete permission.

mutation DeleteKnowledgeSource($input: DeleteKnowledgeSourceInput!) {
  deleteKnowledgeSource(input: $input) { error { message code } }
}

# variables
{ "input": { "knowledgeSourceId": "..." } }

Via the Plain CLI

The @team-plain/cli package wraps the API for scripting. It reads your API key from PLAIN_API_KEY.

npm install -g @team-plain/cli
export PLAIN_API_KEY=plainApiKey_xxx

# Index every URL in a sitemap
plain index-sitemap https://docs.yourcompany.com/sitemap.xml

# Index a single page
plain index-url https://docs.yourcompany.com/guide

Authenticated GitBook sources

Plain can index GitBook docs protected by Visitor Authentication by accepting a JWT on the sitemap URL. Generate a JWT on your side and append it as a jwt_token query parameter:

https://docs.yourcompany.com/sitemap-pages.xml?jwt_token=<jwt>

Plain won't automatically reindex these, since the token expires. When it's about to, delete and recreate the source with a fresh JWT using the API.

Markdown support

Some documentation platforms (like Mintlify and ReadMe) publish a clean Markdown version of each page alongside the rendered HTML. When indexing any page, Plain will first automatically look for its Markdown version first, the same URL with a .md suffix, and uses it when one exists, falling back to the HTML page otherwise.

Markdown gives Plain AI cleaner text and better answers. It's fully automatic, there's nothing to configure, and pages without a Markdown version index exactly as before

Best practices

  • Prefer a sitemap over individual URLs, it stays in sync as pages change

  • Index accurate, current docs only; stale or contradictory pages make answers worse

  • Keep your sitemap honest, removing a page there retires it from Plain AI

  • If your docs change often, reindex from CI on deploy rather than waiting for the weekly run

Troubleshooting

Failed source ingestion

Check the failure reason on the status. Common causes:

  • The page couldn't be fetched, it's unreachable, errors, or blocks crawlers

  • The URL points at a .pdf or other unsupported file; only HTML and Markdown pages can be indexed

  • The sitemap is unreachable or isn't valid XML

  • The page needs authentication

  • The page is too large; pages over 1MB of raw HTML or Markdown are skipped and marked as failed

Fix the cause, then reindex to clear the failure.

Ari isn't using a source

  • Confirm Plain AI is enabled on the workspace

  • Confirm the source is Indexed, not pending or failed

  • For Help Center content, confirm the article is published and its Help Center has customer-facing AI setting enabled