OyeChats
FeaturesSolutionsIntegrationsPricingDocsBlogContact us

Training your chatbot

Two ways to teach a chatbot: crawl a website, or upload files. This page covers both, what is supported, what it costs, and how to keep the knowledge base fresh.

Crawling a website

  1. Enter a URL

    Open Chatbots → Knowledge → Add from website and paste the URL you want indexed. A section root such as https://example.com/help works as well as a homepage.

  2. Review the discovered pages

    OyeChats reads your robots.txt for Sitemap: directives and then the sitemaps themselves. If that turns up nothing usable, it scans the page you gave it for same-domain links and follows them. Either way you get the page list with an estimated credit cost before anything is charged, and your typed URL is always included.

  3. Deselect what you do not want

    Pruning the list here is the cheapest moment to do it. Login pages, cart pages, tag archives and paginated blog indexes usually add cost without adding answers.

  4. Start the crawl

    Progress updates live. Pages are indexed in waves as they arrive, so early pages become answerable before the whole crawl finishes. You can cancel mid-run; pages already indexed are kept.

A sitemap gets you better coverage. The link scan is deliberately bounded so it stays fast and does not hammer your origin: it follows links about two hops from the page you give it and only fetches a limited number of pages to find more. A sitemap is authoritative, so it also reaches deep and orphaned pages that no link scan would find. If you have one, discovery will be more complete.
Client-rendered sites. If a site publishes no sitemap and its links only exist after JavaScript runs. A typical single-page app. Neither of the above finds anything. The crawl then falls back to a recursive browser crawl, which does render the page. That path is slower and gives no page list to review up front.
What it can reach. Only pages a public visitor can reach. Anything behind a login or a paywall is not indexed, and robots.txt Disallow rules are honoured. JavaScript-rendered pages are supported, subject to a per-plan cap on how many need rendering.

Uploading files

Use Add files for anything not on your website: spec sheets, policy documents, price lists, internal FAQs.

ConstraintValue
Formats.pdf, .docx, .txt, .md
Maximum per file10 MB
Maximum per upload request60 MB
Maximum files per upload request50
Scanned PDFs. Text is extracted from the PDF's own text layer. A scanned or photographed document has no text layer, so it indexes as empty. Run it through OCR first, or paste the text into a .txt or .md file.

Before charging, the uploader shows a cost preview. Unsupported extensions and oversized files are rejected at that point, so they never consume credits.

What training costs

ActionCredits
Crawled page5 per page
Uploaded file1 credit per 250 words, minimum 1 credit per file
Automatic weekly re-crawl0, funded by your subscription, whatever changed
Manual re-crawl, updated pages only *(Standard and above)*5 per page that actually changed
Manual re-crawl, full5 per page for every page, changed or not

Plans also cap total knowledge size, in pages, files, and characters. Hitting a cap blocks further ingestion for that chatbot until you remove content or move up a tier; it never silently truncates a document. See Plans and limits.

Keeping it current

Manual re-crawl. Full
Re-fetches and re-indexes every discovered page, and charges for all of them at 5 credits each even if nothing changed. Available on every plan, and it is the only manual option below Standard.
Manual re-crawl. Updated pages only
Re-fetches everything but only re-indexes and charges for pages whose content actually changed. Included on Standard and above. Before it runs you get a diff showing how many pages are unchanged, new and removed.
Automatic re-crawl
A weekly refresh of every previously-crawled URL. Pages whose content genuinely changed are re-indexed; unchanged pages are skipped. Cosmetic churn such as a copyright year rolling over does not count as a change. Included on Standard and above; the toggle is visible but locked on lower tiers.
Re-index a file
Re-processes a single uploaded document, useful after you upload a corrected version under the same name.
A full manual re-crawl bills the whole site. Only the automatic weekly refresh is free. A manual full re-crawl deliberately re-charges every page, on a 300-page site that is 1,500 credits whether two pages changed or none did. If you are on Standard or above, use "updated pages only". If you are below it, leaning on the automatic weekly re-crawl is much cheaper than re-crawling by hand.

Removing content

Deleting a page or file from the Knowledge tab removes it and its indexed passages, and the chatbot immediately stops being able to answer from it. Deletion does not refund the credits already spent indexing it.

Answers reflect the index, not the live web. The chatbot answers from what was indexed at crawl time. Publishing a change on your site does not change its answers until the next crawl or re-crawl.

Getting better answers

  • Index the pages that answer questions. Docs, help centre, FAQs, pricing, policies. Before marketing pages.
  • Skip near-duplicate pages. Ten variants of the same landing page make retrieval less certain, not more thorough.
  • Prefer one thorough page over several thin ones on the same topic.
  • Watch Analytics → Unanswered questions. It is a direct list of what your knowledge base is missing.

Something here wrong or missing? Tell us and name this page. We will fix it.