Training your chatbot
Two ways to teach a chatbot: crawl a website, or upload files. This page covers both, what is supported, what it costs, and how to keep the knowledge base fresh.
On this page
Crawling a website
Enter a URL
Open Chatbots → Knowledge → Add from website and paste the URL you want indexed. A section root such as
https://example.com/helpworks as well as a homepage.Review the discovered pages
OyeChats reads your
robots.txtforSitemap:directives and then the sitemaps themselves. If that turns up nothing usable, it scans the page you gave it for same-domain links and follows them. Either way you get the page list with an estimated credit cost before anything is charged, and your typed URL is always included.Deselect what you do not want
Pruning the list here is the cheapest moment to do it. Login pages, cart pages, tag archives and paginated blog indexes usually add cost without adding answers.
Start the crawl
Progress updates live. Pages are indexed in waves as they arrive, so early pages become answerable before the whole crawl finishes. You can cancel mid-run; pages already indexed are kept.
robots.txt Disallow rules are honoured. JavaScript-rendered pages are supported, subject to a per-plan cap on how many need rendering.Uploading files
Use Add files for anything not on your website: spec sheets, policy documents, price lists, internal FAQs.
| Constraint | Value |
|---|---|
| Formats | .pdf, .docx, .txt, .md |
| Maximum per file | 10 MB |
| Maximum per upload request | 60 MB |
| Maximum files per upload request | 50 |
.txt or .md file.Before charging, the uploader shows a cost preview. Unsupported extensions and oversized files are rejected at that point, so they never consume credits.
What training costs
| Action | Credits |
|---|---|
| Crawled page | 5 per page |
| Uploaded file | 1 credit per 250 words, minimum 1 credit per file |
| Automatic weekly re-crawl | 0, funded by your subscription, whatever changed |
| Manual re-crawl, updated pages only *(Standard and above)* | 5 per page that actually changed |
| Manual re-crawl, full | 5 per page for every page, changed or not |
Plans also cap total knowledge size, in pages, files, and characters. Hitting a cap blocks further ingestion for that chatbot until you remove content or move up a tier; it never silently truncates a document. See Plans and limits.
Keeping it current
- Manual re-crawl. Full
- Re-fetches and re-indexes every discovered page, and charges for all of them at 5 credits each even if nothing changed. Available on every plan, and it is the only manual option below Standard.
- Manual re-crawl. Updated pages only
- Re-fetches everything but only re-indexes and charges for pages whose content actually changed. Included on Standard and above. Before it runs you get a diff showing how many pages are unchanged, new and removed.
- Automatic re-crawl
- A weekly refresh of every previously-crawled URL. Pages whose content genuinely changed are re-indexed; unchanged pages are skipped. Cosmetic churn such as a copyright year rolling over does not count as a change. Included on Standard and above; the toggle is visible but locked on lower tiers.
- Re-index a file
- Re-processes a single uploaded document, useful after you upload a corrected version under the same name.
Removing content
Deleting a page or file from the Knowledge tab removes it and its indexed passages, and the chatbot immediately stops being able to answer from it. Deletion does not refund the credits already spent indexing it.
Getting better answers
- Index the pages that answer questions. Docs, help centre, FAQs, pricing, policies. Before marketing pages.
- Skip near-duplicate pages. Ten variants of the same landing page make retrieval less certain, not more thorough.
- Prefer one thorough page over several thin ones on the same topic.
- Watch Analytics → Unanswered questions. It is a direct list of what your knowledge base is missing.
Something here wrong or missing? Tell us and name this page. We will fix it.