Crawl your site

Pro plan

The site crawl feature lets the AI Coach automatically discover pages on your website, providing a comprehensive map you can use to create scenarios.

How it works

1. Tell the Coach to crawl your site: "Crawl my website" or "Find all pages on my site"
2. The Coach uses Firecrawl to discover all accessible pages
3. Results are displayed as a browsable list of URLs
4. You can select pages to create scenarios from

What gets discovered

• All public pages linked from your site
• Nested routes and dynamic pages
• Page titles and descriptions
• Page structure overview

Browsing crawl results

After the crawl completes, the Coach organizes results into two categories:

• Navigation — top-level pages (level 1 depth)
• Pages — all other discovered pages, grouped into collapsible sections

Each section supports:

• Select all / Deselect all to quickly toggle pages
• Pagination — results are displayed 15 per page for large sites
• Individual checkboxes for fine-grained selection

Creating scenarios from crawl results

After selecting pages, click Generate scenarios to create scenarios for all selected pages at once.

Environment-aware crawling

When you have multiple environments configured (e.g. staging and production), the Coach uses the currently selected environment's base URL as the crawl starting point. Switch environments in the project header to crawl a different target.