New Crawler UI/UX
The Algolia Crawler Visual UI provides an updated, user-friendly way to manage your crawl settings and monitor your indexing process. This guide covers the main features of the new interface.
DocSearch Tab
The Crawler UI now includes a dedicated DocSearch tab. This tab provides everything you need to implement DocSearch on your site, including:
- Implementation code: Copy-paste ready code snippets for integrating DocSearch into your frontend.
- API keys: Your unique Application ID and Search API Key for connecting to your Algolia index.
- Quick links: Access to review your records, explore documentation, and join the support Discord.
Trigger a new crawl
Head over to the Overview section to start, restart or pause your crawls and view a real-time summary.
Monitor your crawls
The Monitoring section helps you find crawl errors or improve your search results.
Update your config
You can update your crawler configuration in two ways:
Visual Configuration UI: Quickly edit common options without writing code using the new Visual Configuration interface.
Code Editor:
For advanced configuration, use the live code editor to directly modify your config file and test your URLs (URL tester).
URL tester
From the editor, you can use the URL tester to debug selectors or see how the crawler interprets your site.
Suggestions
The Suggestions section in the Crawler UI provides actionable feedback to help you improve your crawl and data extraction. After each crawl, you'll see recommendations for:
- Fixing redirect or domain issues
- Addressing ignored or failed URLs
- Adding missing sitemaps
Each suggestion includes a description, a solution, and quick links to relevant documentation or monitoring tools, so you can resolve issues efficiently and optimize your search experience.
