All Collections
Using DubBot
Updating your content in DubBot
Request a Crawl Now
Request a Crawl Now
DubBot allows users to initiate on-demand crawls of their Sites and single pages within their sites.
Blaine Herman avatar
Written by Blaine Herman
Updated over a week ago

DubBot automatically crawls sites every seven days. When an automated crawl is complete, users are sent an email notification with a summary report.

Users with the admin role have a Crawl button that allows them to trigger an on-demand crawl. If you do not have a crawl option and need an on-demand crawl, please contact DubBot using the in-app chat or via email at help@dubbot.com.

To perform a manual crawl for an entire site, select the Sites link in the top navigation. From the Sites summary panel, select Crawl in the Actions column for the corresponding Site.

With the screen displaying the list of Sites to the left along with site URLs, Enabled status, Last Crawl date, Next Crawl date, number of pages and Status, all flowing horizontally to the right with the Site link in the upper right highlighted as well as the first Crawl button in the right hand column.

Please note that crawls are not immediate. Although the queue is typically quick, crawl time will be dependent on the size of the queue at the time of request and the size of the Site. The smaller the Site, the shorter the crawl time will be. You will receive an email notification when the crawl has completed.

Did this answer your question?