Request a Crawl Now | DubBot Help Center

DubBot automatically crawls sites every seven days. When an automated crawl is complete, users are sent an email notification with a summary report.

Users with the admin role have a Crawl button that allows them to trigger an on-demand crawl. If you do not have a crawl option and need an on-demand crawl, please contact DubBot using the in-app chat or via email at help@dubbot.com.

To perform a manual crawl for a site, select the Settings link in the top navigation. From the Manage Sites panel, select Crawl in the Actions column for the corresponding Site.

Select Settings in the top navigation bar, the main content area will then contain the table of available sites. The last column of the Sites table has available Actions for each site. Crawl will be the first button in the Actions column.

If a site on the Manage Sites Panel does not have a Crawl button, it is Disabled. No crawls are possible when a site is Disabled.

Manage Sites panel highlighting a Disabled Site, showing the Crawl button is missing.

Please note that crawls are not immediate. Although the queue is typically quick, crawl time will be dependent on the size of the queue at the time of request and the size of the Site. The smaller the Site, the shorter the crawl time will be. You will receive an email notification when the crawl has been completed.

If you have questions, please contact our DubBot Support team via email at help@dubbot.com or via the blue chat bubble in the lower right corner of your screen. We are here to help!