Request a Crawl Now

DubBot allows users to initiate on-demand crawls of their Sites and single pages within their sites.

Updated over a week ago

DubBot automatically crawls sites every seven days. When an automated crawl is complete, users are sent an email notification with a summary report.

Users with the admin role have a Crawl button that allows them to trigger an on-demand crawl. If you do not have a crawl option and need an on-demand crawl, please contact DubBot using the in-app chat or via email at

To perform a manual crawl for an entire site, select the Settings link in the top navigation. From the Sites list panel, select Crawl in the Actions column for the corresponding Site.

Select Settings in the top navigation bar, the main content area will then contain the table of available sites. The last column of the Sites table has available Actions for each site. Crawl will be the first button in the Actions column.

Please note that crawls are not immediate. Although the queue is typically quick, crawl time will be dependent on the size of the queue at the time of request and the size of the Site. The smaller the Site, the shorter the crawl time will be. You will receive an email notification when the crawl has completed.

Did this answer your question?