An Admin role is required to create a new site for crawl and analysis in DubBot. If you do not see "Settings" in the top navigation of DubBot, you do not have an Admin role. If this is the case and you need to add a Site, please chat DubBot support so that we can work with your organization to determine the best path forward.

Select "Settings" in the top navigation.

"Sites" is the default panel open. If you navigate to any other panel within Settings, you can always access the Sites settings from the left navigation in the Settings panel.

settings menu sites settings in DubBot App

Enter the Site Name. The Site Name displays on the Dashboard, in the Site switcher, and email notifications.

Sites options screen focused on Site Name field in DubBot App

Next, select the type of site that you would like to create.

There are four different options for the type of site that you create.

The options are:

  • Default (Follow site hierarchy starting at URL)

  • Full Domain (Crawl everything on the domain starting at URL)

  • Sitemap

  • List of URLs - which can be made manually or you can upload a CSV

The Default option is the most common Site type. The Default option allows users to crawl a whole site or a specific folder off of a site. The Default option also allows users to setup folder and page exclusions for a site.

Learn more about these Site Crawler options. Please email DubBot support or contact us via the app if you need help determining what the best configuration option is for your site.

Sites settings menu focused on Site Name field in DubBot App

The Crawler Max Depth is the maximum level of the folder structure that the crawler traverses. Generally, 10 is sufficient, but this number is customizable.

Site settings menu - crawler max depth field highlighted - DubBot App

Select "Create" your site to save your site. If the "Enable site for checking" box is enabled, the site is now in the queue to crawl.

To update the Standard checks performed for a specific site or to update the dictionary for a specific site, see Site Setup - Standard Checks options.

To learn more about Advanced options for a site, see Advanced options for Site crawling.

Did this answer your question?