Site already set up? Review Modifying an existing Site's Settings.
Finding the Create Site Button
An Admin role is required to create a new site for crawl and analysis in DubBot.
Select Settings in the top navigation.
βSites is the default panel open. If you navigate to any other panel within Settings, you can always access the Sites settings from the left navigation in the Settings panel.Select the Create Site button and you will be taken to the Create Site Panel
If you don't see the Create Site button on the Manage Sites panel, you do not have an Admin role. If this is the case and you need to add a Site, please chat DubBot support so that we can work with your organization to determine the best path forward.
Site Name
Enter the Site Name. The Site Name displays on the Dashboard, Sites Listing, in the Site switcher, and email notifications.
There is also a checkbox for Enable site for checking. If this box is checked, the site will begin its first crawl when you create the site and with a frequency you select later in the setup process.
Crawler Type
Next, use the Crawler Type drop-down to select the type of site that you would like to create.
β
There are four different options for the type of site that you create.
The options are:
Default (Follow site hierarchy starting at URL)
Full Domain (Crawl everything on the domain starting at URL)
Sitemap
List of URLs - which can be made manually or you can upload a CSV
The Default option is the most common Site type. The Default option allows users to crawl a whole site or a specific folder in a site and allows users to setup folder and page exclusions for a site.
Learn more about these Site Crawler Options.
Please email DubBot support or contact us via the app if you need help determining what the best configuration option is for your site.
Crawl Depth Limit
The Crawl Depth Limit is the maximum number of levels deep the crawler will go before stopping the crawl. Generally, 10 is sufficient, but this number is customizable.
Page Limit
The Page Limit is an approximation of how many pages the crawler will crawl before stopping. This allows administrators to put in an upper limit for page inventory. Leave blank or '0' for no limit.
Create
Select Create to save your site. If the Enable site for checking box is checked, the site is now in the queue to crawl.
More on the Site Settings panel
Create a Site (General Tab) << You are here
If you have questions, please contact our DubBot Support team via email at help@dubbot.com or via the blue chat bubble in the lower right corner of your screen. We are here to help!
β