Skip to main content
All CollectionsRelease Updates2021
Exclude webpages from your site inventory using regular expressions
Exclude webpages from your site inventory using regular expressions

November 15, 2021

Updated over 3 months ago

The Ignored Paths for Specific Sites article has the most up-to-date information on this topic.

DubBot now allows administrators to ignore webpages utilizing a regular expression. Webpages or folders will be excluded from site crawl based on the regular expression added.

When creating or editing a Site's settings, select Ignored Paths panel. Under Ignored Paths section, there is now a section for Ignored Paths by Regular Expression.

If you need help utilizing this new feature, reach out to DubBot support using the in-app chat or email help@dubbot.com.

Learn more about Ignoring Paths for Specific Sites.

Did this answer your question?