Advice and answers from the DubBot Team

Search results for:regular expressions

Exclude webpages from your site inventory using regular expressions

DubBot now allows administrators to ignore webpages utilizing a regular expression… Webpages or folders will be excluded from site crawl based on the regular expression added

Regex and XPath for Custom Policies

Resources for learning how to use Regular Expressions and XPath to write Custom Policies in DubBot… In this article we will focus on Regular Expressions and XPath… Regular Expression Learn Regex Resources XPath Learn XPath Resources Regular

Ignored Paths for Specific Sites

Ignored Paths by Regular Expressions… More complicated exclusions can be created using regular expressions… Webpages or folders will be excluded from the site crawl based on the regular expression added

Remove page from Crawl (from Preview Page display)

See also: Exclude Web pages from your site inventory using regular expressions

Understanding Custom Policy Rules

The options include Word or Phrase, Link Text, Link URL, CSS Selector, XPath, and regular Expression… Page Content Check Behavior For the Search Criteria Word or Phrase and Regular Expression users are

Create a Custom Policy

Search type options are Word or phrase, Link Text, Link URL, CSS Selector, XPath, and Regular Expression… Page Content Check Behavior is a dropdown only present for the Word or Phrase and Regular Expression

Custom Policy Examples

Rules: Search By > Regular Expression Rule > \(?\d{3}\)?-?\s?\d{3}-?… Rules: Search By > Regular Expression Rule > update CityName with your appropriate name(s

View Issues on Your Webpage

notice an issue with a link or set of links that are not actually broken, the specific broken link or a regular

Enable/Disable site

Enabled sites crawl on a regular cadence that is provided in the General Site settings

Added client-side XPATH expression validation to custom policy rules

dropdown, select XPath and next to the Rule input box you will now see an error message if your XPath expression