DubBot now allows you to crawl pages that require a login to access.

Using your authentication method, create a user account and login for DubBot to access your secure content.

In DubBot, navigate to the Advanced tab under Settings:

Scroll down to Crawler Authentication and check the Enable crawler authentication box.

If this is unavailable in your Advanced tab, please contact DubBot's Support team via the app's chat, or you can email help@dubbot.com.

There you will find 3 options to select for your authentication method: Basic, Form, or Microsoft.

Finally, enter the information in the fields displayed and save your changes.