DubBot allows you to crawl pages that require a login to access.
If you find that this feature is unavailable in your Advanced tab, please contact DubBot's Support team via the app's chat, or you can email help@dubbot.com.
Using your authentication method, create a user account and login for DubBot to access your secure content.
In DubBot, navigate to the Advanced tab under Settings:
Scroll down to Crawler Authentication and check the Enable crawler authentication box.
There you will find 3 options to select for your authentication method: Basic, Form, or Microsoft.
Finally, enter the information in the fields displayed and save your changes.
If you have questions, please reach out to our DubBot Support team via email at help@dubbot.com or via the blue chat bubble in the lower right corner of your screen. We are here to help!