We have content management server ( Apps Server) and we do have an IHS that takes care of LB and all the requests go through the web server.
Most of the content or Pages requires username and password and for authentication we are using  third party system( LDAP) .

Here comes the problem, validation working fine on public pages but on authenticated pages it's throwing 403 Forbidden by robots.txt.

Not sure Link checker supports authenticated pages.

Any help is appreciable.