Yes, edit the robots.txt file to specify the pages or directories that should not be indexed by search engines.
If your site has some content that you may not want to be publicly searched for, or if some search engine hits are causing the site to slow down, you can make this change.
Only one robots.txt file can be created for a site. This file can contain a list of url patterns that should not be indexed, against the web crawler name.
The default robots.txt is available in the SITE_DIR/var folder. The robots.txt file is accessible without logging into CollabNet TeamForge, via the domain/robots.txt URL. You can update and commit the robots.txt file into the branding root directory, and the system then places the file in the SITE_DIR/var/overrides folder.