Skip to main content

Including Content

For information on search engine optimization (ranking and results placement), please see writing for search engines.

Why isn't my website in the search engine?

There are a few reasons your website might not be in the search engine index. Please check all of the following before submitting a help request.

For Static HTML Pages...

1. Your site hasn't been added to a collection

Schools and some departments manage their own collections. If you have a manager account in SearchBlox, add your new site to your collection's Root URLs field, then trigger a refresh. If you don't have a manager account, please use this form to submit your site.


2. There might be a robots exclusion

It's possible that your web server is restricting access to your website via robots.txt rules. You can usually check this by reading the file (if it exists on your web server). For sites hosted on nuinfo, the primary Northwestern server, robots.txt is located at If your website is at another address, just replace everything before the last slash, e.g. If this file does not exist on your web server, you have no corrections to make. If the file does exist and your site URL appears in this file's list of disallowed directories, you may need to edit the file to remove the line containing your site or contact a server administrator who can help you with this.

This is the most common type of robots block. For more robots exclusion methods to check, please see the excluding content page.

For dynamic Content (ASP, PHP, ColdFusion, et al)...

The search crawler has built-in algorithms to remove duplicate content as can be generated by filter parameters or multiple views. Please consult with us at if you require assistance.