These are other entries in our FAQ database:

• Scheduling automatic crawls and Google Sitemap file updates
  
• Uploading sitemap files via the built-in FTP client
  
• Options and Settings
  
• Information about the fields in the "URL list" tab (manual, include, crawl, priority, freq, URL)
  
• Installing the MSDE database engine
  
• Set up MSDE / SQL Server access
  

Questions, ideas, problems, wishes? Be informed whenever something new comes up (or any important problems are fixed.)

Google Group

Using filters to control the crawler

Settings: Ban URLs

Here you can place URLs or parts of URLs which should not be included into the sitemap. You do not need to add full URLs, you can just add words, filenames, etc. When these are found anywhere within a link, the link will not be crawled. Some examples would be "download.php" (to not add direct download-links), "redirect" (automatic redirect-links), "no-cookie.htm" (error-pages when cookies are not available, generally add all error-pages), etc.

Settings: Drop parts

When the crawler finds one of these entries within the URL (or parameter) it will just drop that part and try to continue with the new URL. You can use this to drop unnecessary parameters (e.g. "sortorder=asc", "start=0", etc.).

Settings: Remove parameters

This will remove the parameter and it's value if found in the URL. You generally use this for unknown values, like e.g. a Session-ID passed as parameter. You could add "sessionid", "sid", etc. to the table here, based on the way your site passes / creates a session-id. Session-IDs should NOT be included within a sitemap-file.