Rank: Newbie Groups: Member
Joined: 12/17/2007 Posts: 3 Points: 9 Location: Brick, NJ
|
I am having a problem spidering my site. I use a cgi file that generates pages with ?ct= and I do not want to include these in the spidering as it creates too many pages that do not contain relevant contect. When I spider my site now, it generated over 35,000 pages. Many of these pages were those with "?ct=". I also have pages that have ?direct= that I do want to include. If I try to exclude "ct" in the query string, it also excludes any pages with "ct" in them, such as "direct=". Is there a way to exclude "ct" with out excluding "direct"?
|
|
|
|
Rank: Administration Groups: Administration
Joined: 1/31/2007 Posts: 380 Points: 539 Location: Chicago, IL
|
Yes. Please read the documentation. Use the "Ignore Querystring" preference.
|