|
|
Rank: Newbie
Groups: Member
Joined: 6/24/2007
Posts: 9
Points: 27
|
In the last couple of days the spider is running and running thousands of urls that don't exist! My site is broken down into baseball, college, football, etc. and it is doing things like college/baseball. How is it coming up with this? I have to stop it because it just runs and runs. I have less than 1000 pages, but this was in the 10,000 when I stopped it!
Thanks
|
|
|
|
|
|
Rank: Administration
Groups: Administration
Joined: 1/31/2007
Posts: 440
Points: 646
Location: Chicago, IL
|
Does your website pass "sessionid" or some other unique session identifier in the URL?
If so, your website is probably issuing a new session for each page the spider finds and it'll never end.
You need to put that field in the "Ignore Querystring" preference.
If you're not sure, either post your website URL here or email it to me and I'll check it out.
Thanks,
Brian
iArchitect
|
|
Rank: Newbie
Groups: Member
Joined: 6/24/2007
Posts: 9
Points: 27
|
You can click on the www on my post and it takes you to our website. Myteamprints.com
I would appreciate you checking this out if you can because I don't know what you are talking about! :)
I will say that it was working just fine for the first couple of weeks we have been running it. Maybe we changed something?
|
|
Rank: Administration
Groups: Administration
Joined: 1/31/2007
Posts: 440
Points: 646
Location: Chicago, IL
|
Take a look at: http://www.myteamprints.com/college/main.htmYou have a link titled "Enter to win a FREE PRINT!" that links to: http://www.myteamprints.com/college/contest.htmThis page doesn't exist and your web server is redirecting to: http://www.myteamprints.com/error_docs/not_found.htmlThis page has bad links which the application is queueing up to visit. Take the number of links on this page times the number of pages that have links to bad pages and you'll see how it's growing so fast.
|
|
Rank: Newbie
Groups: Member
Joined: 6/24/2007
Posts: 9
Points: 27
|
I fixed those bad links and re-checked the site for problem hyperlinks and find no others. It is still doing this however. My other site mypanoramicprints.com works fine. How can I find the problem. Is there something specific I should put in the query string to omit?
|
|
Rank: Administration
Groups: Administration
Joined: 1/31/2007
Posts: 440
Points: 646
Location: Chicago, IL
|
We will check it out ASAP. Sorry for the delay in responding.
|
|
Rank: Administration
Groups: Administration
Joined: 1/31/2007
Posts: 440
Points: 646
Location: Chicago, IL
|
I will has resolution for you on Tuesday, July 24th. Again, sorry for the delay.
|
|
Rank: Administration
Groups: Administration
Joined: 1/31/2007
Posts: 440
Points: 646
Location: Chicago, IL
|
I just spidered your entire website and it ran clean.
Please try it again...maybe clear your cache first (on Preferences tab).
Please let me know how it goes.
|
|
Rank: Administration
Groups: Administration
Joined: 1/31/2007
Posts: 440
Points: 646
Location: Chicago, IL
|
Locking topic as issue has been explained.
|
|
|
Guest |
