Welcome Guest Search | Active Topics | Members | Log In | Register

Struggling with the trial version: Options
Jdebosdari
Posted: Sunday, October 07, 2007 2:39:01 PM
Rank: Newbie
Groups: Member

Joined: 10/7/2007
Posts: 2
Points: 6
Location: UK
Hi
web address: http://store.ashridgetrees.co.uk
there is a valid robots.txt file

The exported text file is as below:
What am I doing wrong?
Thanks
Julian

Spider started.
Added to queue: http://store.ashridgetrees.co.uk/
Added to queue: http://store.ashridgetrees.co.uk/SHOPPING-CATEGORIES;/
Added to queue: http://store.ashridgetrees.co.uk/Information;/
Added to queue: http://store.ashridgetrees.co.uk/Bareroot-Trees-Shrubs-and-Hedge-Plants;/
Added to queue: http://store.ashridgetrees.co.uk/SHOPPING-CATEGORIES/Bareroot-Hedging-Trees-and-Shrubs;/
Added to queue: http://store.ashridgetrees.co.uk/SHOPPING-CATEGORIES/Standard-Large-Trees;/
Added to queue: http://store.ashridgetrees.co.uk/SHOPPING-CATEGORIES/Roses;/
Added to queue: http://store.ashridgetrees.co.uk/SHOPPING-CATEGORIES/Fruit-Trees;/
Added to queue: http://store.ashridgetrees.co.uk/SHOPPING-CATEGORIES/Soft-Fruit;/
Added to queue: http://store.ashridgetrees.co.uk/SHOPPING-CATEGORIES/Tree-Planting-Accessories;/
>>> Execute Web Request error for http://store.ashridgetrees.co.uk/SHOPPING-CATEGORIES;/: The remote server returned an error: (404) Not Found.
>>> Execute Web Request error for http://store.ashridgetrees.co.uk/Information;/: The remote server returned an error: (404) Not Found.
>>> Execute Web Request error for http://store.ashridgetrees.co.uk/Bareroot-Trees-Shrubs-and-Hedge-Plants;/: The remote server returned an error: (404) Not Found.
>>> Execute Web Request error for http://store.ashridgetrees.co.uk/SHOPPING-CATEGORIES/Bareroot-Hedging-Trees-and-Shrubs;/: The remote server returned an error: (404) Not Found.
>>> Execute Web Request error for http://store.ashridgetrees.co.uk/SHOPPING-CATEGORIES/Standard-Large-Trees;/: The remote server returned an error: (404) Not Found.
>>> Execute Web Request error for http://store.ashridgetrees.co.uk/SHOPPING-CATEGORIES/Roses;/: The remote server returned an error: (404) Not Found.
>>> Execute Web Request error for http://store.ashridgetrees.co.uk/SHOPPING-CATEGORIES/Fruit-Trees;/: The remote server returned an error: (404) Not Found.
>>> Execute Web Request error for http://store.ashridgetrees.co.uk/SHOPPING-CATEGORIES/Soft-Fruit;/: The remote server returned an error: (404) Not Found.
>>> Execute Web Request error for http://store.ashridgetrees.co.uk/SHOPPING-CATEGORIES/Tree-Planting-Accessories;/: The remote server returned an error: (404) Not Found.
Spider completed.

Sponsor
Posted: Sunday, October 07, 2007 2:39:01 PM
Get your Sitemap Generator license today! http://www.keylimetie.com/Checkout/Quick-PayPal/
KeyLimeTie
Posted: Tuesday, October 09, 2007 11:54:24 PM
Rank: Administration
Groups: Administration

Joined: 1/31/2007
Posts: 380
Points: 539
Location: Chicago, IL
I scanned your home page and your URLs have semicolons separating the page/folder from the querystring values.
I have never seen this before and it's causing bad URLs to be stored and spider by the application.
For example:
<a href="/SHOPPING-CATEGORIES;jsessionid=ac112b2a1f435623313f67734f95b4c1df1e8bfdee96.e3eSbNyQc3mLe34Pa38Ta38Pahb0" class="topnavlink">Shop for Plants</a>

Are you putting "jsessionid" in the ignore querystring parameters field?
Try changing it to ";jsessionid"?

If this is a standard method of appending querystring parameters, we might need to update the software to account for it.
Please let us know how it goes and we'll do some of our own testing.

Thanks,
Brian
iArchitect
Jdebosdari
Posted: Wednesday, October 10, 2007 6:25:55 PM
Rank: Newbie
Groups: Member

Joined: 10/7/2007
Posts: 2
Points: 6
Location: UK
Thanks Brian

I was kind of afraid of that. We use an ERP app called Netsuite (www.netsuite.com). This is a "Netsuite" problem. If your crawler emulates Googlebot, then the jsessionids are 301 redirected to user friendly urls. Otherwise the sitemap hets stuffed with jsessionids. But I don't see why that should return 404's?

Best
Julian
KeyLimeTie
Posted: Sunday, October 21, 2007 11:06:53 PM
Rank: Administration
Groups: Administration

Joined: 1/31/2007
Posts: 380
Points: 539
Location: Chicago, IL
Have you changed your website since the last message?
When I try to spider it now, it only finds the home page...no internal links.
KeyLimeTie
Posted: Thursday, November 08, 2007 11:28:25 AM
Rank: Administration
Groups: Administration

Joined: 1/31/2007
Posts: 380
Points: 539
Location: Chicago, IL
Locking topic as no response in some time.
Users browsing this topic
Guest


Forum Jump
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

Main Forum RSS : RSS

None
Powered by Yet Another Forum.net version 1.9.1.2 (NET v2.0) - 9/27/2007
Copyright © 2003-2006 Yet Another Forum.net. All rights reserved.
This page was generated in 0.051 seconds.