Rank: Newbie
Groups: Member
Joined: 9/24/2009
Posts: 1
Points: 3
Location: Richmond Hill, GA USA
|
Is there any options within the program that can be set to dramtically speed up the process?
Does it speed up after it finds all the URL's?
It's been running for 15 hours now and has found 26,000+ URL's but only completed 3650. At this rate per hour, we are talking about 101 hours or longer.
I have 4meg of DDR2 ram using an Intel® Pentium® Dual-Core Processor. I'm thinking of stopping the current process and starting over with a laptop that I can just sit in the corner and check daily/weekly to see process.
I have told it to ignore URL strings with: account,checkout,shopping,information,contact,advanced,reviews and I set the results detail level to NONE
What do you recommend?
|
|
|
|
Rank: Newbie
Groups: Member
Joined: 9/30/2009
Posts: 1
Points: 3
Location: Earth
|
I have the same problem. Running one hour now, 483 completed, over 100,000 left "and growing"
Dual AMD
Win7 64bit
4g ram
|
Rank: Administration
Groups: Administration
Joined: 1/31/2007
Posts: 586
Points: 384
Location: Chicago, IL
|
The software runs 100% in memory and is very fast, but has to wait for every web page request to complete.
The problem is most likely:
- your web hosting provider's connection is slow
- your website software is slowing it down
- your local Internet connection is slow.
Try spidering "http://forums.keylimetie.com". I just did and it took 2 min and 45 sec to spider all 1,238 URLs.
We have also spidered sites with over 1 million URLs and it only took 3-4 days. That's a long time, but 1 million URLs is a lot!
If you want, PM us your URL and we'll take a look.
Thanks,
KeyLimeTie
|
