from the dept.
Google has been web-spidering technocrat.net . Because there are over 7000 postings, that's a lot of access - it's getting close to one every second. One thing I notice is that it spiders the addPostingForm on every article, effectively doubling the number of accesses. Unfortunately, the robot exclusion standard isn't fine-grained enough to be able to say don't spider anything with this name. Thus, I propose to rename addPostingForm to addPostingForm.cgi as a hack to tell robots not to access it. Another thing I could do is add a default argument to all invocations of addPostingForm. That might put off the spiders, and I could switch on the presence or absence of that argument to have addPostingForm bail out without any processing when it's invoked without it.
< | >
|"Any system that depends on reliability is unreliable." -- Nogg's Postulate|
|All trademarks and copyrights on this page are
owned by their respective companies.
Comments are owned by the Poster.
The Rest ©1999
Butch Landingin, ©2000-2002