damn!

dailyheights · October 2005

DH just got a bandwidth warning from its benevolent webhost. Bandwidth warning? 11 days into the month??

Checking around, it looks like you guys have hit these pages 274,000 times in the last 11 days.

Something tells me that number just can't be right... but if it is... where do you guys find the time?

daveb · October 2005

It's probably spiders (the web kind). I'd check your raw logs (once again, the web kind) and see what's what. A number that high makes me think that's the case. I have the same issues with my site at times. I basically disallow most spiders with a robots.txt file (assuming the spider plays by the rules and follows the directives I put there. If it doesn't, I ban the ip or range it uses) while allowing ones like Yahoo, MSN, Google, KINJA and a few others. I also disallow my images folder because that's where the meaty (as in non-carneviento...*sigh*...meat...wind...) files are. Spammers run bots to scrape and repost like the problem you had before and/or refer and comment spam. It sucks up a lot of bandwidth. Sucks. Hate them.

dailyheights · October 2005

yes, attack of the robots. has to be. must investigate before DH goes off the air. again.

captain m · October 2005

Daily Heights is Big Time.....

My GF was doing an article on mens grooming supplies for the holidays and I thought it was super gay and thought about what guys really want for christmas and I googled "Wrecking Ball for sale" and DH came up on the fourth hit.

dailyheights · October 2005

ah ha ha!

world domination on a wind of meat!

captain m · October 2005

I highly recommend RSS feed for frequent flyers, it just gives the updates and it uses less bandwith

dailyheights · October 2005

OK. I found the offender - and daveb has identified it as a spider for a "shopping engine" called Exava/Become. The bot sucked up nearly 20GB of bandwidth in less than 10 days! Banned. (This attack did not affect page views, btw.)

Are they planning to dominate the internet by sucking the bandwidth out of it?

anonymous · October 2005

EXAVA/BECOME BOT COME TO DAILYHEIGHTS FOR STORY ABOUT PROSPECT HEIGHTS GAFFLER AND GAS SIPHONER

EXAVA/BECOME BOT CANNOT HELP EXCESSIVE BANDWIDTH CONSUMPTION

EXAVA/BECOME BOT LIKE TO READ STORY OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER

PLEASE DO NOT BAN EXAVA/BECOME BOT

[MOD daveb] qtrain being a smartass. daveb can appreciate this. [/MOD NOTE]

sje · October 2005

rulala · October 2005

Exava/Become wrote: EXAVA/BECOME BOT COME TO DAILYHEIGHTS FOR STORY ABOUT PROSPECT HEIGHTS GAFFLER AND GAS SIPHONER

EXAVA/BECOME BOT CANNOT HELP EXCESSIVE BANDWIDTH CONSUMPTION

EXAVA/BECOME BOT LIKE TO READ STORY OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER

PLEASE DO NOT BAN EXAVA/BECOME BOT

[MOD daveb] qtrain being a smartass. daveb can appreciate this. [/MOD NOTE]

HAHAHAHA qtrain is awesome..

jack · October 2005

Here's a tip.

There are tons of odd robots out there on the web nowadays. Inlcuding odd robot traffic that bizarrely excessively clicks on ads! Seriously. In my day gigs I've gotten calls about clicks being higher than impressions for very specific ads for no reason.

I have no idea why this happens or for what purpose, but if you have a way of reviewing your logs and checking IP addresses and such, do some research. Find out where those calls are coming from. And IP block them from the site entirely.

I don't know how dailyheights is setup on the tech side, but this typically is controlled via the '.htacess' file on the servers root.

Then again, DH could just be popular.

damn!

Comments

Howdy, Stranger!

Categories