I am definitely opposed to SOPA/PIPA, as the potential damage from these pieces of legislation was clear: They could bring down many websites for illegitimate reasons, thus posing a threat to free speech and ruining the great technical achievement (as well as the huge industry) of the web overall.  And so, in this case, my position coincided with that of the Electronic Frontier Foundation (EFF).

But now, the EFF wants me to sign a petition against the so-called Data Retention bill.

The U.S. House of Representatives is currently considering H.R. 1981, a bill that would order our online service providers to keep new logs about our online activities, logs to help the government identify the web sites we visit and the content we post online. This sweeping new “mandatory data retention” proposal treats every Internet user like a potential criminal and represents a clear and present danger to the online free speech and privacy rights of millions of innocent Americans.

I’m not sure I’m as quick to jump on this bandwagon, as I don’t see a problem with ISPs keeping records for 12 months for the following reasons:

1. ISPs already retain data anyway, for time periods of their choosing.  ISPs arguably need this data to track down users who are “abusin’ the tubes” in various ways, including our friends, the spambots, as well as the more nefarious denial-of-service attackers.  If you don’t like this, do you really like the idea of websites that you visit being brought down or ruined with spam all the time?
2. A uniform standard could be useful, as it should make Internet abusers realize that there’s no safe haven for their activities.  No playing one ISP against another, and you can’t get away with –it any more.
3. Privacy concerns may well be overblown.  It’s not like the ISPs can do much with this data, unless requested by authorities with a just cause.  We’re talking mounds and mounds of barely decipherable data here.  It’s only really useful if you already know what things to look for, and for a very good reason, usually to see what a specific Internet abuser is doing around an exact time period.  Unless we have evidence of ISPs using this data for other purposes, I don’t think we should be alarmed.
4. The idea that companies would have an easier time figuring out who their critics are (and similar horror stories) would seem kind of frightening, until one realizes that ISPs already generally retain data, and a company can’t just walk into an ISP and demand this kind of information — they have to get a court to agree, and that’s not as straightforward as you may think.

What are your thoughts?  Am I missing something?

Within the next few weeks, WebCommons’ flagship local discussion space, Louisville History & Issues will be changing its name.

Here’s an excerpt from the site’s announcement:

Since [last Spring], usage of this site has dropped off considerably, and from many perspectives, the site seems to be drifting. All kinds of reasons could be cited, but I think the biggest problem is that this site has never appeared to have the tight focus it deserves. And I believe this tight focus should be on discussion of local issues.

Too many people seem to believe that this is a “history site”, when that was never my intention. I had meant the site to be about the discussion of local history and local issues, with neither one weighted over the other. While I continue to be very interested in providing discussion space for historical discussion (and two forums dedicated to this won’t go away), I think it’s time for a clear shift to local issues as the main thrust of this site.

Barring the acceptance of another suggestion, the site is tentatively set to change its name to
“Metro Issues :: Louisville”.

As nominations for the WebCommon Spambot Awards haven’t been forthcoming so far, I’m going to extend the nominations period.

Originally, it was for two weeks, as I thought that would be long enough, but apparently, not many have taken notice of this amazingly overdue awards program to participate as of yet.

(Thinks to myself: Hmmm, where are all the harried, frazzled web admins who should rightly want to publicly stick it to the spambots and those who enable them?)

Anyway, for now, I’ll keep open the nominations indefinitely.  Then, once they start to trickle in, I’ll set a reasonable expiration date.  Sound good?

Perhaps some readers are wondering why I detest spambots so much that I’m running an awards program in honor of them.

First off, all I can really do is discuss this in terms of my own experience, as I have no idea where to turn for statistics on the exact effect of spambots on our web.  (If you know of some, please post a response.)  So, I’ll speak from personal experience of having been a webhead since 1995 and in particular, seriously developing and administering websites for about the past decade.  Your mileage may vary, so feel free to upset my apple cart in your responses.

Anyway, here’s six reasons why I detest spambots so much (and maybe you’ll detest them more after reading):

6 – The Spam, Stupid.

When successful, spambots leave behind unsolicited commercial advertisements known as spam in blog comments, forum topics/replies, referrer logs, etc.  Duh.  This is the genesis of why we block particular IP ranges and deploy so much plugins, mods and techniques to stop these suckers.  We don’t want spam amongst our content, because it makes our sites look bad and shoos away visitors.  Spam makes a site look like it’s not administered, i.e., not serious.

See the rest after the jump…

Read the rest of this entry »

On January 15, WebCommons launched a new awards program called the WebCommons Spambot Awards, dedicated to recognizing those entities responsible for the spambots that continually attack our websites, as well as entities, tools and techniques that help us deal with them.

Today we’ll focus on Category 4 – Country responsible for the most bothersome spambots.

One word: Russia.  That’s the end of the post.

Seriously, I feel fairly certain that most web admins reading this have the same situation I do — spambots from Russia are a real problem, so much that many just go ahead and block the entire country (I don’t, but I come very close to that).  But based on my own experience, several other countries give me sick headaches (yes, I’m Darrin Stephens’ mom), including my own, my beloved U.S.A.

I don’t have much to say about this topic really, except that it’s fairly easy to track what country a spambot come from, using IP-to-country matching databases such as ip2nation.  And it doesn’t take long to see patterns of where the worst bots are coming from.

If you have been a web administrator for any significant length of time, surely you can think of several countries where the most or the most bothersome spambots hitting your sites originate (I nominated five myself).  So, please nominate one to five of them in responses to this post, or in responses to the original awards program post.  Or in an e-mail directly to us.

Thank you very much for your participation!

On January 15, WebCommons launched a new awards program called the WebCommons Spambot Awards, dedicated to recognizing those entities responsible for the spambots that continually attack our websites, as well as entities, tools and techniques that help us deal with them.

Today we’ll focus on Category 3 – Hosting company or cloud computing platform  responsible for the most bothersome spambots.

As web administrators, typically we run websites that are generally designed for humans for visit and use to their heart’s content.  Also, we typically optimize our sites for search engines, so that when search engine spiders (good bots, usually) drop by, they can consume our content, process it and store it to aid web searches by humans.  Sometimes, we run websites with programming interfaces (APIs) so that programs (bots) from hosting companies and cloud computing platforms can connect to our site to read (and sometimes write) data.  These programs can even interact with our sites as if they were human, to some extent.

Unfortunately, a lot of these programs are rogue.  There are many kinds of malicious bots, including ones that attempt to find security weaknesses in our sites, usually to find a way to insert rogue data into our databases or load rogue programs into our website directories.  For this exercise, however, we’re concentrating on the spambots.  These beasts are all about  inserting spam in various spots on your site, usually on the public-facing side, like in blog comments, but also sometimes in the administrative logs as fake referrers.

Here’s some good news, though.  If you’re not providing APIs on your site, you have no necessity to even allow access from hosting companies or cloud computing platforms, except in the case of the more useful search engine spiders.  But I won’t recommend blocking the IP ranges of all of these entities, as it’s counterproductive (in the form of a too big  .htaccess file or too many iptables rules to maintain) and figuring out every last one of these IP ranges takes too much detective work, more than most of us have time for.  What I do recommend is finding a way to trap the spambots (another topic) and determining where they come from, and if you’re seeing what you feel are too many from a specific hosting company, block their IP range(s).

(Here’s a hint for those really serious about blocking the worst of the hosting companies that host these spambots and other malicious bots: Check out the Wizcrafts blocklists.)

If you are providing APIs on your site, you can still block problematic hosting companies but set up your site to make your read-only APIs accessible to everyone.  For instance, in the .htaccess file, you can obviate all rules for a specific file.  (This is also another topic.)

If you have been a web administrator for any significant length of time, surely you can think of several hosting companies or cloud computing platforms that have delivered the most or the most bothersome spambots to your sites (I nominated five myself).  So, please nominate one to five of them in responses to this post, or in responses to the original awards program post.  Or in an e-mail directly to us.

Thank you very much for your participation!