The ultimate guide to transforming your business with business insights
Insights That Work

The Firehose


Have you ever heard of the Twitter “firehose” or the “sprinkler” ? If you have, you have probably been doing software development work on connecting to Twitter. The “firehose” and the “sprinkler” are just two of the terms Twitter use to describe the type of connections you can make to Twitter with an application. As the name implies the Firehose is everything anyone tweets anywhere, and very few companies need or can cope with the volume of data that is produced. Most applications filter or search the stream for hashtags or keywords or users. Absorbing the full force of the global twitter population is not for the meek. According to ComScore there were 25 billion tweets in 2010. That’s a lot of anything.

Twitter has an incredible volume of information, it seems you either love it or you hate it. I think market researchers have to learn to love Twitter, or at least not hate it. It is a harbinger of what is to come. Try searching the twitter stream for any word, any product name – anything at all. I just tried “dirigibles”, “puke”, “disestablishmentarianism” and “armadillos”. All of these search terms gave me tweets back. I’m not saying the content was great, but it was varied.

Twitter growth outside the USA continues to rise rapidly, with Spain showing a 151% growth in users in 2010. The Middle East is showing a 104% increase over the same period. Within the US the picture is less rosy, late last year reported that Twitter usage was leveling out in the USA, with “only” 24 million unique visitors in the USA. Still seems like a lot of people to me.

Twitter has a couple of problems to my point of view. The first is hashtags, those interesting words you precede with a # symbol. Very useful, it adds a level of filtering and context to a tweet. Problem is, they consume space. Twitter messages are only 140 characters long, and now we are seeing a significant proportion of that consumed by hashtags. If I look at my Twitter feed, I can see messages with 3 or 4 hashtags. I just saw one with 6 hashtags. That doesn’t leave much room for content.

Then there is re-tweeting. When you forward a message to your followers you get text added showing who originated the message. The problem with this is space again. This puts a brake on re-tweeting of messages unless you spend the time to edit the tweet.

Who knows if Twitter really is in decline. Who cares? What will happen is that something will replace it that does the same sort of thing but better. Remember MySpace? Then along came Facebook. Remember AltaVista? Along came Google. And Google is not expanding in the same meteoric way it was. Whatever happens to Twitter, it has shown that people want to communicate, in a great deal of volume. It simply isn’t logical to assume that none of it has any value to market research. I’m not sure what comes next for Twitter like tools, but I am sure it will be of greater volume and I’m also sure MR can’t ignore it.

Please share...

3 responses to “The Firehose

  1. Totally agree. Will be interesting to see what becomes of Twitter or what replaces it. For researchers, however, Twitter is just part of the “fire hose” of information that we must sort through and integrate if we are going to offer the best solutions to our clients.

Join the conversation