Blog archive


We’re excited about Datasift Historics

A couple of weeks ago Datasift announced Historics. This got us very excited, access to two years worth of Tweets, ready for filtering and analysis!

For those who don’t know, Datasift is one of Twitters partners, they run a service that allows developers to filter the ~250 million Tweets per day, in real time, in order to match specified criteria (such as sentiment, keywords, influencer metrics etc) and make the data more manageable.

Currently Datasift only allows you to filter Tweets in real time, as they happen. If you want to save Tweets for analysis after the fact then you will need to filter and save them as they are created. But what if the filter you set up is too narrow? All those lost Tweets would normally prove much more difficult to extract from Twitter.

Enter Datasift historics, all the Tweets going back until January 2010, ready for filtering and analyzing as one big dataset for the first time. “Great”, you might be thinking, “we can find out the most popular things people had for breakfast lunch and dinner over the last two years”. Yup, that and more!

With this power, we could go back and analyze major events such as the London riots last summer, the Arab spring, the last flight of NASA’s space shuttle program, the birth of the Occupy movement etc. Perhaps more interesting to us are Tweets about our clients and how we can help them understand their audience better.

Here are a couple of examples of what we’d love to do with Historics:

Mash up data from Datasift about a TV program with ratings data. Would we discover that a few weeks before ratings start to drop off that tweet sentiment had already started to turn negative? or is there a groundswell of excitement around an episode that sees ratings soar? What were the topics discussed and trends around that breakthrough episode?

Generate competitor reports, including data about:

  • Trending topics.
  • Volume of inbound tweets to each competitor.
  • Volume of replies from each competitor, along with the speed of the replies.
  • Are there any themes associated with each competitor?
  • Who are the main influencers around each competitor, what are they talking about?

How do McDonalds, KFC, Burger King, Pizza Hut and Wimpy compare to Pizza Express, Ask and GBK? Of course, all of this becomes much interesting over time, if your competitors are offering customer service over Twitter, what kind of service level do they say are they offering and what kind of level are they actually offering and able to sustain?

Of course there are a few detractors of Twitters decision to open up data that many thought had disappeared into the ether. An interesting debate was published on the Guardian at the weekend as to whether it was ‘right’ or not. I’m of the opinion that it was all public once and agree with the saying “if you’re not paying for it, you are the product”.

Are you also excited for Historics or annoyed at the use of your data!? Hollah at me in the comments ;)

This post was filed under Data, Digital transformation, Social media, Technology, The future Comments are currently closed.

One Comment

  1. Belinda

    Sorry to take this down to economics. But I’m wondering how much it’s likely to cost to access this data and what the pricing model is likely to look like – ie which brands / organisations are actually likely to use this insight and when?

    Posted 15th March 2012 at 4:29 pm | Permalink