Blog archive


What can we learn from the real time web and Justin Bieber…

  1. Justin Bieber has not died yet. It is just a daily Twitter rumour.
  2. He has done a single with some girl called Taylor Swift?
  3. I have ‘accidentally’ read far too many Tweets about Bieber in the last few days.

Last weekend our very own Ollie Glasses want to learn about D3, a cool library for visualising data on web pages. It does all sorts of out there awesomery, Ollie stripped it back to basics with a rolling bar chart, and then lit a rocket under it by combining it with the Twitter streaming API and our chums, a platform for adding realtime functionality to websites.

The result is a site that tracks the volume of Tweets about Justin Bieber per second. Of course this could easily be applied to any search term, Bieber just happens to be a good test candidate due to the average volume of around 100 Tweets per second.

The app is formed of two parts, the server and the client. The server is just a simple Python script that sits on Heroku, it connects to the Twitter stream and counts the incoming tweets. Every second we send the current count to pusher and reset the counter. Meanwhile on the client side we have a simple webpage hosted up on S3. A bit of javascript connects to Pusher, awaiting any updates sent to them from our server, when we receive an update we add the data to the end of our D3 chart. With a bit of magic we can make it adjust the scale of the chart according to the maximum currently displayed. It’s a pretty simple demo but shows some good principles of the realtime web.

So what else could we do to this? We can display any kind of arbitrary data that is offered in real time. Some examples might be:

  • Displaying rolling activity of a KPI.
  • Displaying the volume of traffic across your network. Be that a network of pipes, computers, phones, socials.
  • Displaying your latest customer support queries as they come in.

The code is up on Github, so please feel free to fork and tweak for your own needs. (There seems to be an issue where we mostly receive the figure 102, I’m not sure if this is a Twitter issue or with our code, any hints on why this is would be great).

This post was filed under Technology, The future and tagged Comments are currently closed.

One Comment

  1. Nice work!

    Sent you a pull request to try and solve “the riddle of 102″ ;)

    Posted 22nd April 2012 at 2:41 pm | Permalink

2 Trackbacks

  1. [...] the total count each second instead of sending every Tweet. Edd’s written about it on the NM blog and we’ve hosted it with Heroku at [...]

  2. [...] Bieber Fever Metre. We learnt more about the d3 visualisation library and helped us develop a relationship with, the realtime web guys. It also helped clarify our position as technical innovators within the company and how we can integrate the rest of the team into these developments. This might culminate in a hack day but definitely more testing-the-water within the company. [...]