February 4, 2016 Leave a comment
Partly a demo example of how to Omniscope can be used to look at twitter data, also is of interest for how we develop and plan our cities. I just saw some interesting tweets under #futurecities and was an event with speakers on the subject of making use of data and technology to better understand how cities work (Event Info). Unfortunately just following tweets of an event is no replacement for actually attending so you miss out on the real content. Still reviewing other’s tweets still brings some little nuggets of knowledge.
Using the current twitter API, it is difficult to mine tweets in bulk from it due to the volume of tweets across the world. You need some serious hardware and automatically running algorithms to extract the data at set intervals a day across multiple access accounts (though this is against terms and conditions!). Alternatively you can use a licensed data supplier to do it instead.
The restrictions on the API creates 15 min windows in which you can only extract a certain amount of data. It is fine to follow a particular trending topic of by filter for specific types of tweets i.e. location enabled tweets. Using Omniscope I used the data API connector made calls to extract data every 5-10mins for the #futurecities tag.
I made a fairly basic Omniscope visualisation of the tweets and some screenshots below:
Extracting 179 tweets over the period 10am – 5pm covering the day event. Below is a graph showing the tweets over time split by original tweets by attendees and retweets by others.
Taking this data the most interesting aspect is to really look at how tweets are passed around and I used the tweet main text to extract out @user_names and taking the original user to map out in a network view diagram the following interactions. Below is a visualisation of the two types of original tweets in blue and red showing retweets of those original ones. The direction of the arrows indicate from the original poster and directed towards users mentioned in the tweet (most cases the recipient who may be interested in the tweet).
It’s always interesting to see how topics are passed around we often see networks and clustering in tweets where the activity will revolve around a dominate/popular user and see the interactions spread out from there. There also looks to be a self enforcing network where individuals will repeat interactions with each other whether directly or just indirectly via retweets.