[flickr]4132778021[/flickr] I thought it would be useful to give a summary of some of the tools I use/developed at CETIS to monitor the pulse of the web around our and JISC work. All of these are available for reuse and documented to varying degrees. All of the tools also use Google Spreadsheets/Apps Script which is free for anyone to use with a Google account, and all the recipes use free tools (the exception being owning a copy of Excel, but most institutions have this as standard).
Hashtag archiving, analysis and interfacing
What it does: It’s a Google Spreadsheet template which can be setup to automatically archive Twitter searches. The template includes some summaries to show top contributors and frequency or tweets. There are a number of add-on interfaces that can be used to navigate the data in different ways, including TAGSExplorer and TAGSArc.
More info: https://hawksey.info/blog/2011/11/twitter-how-to-archive-event-hashtags-and-visualize-conversation/
Monitoring Twitter searches and combining with Google Analytics
What it does: Archives all tweets linking to to the .cetis.ac.uk domain and combines with our Google Analytics data to monitor influential distributors of our work.
More info: https://hawksey.info/blog/2012/03/combine-twitter-and-google-analytics-data-to-find-your-top-content-distributors/
RSS Feed Activity Data Monitoring
What it does: Gives a dashboard view of the total social shares from a range of services (Facebook, Twitter, Google+ for a single or combination of RSS feeds. At CETIS we also monitor the social popularity of blogs referencing .cetis.ac.uk by using a RSS feed from Google’s Blog Search e.g. http://www.google.com/search?q=link:cetis.ac.uk&hl=en&tbm=blg&output=rss&num=20
More info: https://hawksey.info/blog/2012/06/rss-feed-social-share-counting/
What it does: Gives more detailed activity data around socially shared urls combining individual tweets from Topsy, Delicious, and post comments.
More info: https://hawksey.info/blog/2012/08/blog-activity-data-feed-template/
What it does: Dashboards all the project blogs from the OERRI Programme and monitors when they release blog posts with predefined tags/categories. The dashboard also combines the social monitoring techniques mentioned above so that projects and the programme support team can monitor social shares for individual blog posts.
More info: https://hawksey.info/blog/2012/08/how-jisc-cetis-dashboard-social-activity-around-blog-posts-using-a-splash-of-data-science/
Automatic final report generation
What is does: As an extension to the Post Directory this tool combines project blog posts from a predefined set of tags/categories into a final report as an editable MS Word/HTML document. Currently only the original post content, including images, is compiled in individual reports but it would be easy to also incorporate some of the social tracking and/or post comments data.
More info: https://hawksey.info/blog/2012/09/converting-blog-post-urls-into-ms-word-documents-using-google-apps-script-oerri/
As well as standalone tools I’ve documented a number of recipes to analysis monitoring data.
Twitter conversation graph
What it does: Using data from the Twitter Archiving Google Spreadsheet template (TAGS) this recipe shows you how you can use a free Excel add-on, NodeXL, to graph threaded conversations. I’m still developing this technique but my belief is there are opportunities to give a better overview of conversations within hashtag communities, identifying key moments.
More info: https://hawksey.info/blog/2012/08/first-look-at-analysing-threaded-twitter-discussions-from-large-archives-using-nodexl-moocmooc/
Community blogosphere graph
What it does: Outlines how data from blog posts (in this case a corpus collected by the FeedWordPress plugin used in DS106) can be refined and graphed to show blog post interlinking within a community. An idea explored in this recipe is using measures used in social network analysis to highlight key posts.
More info: https://hawksey.info/blog/2012/10/visualizing-cmooc-data-extracting-and-analysing-data-from-feedwordpress-part-1-ds106-nodexl/
Activity data visualisation (gource)
What it does: Documents how data can be extracted (in this case records from Jorum) and cleaned using Google Refine (soon to be renamed OpenRefine). This data is then exported as a custom log file which can be played in an open source visualisation tool called Gource. The purpose of this technique is to give the viewer a sense of the volume and size of data submitted or created by users within a community.
More info: https://hawksey.info/blog/2011/12/google-refining-jorum-ukoer/
So now go forth and reuse!