By @mhawksey

Keep your Twitter Archive fresh on Google Drive using a bit of Google Apps Script

Update: There is a newer take on this solution which dumps Google Drive web hosting and uses Github Pages instead. You can read more in Keeping your Twitter Archive fresh and freely hosted on Github Pages.

Important: A noted in the comments Google announced last year that it is killing Google Drive web hosting which this solution replies on. The good news is I’ve come up with a solution that lets you keep your archive on the web. If you’ve got an existing archive the only change is a new url  https://script.google.com/macros/s/AKfycbwrXr8ejYjHwGEO6kj8f4WHIh096ARDRHdNOgAXPqGltoa80FU/exec?folder_id=YOURFOLDERID (e.g. my archive is at https://script.google.com/macros/s/AKfycbwrXr8ejYjHwGEO6kj8f4WHIh096ARDRHdNOgAXPqGltoa80FU/exec?folder_id=0B6GkLMU9sHmLRFk3VGh5Tjc5RzQ) Your archive is be a bit slower to load and will have a bar at the top to say it’s not a Google application. I’ve updated the template to include the new url.

Like a growing number of other people I’ve requested and got a complete archive of my tweets from Twitter … well almost complete. The issue is that while Twitter have done a great job of packaging the archives even going as far as creating a search interface powered by HTML and JavaScript as soon as you’ve requested the data it is stale. The other issue is unless you have some webhosting where can you share your archive to give other people access.
Fortunately as Google recently announced site publishing on Google Drive by uploading your Twitter archive to a folder and then sharing the folder so that it’s ‘Public on the web’ you can let other people explore your archive (here’s mine). Note: Mark Sample (@samplereality) has discovered that if you have file conversion on during upload this will break your archive. [You can also use the Public folder in Dropbox if you don’t want to use a Google account]

The documentation wasn’t entirely clear on how to do this. Basically it seems that as long as there’s a index.html file in the folder root and links to subdirectories are relative all you need to do is open the folder in Google Drive and swap the first part of the url with https://googledrive.com/host/ e.g. https://drive.google.com/#folders/0B6GkLMU9sHmLRFk3VGh5Tjc5RzQ becomes https://googledrive.com/host/0B6GkLMU9sHmLRFk3VGh5Tjc5RzQ/

So next we need to keep the data fresh. Looking at how Twitter have put the archive together we can see tweets are stored in /data/js/tweets/ with a file for each months tweets and some metadata about the archive in /data/js/, the most important being tweet_index.js.
Fortunately not only does Google Apps Script provides an easy way to interface Drive and other Google Apps/3rd party services but the syntax is based on JavaScript making it easy to handle the existing data files. Given all of this it’s possible to read the existing data, fetch new status updates and write new data files keeping the archive fresh.
To do all of this I’ve come up with this Google Spreadsheet template:

*** Update Twitter Archive with Google Drive ***
[Once open File > Make a copy for your own copy]

Important: Google have changed some of their backend. If you took a copy of this template prior to 8th Dec 2014 the script will fail with a Script is using OAuthConfig, which has been shut down. Learn more at http://goo.gl/IwCSaV (line 277, file "Code"). To get your archive running again the best solution is to copy the template and setup using your existing Drive folder and API key/secret.
Billy has spotted an inconsistency with the way Drive searches for the files used in the script. This has been addressed in an update to the code but existing news will need to either take and setup a fresh copy or open their existing copy and then open Tools > Script editor and replace the code with version here.
Note: There is currently an open issue which is producing the error message ‘We’re sorry, a server error occurred. Please wait a bit and try again.’ Hopefully the ticket will be resolved soon

The video below hopefully explains how to setup and use (Update The script uses a new authentication flow which makes setup easier (and API registration is not necessary if you already use TAGS v6.0):

A nice feature of this solution is that even if you don’t publically share your archive, if you are using the Google Drive app to syncs files with your computer the archive stays fresh on your local machine.
The model this solution uses is also quite interesting. There are a number of ways to create interfaces and apps using Google Apps Script. Writing data files to Google Drive and having a static html coded based interface is ideal for scenarios like this one where you don’t rely on heavy write processes or dynamic content (aware of course that there will be some sanitisation of code).
It would be easy to hook some extra code to push the refreshed files to another webserver or sync my local Google Drive with my webhost but for now I’m happy for Google to host my data ;s

Exit mobile version