Update: There is a newer take on this solution which dumps Google Drive web hosting and uses Github Pages instead. You can read more in Keeping your Twitter Archive fresh and freely hosted on Github Pages.
Important: A noted in the comments Google announced last year that it is killing Google Drive web hosting which this solution replies on. The good news is I’ve come up with a solution that lets you keep your archive on the web. If you’ve got an existing archive the only change is a new url https://script.google.com/macros/s/AKfycbwrXr8ejYjHwGEO6kj8f4WHIh096ARDRHdNOgAXPqGltoa80FU/exec?folder_id=YOURFOLDERID (e.g. my archive is at https://script.google.com/macros/s/AKfycbwrXr8ejYjHwGEO6kj8f4WHIh096ARDRHdNOgAXPqGltoa80FU/exec?folder_id=0B6GkLMU9sHmLRFk3VGh5Tjc5RzQ) Your archive is be a bit slower to load and will have a bar at the top to say it’s not a Google application. I’ve updated the template to include the new url.
Like a growing number of other people I’ve requested and got a complete archive of my tweets from Twitter … well almost complete. The issue is that while Twitter have done a great job of packaging the archives even going as far as creating a search interface powered by HTML and JavaScript as soon as you’ve requested the data it is stale. The other issue is unless you have some webhosting where can you share your archive to give other people access.
Fortunately as Google recently announced site publishing on Google Drive by uploading your Twitter archive to a folder and then sharing the folder so that it’s ‘Public on the web’ you can let other people explore your archive (here’s mine). Note: Mark Sample (@samplereality) has discovered that if you have file conversion on during upload this will break your archive. [You can also use the Public folder in Dropbox if you don’t want to use a Google account]
The documentation wasn’t entirely clear on how to do this. Basically it seems that as long as there’s a index.html file in the folder root and links to subdirectories are relative all you need to do is open the folder in Google Drive and swap the first part of the url with https://googledrive.com/host/ e.g. https://drive.google.com/#folders/0B6GkLMU9sHmLRFk3VGh5Tjc5RzQ becomes https://googledrive.com/host/0B6GkLMU9sHmLRFk3VGh5Tjc5RzQ/
So next we need to keep the data fresh. Looking at how Twitter have put the archive together we can see tweets are stored in /data/js/tweets/ with a file for each months tweets and some metadata about the archive in /data/js/, the most important being tweet_index.js.
Fortunately not only does Google Apps Script provides an easy way to interface Drive and other Google Apps/3rd party services but the syntax is based on JavaScript making it easy to handle the existing data files. Given all of this it’s possible to read the existing data, fetch new status updates and write new data files keeping the archive fresh.
To do all of this I’ve come up with this Google Spreadsheet template:
*** Update Twitter Archive with Google Drive ***
[Once open File > Make a copy for your own copy]
Script is using OAuthConfig, which has been shut down. Learn more at http://goo.gl/IwCSaV (line 277, file "Code")
. To get your archive running again the best solution is to copy the template and setup using your existing Drive folder and API key/secret.The video below hopefully explains how to setup and use (Update The script uses a new authentication flow which makes setup easier (and API registration is not necessary if you already use TAGS v6.0):
A nice feature of this solution is that even if you don’t publically share your archive, if you are using the Google Drive app to syncs files with your computer the archive stays fresh on your local machine.
The model this solution uses is also quite interesting. There are a number of ways to create interfaces and apps using Google Apps Script. Writing data files to Google Drive and having a static html coded based interface is ideal for scenarios like this one where you don’t rely on heavy write processes or dynamic content (aware of course that there will be some sanitisation of code).
It would be easy to hook some extra code to push the refreshed files to another webserver or sync my local Google Drive with my webhost but for now I’m happy for Google to host my data ;s
Jedidiah Rex
I tried making a copy of the spreadsheet, but that option was not available to me.
Martin Hawksey
Hi Jedidiah, first thing to check is you are logged in to your google account when you open the spreadsheet
Jedidiah Rex
Hi Martin – I was logged into my google account. I have tried it since and been able to make a copy. Thanks.
Alan Levine (@cogdog)
I’m not getting any action, have triple checked settings. U’ve entered my folder ID twice but the status says “Folder ID not entered yet” When I runt he script, it just stays at Running script updateArchive – it looks like its not getting my folder ID?
Martin Hawksey
Think it was my dodgy coding not updating the location on the readme. If you open the Log sheet has it recorded any updates?
Alan Levine (@cogdog)
Nope, updates are blank. Should I share the doc with you?
Martin Hawksey
[closed – for some reason authentication with Twitter didn’t work 1st time. Added some extra UI to help]
Benjamin
Looking for a link to archiving twitter on a website (other than on Google Drive)…
lilla
Hi,Martin it is great what you’ve done. thanks for this beautiful script.
I tried to follow your instruction. Everything loooks good till at last what appears is yours, not mine. Check this. https://googledrive.com/host/0B1yKt8K-OtLOUDFvY1laMjNMS1E/
So, how to change the name account?
thanks
Martin Hawksey
Hi Lilla – did you start with an upload of your own archive from Twitter?
That Twitter Archive? Thanks to Martin Hawksey (Genius) ITS ALIVE - CogDogBlog
[…] It took him like 24 hours to build this. See Keep your Twitter Archive fresh on Google Drive using a bit of Google Apps Script. […]
How to Make Sure Your Twitter Archive Is Always Up to Date « VidenOmkring
[…] Now, just use Google Apps Script to make sure your data updates with every new tweet, and boom: a full, cruisable archive of all your tweets that is inexplicably absent on twitter.com. You can get a full explanation of the process here. [JISC CETIS MASHe} […]
peelie
hi
am getting Unexpected error: (line 35) when i try to authorize function?
var result = UrlFetchApp.fetch(“http://api.twitter.com/1.1/account/verify_credentials.json”, requestData);
thanks in advance
Martin Hawksey
Sounds like you API key/secret are not set properly. Make sure there’s so spaces before/after them when you enter them into the configuration dialog (I’ve added a trim to prevent this in the future)
peelie
hi again
used your new spreadsheet template and now getting a different error message says:
—
Apps Script
Unexpected error:
—
have tripled checked the api key/secret.
and ideas?
thanks
peelie
ak ignore last message working now, i forgot to add the callback url, doh!
thanks again great script!
Mark Sample
Holy smokes, this is great! I had trouble getting it to work at first, and I wanted to share the solution, in case your readers encounter the same problem. Depending upon one’s Google Drive settings, your Twitter archive folder might be converted to “Google Docs” format during the uploading process. This breaks your files and makes them unreadable (as I found out!). So make sure that in your Google Drive upload settings, you have all the file conversion rules turned off.
Here’s my own now-working archive: https://googledrive.com/host/0By7OircJ9labZktFR0xkT1ExUXc/
Martin Hawksey
Thanks Mark! I’ve added a couple of notes in the template/post to highlight this.
Alan Levine (@cogdog)
That good to knoe Mark- I used the Google Drive desktop app which takes care of that. I know point my subdomain to GDrive http://tweets.cogdogblog.com/ The storage and capability here has me reeling with new ideas.
How To Make Sure Your Twitter Archive Is Always Up To Date | Gizmodo Australia
[…] Now, just use Google Apps Script to make sure your data updates with every new tweet, and boom: a full archive of all your tweets that is inexplicably absent on Twitter. You can get a full explanation of the process here. [JISC CETIS MASHe] […]
drikkes
Wow, great. That even worked for a coding/web-admin dummy like me. Thank you so much. Because I was too stupid to set up Tweetnest.
Francisco George (@paco229)
Great sheet Martin,
I have a problem though when I want to start the sync, it keeps telling me “application has not been authorised” or something like that…I repeated the process again and again and no way to have it work.
For the moment I just managed to get my tweets online.
https://googledrive.com/host/0B_RajZialboVTFNhYi1HcmpvVkE/
if you have any hint…
Martin Hawksey
In Tools > Script Editor when you then clicked Run > authorize did any error msgs come up?
Andrew Laws
Hi Francisco George,
I had the same problem – if you do as Martin suggested and Run > authorize part of the script, you’ll get a dialog box (can’t remember what it said now) with a button. If you click the button, it takes you off to Twitter, and you’ll be able to authorize the script there.
It fixed the problem for me at least – hope it works for you too.
Nice work Martin!
Francisco George (@paco229)
Hi Martin,
Thanks a lot, it worked!
I somehow din’t quite understood this step…and now with your hint everything worked fine.
You should maybe submit your solution to Twitter, it would ease their load if by having this solution implemented as they would not have every one downloading their archive too frequently.
Very nice work!
Francisco
PS: is there anyway to change the frequency of the automatic update and set it for a for a every 8 hours update for example?
Martin Hawksey
UX isn’t my strong point. I’m not sure what Twitter would make of this ;s
[You can update frequency by opening Tools > Script Editor and then Resources > Current script triggers. In the dialog box you should see a function set to fun daily. Just adjust the frequency from one of the drop downs and save. I wouldn’t recommend going any lower than every 15 min to avoid api/usuage limits]
Patrick Torsney
Hi Martin, sorry if I’m being dense but what is the current update frequency set at – daily? I tried the route you mention above Tools > Script Editor > Resources – I don’t have ‘current script triggers’, instead I get the option ‘current project triggers’. The dialog box this brings up shows one trigger but it isn’t editable at all and I can’t see anywhere that mentions update frequency
It’s not the end of the world – I would be happy with once a day if this is what it is
Btw, love it. Thanks very much 🙂
Martin Hawksey
Hi Patrick – in the spreadsheet there should be a Sync Twitter Setup menu where you can toggle the auto-refresh
Patrick Torsney
Hi Martin, thanks I found that in the end but thanks for getting back to me. Very chuffed with this – great bit of work, well done 🙂
老 牛 | 推文检索
[…] 这里是设置教程页面。 […]
Cement Science
OMG, this is really interesting.
shiruken
Is the formatting of the added (updated) tweets exactly the same as the original Twitter Archive? I had a script written to parse the original archive JSON by stripping the first line (per the README in the archive) but that seems to fail with the updated archive files. It seems that in the original archives, the first line reads as “Grailbird.data.tweets_2013_01 = ” whereas yours outputs “Grailbird.data.tweets_2013_01 = [” with that opening bracket appearing on the first line rather than the second. It’s probably a simple fix to your script but it does break JSON parsing if the first line is stripped.
Martin Hawksey
ah hadn’t spotted that. I’ve updated the template. The changes are line 109 which is now
tweet_file.replace(“Grailbird.data.”+var_name+” = \n”+ JSON.stringify(Grailbird.data[var_name], null, ‘\t’)); // replace old file
and line 116 which is now
new_tweet_file.replace(“Grailbird.data.”+i+” = \n”+ JSON.stringify(newData[i], null, ‘\t’)); // replace content with new data
Francisco George (@paco229)
Does this update is meant for every one of us? Or is it only relevant for the script Shiruken is using?
Martin Hawksey
Yes only relevant if you are doing your own post processing
shiruken
So I think something may be broken now that a new month has rolled around. The script generated a “2013_02.js” file in the root of my Google Drive rather in the appropriate directory which resulted in future updates failing.
Francisco George (@paco229)
Well for the moment I can’t really test it, DRIVE and Youtube are not responding.
Francisco George (@paco229)
Ok…My “2013_02.js” has been created in the correct directory. under data>js>tweets
shiruken
Very weird. I moved the file back into the appropriate location and everything seems to be working fine. Guess I’ll find out next month if it’s actually a problem.
Francisco George (@paco229)
Maybe it is due to the short downtime of drive.google.com and youtube.com that prevented a correct sync. My guess
Francisco George (@paco229)
but you are right too, the file “2013_02.js” appears in my root too but along it, it says it is located in the “Tweets” folder(the public folder I use)
Martin Hawksey
Appeared to work okay for me. Like @paco229 the new file appeared in the root directory and data>js>tweets (which is surprising as I thought Drive had done away with files having multiple labels/locations). I’ve added an extra line in the template to prevent this. For existing copies of the script insert the following line after line 115:
new_tweet_file.removeFromFolder(DocsList.getRootFolder());
so it should look like:
new_tweet_file.addToFolder(tweetsFolder); // move new file to the data/js/tweets/ folder
new_tweet_file.removeFromFolder(DocsList.getRootFolder());
Francisco George (@paco229)
It is back now
Twitter Analytics 2.0 | Colin Sullender
[…] few days before I finally gained access to the Twitter Archive feature, a Google Apps Script was released to automatically update the Twitter archive with new tweets. The latest update to Google Drive […]
자동 업데이트 되는 나만의 트위터 백업 ( Twitter Archive ) 사이트 만들기 | Guru's Blog
[…] […]
peelie
hi
my archive is giving me a 404 not found error.
when i try to manually update archive using your script i get this error message:
TypeError: Cannot call method “getContentAsString” of undefined
thanks in advance!
Martin Hawksey
Sounds like your folder is not shared/missing?
peelie
hi
had to eventually upload new Twitter folder, and set share permissions to “Public on the web – Anyone on the Internet can find and view”
thanks
Willem Karssenberg
Thanks for sharing this routine!
Defenitely going to try this out.
Nice to see our conversation back in your archive.
I’ve changed some backgrounds in mine:
http://www.karssenberg.nl/twitter/
Patrick Torsney
Hi Willem, how did you manage to change your background?
Willem Karssenberg
Thank you so much Martin! I’ve got it all up and running, the way you described.
Only the search function does not work yet. May be it needs some time indexing?
You can view my archive at: https://googledrive.com/host/0By2d5B0yP7tLbmdsVkRyS2pQQk0/
This is so nice!
Martin Hawksey
Hi Willem, search is done via the browser (no clever indexing). When I tried doing a search in your archive the console said the file data/js/tweets/2008_05.js was missing
Thanks,
Martin
Willem Karssenberg
Thanks so much, you found the bug! Actually, the file was not missing, but existed two times! Ni idea how that happened. May be a hickup whith ftp.
Anyways, search works now thanks to your attention.
Lots of people here in the Netherlands tried your script already. It’s been retweeted quite a lot!
Francisco George (@paco229)
Hi Martin
I’m getting this message from [email protected] on intervals of one hour(my refresh rate)
26/02/13 18:51 updateArchive We’re sorry, a server error occurred. Please wait a bit and try again. (línea 72, archivo “Code”) time-based 26/02/13 18:51
Any idea of the problem?
Thanks
Martin Hawksey
Hi – There is an open issue related to this. I hoping Google will have it resolved soon!
Martin
Back that Shiz Up: Download Your Twitter Archive | Hello, my name is Chris Lam.
[…] there is a Chron article and Martin Hawksey’s script code to pull your tweets into a Google Doc! Read carefully and voila – you’ve got yourself a current/updated log of all your […]
Harmen de Vries
When starting “update Archive Now” I’m getting to see (in Dutch) an error message telling me that function getFolders cannot be found in object false. There the script stops. What’s going wrong ?
Mike
How to fix this Twitter Api not configured???
Martin Hawksey
did you get a twitter api key and secret? If you did, did you add them via the Twitter sync menu? If you did, did you open Tools > script editor and run authenticate?
Harmen de Vries
Yes, I took all these steps without getting any error message, everything went smooth as far as these steps are concerned.
Harmen de Vries
Sorry, reaction on the wrong thread …
Martin Hawksey
what happens if you run the authenticate function again? (In particular does a dialog box appear taking you to Twitter to authorise the connection or nothing)
Martin Hawksey
Harmen – I’ve confused myself my responses were supposed to be for Mike. Your problem sounds like all the folders/files aren’t uploaded to Drive properly
Harmen de Vries
I’ll try to fix that Mike, thanks.
Mike
Sorry, I don’t know how to get the key and secret, I stuck there. I followed the instructions on video, by clicking authorize and run, but it appears a box – Twitter Api not configured.
Martin Hawksey
In the spreadsheet menu bar there should be a Sync Twitter Setup menu option. From it click API Authentication for instructions on setting up the API connection
Mike
Thanks a lot! I just realized I haven’t create that things, that’s the reason I can’t authorize the app. Everything is fine and my twitter archive is able to be updated now. Thank you again. =)
Mike
Between, it only updated the latest tweets, how about some old tweets that I have deleted, but it still appear on my archive?
Jennifer Vinopal
Hi, Martin. Great script and easy to set up. Thank you for sharing it. I credit you on my website.
Agora você já pode fazer download de todos os tweets do seu Twitter | Bastidores do #BunkerWeb
[…] do arquivo novamente. Caso você queira ter um arquivo sempre atualizado, a @rosana compartilhou este link que mostra como sincronizar o seu arquivo de tweets com o Google Drive utilizando […]
Adam Clarke
Is it possible to edit the title tags within the index.html & how would I go about using a bespoke address instead of the long googledrive address.
Great script and so easy to set up.
Thanks
Adam
Martin Hawksey
Hi Adam, I’ve done some minor editing of the index.html for my own archive but haven’t dug that deep. I also use a redirect on my webserver http://tweets.hawksey.info (doesn’t cloak the address, I’m sure there are ways of doing it)
The @BAILII Twitter archive now updates automatically on a daily basis | ilegality
[…] this is down to a great Google Apps script created by Martin Hawksey that I came across and some time spent setting it up, getting the app […]
Eric Leamen
Hi Martin,
I’ve tried this a few times, followed step by step and to a T, but every time I get an “oAuth Error” after running the “authorize” script. Any ideas?
jojo
i had the same problem. right click on “My Drive/tweets” folder
click “Details”
then edit the Sharing visibility and change the folder from private to public.
if u succeeded you’ll see a small word beside your folder name “shared”
then its done
drikkes
Running this since weeks and I have to say I’m very satisfied with it. The search is great and really works. But there’s one question I worry about: Will this still works, when Twitter will change to its 1.1 API completely?
Martin Hawksey
Fear not – when I wrote the script I was aware of the impending changes so it entirely uses version 1.1 of the API
drikkes
HurraY! And many thanks.
Patrick Torsney
Double hurray!
Patrick Torsney
I hope this isn’t a cheeky question, but does anyone know how I would go about putting my own Twitter background on this for when people are searching tweets? At the moment it’s pretty uninspiring and blank. Thanks
Martin Hawksey
Easiest way is to change the img/bg.png image. For more tweaking most of the styling is done in the css/application.min.css file. It’s minified so not easy to navigate (sites like http://procssor.com/process can decompress it if you are doing lots of edits)
Patrick Torsney
Phew, you were right about it being minified! Thanks Martin I’ll have a bash and thanks too for keeping up with everyone’s comments, it’s great style. Much appreciated
Facilidade Americana com jeitinho brasileiro @Twitter | M A G Monitoring 4 your GAP
[…] Fonte: YouPix – Jisc Cetis Mashe […]
Mike
Hi, it’s me again! I would like to ask you whether that’s a way to remove DELETED tweets on twitter archive from google drive?
Martin Hawksey
Easiest way is to re-request archive from Twitter and replace the files. The script will just continue updating the data files it has available
Francisco George (@paco229)
Hi Martin,
Since this morning I’m getting this error report. Any advice?
Details:
Inicio Función Mensaje de error Activación Fin
16/03/13 19:51 updateArchive TypeError: No se puede leer la propiedad “id_str” de undefined. (línea 95, archivo “Code”) time-based 16/03/13 19:51
Francisco George (@paco229)
The error code from Appscript has changed 3 hours ago now it states
17/03/13 6:51 updateArchive ReferenceError: “tweet_index” no está definido. (línea 80, archivo “Code”) time-based 17/03/13 6:51
Martin Hawksey
Hmm looks like your archive files have become corrupted (I’m guessing a write process wasn’t completed properly). Easiest solution is to request your archive from Twitter again and replace the files on drive. If you use the same folders setting up the script again is not required
Rodney Gibbs
Thanks! This works like a charm. Here’s an archive of the Texas Tribune, where I work: https://googledrive.com/host/0B0ATO9kgH3hJdjVTWnpER1QzYlE/
Martin Hawksey
Hi Rodney – I know a number of graduate researchers swing by this blog and I’m sure they find it a valuable resource 🙂
Martin
Twitterの自分の過去ツイートをGoogleドライブにバックアップ(&公開)する方法 | きじとら
[…] Hawksey(@mhawksey)さんという方がドンピシャなものを公開してた。 […]
Download de tweets: Twitter, Youpix, Google Drive | Sobre Marketing Digital
[…] fazer sempre o download do arquivo novamente. Mas, se você quiser ter um arquivo sempre atualizado neste link você terá as respostas que procura como como sincronizar o arquivo de tweets com o Google Drive […]
What use is a personal tweet archive ? | Webstory: Peter Webster's blog
[…] so often. And, thanks to @mhawksey, there is a simple way of doing this, using Google Drive. Martin explains all here, with a handy video set-up […]
Lawrence
Hi, Thanks for this wonderful guide. I have some issues though. When I get to the API Configuration step, I am not sure what to put in for the Application Details Page. I see: Name, Description, Website, and Callback URL. What is the website we should set to?
Also will this work with a private twitter account? I don’t want to share my tweets and want my own personal archive.
Thank you!
Mike
Hey, I have downloaded twitter archive several times, it was complete. However, my recent download was incomplete, it started at my latest 3000 tweets. Why it happened to me?
Martin Hawksey
Is this your download from Twitter?
Mike
Yes, it is.
Martin Hawksey
This sounds like a question for Twitter. I’ve no control over what is in your sorehead from them
Renegade Robin2go | Archiving the twitters.
[…] here, Martin has done all the work for you. His blog post explains how the process works. It’s really straightforward, and you can work straight off […]
Chris
I’ve just figured out how to create a twitter archive page hosted by Google Drive thanks to your great article and video. Just a quick question- is it possible to insert/include the actual web page- the one that the twitter archive is hosted on- in a new Google Document or Spreadsheet on Drive? Thanks for any advice!
Seungow
When starting “update Archive Now” I’m getting to see (in Dutch) an error message telling me that function getFolders cannot be found in object false. There the script stops. What’s going wrong?
helen
dear martin
I tried above and seemed to get things working but not viewable. what i mean is that the twitter data was upadating each time I tweeted – the new tweets were supposedly being added but I couldn’t open the archive to see these no matter what I tried.
I reloaded the archive up again and even upgraded my google apps account to make it possible to change settings of folders/files to public (did you know you could not do that on a google apps free edition?). I am now getting the same error as Seungow
Cannot find function getFolders in object false. – despite going through all the steps of the spreadsheet.
If you could shed any light on this I’d be grateful. Not so au fait with scripts unfortunately.
thanks, Helen
helen
please forget my previous problem. everything is working now and i think it’s amazing. have to get my head around how great this is and how much it can help my clients.
you are a genius!
thanks….
dhkeller
This is absolutely one of the most useful tools I use. Thanks!
Has anyone tried using this system to add additional twitter handles to the tweets that are displayed? I currently archive my own tweets, but want to add my family members in one central archive. I may try on my own, but thought I’d see what other people thought of my chances before I destroy my current archive. 😉
Martin Hawksey
From recollection it is feasible but would require rewriting the collection routine. Another way to do it might be to use this with the search operator ‘from:mhawksey OR from:dhkeller’ etc. (without quotes) The display of data however isn’t as pretty and it won’t be a fully archive
Michael
Thanks for the fabulous tool, been using it for the last few months & it was working like a charm..
Now for the past couple of days, I’ve been getting this error emails from Google drive
Your script, Copy of Sync Twitter Archive v1.0, has recently failed to finish successfully. A summary of the failure(s) is shown below. To configure the triggers for this script, or change your setting for receiving future failure notifications, click here.
The script is used by the document Update/Host Twitter Archive with Google Drive.
Details:
Start Function Error Message Trigger End
11/20/13 12:11 AM updateArchive TypeError: Cannot call method “getContentAsString” of undefined. (line 73, file “Code”) time-based 11/20/13 12:11 AM
Sincerely,
Google Apps Script
I went into the spreadsheet & tried to do a manual refresh, it doesn’t work with the same error. I am no expert by any means but it seems either the Twitter API or Google Apps API has changed?
I’m just asking if you have encountered the same error and if you plan to update the sheet to fix it. 🙂
Thanks and keep up the good work.
Martin Hawksey
Hi Michael, I know of one other recent case similar to yours. It looks like a glitch Google’s end. Usually it appears to clear itself and start working again. One thing you can do is search your Google Drive for a file called user_details.js. If this is missing or corrupt the script won’t work. The fix is to get a fresh copy of you archive from twitter and replace the file (it’s in the data/js folder).
Thanks,
Martin
Eric
Heya Martin. Thanks again for this excellent resource. It’s really amazing, and I love being able to host an automatically updated archive of all my tweets.
I’m having the same problem as Michael. I’ve tried replacing user_details.js, as you suggest, but I’m still receiving the same error. Is there something else I can try, or should I just delete everything and start again from step one, hoping that a fresh go will fix this odd and out-of-nowhere problem?
MIkey H
Hey Martin,
This has been working great all year, and about 5 days ago I started to get this error when it tries to update:
‘TypeError: Cannot call method “getContentAsString” of undefined’
I’ve requested a new backup from twitter but it seems to be taking longer than usual. Any ideas what’s going on?
So far I have deleted my old set of files and tired starting from fresh but I still get the same issue:|
any help would be appreciated.
Cheers,
Mikey.
PS. Would there be any way to get it to download any images we post? I understand there might be storage issues after a while, just curious
Iwan
Hi there,
I have the same problem as Mickey and Michael. “TypeError: Kan methode getContentAsString van undefined niet aanroepen. (regel 73, bestand ‘Code’)” Is there a fix yet?
Mikey_H
*UPADTE*
Right, I’ve’ve finally received a new archive from twitter. I deleted everything and started again. Same issue persists. Over 4 weeks now so I’m not sure is it’s a permanent thing with drive or not 🙁
Billy
Hey Everyone having the “TypeError: Cannot find function getContentAsString in object” error, I figured I would put that string in for people searching for this FIX.
First off, thank you Martin for the amazing script and idea, I love my Twitter archive, but just like many people I’ve seen commenting, it hasn’t been working for a little while now. I was having the error absolutely every time, so I decided to try some things out, and I figured out the cause of the problem, and a workaround for now, but I thought I would post things here to see if we could fix it properly.
Around line 70 of Code.gs, there are the json calls to bring in the data from the existing archive, starting with this:
var user_details_file = js.find(“title:user_details.js”)[0];
The issue is, for some reason, the built-in .find() function is not liking the ‘title’ part of the query. If you remove ‘title’ from those three .find functions, and the one further down (around Line 90, just look for ‘find’ and ‘title’ near each other), the queries still complete, and the archive works properly again!
Any idea why Google Scripts isn’t accepting the ‘title:’ argument for a search anymore?
Martin Hawksey
Billy – Big thanks for taking the time to look at this and finding the bug. Seems like this is inconsistent as my archive has been happily updating using the tite: search. I guess it must be the way Google indexes the search in Drive.
I’ve updated the code to try an take account of this (untested). Unfortunately this will only take affect for new copies of the script. Existing users experiencing problems should either take a fresh copy or open the new version then open Tools > Script editor and copy the code across to their own version.
Thanks again,
Martin
Billy
Looks like a good solution to me, but it looks like the .find on line 92 of your Gist is going to toss the same error. Since I’m finally playing around with git again after a little while, I quickly forked your code:
https://gist.github.com/WillPresley/7772938/revisions
Since I was having the issue constantly, I figure I’m a good test-case, and your updated code (with my tiny edit) works perfectly!
Sam
I’d also be very curious about how to get the Twitter images pulled. The self-hosted archive is definitely cool, but as long as the images themselves are still hosted on Twitter rather than in my GoogleDrive, the “archive” is incomplete. Anybody gets it working, let me know! 😀
Mikey_H
@Billy – well done mate. Went in and edited it manually (been years since I’ve gone anywhere near a piece of code) and it worked a treat.
@Martin – I didn’t realise how much I rely on this to search old tweets. A big thank you for spending the time figuring out how to do it. I’ve been looking into ways of getting uploaded images (twitpic/yfrog/twitter etc.) and archiving them. Doesn’t appear to be any where near as simple as I thought it would be. Think I’ll leave it until I have some time, over Xmas maybe.
Anyway thanks again to the both of you.
Mikey
Francisco George (@paco229)
Hi Martin,
After a looooong time working smoothly since 24h I get every hour(my refresh rate) this message from Google:”TypeError: No se puede llamar al método “getContentAsString” de undefined. (línea 73, archivo “Code”)”
Is it a google error once again?
Martin Hawksey
Dang! Here’s a different route https://gist.github.com/mhawksey/7770536 (the change is relatively minor switching from the Google Apps Script DocsList service to the DriveApp Service). Let me know if this works
Putu Ebo
Thanks Martin, it’s works!
Willem Karssenberg™ (@trendmatcher)
@Martin,
Since two days I get an error each from Google that the script fails.
What can I do to repair it?
I really love my archive and use it a lot, so I hope this is temparary?
Hope (and trust:) you can help me out…
Martin Hawksey
Hi, others have reported an issue. I’ve posted a fix in the comment thread
Willem Karssenberg™ (@trendmatcher)
@Martin,
Thank you so much again!
I had overlooked the updated script.
Now works like a charm again.
Cannot tell you how happy I am since I use my archive a lot.
Many people are jealous of what I have!
Of course I always point them to your script.
pakddo
@Billy & Martin
Modified Script works fine. Thank you for your attention.
dhkeller
I have secured a new archive, and am recreating my “updatable” archive using the new code from GitHub. But now I am getting a persistent “Twitter access has not been authenticated” error. This is *after* I had authorized it using the “authorize” function, which did accept a authorize click from Twitter. My consumer key/secret have been triple-checked, but nothing.
But —> Now I see a message from Twitter on its OAuth settings page that says “api.twitter.com now requires SSL/TLS for all connections as of January 14th, 2014. | Read more →”
Red More: https://dev.twitter.com/discussions/24239
Could this be the problem?
Robert Jan
Hi Martin,
Just so you know: the ‘script failure’ is back:
ReferenceError: “tweet_index” is not defined. (line 81, file “Code”)
drikkes
It ran a long time smoothly, but now I got a ReferenceError: “tweet_index” is not defined. (line 80, file “Code”). What can I do about it?
mhawksey
Seeing this error myself – looking into a fix
drikkes
Did you found a solution? I now update the file manual once a week. Doesn’t take much time, but fixing the automated solution would be awesome.
herecomesthesun
first i have to thank about this feature! but i have a trouble. when i click ‘update archive now’, after some secs, google messages “File exceeds the maximum file size.” and i cannot update it. I think this is related with google drive but i don’t know well. how could i fix it? thanks.
Eric
Thanks so much for this. I’ve been using this method since you first posted it. I do have a question though.
When viewing my archive, any tweet made with an avatar other than the most current one shows the avatar as a broken jpg. Looking through your archive, it seems that it doesn’t do this for you, so I’m wondering if there’s any way around it.
It’s not the end of the world, and I obviously don’t care if the CORRECT avatar is shown for older tweets, but I’d love it if my current avatar showed for all the tweets in my archive, not just the newer ones.
Oh, and for the record, this also happens with retweets; if someone has changed their avatar since I retweeted them, the retweet shows a broken jpg as well.
Any help would be appreciated! My archive can be found at http://napor.us/twit
Martin Hawksey
hi – I’m not aware of a way around this. It would be interesting to see how if you re-exported your archive from Twitter if they use the latest avatars
Dr. Damian Bariexca (@_drdamian)
Hi Martin – Tried to comment earlier but not sure if my comment is in moderation or just didn’t go through, so trying again:
My Twitter history goes back to 2007, so the right-hand column now takes up the entire height of a smaller screen (e.g., laptop or netbook). I have text and a ‘follow’ button underneath the list of months/years, but one cannot see these on a smaller screen – scrolling down only scrolls through the tweets, not the side of the page.
Any ideas for a workaround? Here’s a link to my archive in case I didn’t explain it clearly: http://goo.gl/IqDSGN Thanks!
Martin Hawksey
Hi – in your js/applications.js file if you add the line
$('.footer.muted').prependTo('.sidebar-nav');
afterGrailbird.init();
at the end of the file it’ll move the description to the top of the sidebar. To move the follow button up it would be better if you could reference it<center id='follow-me-but'>...
and then you could use another similar line$('#follow-me-but').prependTo('.sidebar-nav');
cesarea
Hello, Martin.
I have been using your twitter backup method. however, my twitter achieve page seems to be an empty space since yesterday.
I checked a logs, the tweets seem to be backed up with no problems. But the archieves are not shown.
I do not fix the problem, though I changed the twitter consumer key and consumer secrets.
Now I also find the same problem your twitter achieve page.
Do you know what’s the problem?
Martin Hawksey
Hi – there appears to be a problem with running JavaScript files on Google Drive https://plus.google.com/app/basic/stream/z134sdepnxaqujxjp04cgbip2tnvcp5jbh4. The word is that this will be fixed. If it’s not back soon I’ll look for a workaround
cesarea
Thank you!
I could fix the problem when i changed the file extension to .js →.txt→.js.
After synchronized my tweets, I checked that the twitter archive is working without a problem.
Martin Hawksey
… appears Google have resolved this issue and Drive is happy running the Javascript files associated with the archive
Jonathan
I apologize in advance for wasting your time with this question. I went through this tutorial and believe I successfully installed the archive on Drive. I forgot to bookmark the URL. I cannot find the archive online. I can find the source files on Drive, but can’t find it “online.” Do you know how I can find the URL without deleting and re-doing the hack? Thank you.
Jonathan
Martin Hawksey
Hi Jonathan – if you can still find the spreadsheet you used to set this up towards the bottom it includes the magic link.
Rebecca Onion
Hello,
First of all: Thank you!! so much for this hack. I have been relying on my archives for 2 twitter accounts for almost two years now and I love having them. This is the first issue I’ve had in all that time, which is pretty remarkable and great.
They’ve failed to update for a week and a half (I have them on autoupdate, daily). Here’s the error I get when I try to manually update:
“DocsList” is not defined.
Any ideas?
Thanks.
Rebecca
Martin Hawksey
Hi Rebecca – Google changed the way you interact with Google Drive. This has been addressed in an update to the code but for existing versions of the template you will need to either setup a fresh copy or open your existing version and then open Tools > Script editor and replace the code with the version here.
Mikey_H
Hi Martin,
Been a while since I’ve had to comment on here but I’ve hit a new issue. Backups stopped working with the following error
‘updateArchive – Authorization is required to perform that action.’
The backup stopped working about a month ago and I’ve tried;
– editing the code with the new bit from Billy
– resetting API keys
– deleting everything and starting from scratch
As soon as i try and authorise i get this error;
‘Twitter Authorisation
Please be aware that if you are logged into multiple Google accounts authentication may fail. You can either log out of other Google accounts or use an incognito window.
Oops something went wrong. Twitter says:{“errors”:[{“code”:32,”message”:”Could not authenticate you.”}]} ‘
I’ve tried setting up a new twitter app but i continue to get the same error… any ideas?
Many thanks
Mike
Martin Hawksey
Hi Mike – in your Twitter app make sure sign in with Twitter is unchecked. Let me know if you are still having problems
Mikey_H
Thanks Martin, it was my mistake not making the drive folder ‘public’ was causing the issue! bit random but somehow it defaulted back to private. All working now.
mura nava
hi there
i am getting this error message from Google:
——-
updateArchive Script is using OAuthConfig, which has been shut down. (line 269, file “Code”)
—–
thanks
mura
thanks
Robert Jan
Hi Martin,
As of this morning July 7th your script stopped working, apperently due to oAuth changes. Error message:
Script is using OAuthConfig, which has been shut down. Learn more at http://goo.gl/IwCSaV (line 277, file “Code”)
The explanation being Greek to me, could you come up with a solution? 🙂
Cheers,
RJ
Martin Hawksey
Google have changed some of their backend. If you took a copy of this template prior to 8th Dec 2014 the script will fail with a
Script is using OAuthConfig, which has been shut down. Learn more at http://goo.gl/IwCSaV (line 277, file "Code")
. To get your archive running again the best solution is to copy the template and setup using your existing Drive folder and API key/secretColin Sullender
I re-copied the template and pointed to my existing Drive folder but when I try to do the Twitter Auth nothing ever populates the pop-up window.
Martin Hawksey
So Sync Twitter Setup > API Athentication = blank popup? What browser do you use?
Colin Sullender
Yup. Using Google Chrome Stable (Mac). Tried it with extensions disabled and still not showing up. Swapped to Safari and it appeared so it appears to be something with Chrome for Mac.
Martin Hawksey
hmm – can’t replicate. When you first used a menu option did you get a ‘Authorisation Required This app needs authorization to run.’ prompt and authentication request? If not I’d try again with a fresh copy. Also check if any popups get blocked
Alan Levine (@cogdog)
Thanks Martin! I saw the notice last night, you are quick. I spent a bit of time turning off updates on a pile of old twitter TAG sheets too,
FYI, the new sheet works with existing authorizations if one is using your other fine sheets, so I did not need to enter the Oauth keys used on the old archive sheet.
This is one of your finest magical bits of internet wizardry. Twitter’s search is like spinning some magic wheel of fortune. I use my archive so often to find archived tweets or conversations.
You deserve an extra G&T for this.
Martin Hawksey
@cogdog Mark Sample (@samplereality) needs to take a bow for alerting me. Thanks for the note on if you are already using TAGS the ‘magic’ follows you .. 🙂
Colin Milligan
Thanks Martin, great to have this, and great to get the update so promptly.
Do you have any idea why the main tweet panel (left hand side, under the date) doesn’t render in Firefox (though not Chrome or IE). I had a look in the comments above, but can’t find anything, so not sure it has been mentioned before. It feels like a CSS issue (ie not your code) but thought I would ask anyway.
Martin Hawksey
Hi Colin – not experiencing an issue when I look at my own archive in FF. If you open http://tweets.hawksey.info do you have the same problem?
Colin Milligan
Hello
Your tweets panel looks the same as mine – missing – so the problem must be with my Firefox. Shall investigate!
thanks again for updating the archive script. All the best
Martin Hawksey
What operating system and version of Firefox do you use?
Colin Milligan
Sorry for the delay – holidays ‘n all.
The good news is that I tracked down the problem to AdBlockPlus (current firefox, windows 7+ updates). Disabling it for that page displays the archive.
Rhodri Marsden
hi Martin,
I’ve found this tool incredibly useful over the past couple of years, but I’ve been having trouble getting it up and running again after the OAuthConfig shutdown. I ended up just starting from scratch with fresh downloads / installs of everything, but when I do a manual “Update Archive Now” I get the following pop-up message:
“No item with the given ID could be found, or you do not have permission to access it.”
Any ideas as to why that might be? Thanks in advance for any thoughts!
R
Martin Hawksey
Entered the correct folder path?
Rhodri Marsden
I’m a massive doofus. I wanged the whole URL in the Archive Path instead of just the folder ID. Apologies. Many thanks!
Dr. Kelvin Thompson
Hi Martin:
Just a little note of relieved appreciation. I used your sheets for ready access to my Twitter archive back in early 2013. In November 2013 my sheets stopped syncing. Figuring that this was due to a change at Twitter, I suspected that I would need to troubleshoot/upgrade my sheets. I had no time to do so and so for a year and a half this stayed a very back burner/back of my mind task.
Today I finally dug back into your blog/comments and was able to successfully restore functionality of my searchable Twitter archive. I am overjoyed!
Thank you again for this wonderful contribution and for your ongoing support!
You are truly an Internet Hero! 🙂
All the best!
Kelvin
Patrick Mackaaij
Google is “Deprecating web hosting support in Google Drive”:
http://googleappsupdates.blogspot.nl/2015/08/deprecating-web-hosting-support-in.html
So I think this solution will stop working August 31, 2016.
Any idea which of the “wide variety of public web content hosting services [that] have emerged” would accommodate updating the Twitter archive like this nice solution does/did? 🙂 Maybe Dropbox indeed?
Martin Hawksey
Good news I think I’ve got a prototype workaround that doesn’t rely on Google Drive web hosting. Extra good news no changes are needed to existing archives, just pointing people to a different url. My archive can now be accessed at https://script.google.com/macros/s/AKfycbwrXr8ejYjHwGEO6kj8f4WHIh096ARDRHdNOgAXPqGltoa80FU/exec?folder_id=0B6GkLMU9sHmLRFk3VGh5Tjc5RzQ (adding your own folder_id to the url should work)
Patrick Mackaaij
Seems to work indeed, thanks! Though Google Drive was still working today as well so we’ll have to see 🙂
My url that forwards is tweets.mackaaij.org
Alan Levine
Thanks for this, I just get the Red Alert from Google (who must be bored and feel the need to yank another service just for grins)– it seems to be working well.
Is there another approach for hosting on Dropbox? It’s mentioned, but not sure how you did that.
Martin Hawksey
Going down the Github Pages route. I’ve a prototype script that is working, just adding some interface so that it’s just a button clicking exercise for the user. Biggest part of the curve will be getting got novices to commit their initial archive.
Nice thing about Github Pages is if have your own donation registered you can add a CNAME to your DNS.
I’ll drop a line in this comment thread when there is something to try
Scott Crevier
Martin, this is great. Thanks so much for sharing with us all. I set it up a few weeks ago and it’s working great. Would you happen to know how to have the date & time display along with each tweet? I realize it would be a change to twitter’s code and not necessarily yours. I just started browsing through the JavaScript code and the solution is not apparent to me.
Martin Hawksey
Update: There is a newer take on this solution which dumps Google Drive web hosting and uses Github Pages instead. You can read more in Keeping your Twitter Archive fresh and freely hosted on Github Pages here https://mashe.hawksey.info/2016/08/keeping-your-twitter-archive-fresh-and-freely-hosted-on-github-pages/
im
I tried this script before and worked for long time until Google changed its authentication method.
Now I’m trying the new one but stuck at Twitter authentication dialogue with same API key and secret. When I click “next”, there’s nothing happened to move on. What’s the possible issue?
Martin Hawksey
Hi – this solution no longer works as Google discontinued Google Drive hosting but there is an updated version here https://mashe.hawksey.info/2016/08/keeping-your-twitter-archive-fresh-and-freely-hosted-on-github-pages/