Twitter Archiving Google Spreadsheet TAGS v5

352

Update: TAGS v6.0 is here! You can jump straight to the new site. If you are using version 5.x here are instructions on how to upgrade (there is an old Sheets version of TAGS which is recommended for use with TAGSExplorer).

For a couple of years now to support my research in Twitter community analysis/visualisation I’ve been developing my Twitter Archiving Google Spreadsheet (TAGS). To allow other to explore the possibilities of data generated by Twitter I’ve released copies of this template to the community.

In September 2012 Twitter announced the release of a new version of their API (the spreadsheet uses this to request data from Twitter). Around the same time Twitter also announced that the old version of their API would be switched off in March 2013. This has required some modification of TAGS to work with the new API. The biggest change for TAGS is that all requests now need authenticated access.

So here it is:

*** Twitter Archive Google Spreadsheet – TAGS v5.1 ***
[If the first link doesn’t work try Opening this Spreadsheet and File > Make a copy]

Important: Uldis Bojars (@CaptSolo) has highlighted an issue with TAGS V5.0 which meant it would not page all the results. This has been fixed with the release of v5.1. If you are running version 5.0 of TAGS it’s highly recommended you update the code.

Note! Search API isn’t 100% reliable:

We find that the search API over-represents the more central users and does not offer an accurate picture of peripheral activity; we also find that the bias is greater for the network of mentions – González-Bailón, Sandra, et al. “Assessing the bias in communication networks sampled from twitter.” Available at SSRN 2185134 (2012).

Instructions for setting up TAGSv5

Instructions are included on the Readme/Settings sheet of the template. If you are having problems it’s worth checking Instructions for Tags v5.0 written by Stacy Blasiola (@Blasiola) or this modified version by Karen Smith & Shanifa Nasser made for Open Data Day Toronto available as CC-BY-SA.

What will happen to my existing TAGS sheets that aren’t version 5.0?

When Twitter turn off the old API (test outages this March) all authenticated and unauthenticated search requests will stop working.

How do I upgrade existing versions of TAGS spreadsheets (v3.x to v4.0) to keep collecting beyond March 2013?

As I can’t push an update to existing copies of TAGS you’ll have to manually update by opening your spreadsheet, then opening Tools > Script editor… and replacing the section of code that starts function getTweets() { and finishes 134 lines later (possiblly with the line function twDate(aDate){ ) with the code here. [And yes I know that’s a pain in the ass but best I could do] … or you can just start a new archive using TAGSv5.0

More additional tips and info when I get a chance

352 Comments



  2. Thanks Martin – good job :-)



  5. Thanks for this Martin. But I’m getting the “Script function onOpen could not be found” error message when I try to use the spreadsheet. I’ve tried making a new copy but with no better results. Any ideas? Should I try again later?


  6. Wait – it is working, but I was pressing the wrong button. I’ve got two buttons – one is labelled Enable Custom Menu – that gave the error. There is a blank button next to it that worked (?!).

    But now I’m getting an authentication error: OAuth Error (line 376, file “Google(c) Code”).


  7. hi

    was trying to use TAGS explorer but getting error message “Error in query: Access denied Access denied”

    thanks


  8. Hi Martin,
    thank you for your tremendous work and the new version of your TAGs sheet!
    Similar to Sharon I also got the “OAuth Error (line 376, file “Google(c) Code”)” error. This error occurs when I try authenticateTwitter in the Script Editor.
    Thanks
    Jonas


    1. OAuth Error (line 376, file “Google(c) Code”)

      Sounds like your key/secret are entered incorrectly or a mistake in the callback url entered in dev.twitter.com


      1. Thanks for your reply! It was indeed the missing callback url. If anyone’s interested, the solution to this error is the callback url “https://spreadsheets.google.com/macros”.
        thanks again!


  9. Like Peelie I get errors with the 2 Web view links at the foot of the dashboard. I’ve also tried entering the URL/key at http://hawksey.info/tagsexplorer/- it just sits there looking at me!


    1. For those views you need to make sure the spreadsheet is File > published to the web


      1. Doh…now I feel really stoopid! Thanks for the speedy reply.


  10. How many Tweets does this archive? I can’t get it to archive past about 80, instead of the 1500 like the last spreadsheet.

    Thanks!


    1. I imagine you are hitting the Twitter Search API 7 days limit (anything older isn’t available). I’ve got v5.0 running on #edcmooc and it’s beginning to creak at 15k tweets


  11. Hi Martin,

    Thanks again for setting this up.

    I am running into issues exporting the spreadsheet. I’ve tried all of the options, but they all return the same error: ERROR: Export to .csv failed.

    Any ideas why this is happening?

    Thanks!
    Stacy


    1. It happens when the number of tweets is greater than 10k, more or less. What you can do is to copy the file, and delete half the tweets in the original, and the other half in the copy, and export both…
      Frederic



  13. Hi,

    Thanks for this. I may be a little confused, but do is the newest version of TAGS (ie v5) sufficient to work or do we have to change the script in like it says?

    Thanks
    Ashley


    1. Hi Ashley – I’ve clarified the text slightly. v5.0 requires no update to the script.


  14. THANKS a bunch for this work! Do you have a location for contributing to the work fund?! I definitely am happy to donate a bit to the cause.


  15. Just want to thank you very much for putting this up and making the public search interface available



  17. Hi Martin,

    I’ve gotten through everything smoothly, but when I try to perform a search, the “Running script collectTweets” box just remains on-screen indefinitely and nothing ever appears in the archive. Any advice on what I’ve done wrong?

    Thanks so much


  18. So, I followed all the steps, set everything up correctly… but I have a question about how much data is pulled. I am noticing an inconsistency in the number of tweets by a particular person via a particular hashtag. Also, the google doc “dashboard” lists a little over 1500 tweets and shows that the first tweet was made 2 days ago (Feb. 28). And the analyzer only shows from March 1st…
    And I just noticed that the google archive is missing a good number of Feb. 28 tweets… (also, is it possible to change the time zone?)

    Anyway, sorry to complain, want to say that even though I’m having problems this is amazing.


    1. The twitter API is limited by the last 1500 tweets and/or 7 days. So if you made a search on the 1st March it’ll pull the last 1500 tweets upto a 7day limit (which ever one it hits first). Do you think that explains your problem? Not sure about timezone, you might be able to go into Tools Script editor and change something in File > Project settings(?) but I’ve low confidence that will work (it may need to be tweaked in the code)
      Martin


      1. Thanks for the response…

        Curious, is there a way I can embed the archive or explored on my webpage?


  19. Dear Martin,

    I was so excited to find this and I have followed the instructions carefully and I am desperate to make use of it and share it with others. Two surprise things:

    1) at min 3.25 in your video the authenticate twitter step – I did that, but I got no popup to authenticate on twitter – I did get the ” the yellow “running function pop up at min 3.28, but no “authorization required at min 3.28 or the subsequent twitter authentication window.

    As a “hack”, I did see on the developer page that I could create my own token right from the Twitter Developers site – “Use the access token string as your “oauth_token” and the access token secret as your “oauth_token_secret” to sign requests with your own Twitter account. Do not share your oauth_token_secret with anyone.” and I Verified the following was in my twitter profile: – Julie Archive by
    A way to archive my tweets
    Permissions: read-only
    Approved: Sunday, March 3, 2013 9:34:02 p.m.

    -If this is not correct this may explain why my spreadsheet isn’t working (see #2 below).

    2. At mins 4.10-4.15 I ran a search for the hashtag #ihateschool – i went to menu TAGS, RUN NOW! and it’s just hanging at “Running a script collectTweets Dismiss”

    And Nothing is going into the archive window.

    Any help you could offer…?
    Thanks,
    Julie


  20. Hi Martin, any idea why the Twitter application returns “Not a valid URL format” when I enter the spreadsheet url set out above?


    1. Hi Martin – First thing I’d check is there is no space before or after the url. If it’s still not working let me know
      Martin


  21. Great resource, thanks!! Do the advanced functions such as get followers work? I did try and it looked like it was working but a new tab was not created with the data.


    1. Hi Mary – the sheets data are written to are hidden be default. Click on the icon bottom left with 4 thick black lines to unhide them
      Martin


  22. I have used your TAGS spreadsheets before. I followed all the directions on your video and have the new Twitter key and secret. When I try to authenticate twitter, it flashes quickly with “running function authenticate twitter” but then nothing happens after that.


  24. First of all, thank you for this tool. It is probably the most useful twitter tool I have ever come across.

    Second, is it possible to use the advanced search operators ‘near:’ and ‘within:’ ?

    I have tried this in several different combinations with single, double and no quotes and cannot seem to get it to work.

    Thank you.


    1. Hi Scott – Unfortunately the Search API doesn’t support near: and within: It is possible to add a geocode parameter manually in the source code. If you open Tools > Script editor around line 120ish you’ll see a number of params are set eg params.count = advParams.count || 100; you can add another line with something like params.geocode = “37.781157,-122.398720,1mi”; setting your own latitude, longitude and radius (see documentation for details)


      1. That worked perfectly, thank you!

        A great tool just got even better.


      2. Can you provide any more detail on this geocode solution. I edited the code as suggested and continually come back with no results. Any help is appreciated.


      3. The geocode filter was exactly what I was looking for. Thanks for such a great tool and for the support :)


  25. Martin, this is a *great* application – as a social media user and researcher it’s great to have a customized search that I can access through Google Drive. However, I just wanted to confirm that the -7 day limit was a Twitter API issue, yes? We’re trying to pull hashtags from catastrophic events (such as the High Park Fire in Colorado, #highparkfire) but it seems that we can’t get those, as they would have been from June 2012.

    We’re able to use the “regular” Twitter search to get access to them (https://twitter.com/search/realtime?q=%23highparkfire&src=typd) but it seems that there is no way to pull these into an Excel sheet or file such as Tag v5.0?


    1. You are correct the api is limited to last 7 days. Twitter recently started rolling out the inclusion of older tweets in it’s search results but this data is currently not accessible to anyone else. Datasift have a a historics sou for older data. The data isn’t free but very cheap


      1. Thanks so much for the reply, and for the great program. I’m having some trouble getting the visualizations to show up in the Google Doc (lots of “#name” errors) but the TAGSVisualizer is pretty slick.

        I’ll follow this project often and recommend it to others – and you’ve provided our lab with an invaluable Tweet archiving tool that we will be sure to credit you with in any publications. =)


  26. Thank you Martin for doing this. Brilliant resource.

    I have tried setting it up twice and when i run it i get the following message “Exceeded maximum execution time” after a while, using different search terms.

    Any ideas?

    Alan


  27. Hi Martin,

    I’ve set up one of these archives and it went fine, but today I’ve had another go with V5.0 and I’m getting an error message.

    After Twitter API Authentication is completed (I made a new app, but I’ve also tried it with the old app), I authorise the API, put in the search term (a hashtag) and click Run Now.

    I then get asked for ‘Twitter API Authentication required’, and a red erro message, “TypeError: Cannot read property “length” from undefined.”

    Any ideas what I’ve done wrong?

    Many thanks, Su


    1. Hi Su – when you run the script authenticateTwitter does it redirect you to Twitter where you approve the connection?


          1. Yes, made a new application as per instructions.


          2. The Twitter API settings have the correct callback url (in the callback field)?


        2. Hi, I am experiencing the same issue. Have been using the tool for about a month now and suddenly it has stopped working and is giving me the error “Twitter API authentication required” — my access token and secret are set, and I have triple-checked the callback URL.

          Then today I have also started receiving the error “TypeError: Cannot read property “length” from undefined.”

          Any thoughts?

          Thanks so much!
          Kelly


          1. Hi Martin,
            Yes all the settings were fine, i went through them again.
            Then this morning I clicked Twitter API Authentication and ‘OK’ again, and chose ‘Run Now!’ and the script just worked. No idea what I did, it just worked fine this time.

            Thanks for your patience!


  28. Yes it did, though I’ve just had another go via the script editor and it just sits there when I ask it to authenticate twitter.


  29. This is happening to me too. I just set it up to try to run it again and I got the same thing: TypeError: Cannot read property “length” from undefined


  30. Hey – All, I got this to work finally! I was having trouble with the “run thing” issue as well – then i just went to the link provided by Martin here – https://docs.google.com/document/d/1N838xH6NeRa6_kZQVkB9tBCliJo_8drvPuM8Ds6tsHg/edit

    And I followed those instructions exactly. I didn’t get the results I was expecting so then I went back to Tools, Script Editor and I ran the collectTweets function and now it is working.



  32. Hi

    A previous version worked for edges and a simple network analysis.
    I tried to fill “edges sheet” but it didn’t work out. I know NodeXL does social network analysis, but it does not update archive.

    Thanks much for your excellent work!


    1. I’m also having some trouble with the “getCommunityGraph” function for filling the edges – it does fill the edges but i can’t make out what exactly is being displayed, whether they are “following” vertices or retweet vertices.
      Sometimes i come up with a “#Value” in the cell, but can’t figure what the error is.
      Didn’t understand how the “getCommunityGraphfromsaveddata” works, as well.

      What I’m actually trying to figure out is the best way to import retweet relations into Gephi, any advice would be much appreciated!

      Besides that, i too must pay homage to the incredible work you’ve done! You rule man! Thank you veeery much!





  36. A big thank you for this great tool. Please, can the archived data be exported into Gephi (or any other network analysis tool) for analysis? Once again, THANKS!



  38. Hi Martin

    I have a problem with authentication, and I can see that is not only my problem. I did everything exactly as described, but when I go to Tools > Script Editor, and run authenticate Twitter, nothing pops up to get me to the Twitter authentication page. Hence, the app does not work. Any idea what is wrong?

    Thanks

    Hossein



  40. Hi Martin,

    firstly thank you for sharing your work, I have been using using the various incarnations of TAGS spreadsheet (v.3 – v5.05) extensively in my day-to-day analysis. Lately, I have been looking at the code closely and have a question. I am currently searching 24 @keywords and limiting the number of tweets too 100. As soon as I add the 25th @keyword I get the “Execution Timeout Exceeded”. Changing number of Tweets doesn’t make a difference. After a bit of research I came across the following link;
    http://stackoverflow.com/questions/7854573/exceeded-maximum-execution-time-in-google-apps-script
    And wanted your advice as to whether in principle/theory it would be possible to modify the collectTweets function to run in a similar way to the runMe() function in the link above. A timed batch process of var SEARCH_TERM and do it in chunks.
    Hope this makes sense, and thanks again for your contribution.


    1. Hi, quick Q are you using a single spreadsheet for you 20+ keywords? If so the issue is probably the url sent to Twitter (which will include your search term) is too long.
      Thanks,
      Martin


      1. Morning,

        Yes, you are correct, the keywords are in a single spreadsheet (below keywords take 4secs to collect&getTweets). Here is my working keywords set, = 439 Characters;

        @Universal_WE OR @mci_group OR @PMMOurWorld OR @Zibrant OR @GPJ_EMEA OR @GeorgePJohnson OR @jackmorton OR @TROexperiential OR @CarlsonWagonlit OR @Banks_Sadler OR @aileen_reuter OR @BCDTravel OR @rpmltd OR @VenuesEventMgt OR @wandoevents OR @logistikgroup OR @cievents OR @FirstProtocol OR @WRGCreative OR @ubm OR @Clarion_Gaming OR @ImaginationLabs OR @aymtm OR @BSIEvents OR @cascade_alan OR @crownlondon OR @drpgroup OR @FirstEventLeeds

        Is there a better way of analysing multiple keywords? Should I look to break this down into multiple TAGS files and try to analyse across multiple files?

        Thanks,
        Michael.


        1. I’m not aware of an easy way to get around the query length other than breaking it up over spreadsheets. It would be possible to add some extra code to do this (unfortunately I don’t have time to do it)


          1. I’ll keep it to Top 20 for now, thanks for getting back to me.

            I monitor some fairly high traffic hashtags that required me to add a couple of functions that deleted tweets based on numRows & numDays. Hope its ok to include them here in case anyone wants to create a new script file;

            function deleteOldDays() {
// THIS FUNCION DELETES ROWS OLDER THAN 7 DAYS
// I SET AS A TIMED TRIGGER > DAILY
var ss = SpreadsheetApp.openById("SPREADSHEET_ID"); // Number between key= & # in current spreadsheet
var sheet = ss.getSheetByName("Archive");
var datarange = sheet.getDataRange();
var lastrow = datarange.getLastRow();
var currentDate = new Date();
var oneweekago = new Date();
// CHANGE THIS NUMBER TO INCREASE OR DECREASE DAYS
oneweekago.setDate(currentDate.getDate() - 7);
for (i=lastrow;i>=2;i--) {
var tempdate = sheet.getRange(i, 1).getValue();
if(tempdate < oneweekago)
{
sheet.deleteRow(i);
}
}
}
function deleteOldRows() {
var ss = SpreadsheetApp.openById("YOUR-SPREADSHEET-ID"); //Remember to replace this with your ID
var sheet = ss.getSheetByName("Archive");
var rowsToKeep = 12000; //CHANGE TO YOUR DESIRED NUMBER OF ROWS TO KEEP. 
var rows = sheet.getLastRow(); 
var numToDelete = rows - rowsToKeep  -1; 
sheet.deleteRows(2, numToDelete); 
}

  41. Hello Martin. Thanks for what you have create.

    I would create two files for different keywords. Do I just need to make two copies in putting different keywords inside it and it works or there is another manipulation? I’m afraid that the two files with the same settings and authentication are not distinct. I need to do two separate searches with different keywords. It is for this that I want two separate files.

    (Sorry for my english, I’m french)

    Thank you !


    1. Hi, you can use the same file and in the search term use OR. For example, cats OR dogs



  43. Hi Martin,

    I’m getting the following error on V5:

    Line 157 Cannot read property “followers_count” from undefined.TypeError

    I am able to test the collection successfully.

    Any ideas why that might be happening?

    Thanks,
    Bill


    1. Is this using a copy of the template or trying to update an existing spreadsheet?


      1. A copy of the template. I created it by clicking on the *** Twitter Archive Google Spreadsheet – TAGS v5.0 *** link.

        Thanks,
        Bill


  44. Hi Martin,

    I’m getting the following error on V5 after running the authenticateTwitter function on the script editor:

    Request failed for returned code 404. Server response: {“errors”:[{“message”:”Sorry, that page does not exist”,”code”:34}]} (línea 376, archivo “Google(c) Code”)

    I have already authorised permission with my twitter account.

    Do you know why that might be happening?

    By the way, I have another problem with the conversation visualization in TAGSExplorer. I have already publish the Google Spreadsheet in the Web but when I try my URL in the TAGSExplorer, the web is loading for ever with the message “Loading data”. I think that there aren’t too many tweets to be loading so slowly… (only 246), so I suppose it may be another thing… Any idea to solve this?

    Thank you very much and good job!

    Victoria


  45. Hi, thank you for TAGS, you may save me weeks of work!
    I’ve followed the instructions (got special menus, authorized the app), but when I click on “run now!” it just says “Running script collectTweets Dismiss” and stays like this, i can still do things in the sheet, but it doesn’t give me any tweet in the archieve.
    What can I do to make it work?


    1. Hi, As far as I can see this is to do with Google infrastructure and have run out of optimisation solutions. For some reason on the initial collection I find better performance if I open tools > script editor and then Run > collectTweets. Once an archive is running they generally update without any fuss. Seems to be an issue around when the spreadsheet is first copied.


      1. thank you, I just opened the script editor and ran the collectTweets from there, a popup asked the app authorization and after it started working normally


  47. Hi there, can I check if this only pulls in organic tweets and not promoted tweets?




  50. Great job, @Martin and thank you for sharing it with the world!

    Couldn’t find an answer to this, so asking here- does TAGS support a combination of logical operators for the search term. Specifically, I am looking for something like: a AND (x OR y OR z).

    Cheers,

    Ned


    1. Hi Nedyalko – yes you can use combinations of operators. One thing to watch is your query string doesn’t get too long (the maximum is 1,000 characters – this includes some parameters sent by the script so it’s a little less than this)
      Martin



  52. I use your script, but I wanted to create new one and it seems that something is wrong. I have only “running script” but no data is pulled from TW. Do you have any idea?


      1. I’m having problems with this, too. I’ve authenticated several times, but cannot get the custom menu to install. It doesn’t matter how many times I click on the button, give authorization, and click “accept”, it does exactly the same thing, and I just end up in a loop. Baffled.


          1. I had the same problem with the message “running script” and no data pulled from the spreadsheet. In order to solve it I tried to reload the spreadsheet, then went to Script Editor and authenticated again. Then closed the editor and from the spreadsheet I re-ran the script to get the data, and this time it worked like a charm!





  56. Thanks for this awesome tools! :D
    Tweet mining is so cool too…


  57. Hey Martin,

    I just have a question about rate limitations. I know that it is possible to search up to 15,000 tweets or tweets from the past 7 days (which ever limitation is reached first), but would you be able to tell me if any further rate limitations exist specific to this software? For example, I know that I was having a problem at one point where I made too many searches in one 24-hour period, so my searches were unable to run any further. Is there an exact rate limitation in this regard? Maybe the 15K/7-day rate limitation applies in this scenario? Thank you!

    All the Best,
    Brendan


    1. Hi Brendan – the api docs state 180 calls per hour. The script fetches 100 tweets per call. TAGS Advanced menu has a ‘Test Rate’ menu option for you to monitor remain calls


      1. Ah, perfect! Exactly what I was looking for; thank you!


  58. Hi Martin – I’ve been experimenting with the TAGS v5.0 script to explore how to archive our #cdnedchat sessions. I have had trouble getting the archive to show times during the turnover from end-of-day to next (23:59:00 June 3rd to 00:59:00 June 4th) in the search.

    I’ve tried changing the settings up (2 days, 1 day, 3 days) but that particular hour is just not showing up in the search. As it happens, that’s the exact hour when our chat happens online.

    Any way for me to fix this? What do I need to adjust in the readme/settings?

    Thanks!


    1. Okay, I went back to Tools, Script Editor and I ran the collectTweets function and now it is working – all tweets are showing up. Just have to figure out how to get the analytics to work with the “paged” archives. Slowly but surely! Thanks for such an amazing resource!


      1. Hi Tanis – glad you got it working. The paged collection is a feature from the very early versions of this template and I haven’t really been looking after it. My recommendation would be to use the single archive version. If you want to give users easy navigation per chat the MLAA developer TAG Viewer is a great option. Here’s an example on how the TAGS sheet for #cdnedchat is rendered. Alternatively there is this searchable interface I’ve created

        Martin


        1. Brilliant! Thanks for the suggestions regarding archiving multiple chats in one archive – love these resources you have created!



  60. Thanks for this great tool. Really a brilliant thing. I also got a question. I just ran a sweep but was wondering why it doesn’t seem to go very far back in time. Is there a way to make it sweep tweets from the past few days? Right now it only goes back for about a couple of hours even though I know that there is much more. Any ideas?
    Thanks again.
    Micha



  63. Since yesterday the script is not working. There is a problem with the variable followers, that I am suppose that is due to the retirement of the API v1. I have been using your script for several months now, and I wanna thank you for that.


  64. Martin this is the error that I get:

    Apps Script

    Line 157 Cannot read property “followers_count” from undefined.TypeError

    I tried to set up a new gsheet but I get the same error. It was reported that twitter has made some changes on how to get the followers, but I didn’t have so far the opportunity to search for more details.

    Thank you again, for your work.


  65. Thanks for Tagsv5.0! It is of immense importance to me. I am PhD at the Wageningen University & Research centre (the Netherlands) and I am doing a research on the influence of twitter-acitivty on the governance of sustainability in agro-food systems. I’ve made five documents for archiving tags, but now I encounter two problems: 1) only my first two are working (1 shared, 1 private), the others do not archive data, and 2) Tagsexplorer isn’t working for any: it says ‘loading data’ (for 8 hours).. I hope you can give me any direction for solving these problems. Keep up the great work!


  66. I am facing same error as M.M. Script is not working. Plus, I get a redbox saying something about the length property.

    Thanks for your work


  67. Thanks for this great tool. I am a PhD researcher and this is of great value to me!

    If you have problems with TAGSExplorer: it might be because you’re using Internet Explorer, try Google Chrome instead. It worked for me :)

    I can’t make new archives. I’ve two running properly, collecting about 40 tweets a day for a few weeks now. When I make a new one and click ‘run now’, it says

    Apps script:
    Twitter API Configuration Required
    And then, when I close that window, it says in red:
    TypeError: Cannot read property “length” from undefined

    I’m sure I’ve followed all steps, including the Twitter API authentication procedure (you can use your key and secret multiple times right?) and authenticate Twitter via Tools, script editor (you need to do that for all docs?). I haven’t archived many data yet, so I guess it’s not the amount of data I am pulling in. Is there a limitation for the amount of archives to have running, or the times you do the Twitter API authorization configuration, or just for the amount of data pulling in?

    I hope someone can help me with this somehow. Many many thanks


    1. Problem Solved: When I did the Twitter API authentication via Tools > script editor > run > authenticate twitter it worked. I guess I did something different before..


      1. Thanks! You saved me a good amount of head scratching!

        This tool is wonderful! I’ve popped it on my blog.


  68. Any suggestions for getting historical data in the same format? I need very little and basic data starting from November 2011 with the same search term as I used with TAGS. Anybody already done that? I would love to hear any suggestion. Thanks a lot.


  69. Hi, I’ve just made a copy of version 5.0, but when I click on the ‘enable custom menu’ button a window opens and says, “Oops, Script function onOpen could not be found”

    Is this a problem with the software or is it my google drive causing it to do that?


    1. Hi Andy,

      Google sometimes needs a couple of seconds to catch up with the template copy to attach the script. The advice I’ve seen elsewhere is to keep hitting the damn button until t works ;)



  71. Thank you very much for this! I hope this is not a silly question but only the first search term I enter shows up in the archive. Is it possible to enter more than one search term or do I have to do separate sheets for them? Also, my understanding is that Twitter allows for searching further back than 7 days. Is this a possibility here? Thank you very much again!


  72. Hi Martin,
    Is there an option for language filter in the archive? I’ve notice the French are using the same hashtag as us. If I could filter out the french language tweets…


    1. Hi Su – requires some code tweaking. In the Tools > Script Editor change the lines around number 87 from

      if (sinceid){
      var data = getTweets(SEARCH_TERM, {"sinceid": sinceid}); // get results from twitter sinceid
      } else {
      var data = getTweets(SEARCH_TERM); // get results from twitter
      }

      to

      if (sinceid){
      var data = getTweets(SEARCH_TERM, {"sinceid": sinceid, "lang": "en"}); // get results from twitter sinceid
      } else {
      var data = getTweets(SEARCH_TERM, {"lang": "en"}); // get results from twitter
      }


      1. Hi Martin, that’s great – thanks for letting me have the code. I’ve made the changes and can’t see any more French tweets. I’ve deleted the ones from the weekend.

        I’m liveblogging the Government’s Construction Summit tomorrow – will be also promoting TagsExplorer and TagsArchive to help non-tweeters see what’s going on.




  75. Hi, very awesome script and interface! I’ve ran into a problem. I set up a couple of scrapes for my thesis research. All working fine, but one isn’t working and states dates from 30/12/1899 till 30/12/1899. What could this be?


  76. Hello Martin, when i run the time-driven triggers to collect tweets .. is there a way to filter the new ones from the old ones ? Basically .. i want to be able to capture all tweets on a file between 1pm – 3pm and another separate file from 3pm – 5pm and a separate file from 5pm – 10pm & so on and so forth . .. is this possible to do ? Or does TAGS just update the tweets to the spreadsheet ?

    ALSO, is there a limit to how many tweets it looks for ?

    Lastly, on the spreadsheet .. the times it says the tweets have been posted are incorrect as to when i open my tweet deck. How do i change the time zone so i see the correct time same as i see on tweet deck ??

    The time is off by like 7 hours or something like that ..

    Thanks,
    Bill


      1. This is very helpful!

        One question; how can I change the time zone so the times are accurate for EDT (the time for New York City)?

        Perhaps an ideal would be if there were a way to have the date in one column (in US format, which would be 7/12/13 for July 12, 2013) and the time (in EDT) in another column? Would this be something easy to adjust?

        Thanks for this wonderful script!


        1. As a couple of people have asked. One way to get different timezone date display is in the Spreadsheet go to File > Spreadsheet settings and select date locale and timezone and then in Tools > Script editor open File > Project properties and change the timezone there as well. This should convert any existing and all new date/time data in the time column. The created_at column is unchanged as this is the metadata passed back from Twitter and required for tools that visualise a TAGS archive





  80. Hi Martin,

    Thanks for the amazing tool. I have one question though. How can I collect tweets based on geolocation? I want to collect some data from Bucharest and Brasov, Romania, but not by using hashtags. Is it something that I need to modify?

    Thanks,
    Cristina




  83. Brilliant tool – thank you, Martin. I had been using “The Archivist” to archive #globalclassroom twitter chats but, now that it’s subscription only, your script is the perfect replacement!


  84. Fantastic tool, thanks Martin! I’m trying to figure out a way to strip line breaks out of the tweets when they are imported to the Google Docs spreadsheet.

    I’m exporting this data via a web query to Excel and line breaks in Tweets cause the text to broken in to separate cells.


      1. Thanks for your guidance Martin. I have managed to strip line breaks from the Tweet text by adding some code to the TAGS.gs script. Around line 187 (after //end of while loop), at the end of the gettweets function I added the following code:

        // remove line breaks

        for (var i = 0; i < data.length; i++) {
        data[i]["text"] = data[i]["text"].replace(/\n/g, " ");
        }

        Perhaps this might be a useful option to include in future releases?


        1. Thanks for this. Given the workflow Richard has highlighted this makes sense as an option.


  85. Hi Martin,

    This question has been asked before by Bill but I dont think found the answer to it. Is there a way just to scrape the tweets in a particular time frame. That is for instance from last Sunday 10pm to 11pm EST US time or Monday 2am to 3am UTC? When I started my search last night at 11.20pm I only was able to get the last 5 minutes that is from 10.55pm and a lot of “clutter” that followed after. Now I wonder if there is still a way for me to get the tweets from yesterday 10pm to 11pm? Your help is greatly appreciated.
    Thanks.
    Micha


    1. Hi Micha – unfortunately not down to the hour the best the Twitter API can give is a day range (from last 7 days). On the readme/settings sheet there is an option to get yesterday, -2 days, -3 days etc. To use this you need to make sure there aren’t newer tweets in the archive than the date range you are trying to get. e.g. if you change the setting to yesterday and you’ve already got tweets from today in the archive when you hit TAGS > Run Now! it won’t insert the older tweets.


  86. I have followed the directions to the T and still can’t seem to bring any tweets back. On the Dashboard it says “Cannot read property ‘title’ of undefined”. I’m not sure what that means. I would really love to use.

    Let me know if I’m missing something.

    Thanks!



  88. I have read your Twitter Archive post on moz.com, I admire your post and happy to read and implement in my twitter strategy. I hope you do more like this.I know that you put the effort on this. We are happy to use this sheet :)


  89. Thanks for making this tool publicly available, it’s much appreciated.

    I am however having trouble getting everything to work correctly. I’ve followed the setup instructions yet each query I try to run gives me a “exceeded maximum execution time” error.

    I’ve authenticated twitter from the Script Editor and plugged in my Twitter Oath Key & Secret.

    Am I missing something? Any help you can provide would be wonderful. Thanks!




  92. Thanks for this great tool, Martin. I haven’t figured out how to get anything in the “geo_coordinates” column. Is there a trick here? I’ve turned on locations for my tweets, but that doesn’t seem to populate that column. Any advice?


  93. Thanks for this awesome tool, Martin! Do you know a possibility to get older tweets, like from march? Operators ‘since’and ‘until’ don’t work, and I know there’s the time period of 7 days but is it possible to search further back? Thank you so much!





  97. Here is another comment in case you want to use TagsExplorer with your own spreadsheet. I tried to instructions but I was unable to see the visualization (spreadsheet was not loaded) until I published it to the web (even if it was shared). I think this step should be added to the list of steps. Thanks, this is an awesome tool.


  98. Hi, Martin

    Thanks for this great tool. I use SNA in my PhD project and, when it comes to extract data from Twitter, NodeXL seemed to do the trick… until the moment I found out about TAGS magic. So, I took it for a test drive, but the SNA script is not working for me. Every time I try to run some functions of TAGSAdv.ps (from the script editor) I get an error:
    1- GetCommunityGraph: “TypeError: Cannot read property “length” from undefined. (line 43, file “TAGSAdv”)”
    2- replaceIDsWithNames: “The coordinates or dimensions of the range are invalid. (line 98, file “TAGSAdv”)”
    3- GetCommunityGraphFromSavedData: The script runs but nothing happens!
    4- displayIDSinCell: “Invalid JSON string. (line 150, file “TAGSAdv”)”
    Everything else seems to be working fine (and I could export the relevant data to UCINET, for example), but without the SNA dimension TAGS loses its allure to me…

    Keep up the good work and thanks for your help!


  99. Martin,

    We’re having an odd problem on one of our TAGS sheets. We haven’t received any updated since Tuesday night, including a few direct replies which are usually very reliably caught.

    I have tried running again manually, reauthenticating, etc. but no error is showing and no additional tweets are being collected.

    Have there been any issues with the API/changes made since the conversation tweak Twitter made this week? Any/all advice very much appreciated ;)

    Thanks, Nicola.


    1. Hi Nicola – when you look in dev.twitter.com is your app still okay? Things I would try: Delete last tweet row in archive and try again; another thing you could try is open Tools > Script editor, Run > collectTweets and then View > Logs


      1. Thanks Martin.

        So, no indication of issues in dev.twitter.com. App still seems fine.

        Deleting the last line and rerunning resulted in that line being reinstated. But none of the more recent tweets being added. The missing tweets are both from our account and replying to it and would normally be caught by the TAGS sheet (see anything post 27th Aug here: https://twitter.com/search?q=digimap4schools&src=typd).

        Running collectTweets from the Script editor appears to be fine and the Log is blank.

        Sorry, this seems to be one stubborn issue :-O

        – Nicola.


          1. Ah, thanks for this. I shall have a full read. So I take it that TAGS uses the Search rather than the Streaming API? That’s useful to know anyway ;)

            In that case I think we’ll probably handle the omitted tweets as we have a couple of times before – by manually adding in the key date/tweet ID/content/user info etc.

            Huge thanks for all your help! :)

            – Nicola.


  100. Hi again, Martin

    Sorry for insisting, but the problems persist (errors while running some functions) and I cannot render any SNA related data. I have a complete verteces sheet, but blank edges and no metrics. What am I doing wrong here?

    Thanks!






  105. I absolutely love TAGS and have been using it over the past 18 months to scrape many of the hashtags for a local public television station. Our budget is limited and your software has become an amazing asset to keeping up with the times.

    We are running into a recent problem every since the latest update of TAGS back in the spring. We are now experiencing duplicates in our archive. The archive can have tweets from say 9-17-2013 to 9-10-2013 and then it will start back at 9-17-2013 again. What would be the cause of this hiccup within the archive?

    Thanks,
    Reggi


    1. Hi – I’ve also noticed recently problems with duplicates with the new API. Technically this shouldn’t happen, but I wonder if as part of the new API they’ve gone for speed over accuracy. In newer versions of TAGS there is a delete duplicate option from the TAGS menu to help clean up the archive


      1. Hi Martin,
        just wanted to chime in, because this option doesn’t work in my archive.
        The error message I get (just deleted the whole archive and created a new one with the same result) is: TypeError: Cannot find function trim in object 380014964331007900
        Do you have an idea how to fix that?
        Thanks!



  107. This is brilliant! Thank you! Any chance you will be doing one for Face Book?

    Thanks again!




  110. Hi Martin,

    I’ve ran into a strange thing in the tool. I want to collect all tweets to and from ‘@Connexxion’. To:Connexxion is working perfectly, but from:Connexxion gives me “Twitter said: null”. Actually, every day there are several tweets from this account so this shouldn’t be a problem.

    Thanks in advance!


    1. I have the same problem. I have created 5 spreadsheets now, all with this same result, creating them as I always have.


    2. I’m having the same problem. I want to collects tweets from @IAM_SHAKESPEARE and keep getting “null” to any from:IAM_SHAKESPEARE requests

      How can I fix this?



  112. I seem to get this with every spreadsheet copy I make now…

    Cannot read property ‘title’ of undefined

    – Twitter app is set up
    – App consumer and secret are set up properly
    – Ran twitter authenticate (And I dot the popup and agreed)


      1. all it is is “#ORUHangout”


  113. Also, thank you so much for your effort in this. I REALLY appreciate you taking the time to create and maintain this as well as engage with the users. You rock.


  114. Hi Martin,

    Thank you so much for your great app. It helps me a lot in my research. :-)

    However, could you please tell me how can I change the time (actually i am in France)?

    Thank you again,



  116. Hi Martin, this looks a great tool, but I have filed to get it working. I get an error in the summary tab, in the right box, highlighted in red “sc(..)is undefined”
    I have added the authentication codes form Twitter and run the authenticate script, but was not sure if I needed to use the access tokens then offered by twitter. Do I need to use them in some way?
    Thanks
    asher


  117. Hi,
    I have been trying unsuccessfully to get TAGS 5.0 to work on and off for about 48 hours now, even going so far as to make new copies, refresh existing sheets, repeat authentication, different twitter authentication keys/accounts and also use basic search terms (e.g. ‘food’) and small quota requests.

    But each time, even if I just try to do a ‘Test Collection’, the TAGS sheet hangs at the notification box ‘Running script. Dismiss’.

    My TAGS 5.0: https://docs.google.com/spreadsheet/ccc?key=0Al135Uzr-xDZdHdLbHRsWXVDTFhEZjN5ejZCUDd2UUE&usp=sharing

    Can anyone let me know if their TAGS 5.0 has worked in the last 7 days? Just so I know where I’m going wrong.

    Thanks,
    Sam


  118. Hi Sam, sounds like the same problem I had. I too tried multiple installs/copies with fresh authenticaitons
    Martin, sorry if I /we are making an obvious mistake.
    asher


  119. Hi Martin,

    Great Sheet! Excellent work!

    I’m trying to look for search terms that have a single quote-mark in them. e.g. here’s the set of terms I want to search and archive

    (#DevilsDue OR “Devil’s Due”)

    The scripts never complete running with this query.

    When I search for #DevilsDue I get resuts.

    When I search for Devil’s Due I get no results, so I assume the ‘ is the culprit.

    Any ideas what may be gong on here?

    Thanks again – great work!
    Nogoyo


  120. Leaving the script untended overnight I get “Exceeded maximum execution time” error, but no tweets.

    asher


      1. Hi,
        Asher, Bill – thank-you so much for pursuing this. By your joint help, I managed to get the script working (by watching Bill’s very useful video, I found I had forgotten to add the Callback URL).

        Everything is working perfectly now, and like Asher, I want to thank-you Martin for creating this script, and the supportive community around it – really excellent work. And thanks Bill for your great additional resource.

        Best,
        Sam


  121. @Bill, thanks, your tutorial really helped. I now have a working copy, I had missed an element of the authentication. Your video is a great support, much appreciated.

    asher


  122. @ Martin.
    Martin, thank you for the work you have put into this fabulous tool. I have been trying to find an effective way for gathering tweets for quite some while. Your tool is exactly what I wanted. I also love the community of support you have around it. Nice one!

    asher



  124. Excellent! @Martin – Thank you very much for the tool. This is very helpful for my research.

    (PS. One observation – There are numerous duplicate tweets in the Archive sheet. I hope this will get sorted out soon.)




  127. I’ve used these TAGS scripts since V3, and jolly good they are too. Here’s a puzzle though, I set up one to collect a hashtag, #cilip2014. I know there’s a seven day limit imposed by Twitter, but for some reason it doesn’t collect anything earlier than one time stamped Thu Oct 24 18:37:17 +0000 2013….this was actually half way through the event whose tweets I was trying to archive, so I know there are earlier ones that day. I’ve tried tweaking the collect date from default to -7 days, and running again, in case that would force it to find the earlier tweets, but still it fails


      1. Martin – thanks for TAGS – it is brilliant, but I have hit a snag. It has been collecting #rwpchat since December (and doing the job beautifully), but this week has not collected new tweets since Monday (and there are a couple of hundred new tweets). I keep getting the message “Exceeded maximum execution time”. Any suggestions about how I can get around this? Thanks in advance.
        Ellen


        1. Hi Ellen – I recently had similar problems and made a minor adjustment to the code. You should try opening Tools > Script editor.. and replace the code on the first page with the code here https://gist.github.com/mhawksey/7230553 (New copies of the template include this modification)


  128. Thank you for this fabulous tool. My only problem is re-using one spreadsheet for more than one keyword. The first time I set up a sheet with a new keyword, everything is good and the archive etc populates perfectly. However I am very excited about its potential and wish to run lots of different queries!

    If I then try to do a new keyword by overwriting the text in the readme/settings page, and again press Run, it says the query is running but the data in archive and the other sheets does not update.

    Do I need to create a new spreadsheet for every new query, going through the setting up tags dropdown and authenticate process from scratch? Or is there some way of completely refreshing the data?


    1. Hi yes currently you need to setup a new spreadsheet for each query. When the script runs it looks for the last tweet in the archive and uses this to only get newer results so if you change the query it will only collect tweets newer that the tweet from the old query (if you see what I mean). It’s not ideal but I’m working on an improved solution






  133. Stunning good app Martin – many thanks for creating it.

    It is all working perfect apart from one thing – when I publsi h the Dashboard to the web I get a Cannot read property ‘title’ of undefined error on the Top Tweeters’ graph (although I can see it on the GDrive instance).

    Many thanks in advance.

    John


    1. Hi John – Found similar myself. If you go into the chart edit mode and then come of it it seems to fix it (guessing bug)


  134. Hi! I’ve been trying to set up, but Twitter authentification fails (Url Invalid) using the Callback URL provided. Any solutions about this?
    Thanks!



  136. Hi Martin,
    Awesome tool and very helpful in my research. Have a query if you could resolve…
    Google spreadsheet reaches its size limit frequently and need to be manually emptied out and download every few hrs. Any better way for long-term tasks?


  137. I have used TAGS v5 and am very happy with it. Keep us updated should there be new developpments





  141. Hi Martin,

    I’m interested to map tweets so I’m wondering if it’s possible to collect only the geocoded tweets.

    thank you.


      1. I did it before but it shows all the tweets within the radius, the geotagged ones and the non geotagged ones but having a location in the profile within the radius (the ones that the geo_coordinates cell appears empty).

        e.g. I’m looking for tweets from Barcelona area (41.40,2.15,100km). There are Indonesian Barcelona supporters using Barcelona (or even Camp Nou) as a location. Even they’re tweeting from Indonesia their tweets appear in my search.

        So I’m only interested in the 1st ones, I don’t know if it’s possible though.







  147. Can some one please help to access Tweets from R software



  149. I am seeing the following error;
    collectTweets TypeError: Cannot call method “getSheetByName” of null. (line 62, file “TAGS”)

    How can I remedy this?











  159. Error: “Twitter said: null”

    Thanks for writing this script! I’m getting the above message when doing a test collection. If I run the script with a search term, I simply get a sheet with no results. Any ideas?

    I’m hoping to use the script to compile a list of twitter users (say, bloggers with an interest in pottery). I’m hoping that using the search term “blogger AND pottery” would return the list I see in Twitter’s search.


    1. Had the same problem, here’s what worked for me:

      1. Created the spreadsheet again (clicked on the second link given above, then File> “Save Copy”

      2. Followed instructions 1 and 2

      3. Clicked on Tools > Script Editor > Run > Authenticate Twitter. A Window opened “You are about to authenticate your first App” – clicked on button (this last third step I had done before, but the window hadn’t opened – may I hadn’t been logged in to Twitter)




  162. There’s one thing missing from all instructions.

    I set up an instance of TAGS 5.1, followed all instructions, set up a search for a hashtag and did a one-time run, and viola! I got a bunch of tweets. Fantastic!

    The next day, I wanted to run it again and… wasn’t sure exactly what I needed to do. I just went to the TAGS menu and clicked the option to RUN. The script ran, but I got no tweets. Then I tried doing various steps over again, but never got anything else to run. I got no error messages, but just never got anything in the archive.

    So which, if any, of the steps in the setup are required to do again before another run?

    If there are no steps that need to be duplicated, I suspect that maybe my problem is related to the fact that I cut all the data out of the archive tab and moved it somewhere else. I assumed that TAGS could run again and just repopulate the blank archive tab, but one of Martin’s posts suggested that it looks for the last tweet to figure out where to restart. I wonder if there is a way to remedy that? Like maybe, copy all the data out, but leave the last tweet?

    -Michael






  167. Hi Martin,

    Did you remove the ability to archive favorited sheets in 5.1? I’m not seeing it, unless I’m missing something.

    I’ve been running 5.0 since you added that functionality. I don’t want to lose it, as it is crucial for me.

    Hoping I didn’t overlook something in the comments addressing this issue.

    I’ve also noticed in 5.0 (and IFTTT when I used to use it) that if I retweet a tweet, and then favorite an earlier tweet, 5.0 doesn’t pull the favorited tweet. It’s like the retweet creates a time barrier. Maybe just a quirk of Twitter’s API? Anyway, I thought I’d raise it.

    Love the sheet. Hoping I can upgrade. And would like to spread the word to a class I’m teaching, but favorites are key to that.

    Thanks!


      1. Thanks! Baking it into 6 would be great. I’m trying to get some Columbia University students using it in a media innovation class I’m helping lead, based around the favoriting functionality. And, losing that would impact how I’m tracking projects online.




  170. Hi, I am new to this awesome API and am doing a cool project here at the university. Question. is there a limit to the number of handles I can list in a query? I have a list of about 100 twitter individuals that I would love to follow and download and study. Thank you for your help and advice.


    1. yes there is a limit to the query length (I can’t remember exactly but the answer is in this thread somewhere)





  174. Hi Martin,

    I responded to another partially-relevant comment elsewhere, but thought it might be too buried for you to see.

    First off, thanks so much for this tool!

    I’m trying to collect only those tweets that have geocodes. I have found elsewhere how to collect only those tweets associated with locations within a predetermined distance from a coordinate point (params.geocode = “-x,y,zmi”). But I need to specifically filter for tweets with longitudes and latitudes. How would I go about filtering specifically for them?

    Thanks again for your hard work and for any advice you can provide.


    1. Doesn’t appear to be anything in the API to prevent place association. Is the data good enough for post processing?


      1. Yes it is and I have already written in a place association. What I am trying to do now is filter for only those tweets with geo.coordinates[0] and geo.coordinates[1] data. Do you have any ideas on how to do this?





  178. Hi Martin,
    thank you for this great tool!
    Is there any possibility to check the connections between the archived users?
    I would like to know how many people from the list (sheet: summary) a user is following.

    I didn’t find a solution during my web-research, but I think it might be possible through this api: https://dev.twitter.com/docs/api/1.1/get/friendships/show

    Do you have any idea how to solve it? (sorry, i’m no coder – so i’m not aware if it’s a big deal.)



  180. Thank you for sharing this, I’m VERY grateful for your work! Right now, I’m getting an error on the sheets I’m using saying “TypeError: Cannot call method “getSheetByName” of null. (line 62, file “TAGS”)” I’m not sure what that means and how to fix it. I’m not very well versed in this, but have found this to be INCREDIBLY helpful when it was working, now I’m having to do it by hand, because of these errors. Fixing this would save me a LOT of time. Thank you in advance.






  185. How i can make this in mysql and php? because google drive work very slow for me when i try to search with this script, can anyone help me?

    Regards.


      1. I test it but its oriented to self search, not for tracking hashtag or external data, like your script :(


        1. Martin, do you know another system like this? like yours but mysql php?


          2. Ill gonna tested, let u know! thanks m8 :)


      1. Already test dmi-tcat and need more details than that, any other idea? i see u copy link twice is that correct?


  186. I love this sheet and you for creating it. SO helpful.
    I’ve had great luck with it so far, but for some reason today it’s only returning 100 Tweets when I search for a specific term (a hashtag that I know has more than 100 tweets). Do you know why it’s limiting the archive tweets to only 100? I left it at the default 1500 results on the settings page.
    Thank you for your help!


      1. Yup – 5.1. Thanks!


  187. fantastic sheet, thanks so much for creating it. I managed to fill it up rather quickly (gluttonous clearly) – google have announced larger spreadsheets are now available but I wasn’t sure how to transfer the data into a new sheet?
    again, very grateful for you work!


  188. Still getting only 100 results. Anyone else have this problem?


    1. Also only getting 100 results, even on lists of accounts that have previously generated 1000+. No clues as to why, unfortunately.


    2. Yup! same problem here. Did you find a way to fix your problem? We made two archives work in february, but this week, using v51, we’re only able to collect 100 tweets in retrospect – and after that, it’s only adding newer tweets in hourly updates. Tried paged searches for e.g. -7d, -6d, and older data are positively retrievable. But still, no initial or continuous inclusion of those. Strange.


      1. Sadly, I never heard back and had to go with a paid version.







  194. Hi Martin,

    thank you very much for sharing your work, works fine for me so far. I was wondering if it is possible to set the starting point for the data collection even below -1 day. That would be a great help for me dealing with discussions, which generate thousands of tweets a day. The perfect scenario would be that the run now command just starts gathering data from the moment of its first execution. Is that somehow possible?

    Thanks
    Cornelius



  196. I have archived six different documents on the same hash tag, is it possible to merge them without losing the metrics such as number of unique users etc


  197. Dear Martin,

    Is there any way to retrieve the number of times a tweet has been favorited. I would like to retrieve tweets, and 1/2 days later, using just its id, retrieve the number of times (and by who/date-hour) it has been favorited during these 2 days.

    Thanks again for your great tool and help!

    Frederic


  198. Hi Martin,

    Thanks for setting this up!

    I am currently having a few technical difficulties. First, I am getting a error message when clicking on the web views link. I did file>publish to web and I am still getting the error message.

    I am also having a problem using the mozcontemplate – http://builtvisible.com/wp-content/uploads/2013/07/template.xlsx (SEO Tools is installed)
    Once I open the template and enable editing I get 5 different error messages.
    Error Messages received:
    1.Unshort URL error
    2: Twitter count error
    3.Google Plus Utils error
    4. Xpath On URL error

    Once I clear the errors messages, I import data from web (using the link in the tags google doc-file>publish to web section). The data is populated in the twitter archive sheet. The data in the charts sheet do not refresh and the analysis sheet once information is dragged down, everything to the right of the extracted URL does not populate.

    I am at a lost and looking forward to your advise.

    Thanks,

    Karla



  201. Many thanks for this Martin, It is really simple to follow and I am hoping to use it for a conference this week. However I am having a small issue with the fact it will collect the #enconf14 but will not pick up @enconf if I put them in the box as

    #enconf14 AND from:@enconf

    I realise you are not a Q&A service! But could you let me know if I am doing something obviously wrong? I have googled but can’t seem to find anything.

    Many thanks
    Kathryn


    1. Kathryn, try the search without the @ symbol.


  202. Hi Martin, I set everything up smoothly. However when entering my selected tags into the search bar and moving onto the archives page, nothing is popping up. It does say ‘running script’ at the top however. Does it take a while for data to pop up if a commonly used tag? Or has something gone wrong here? Thanks!


    1. This happens to me from time to time. Sometimes, authorisation does not always work with the Twitter API, and you have to try a few times to make sure you see both pop-up boxes asking you to authenticate in the main TAGS>Twitter Api authetication, AND the Tools>Script Editor menus. What I advise is to:
      1) Close and reopen the sheet (to kill any scripts that might have crashed),
      2) Once the sheet is reopended, go back to Tools> Script Editor, and Run>authenticate Twitter again (make sure the white box connecting to the Twitter API opens and asks you to allow the script app to use your Twitter account.
      3) Then go back to the sheet and run the collection process from the TAGS>Run now menu, and it should work after a few moments.



  204. Hi Martin, thanks for sharing your FANTASTIC tool! It’s a great help for my PhD as I am working on very similar graphs, but they are not interactive yet (http://www.md2c.nl/twitter-community-overview/). However, I could not get the getCommunityGraph() working: I got vertices, but no edges. Any idea?


  205. Hello, Martin. Thank you for your great work. I really like to use it.
    But I cannot use the function of collecting tweets on specific time within 7days.
    I set 1) Tags> Update archive every hour 2) Tool>Script editors> project’s triggers>specific date and time, but it was not operating.
    Would you let me know the right way to use this function?
    Many thanks.
    Seungho






  210. Great tool! You might want to get also the https:// links counted by tweaking your formula a little: =ARRAYFORMULA(SUM(IF(ISERROR(FIND(“http://”,LOWER(‘Archive’!C:C))),0,1);IF(ISERROR(FIND(“https://”,LOWER(‘Archive’!C:C))),0,1)))


    1. good point, forgot about https. Given (i think) everything gets wrapped in t.co you could probably just find “://t.co/”


  211. Hi,
    First, thanks for your great work. It has been very beneficial to my reasearch. I however noticed that the columns “in_reply_to_user_id_str”, “in_reply_to_screen_name” and “geo_coordinates” remain empty in my spreadsheet. Is there a reason or something I’m doing wrong? Thanks in advance.

    Cheers, philipp


    1. Hi Philipp,

      “in_reply_to_user_id_str” and “in_reply_to_screen_name” will only be populated if the tweet was a reply to someone.

      Similarly with “geo_coordinates”, this will only be populated if the user adds their location to the tweet.

      Daniel


  212. Hello,

    I am new to this twitter scraping spreadsheet. When I test the collection or run the script, it throws the following error:

    “Message details
    TypeError: Cannot read property “length” from undefined.”

    Any idea(s) on how to solve this?


    2. SOLVED – PEBKAC – incorrectly performed the following:

      3. When your Key and Secret are saved you need to open Tools > Script Editor … and run the ‘authenticateTwitter’ function


  213. Having trouble with triggering:

    getTweets>Time-driven>Hour timer>Every hour

    Execution failure notifications
    via email at [email protected]>immediately

    No notifications have been emailed, and no updates have occurred without manually using “Run Now!”. Any ideas?


    1. Ok, so a failure notification email was eventually sent. I guess setting this to “immediately” is not working.

      The email says:

      Summary:

      Error Message Count
      Authorization is required to perform that action. 11

      Followed by a list of 11 instances were getTweets tried to trigger but failed because of “authorization required to perform that action”

      I do not know what authorization I need to give this, as I’ve got it working for “Run Now!”. Any ideas?


      1. hi – when you open tools > script editor and run authenticateTwitter does it popup an authorisation window?


        1. Well, it did open a popup window for authorisation in the past. Now it does not, though.

          Instead I now get a script warning:

          ScriptProperties API is deprecated.
          File: TAGS Line: 27
          The API has been marked as deprecated which means that the feature should be avoided and may be removed in the future. Consider using an alternative solution.

          Line 27 being the following from authenticateTwitter:

          ScriptProperties.setProperty(‘active’, SpreadsheetApp.getActiveSpreadsheet().getId());


          2. Deprecated but still works for now. Next version of TAGS is close to completion and is a major code rewrite to use all the latest apps script services. Hopefully out next week but you can use my contact form http://mashe.hawksey.info/contact-form/ for early access (all I need is an email to share the new template with)





  217. Hi, I’m trying to find a way to download an entire hashtag (#thoughtvectors) for research. Is the number of tweets I can archive using TAGS v5 limited to the last seven days? Thank you!


    1. yes – but someone else might be archiving it using tags (worth pinging @cogdog)


      1. Thanks a lot, I’m exploring few other tools, but will connect @cogdog if I can’t find a good tool



  219. Having same issue as @LondonVotes. You say it still works tho? This is my first time using. I hit “Run Now!” and it still says “Running Script” after about 20 minutes. How long does it normally take?









  228. I love this script, but the loading of the datavisualization takes too long en mostly it times out. I tried with a fresh archive (500 tweets), but no change.




  231. Dear Martin,
    Thank for setting up this. I am having trouble running the script. It doesn’t seem to work. I don’t get the same window as you, and the list on archive.:( Am I doing something wrong?

Comments are closed.