All of the tweets for this project have been processed and consolidated into a single file that can be downloaded with this link: https://s3-us-west-2.amazonaws.com/healthcare-twitter-analysis/HTA_noduplicates.gz 1.85 Gb zipped / 15.80 Gb unzipped Each of the 4 million rows in this file is a tweet in json format containing the following information: All the Twitter data in exactly the json format of the original Unix time stamp All the Topsy data originating file name score author screen name URLs 60% of the records have geographic information ... Latitude & Longitude Country name & ISO2 country code City For country code "US" Zipcode Telephone area code Square miles inside the zipcode 2010 Census population of the zipcode County & FIPS code State name & USPS abbreviation The basic technique for using this file in Python is the following: import json with open("HTA_noduplicates.json", "r") as f: # convert each row in turn into json format and process for row in f: tweet = json.loads(row) text = tweet["text"] # text of original tweet ... # etc. Python provides very powerful analytical and plotting features but R is also very handy; R does not work well with large datasets but Python can be used to create a targeted subset file that R can read (or Excel, or anything else for that matter). For long-running jobs, I used Amazon Web Service's EC2 running Ubuntu 14.04, accessed via PuTTY and WebSCP; for local processing I used a Windows 7 laptop with the data on a terabyte external hard drive. The Status Report in the main repo contains a comprehensive explanation of the dataset examples of analyses done with this dataset a list of references to other healthcare-related Twitter analyses instructions for using Amazon Web Services sample programs using this file with Python, R and MongoDB.