{"id":920,"date":"2016-12-07T00:06:04","date_gmt":"2016-12-06T15:06:04","guid":{"rendered":"http:\/\/www.robfahey.co.uk\/blog\/?page_id=920"},"modified":"2019-01-31T17:44:49","modified_gmt":"2019-01-31T08:44:49","slug":"social-media-data-workshop-waseda","status":"publish","type":"page","link":"http:\/\/www.robfahey.co.uk\/blog\/social-media-data-workshop-waseda\/","title":{"rendered":"Using Social Media Data in Political Science Research (2016)"},"content":{"rendered":"

Workshop @ Waseda University, 2016\/12\/07<\/span><\/h3>\n

This page includes a listing of all of the software and packages mentioned during the workshop, as well as a few others you may find useful. I’ll link to detailed guides I’ve written about this topic, and update the lists as I write more in the coming months.<\/p>\n

\n

Core Software<\/h2>\n

Python<\/a> – the programming language used for all of the examples and packages in this workshop.
\nI also recommend the PyCharm<\/a> interface for writing and organising your Python code. There’s a free educational license that you can sign up for.<\/p>\n

As for databases, I suggest using MongoDB<\/a> for storing social media data – but certain types of project may benefit from a more structured SQL database like MySQL<\/a> or PostgreSQL<\/a>. You may also simply have experience with SQL and prefer to use it.<\/p>\n

If your project is very large, Google BigQuery<\/a> (part of Google’s Cloud Platform<\/a>) is worth looking into, but its costs can scale up very quickly. (Edit:\u00a0<\/em>Just after the workshop, Amazon announced a new cloud service called Athena<\/a>, which is very similar to BigQuery; I haven’t tried using it yet, but\u00a0it looks like a viable alternative.)<\/p>\n

\n

Python Packages<\/h2>\n

These are add-on packages for Python which make your life much easier when accessing, storing, handling and processing social media data.<\/p>\n

Twython<\/a> is a simple, easy to use interface for the Twitter API;\u00a0pymongo<\/a> is a similarly simple interface for MongoDB. These two are all you need to start downloading and storing social media data from Twitter.<\/p>\n

numpy<\/a> and pandas<\/a> are a matched pair of packages which give Python similar statistical and scientific computing abilities to R. They’re required for a number of the other, more advanced packages on this list.<\/p>\n

matplotlib<\/a> is a very popular package for creating graphs of your data. I also like Bokeh<\/a>, a more modern graphing package which creates interactive graphs you can manipulate in your web browser.<\/p>\n

SciKit-Learn<\/a> is a machine learning package for Python which includes good\u00a0text analysis functionality, as well as implementing a large number of classifying and clustering algorithms.<\/p>\n

NLTK<\/a> (the Natural Language ToolKit) is a package of tools\u00a0specifically aimed at text analysis. It’s better at certain things than SciKit-Learn; for example, it can tokenise text in a more advanced way that allows for “stemming” European languages – i.e. using the base form of each word to avoid duplication in the corpus. Paired up with SciKit-Learn it makes a formidable machine learning system.<\/p>\n

\n

Japanese Language Handling<\/h2>\n

Refer to my guide to\u00a0setting up and using MeCab and its dictionaries with Python<\/a>.<\/p>\n

\n

Some Relevant Papers and Guides<\/h2>\n

Barber\u00e1, P. (2015). Birds of the same feather tweet together: Bayesian ideal point estimation using Twitter data<\/a>. Political Analysis<\/i>, 23<\/i>(1), 76-91.<\/span><\/p>\n

Brandon Rose’s excellent\u00a0guide to simple document clustering in Python<\/a>.<\/p>\n

Ceron, A., Curini, L., & Iacus, S. M. (2016). iSA: a fast, scalable and accurate algorithm for sentiment analysis of social media content<\/a>. Information Sciences<\/i>.<\/span><\/p>\n

CodeAcademy<\/a> has great free tutorials for a range of languages and technologies including Python, for those who aren’t familiar with programming.<\/p>\n

Hopkins, D. J., & King, G. (2010). A method of automated nonparametric content analysis for social science<\/a>. American Journal of Political Science<\/i>, 54<\/i>(1), 229-247.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

Workshop @ Waseda University, 2016\/12\/07 This page includes a listing of all of the software and packages mentioned during the workshop, as well as a few others you may find useful. I’ll link to detailed guides I’ve written about this topic, and update the lists as I write more in the coming months. Core Software …<\/p>\n

Read More<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-920","page","type-page","status-publish","hentry"],"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/P71QYy-eQ","jetpack-related-posts":[{"id":1092,"url":"http:\/\/www.robfahey.co.uk\/blog\/sentiment-analysis-in-python\/","url_meta":{"origin":920,"position":0},"title":"Sentiment Analysis in Python (2019)","author":"Rob Fahey","date":"29\/01\/2019","format":false,"excerpt":"(Data Science Week @ Waseda 2019) Slides from the Workshop You can access the Powerpoint slides from the workshop on Slideshare: https:\/\/www.slideshare.net\/RobertFahey3\/sentiment-analysis-in-python-waseda-data-science-week-2019 Example Code and Data sentiment_workshop.zip - this file contains the example code and data used in the workshop. Unzip the file into a directory you can easily find,\u2026","rel":"","context":"Similar post","block_context":{"text":"Similar post","link":""},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1243,"url":"http:\/\/www.robfahey.co.uk\/blog\/publications\/","url_meta":{"origin":920,"position":1},"title":"Publications","author":"Rob Fahey","date":"09\/05\/2022","format":false,"excerpt":"Journal Papers Hino A, Ogawa H, Fahey RA, and Liu L (2022), Japan: Political Developments and Data in 2021. European Journal of Political Research Political Data Yearbook.Jungkunz S, Fahey RA, Hino A (2021), How populist attitude scales fail to capture support for populists in power, PLOS One 16 (12): e0261658.Fahey\u2026","rel":"","context":"Similar post","block_context":{"text":"Similar post","link":""},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":117,"url":"http:\/\/www.robfahey.co.uk\/blog\/sns-polcom-analysis\/","url_meta":{"origin":920,"position":2},"title":"SNS PolCom Analysis","author":"Rob Fahey","date":"11\/12\/2014","format":false,"excerpt":"This is a holding page for a set of Python \/ MongoDB \/ Google BigQuery\u00a0tools I am building to analyse the use of social media\u00a0platforms by political actors in Japan.","rel":"","context":"Similar post","block_context":{"text":"Similar post","link":""},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1161,"url":"http:\/\/www.robfahey.co.uk\/blog\/social-network-analysis-unimi\/","url_meta":{"origin":920,"position":3},"title":"Social Network Analysis – University of Milan (2022)","author":"Rob Fahey","date":"28\/02\/2021","format":false,"excerpt":"Autumn Term Week 1 (October 17-21) DateClass MaterialsAssignmentMonday, October 17 (10:00)SlidesAssignmentTuesday, October 18 (On-Demand)On-Demand LectureSlidesAssignment Week 2 (October 24-28) DateClass MaterialsAssignmentMonday, October 24 (10:00)R Script & DataAssignmentTuesday, October 25 (10:00)R Script & DataAssignmentWednesday, October 26 (On-Demand)On-Demand LectureR Script & DataAssignment Week 3 (November 7-11) DateClass MaterialsAssignmentMonday, November 7 (10:00)SlidesR ScriptAssignmentTuesday,\u2026","rel":"","context":"Similar post","block_context":{"text":"Similar post","link":""},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":11,"url":"http:\/\/www.robfahey.co.uk\/blog\/about-me\/","url_meta":{"origin":920,"position":4},"title":"About Me","author":"Rob Fahey","date":"09\/01\/2012","format":false,"excerpt":"I'm Rob Fahey. I'm an Assistant Professor at the Waseda Institute for Advanced Study (WIAS) at Waseda University in Tokyo, where I work on contemporary politics and political communication, and a Visiting Professor at the Department of Social and Political Sciences at the University of Milan, where I teach courses\u2026","rel":"","context":"Similar post","block_context":{"text":"Similar post","link":""},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":37,"url":"http:\/\/www.robfahey.co.uk\/blog\/contact\/","url_meta":{"origin":920,"position":5},"title":"Contact","author":"Rob Fahey","date":"09\/01\/2012","format":false,"excerpt":"The more you research the impacts of social media on society, the less inclined you are to actually use it. I'm often on Instagram (though it's more for personal life than work-related updates), and occasionally on Twitter, though I've been taking a break from it since opening it on my\u2026","rel":"","context":"Similar post","block_context":{"text":"Similar post","link":""},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"http:\/\/www.robfahey.co.uk\/blog\/wp-json\/wp\/v2\/pages\/920","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.robfahey.co.uk\/blog\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"http:\/\/www.robfahey.co.uk\/blog\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"http:\/\/www.robfahey.co.uk\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.robfahey.co.uk\/blog\/wp-json\/wp\/v2\/comments?post=920"}],"version-history":[{"count":10,"href":"http:\/\/www.robfahey.co.uk\/blog\/wp-json\/wp\/v2\/pages\/920\/revisions"}],"predecessor-version":[{"id":1101,"href":"http:\/\/www.robfahey.co.uk\/blog\/wp-json\/wp\/v2\/pages\/920\/revisions\/1101"}],"wp:attachment":[{"href":"http:\/\/www.robfahey.co.uk\/blog\/wp-json\/wp\/v2\/media?parent=920"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}