Reflections on Twitter as a Historical Source

Last week, I detailed my clunky and ad-hoc method for collecting and analysing old tweets. I have now finished my data collection (I read almost 26,000 tweets in total), so it seemed like a good time to reflect a little more on the experience of the process and what I found, rather than just how I did it. The tweets I read were all written during 4 days in November and December 2010. During this period a nationwide campaign was trying to persuade the British government not to make dramatic changes to the way that higher education was funded, which included raising university tuition fees to up to £9000 a year.

parliament-student-protest-045

The Student Tuition Fee Protests in 2010 are the most recent case study in my PhD, which has presented methodological challenges as well as opportunities (Photo: Urban75).

The Student Tuition Fee Protests in 2010 is the only one of my case studies (the others are the Gordon Riots (1780), the Hyde Park Railings Affair (1866), and the Battle of Cable Street (1936)) that I lived through and participated in. I have my opinions about the issues contested in each of the other case studies, but researching events that you yourself experienced is very different. I was a second year undergraduate in late 2010, my younger sister would be affected by the proposed increased fees, and I cared very much about what happened. Reading through tweets from the four days of protest in London brought back a lot of emotions; the desire to do something; hope that we could make a difference, disbelief that anyone thought the proposals were a good idea; betrayal at the Liberal Democrats’ U-turn; anger at those who dismissed students as ignorant, lazy and apathetic; all soured by the knowledge that we didn’t change anything. Compounding this is the tendency people have to be more arrogant and abrasive on the internet than they ever would be in person. Because of this some Tweets were quite offensive, and it was hard not to take it personally. I found myself fighting the urge to reply to some of the most irritating Tweets, repeatedly reminding myself how strange it would be to get a reply to something written 6 years ago. Reading the tweets caused me to re-live many of the feelings I experienced back in 2010, which meant that this research was often quite draining emotionally.

049

Some tweets expressed extreme opinions, which I often found difficult to read.

One of the biggest problems I have faced so far in my PhD research is that the further back in time you go, the less archival material there is which records the perspectives and experiences of ordinary people. This is a challenge for many historical researchers, but it has been particularly difficult for me because the wealthy elites don’t tend to be the people participating in protest and dissent. The internet is relatively accessible, with only 11% of British adults having never used the internet (Office of National Statistics, 2015). This does not mean that 89% of British people use Twitter, but it does give me the opportunity to see what ‘ordinary’ people were saying about the protests, which is a rare treat for me. Twitter revealed some wonderfully fine-grained details about the protests and what it was like to be there. For example, a woman called Rosie McKenna broke her glasses and hurt her leg whilst being kettled by police on the 9th of December. It was great to be able to develop such a clear picture of what it was like to be part of the protests, rather than having to rely heavily on imagination.

040

Twitter preserves the experience of protesting in finer detail than traditional archival sources.

Another aspect of the research that I really enjoyed was seeing how various processes present in my other case studies played out through this modern technology. A common feature of protests and social movements is conspiracy theories; people speculate about who the ‘real’ organisers of a protest event are, or who might be manipulating the course of events to suit their own aims. The Gordon Riots, for example, were blamed on the American, Spanish or French governments. Scholars have argued that these theories developed because at that point it was not generally believed that the lower classes were capable of organising themselves in such a manner; they need someone to tell them what to do (Leon, 2011; Tackett; 2000). Conspiracy theories persist, however, despite modern society holding a less patronising view of the working and middle classes.One of the best known events of the 2010 Student Protests was the occupation of 30 Millbank, the building in which the Conservative Party campaign headquaters were housed. The response of the Metropolitan Police on this occasion was rather slow and inadequate. The most likely explanation is that they were surprised by the strength of feeling amongst the protesters, and had not prepared for trouble on that scale. However, it was suggested by some Twitter users that the police had deliberately responded slowly, because policing was facing its own budget cuts under the austerity regime, and wanted to demonstrate their usefulness to the government. The saying ‘the more things change, the more they stay the same’ springs to mind…

060

Twitter gives modern conspiracy theories related to protest a new platform on which to be transmitted and debated.

After a long period of writing, I really enjoyed getting to doing some research again, and exploring a new source of data. Working with Twitter was tiring, physically as well as emotionally (I had to take regular breaks because of the strain on my eyes), but also very rewarding. It has provided me with evidence to back up my arguments, as well as leading me to develop some new ones, and I feel like my PhD will be stronger because I tried this new (to me) research method.

Sources and Further Reading

León, Pablo Sánchez. “Conceiving the Multitude: Eighteenth-Century Popular Riots and the Modern Language of Social Disorder.’ International Review of Social History 56, no. 3 (2011): 511–533.

Tackett, Timothy. “Conspiracy Obsession in a Time of Revolution: French Elites and the Origins of the Terror 1789–1792.” The American Historical Review, 105, no. 3 (2000): 691–713.

#demo2010: Harvesting Old Tweets as a Research Method

The long time scale of my PhD means I have to deal with vastly different sources in my research. For the Gordon Riots (1780), I use mainly eyewitness accounts and court records. For the Battle of Cable Street (1936), I have access to images and videos of what happened. For the 2010 Student Tuition Fee Protests, the choices are almost endless. One of the sources I decided to utilise was Twitter, the social media website that allows its users to post updates of up to 140 characters. Every type of source presents different challenges for the researcher, and I found the unique challenges of Twitter rather difficult to cope with at first. This post is about the method I developed for my research, and I hope it will act as a catalyst for discussion amongst other scholars dealing with similar issues. My research was conducted on a computer with a Windows 10 operating system, and I do not know how well my method would translate to a different operating system.

Whilst there are programmes which collect tweets in real time as they are tweeted, many of which are open access, there are fewer designed to harvest pre-existing tweets. Those there are are aimed at a commercial rather than academic market, and their cost is beyond the scope of my research budget. So I had to develop my own ad-hoc, ‘low tech’ method of harvesting old tweets, using Twitter’s Advanced Search function.

In 2014 Twitter began allowing users to search for tweets more than 7 days old in its Advanced Search function (accessed from the options menu of a bog standard Twitter search result page, or by googling ‘Twitter Advanced Search’. You have to have a Twitter account to use this function). Advanced Search lets you combine a whole variety of search parameters, including date, location, hash tags, Twitter accounts, key words, sentiment (whether a Tweet is positive or negative). You can even input words you don’t want to be included.

Twitter Advanced Search

Twitter’s Advanced Search function looks a little like this.

Once I decided I was using Advanced Search, I had to decide on search parameters. The Student Tuition Fee Protests were a series of demonstrations, occupations and marches on both a national and local scale that took place between the 10th of November and the 9th of December 2010. I wanted to see Tweets from the four days of action that took place in London, on the 10th, 24th, and 30th of November, and the 9th of December. I started by searching for tweets that had been geotagged with London on the revelant days. Only a small percentage of tweets are geotagged, but it provided me with an idea of the hashtags and keywords that were were being used in regards to the demonstrations. I used this to decide on my search parameters. For example, for the demonstration on the 9th of December I searched for ‘Any of these words: protester, protesters, students, tuition, fees, protests’ and ‘These hashtags: #demo2010 #dayx3 #fees #solidarity #studentprotest #ukuncut’. For each demonstration, I used a slightly different combination of hashtags and keywords, in an attempt to find as many relevant tweets as possible. I acknowledge, however, that I probably did not find every tweet about the demonstrations. I also altered the dates as appropriate, then started the search.

Twitter Search Results

This is the top of the search results page I got for the protests on the 9th of December 2010.

Now for the long-winded part. I have not found a way to download multiple tweets at once. You can use your browser’s print function to save the search results as a pdf, but there are several disadvantages to this. You cannot expand the tweets to see what time they were tweeted, and it will only save the tweets that have loaded- you have to scroll all the way down to the bottom of the search results to save them all, and this can take a long time when searches yield more than a few thousand tweets. I did save the search results as a pdf, so I can go back to them at a later date if I want to, but only once I had read them all.

And that is how I analysed the search results, by reading every single tweet. Any tweets that I thought might be relevant to my research, I saved as a jpeg using the Snip tool, with it’s own individual number (001, 002, 003, 004 etc.). I also pasted each tweet into a word document, so I could go back to them later without having to open each individual jpeg. I coded the saved tweets as I went along, making a note of the tweet’s number and the key theme it related to. I also kept a count of how many tweets I had read as I went along. I wouldn’t say it was very reliable, but I can at least say roughly how many tweets I analysed for each demonstration. For example, I read almost 8000 tweets related to the demonstration on the 10th of November 2010.

Snip Tool.png

The Snip tool allows you to make an image from your computer screen. It works a little like print screen, but you are able to select a particular area that you want to capture, like this tweet from the 9th of December 2010.

dav

Sometimes, the simplest way of doing things is the best. I counted every tweet I read, and coded the most relevant ones using a good old fashioned notebook and pen.

So there you have it; my ad hoc, low tech (for Twitter!) method for collecting and analysing old Tweets for academic research. It is a rather clunky method, and I suspect that someone with more technological know-how than me could improve it dramatically, but it has allowed me to see how social media was being used during the 2010 Student Protests in London. If you have experience with this sort of research, or just have an opinion on it, then I would love to hear from you!

Following the Chartists around London

Last Monday, I took part in an event organised by Dr. Katrina Navickas of the University of Hertfordshire and British Library Labs called Following the Chartists around London. Dr. Navickas won a competition run by the Labs to develop a project that makes use of the British Library’s digital resources. As a result she is currently working on the Political Meetings Mapper, a project mapping all of the Chartist meetings listed in the Northern Star, one of the most popular Chartist newspapers. The Following the Chartists event was part of this project.

Katrina Navickas, in full Chartist costume, introduces her Political Meetings Mapper project.

Dr. Katrina Navickas, in full Chartist costume, introduces her Political Meetings Mapper project.

The afternoon began with lunch and a series of talks. Mahendra Mahey, manager of the British Library Labs project, introduced the British Library Labs and their work. Dr. Navickas explained the Political Meetings Mapper and gave a brief history of the Chartists. Dr. Matthew Sangster (Birmingham University) talked about his website romanticlondon.org, which uses contemporary maps and representations to explore romantic-era London. Finally, Professor Ian Haywood (Roehampton University) discussed the visual representations of ‘monster’ meetings- large, outdoor political meetings. The Chartists used this tactic frequently. We then embarked on a rather damp walking tour of Bloomsbury and Soho, visiting the sites of Chartist meeting places. In some cases, the pubs are still there, in others they have become stationary shops, or the building sadly no longer exists. The tour ended at the Red Lion in Kingly Street in Soho, which hosted meetings of both the Chartists and the London Corresponding Society.

Following the Chartists around London walking tour route (Source: Katrina Navickas).

Following the Chartists around London walking tour route (Source: Katrina Navickas).

We weren't about to let a little bit of rain stop us!

We weren’t about to let a little bit of rain stop us!

At the Red Lion we re-enacted a Chartist meeting that took place in December 1838. This is where I came in; I played the roles of a female Chartist of St. Pancras/ Mr. Cardo, who proposed the following resolution:

This is the most important crisis that has existed for the working classes. At the present moment we possess a power most mighty in its operation, one that is to be viewed by us with the highest feelings of delight and by our enemies with dread and alarm (Cheers.) … the Radicals are determined to be staunch to a man, and the people united will carry the day.

RESOLUTION: That this meeting considers a perfect union among all the Radicals absolutely necessary for the accomplishment of Universal Suffrage.

A recent Chartist conference in Edinburgh had proved devisive, and there was a sense that all the various groups and factions needed to get back on the same track, quickly. Only with a united front could universal (male) suffrage be won. Mr. Cardo’s motion was passed unanimously by our meeting.

Me, Samantha Walkden and Alexandrina Buchanan, some of the volunteer Chartists.

Me, Samantha Walkden and Alexandrina Buchanan, some of the volunteer Chartists.

The whole afternoon was great fun. I thoroughly enjoyed wearing a bonnet and apron, even if we did get some funny looks as we wandered around London. The talks highlighted the potential of digital research methods in relation to archives. Around 2% of the British Library’s collections have been digitised, which may not sound like a lot, but considering the Library holds well over 150 million items, it is a huge amount. Dr. Navickas has used computer programmes to transcribe newspaper articles, date meetings, and create maps that begin to interpret the data. The transcription software still needs a human to check its results, and all of this could have been done by hand, but it would have taken an awful lot longer. When it is finished, I think the Political Meetings Mapper will be a valuable tool for academics, students, and the simply curious; a resource which others can use to develop our understanding of the Chartist movement.The walking tour and re-enactment demonstrated how the Political Meetings Mapper could be used.

The British Library Labs project is doing valuable work raising awareness and promoting engagement with the Library’s digital collections. I learnt a lot about the possibilities of digital research methods, and would love to try and work it into my own work somehow!

If you want to do the walking tour yourself, see Dr. Navickas’ guide here.