Last week, CSISP hosted a workshop entitled Digital Tools for Qualitative Research, made possible by the ESRC DTC multidisciplinary fund. The goal of the workshop was to address entrenched debates related to ‘big data’ and digital sociology, which often fall along classic quant versus qual lines. Quantitative and qualitative research represent a longstanding division of labour within the social sciences, with very distinct histories, values and epistemologies. It has been argued that the rise of social media and “transactional data” may require a re-negotiation of these established roles, but how can these new data sources be approached in ways which satisfy the commitments of both traditions?
The two day workshop brought together 25 participants from across the UK and Europe, and from a variety of disciplines (Sociology, Computing, Media and Communications, Design) and approaches to data: empirical studies of social media, media theory or observing practitioners of big data methods. The first presenter, Tommaso Venturini from Sciences-Po’s Media Lab argued that one way of undoing quant-qual divides, following work in Actor-Network Theory, is through networks – which allow the researcher to zoom from the individual case to the aggregate without reifying either. He presented a suite of scientometric tools (Sciencescape) which analyse citation networks and key words over time. These tools illustrated clearly how a relational approach to data can undo the fictions of scales, but it was remarked that citation data is a rare case in which the data sets are complete and relatively stable. It was unclear if networks could so easily map onto the associations of ANT in other contexts, like social media.
Bernhard Rieder from the Digital Methods Initiative in Amsterdam in contrast argued that quantitative techniques can be used in very qualitative ways. Using a case study of a political Facebook page, he demonstrated that the inclusion of numbers, such as ‘likes’ are not self-explanatory and must be qualified – even in the back-end of platforms there are conflicting and contradictory figures for the same object. Citing John Tukey, he proposed an “exploratory” and visual mode of data analysis using scatterplots, frequencies and networks. For Rieder it seemed to be the mindset, rather than the tool, which brought quant and qual closer together.
Noortje Marres talked about the methodological “uncanny” which arises in social media research when social and cultural researchers realise that methods not unlike their own are embedded in the platforms they wish to study, as in the case of social network analysis (e.g. – mentionmapp). She argued that as a consequence it is impossible to ‘apply’ social methods to study social media but neither should we adopt “the methods of the medium” if we are interested in doing social research – instead we must actively configure and reconfigure our methods at the interface. Illustrating by discussing the use of textual analysis to study privacy debates with Twitter, Marres noted the difficulties in detecting the “issuefication” of privacy following the Edward Snowden revelations, in ways that do not just re-produce Twitter’s own definition of what counts as a happening issue. Leed’s Helen Kennedy, however, raised the point that these sort of studies produce very different sort of results than interview based or ethnographic studies of social media usage and asked how these approaches could be brought together.
David Moats suggested that the divide between between quant and qual should be rethought instead as a question of the adaptation of methods to objects of study (both quant and qual may be guilty of being imposed on an object). He proposed, following Latour’s analysis (1995) of natural science methods that quant and qual can collaborate in bridging the translations from the particular, unique object(s) and their generalizable comparable form. He then showed two in-progress studies which placed social media texts within different visualisations which allowed for qualitative textual analysis and viewing macro relationships in the same diagram.
The rest of the workshop was devoted to group work on projects chosen by the participants, which represented a variety of approaches to the quant qual problem. You can view the working documents of two of the groups on the Issue Mapping platform: Youtube Group; Privacy Group.
At the end of the second day, the groups presented their projects and Alex Wilkie from Design gave some insightful feedback and avenues for future work. Some participants still felt that the requirements of quantitative studies still institutionally dominate the voices of qualitative researchers while others remarked that the quantitative is often too easily trumped by a single qualitative counter-example. It was proposed that despite closer collaboration and mixing of quant and qual there were still ends of a spectrum from particularity versus generality which can can only be occupied by either quant or qual techniques – but it was also pointed out that even these different registers of analysis may be a consequence of the study rather than existing apriori. While most of the participants were willing to speak across the aisle and collaborate, it was clear that quant and qual debates come with a lot of baggage, and it will take more open discussions like these to unthink these divisions.