This article was originally written by Avanade alum Roberta Capellini.

It is commonly said that there are two kinds of people in the world: those who set one alarm clock and those who set one every five minutes; those who love coffee for breakfast and those who love tea; and those who trust small data only and those who trust big data only. For others, have a look at the book, “2 Kinds of People: A Visual Compatibility Quiz” by João Rocha.

Now, the question is: does such dichotomy still make sense? No complaints at all for coffee and tea or the number of alarm clocks, I’ll just speak about small and big data. 

In essence, that was the topic around which the debate “Exploring Data: The Synergy between Profiling and Empathizing” has been unfolded. It took place during the convention “Humanizing Digital Technologies Through Design Thinking” organised by the Observatory Design Thinking for Business, School of Management, Politecnico of Milan, on May 8th 2020. 

Starting with the question, “Which are the most precious data in design thinking projects? Small or big data?”, several design thinkers brought their points of view, including me. If I had to tweet my answer, it would be: None. Design thinking is for both. But let me explain why. 

Here, the importance of data, both small and big, is not under discussion. Doing research, collecting and interpreting data is the key to producing actionable results, to best inform design decisions and to create innovation. 

At Avanade, we tackle this challenge through a mixed-method approach, leveraging the best of both worlds, small and big data. At its core, a mixed-method approach means reaching a comprehensive conclusion by linking different bits and pieces of data, generated by different methods (Brewer & Hunter, 1989). Hence, our vision is to blend the breadth of big data with the depth of small data and mix them with care (Blok & Pedersen, 2014).

The conditio sine qua non for our mixed-method approach to be successful is to understand that small and big data serve different purposes and answer to different research questions.

Small data - or thick data as defined by Geertz (1973), is typically generated by qualitative methods, when you go deep with a few users, by observing or interacting with them in their daily lives and addressing questions that focus on why they acted in some specific way (Maxwell, 2008). There is narrow and rich information that can be captured about people, such as their values, goals, emotions and behaviours.

The definition of Big Data is constantly changing (Kitchin & McArdle, 2016). Here, we define it as massive quantitative data sets collected through different sources - eye tracking and sensors, data logs of an app, web searches and social media - that can generate insights that apply to a large portion of a or even an entire population. The focus is to enable a powerful understanding of user behaviours, patterns and anomalies with higher speed and accuracy, addressing the questions of what is happening.

Being a quantitative researcher at heart, I believe a point of focus should be dedicated to surveys, recently defined “small big data” and placed in between the two types of data, and other data collected through experiments in laboratories, since they are quantitative tools but differ from big data in terms of volume, velocity and variety (Gray et al., 2015; Callegaro & Yang, 2017). 

Going back to our dichotomy, small and big data appear to differ in the problem-solving functions they serve: what vs why, at scale vs in depth. Our mixed-method approach can offer both. By incorporating them into our design thinking initiatives, we can understand human behaviours, revealing not just what is happening but also why it is happening, capturing not just a deep knowledge but also scalable. The key is to be able to sequence the two types of data differently depending on the kind of issue faced. Let me provide two quick examples.

Recently we were working on an innovative concept of the future digital workplace of a company. The situation at time 0 was like a whitespace scenario. Thus, we strategically decided to start the discovery phase with qualitative research that generates small data. We did interviews and focus groups, we talked to employees and stakeholders in order to uncover unmet needs, capture attitudes and expectations that could drive our innovation. The qualitative data represented for us a foundation for the following phases; next, we wanted to gather more robust and representative insights, to be more confident in our results and to be able to generalise them to the potential user population. That’s why we also analysed quantitative data collected through surveys and big data. Using clustering techniques, we were able to identify four need-based groups and then build our Personas, having a clear picture of the people we were innovating for. 

Conversely, almost at the end of 2019 a different client asked us to come up with a solution to improve their e-commerce. Basically, we had to redesign specific elements of the website to drive meaningful impact for the company. The situation at time 0 was different from the previous example: the client was asking for an incremental innovation (e-commerce redesign), and we had at our disposal a huge amount of big data. As a consequence, we strategically decided to analyse those data using R and Power BI, having a look for instance to the conversion funnel, keywords searches, numbers of views and gathering a lot of insights on what was not working properly on the e-commerce. After having a clear view of what was broken, we decided to collect small data through users interviews to interpret and make sense of those insights.

To sum up, the choice of whether and how to integrate small and big data is mostly strategic and based on business/project goals. 

It’s clear to me that an organisational change is necessary within companies. The key to going beyond the aforementioned dichotomy toward a mixed-method approach is to build silo-breaking, interdisciplinary teams. This means the need to engage both researchers, designers and data scientists in all stages of the design thinking process and derive benefits from the disruptive ideas that can be encouraged by bringing those different disciplines together (Wettersten & Malmgren, 2018).