Understanding the needs, behaviors, experiences and goals of the next billion job seekers in India
Timeline: 6 weeks
Sample Size: 12
Vahan’s product is a one-of-its kind jobs platform targeted at the next billion job seekers in India.
With millions of Indian job seekers already using the platform, and several tech startups among its clientele, the projected growth and potential for the Vahan product is significant. On the flip side, the marketplace Vahan focuses on is also characterized by a high attrition rate.
Vahan aspired to finetune and strengthen their offering through goal directed design – Fueled by a stronger and deeper understanding of the next billion job seekers.
Over March and April 2020, Anagram Research conducted a research study aimed at gaining a deeper understanding about the context, background and capabilities of different types of targeted job seekers.
Research also aimed to understand job seeker experiences – Including practices, processes, ecosystems, attitudes and behaviors.
This in-context research study for Vahan was completed in about 6 weeks, using a systematic process:
- Participant Recruitment: Job seekers participating in the study were recruited and scheduled in advance by professional recruiters.
Recruitment was done on the basis of a screener designed by Anagram Research, with inputs from the Vahan team.
- Home Visits (In-Context research): Detailed 90 minute semi-structured interactions were conducted with each participant.
Interviews were structured into broad themes rather than being tightly scripted, and included interviewing, observation and quick research activities. Doing this helped to keep the discussions relevant, break the monotony of discussion and elicit in-depth responses on core topics of interest.
- Data collection: During fieldwork, we captured raw data in the form of field notes, audio transcripts, video recordings and photographs.
- Synthesis: Two levels of synthesis were completed.
Quick synthesis – To capture immediate inferences and key observations;
Detailed synthesis post fieldwork – Data coding and persona scales were used to cluster people with similar attributes to derive ‘design personas’.
The outcome of the research was a set of job seeker Personas and insights into the barriers and motivations around using online channels for job seeking.
When used actively and consistently, design personas are distinct from marketing personas and can be a very effective and inspirational tool to guide design.
Therefore, our deliverable included a nifty set of Brainstorming Cards to help the Vahan product and design teams imagine future scenarios for their product.
These cards were based on actual and specific user needs and desires.
To help the Vahan team derive maximum benefit from the persona research, our research report also included guidelines on how to use / not use the personas and the brainstorming cards.
Devika and her team did a great job of understanding our needs in detail before diving into the project.
They were very thorough in their inquiry during the discovery and the research phase.
As a startup it is important for us to get learnings from the field in a more iterative way which is something they facilitated before delivering a final report on their findings. They empowered us with the right tools to put the research to use which made it easier to act on the findings.
They’ve also been available post completion of the project to answer follow up questions.
I would recommend them without hesitation!
Madhav Krishna, Founder and CEO, Vahan
Enabling a better future for medium – smallholding Indian farmers
Timeline: 5 weeks
Sample Size: 9
Locations: Villages in Maharashtra and Madhya Pradesh
The innovation team at a global consultancy was engaged on a project for a client in the food and agriculture domain. The client was keen to develop solutions geared towards enabling a better future for medium – smallholding farmers in India and another country in SEA.
While in-house researchers from the global consultancy conducted research to inform innovation in SEA, they partnered with Anagram Research to conduct a quick research study about Indian farmers.
The overall research effort was quick (completed in about 5 weeks), in order to provide timely inputs and inspiration in context to the overall project timeline.
- Farmers participating in the study were recruited and scheduled in advance by professional recruiters, on the basis of a detailed participant recruitment screener.
- Detailed, 120 minute semi-structured in-context interactions were conducted with all participants.
Quick research activities were used to break the monotony of discussion and elicit more thoughtful responses.
- Interviews were conducted in local / regional languages, with the help of a local language moderator and a simultaneous translator.
A researcher was present at all interviews. The researcher’s role was to guide the moderator, observe and take notes and also probe participants when required.
- Rough notes, audio transcripts, video recordings and photographs were captured during fieldwork.
- Two levels of synthesis were completed.
Quick synthesis captured immediate inferences and key observations.
Detailed synthesis was conducted post fieldwork, using the affinity diagramming method.
Anagram Research created a detailed insights presentation that included an in-depth capture of the Indian farmers context as well as farmers goals, aspirations, attitudes and values.
The report highlighted similarities with the findings from the study conducted in SEA, as well as insights distinct to Indian farmers.
In addition, Anagram Research presented a set of opportunity spaces and overarching guidelines that were based on the primary research.
Insights from the Anagram Research study helped the innovation team build empathy, as well as strengthen and guide product decisions and innovation directions proposed to the agri-business client.
Over the years, Anagram Research has mindfully recommended and used a remote research approach on a number of projects.
Our recommendations to use a remote research approach have been made in response to varied objectives:
- Logistics or scale…
- To accommodate participant preferences or profiles…
- To enable a large client team visiting India for a short period to observe research with a range of users in a short span of time…
- To help global researchers gain a cultural context before the start of fieldwork…
- To validate or build on insights emerging from in-context research…
Here’s a glimpse into some of Anagram Research’s experiences and approaches to remote research:
- Longitudinal Research
A client wanted to understand the positive and negative experiences that target users faced on their app and on competing apps.
In addition, they wanted to gain an in-depth understanding about user behaviors, attitudes and expectations.
Since app usage typically varied over time and across contexts, Anagram Research recommended a longitudinal research approach.
A Diary Study with 48 participants was conducted across 4 Indian metros, over a period of 7 days.
As participants were tech savvy, the study was designed to be conducted remotely.
Doing this helped ease several challenges typical to traditional Diary Studies.
In the past, we have used software like FocusVision Revelation for remote Diary Studies.
For this project, we used a bootstrapped approach – Interacting with participants over WhatsApp and phone calls.
On each day, participants were asked to explore the app they were using and answer a few questions from the researchers.
Responses were in the form of text messages, images (screen captures) or short video clips.
While the Diary Study method is well suited to a remote format, group discussions are better done face-to-face.
The Triads were held at centrally located, professional research facilities.
The output of this research was an insights presentation that included a set of overarching insights and recommendations, supported by detailed slides and brief written recommendations.
Our report also included a section on usability / experience issues faced by users.
- Direct Interviews
Over the years, Anagram Research has conducted remote one-on-one interviews for several different projects.
These interviews are conducted over the phone or using a web conferencing platform.
The value that a remote format can bring in certain circumstances was nicely illustrated on two separate projects, where we used both remote and face-to-face formats:
In a study with non-professional caregivers to the elderly, interviews conducted remotely had a flavor that was distinct from face-to-face interviews.
Caregivers we spoke to remotely, seemed less uncomfortable initiating a dialog around the unspoken burdens of caregiving.
Caregivers we met in-person on the other hand, found it harder to verbalize this openly, preferring to focus instead on other aspects of caregiving.
On another study to map employee experience journeys, it was important for us to reassure participating employees about confidentiality.
We attempted to address this – By not video recording interactions, by switching off audio recording during sensitive discussions and by emphasizing our status as a neutral third party.
Despite these measures, employees we spoke to remotely over calls, were strikingly more candid than employees we interacted with on our client’s premises.
Remote research can be more ‘convenient’ and cost effective than traditional research methods.
However, while there are many instances where remote research is the best choice, it is not necessarily the best approach for all research objectives and questions.
Remote or In-Context?
You don’t need to have it all figured out before you contact us!
We are happy to talk to you to understand your research or business objectives, and make recommendations on the best approach.
Testing and Evaluation
Anagram Research has conducted evaluation research studies for several Indian and global tech companies. Our research has helped our clients improve their product user experience and usability or helped them ensure that their products and services reflect user expectations and mental models.
Selected examples of evaluation research projects that we have been involved in, have been featured here:
A leading Indian mass media company partnered with Anagram Research in 2017 to conduct a foundational research study.
Insights from this study were one of the key inputs towards informing a ‘Mobile First’ strategy for the different news brands owned by this company.
This research also helped to shape the redesign of the umbrella brand’s business newspaper for digital media.
In 2018, Anagram Research engaged for the second time to test a high-fidelity prototype of the business newspaper.
The objective of this study was to identify all concept, experience and usability issues prior to going live.
Like many of our evaluation research studies, this study was conducted at professional research facilities.
A one-way mirror, video and audio setup enabled our client to comfortably observe participants as well as view usage of the mobile prototype being evaluated.
Encouraging the extended team to observe the research helped evangelize user centered design within the organization.
Anagram Research used evaluation research techniques like free exploration, talk-out-loud and scenario-based tasks to gain in-depth and accurate user feedback.
The redesigned mobile app launched in 2019 has unseated its long-time competitor and recorded 33 million visits by the end of 2019.
While a professional research facility is a suitable and convenient environment for some evaluation research studies, being in the product’s context-of-use is important in others.
Peel Technologies is the startup behind the Peel app – Which turns a smart phone or tablet into a smart TV remote.
The product team at Peel wanted to gain a better understanding about how users in India consume digital content.
In addition, they wanted to evaluate the user experience on the Peel app and get user feedback on some new concept prototypes.
To meet these objectives, Anagram Research conducted seventeen, 120-minute in-context interactions with users in Delhi, Hyderabad and Mysore.
In this instance, conducting the research in-context was also invaluable – resulting in several strategic insights that would not have surfaced if the study had been conducted at a research facility.
During the research it felt as if Devika was just as invested in the success of our app and product as we, the ones who had spent years building it, were.
Because of this, even a year later, we all still treat several of the insights that came out of our India research as cornerstones upon which we feel secure building our product direction for the future.
(Former client at Peel Technologies)
Evaluation research is a key component of the iterative design process.
Over the years, Anagram Research has facilitated research across all stages of product development – From concept and prototype testing to the evaluation of live apps and websites.
In addition to in-context and lab settings, Anagram Research has experience conducting evaluation research with remote participants.
A potent method within the research toolkit, secondary research can provide valuable and inspiring research-based insight.
Although an undervalued method that is often omitted from research roadmaps, over the years Anagram Research has engaged in secondary research in multiple different ways:
- Stand-alone projects
We have inspired our clients and identified areas for improvement and innovation through trends analysis, competitor research or benchmarking.
The secondary research projects we undertake are customized to align with specific objectives, budgets and timelines. Our engagements have included quick projects that could be accomplished in a week, as well as complex initiatives requiring a dedicated commitment of a month or more.
Our expertise and approach to secondary research has evolved over the years – We have learnt to anticipate and navigate challenges, conduct research systematically and efficiently, and adapt or construct frameworks to make our findings relevant and easily digestible.
- A framework that we recently used for a Trends Map, was inspired by Raymond William’s ‘Marxist Theory of Culture.’
Although originally used in the field of cultural studies, the ideologies referenced in his work (‘Dominant’, ‘Residual’ and ‘Emergent’), beautifully translated into a timescale for our Trends Map.
- A framework that we recently used for a Trends Map, was inspired by Raymond William’s ‘Marxist Theory of Culture.’
- Secondary Research to gain perspective
While we work across industries, some of the projects that come to us are very domain specific. Gaining a deeper understanding about the domain or topic of research during primary research planning helps sharpen and contextualize our approach.
We gain perspective and immerse ourselves in these projects by conducting stakeholder interviews or by referencing relevant information, expert opinions and diverse perspectives in the form of reports, articles, blogs or previously conducted research.
- Mixed Method Approach
At Anagram Research, we often use secondary research to substantiate and complement our primary research insights.
- In a project exploring different forms of online abuse, we interacted with 30 users across India, to understand their experiences as witnesses or victims of online abuse.
Primary research comprising a week-long diary study, followed by in-depth interviews resulted in rich data.
During analysis, secondary research helped us to contextualize some of our findings and enhance credibility of our primary research insights.
For example, the ‘Bad Girl’ poster created by design students of Srishti School of Art, Design and Technology is a parody of Indian school posters from the 80’s and 90’s. It depicts tongue-in-cheek examples of things men might get away with, but women would be judged for.
We referenced the poster in our insights report, to illustrate the extent of patriarchal attitudes and mindsets towards women in India. Minus this example, some of our primary research findings might have been harder to relate to for a foreign audience unfamiliar with India’s socio-cultural context.
- Secondary research on the same project highlighted forms of digital abuse like ‘Hindutva Pop’ and ‘MTV Troll Police’.
We had not heard about these formats during user interactions, and these findings supplemented our primary research insights or tied in to themes that had emerged during primary research.
For example, we had learnt through primary research that inherent negative emotions stemming from the stress of dealing with life and its challenges, is one of the catalyst’s to online abuse. The reality show ‘MTV Troll Police’ corroborated this insight – Interviews with trolls featured on the show highlighted that trolling is linked to frustration, boredom and the need for attention.
- In a project exploring different forms of online abuse, we interacted with 30 users across India, to understand their experiences as witnesses or victims of online abuse. Primary research comprising a week-long diary study, followed by in-depth interviews resulted in rich data.
- Analogous Inspiration
We use secondary research in the form of ‘analogous inspiration’ to encourage and inspire clients to look at problems and pain points through fresh eyes.
- In a project for a multinational electronics and home appliances manufacturer, we created ‘Brainstorming Cards’ to highlight our primary research insights in a more tangible format.
Problems and needs identified through primary research were placed on the front of printable cards. ‘How might we’ questions were placed at the bottom of each card to prompt the design team to think about each problem as an opportunity for design.
The cards could also be flipped for more inspiration – The back of each card contained related pictures and snippets from fieldwork, or analogous or existing solutions in the market that were identified through secondary research.