By Crispin Beale
Photo courtesy of mTab
One of the mainstays at the Thanksgiving table is always stuffing. And whilst it's a delicious component of the family meal, you can have too much of it…likewise, “stuffing” can drain the efficacy and efficiency from market research. Last Thanksgiving, we reviewed, with research experts from around the globe, things that we should be thankful for; this time around, we feature market research and insight experts worldwide on what they feel market research is too stuffed with! Whether it's hype around a particular technology, reliance on a stale methodology, or ignorance of a valuable approach, this feature shares the stuffings that leading experts in the global insights profession think should be tossed from research. Hopefully you find something to be thankful for in this bumper edition stuffed with views and top-tips. Happy Thanksgiving.
Ben Page, CEO, Ipsos, France
“The research industry is stuffed with too many people who really want to be in technology or consulting. Both of those are vital to making research work and make a difference - speed, power, and impact - but we run the risk of throwing the baby out with the bath water too often. Ultimately we remain fundamentally about understanding human decisions and behaviour and what influences them better than anyone else in our ability to drive change. If we forget that, we may as well be software providers.”
Annie Pettit, Chief Research Officer for North America, E2E Research, Canada
“I’m hugely grateful for statistics, but we overeat when it comes to significance testing. It’s easy as apple pie to create (or prevent) statistical significance by using specific sample sizes or to discover astounding insights that will never replicate because they are false positives. Harvesting tabulations for that cornucopia of tiny p-values is a misuse of our time, which should instead be spent feasting on meaningful differences. If you need a significance test to tell you whether a difference is your next gravy train, it’s not. Effect sizes will fall nicely into place instead.”
Sinead Jefferies, SVP, Zappi & Chair, Market Research Society, UK
“It's no secret that people working in insights are overwhelmed with data. Never mind the turkey; we've got data stuffing up everything all over the place! Just like a top chef can select a few choice ingredients and turn them into a spectacular turkey recipe, the best insight professionals work with all the available assets to create a masterpiece. We're at a really pivotal moment in research right now, whereby the combination of excellent insight thinking and increasingly powerful technologies are allowing us to properly pull together the right data points and indicators to provide a clear, simple, and compelling story and do so in a faster, more affordable and predictable way - making it much easier for business to increase their confidence levels in making critical business decisions.”
Tony Costella, Director Global CMI, Heineken International B.V., Netherlands
“I feel Market Research is still too stuffed with technical details that can get in the way of the story and can switch some people off. We need to separate discussions on the methodology and techniques used (we can discuss these first up-front with the insights professionals) from the insights, recommendations, and powerful stories that will motivate decision-makers. The role of MR is to provoke and inspire – not to defend the approach.”
Justine Clements, CM Insights & Innovation, Samsung Electronics, Australia
“In market research, there is an obsession with volume; clients want big puffy data sets, well-seasoned and cooked to produce way too many charts and individual numbers that few can consume, let alone understand. Often more data, charts, and numbers make clients feel confident, either because they feel they have analysed everything or they have the numbers as proof. But it rarely helps anyone become more informed about the customer, nor does it help them make any meaningful changes that actually impact the customers' experience for the better. Much better to have fewer numbers and a story that provides a greater understanding and drives real change.”
Mark Langsfeld, CEO, mTab, United States
“I think the insights industry is stuffed with fragmented and siloed expertise. Research and insight professionals are too isolated. Bringing them together elevates efficiency, efficacy and expertise. While we’ve taken an initial step towards this by unifying experts from across the globe with the Insight250, the industry needs to unite to share expertise and perspectives to advance research for the common good. Bringing the content, community, and collaborative dimensions of research together. I am thankful to be on the frontline of this to elevate market research.”
Parves Khan, ESOMAR Director General & CEO, Netherlands
“We’re often too stuffed with fact-finders. We need them for sure, but we also need to build skills that nail the underlying business issue that needs to be solved. When we position ourselves as a strategic business partner to our stakeholders/clients – that’s when we get that seat at the table!”
Ed Clarke, Director of Research & Insights, Microsoft, United States
“Having run many tracking studies (both brand and customer satisfaction) across a lot of industries, I’ve seen a preponderance of people that either overly rely on their results or misinterpret the results altogether. The most common mistake researchers and stakeholders make is to react to results that are significantly different, but may not be meaningfully different. A measure that goes up or down by 2pp may be significant, but you have to ask yourself if it is meaningful to the business. Measures can also move up and down based on sampling error or other factors that are not related to either real sentiment change, or the result of a real experiential change. It’s important to look at measures over longer periods of time to determine what’s meaningful, or a real trajectory shift.”
Roland Abold, Vice President of Sales & Commercial Excellence, GfK, Germany
“Modern market research tends to overemphasize the importance of sophisticated statistical analysis and predictive algorithms in comparison to the quality of input data. Innovative tools and data science are important to efficiently process data and generate meaningful insights and recommendations. But ultimately, they are totally useless if the underlying data source is of bad quality or not fit for purpose.”
Danny Russell, Owner, Danny Russell Consulting, United Kingdom
“The stuffing that should be tossed from (commercial) research is the almost professorial fascination with the inputs; at the expense of focusing on the outputs. Normally, a huge amount of energy is expended at the start of the project on the technicalities of sampling and questionnaire design. But far too little thought as to how to ensure that the right people in the business are served with the insights, in the right format at the right time, so that consumers can be better served.”
Alexander Edwards, President, Strategic Vision, United States
“When the Thanksgiving research project is stuffed with Hubris, the smell, taste, and the results are always sour. Whether it comes from an arrogant research director or a client who simply wants to check a box so they can justify doing their own thing, it makes for a bad meal, enhances arguments with family at the dinner table, and leaves an unfulfilled desire to find relief in a comfort room.”
Arundati Dandapani, Founder, Generation1.ca, Canada
“Long questionnaires, unnecessary profilers, bad language, and jargon are extra. If we are trimming the fat, then we should also cut bias, and that’s mostly misunderstood because it’s so specific to each study’s objectives. Did we say egos and misinformation should be off the menu? The MRX meal should stir up high levels of camaraderie with a strong code of conduct if we want best-in-class dinner-table insights that go viral and make all stakeholders happy.”
Seyi Adeoye, CEO, Pierrine Consulting, Nigeria
“Less is really more! There is a tendency amongst market research professionals to always want to load every questionnaire or discussion guide with both need-to-know questions and nice-to-know questions. This is either coming from the client hoping to use one study to answer business questions that typically will require several studies or the researcher who is hoping to capture as many nice-to-know elements of a study; as against keeping each study sharp and focused. A well-designed study with focused instruments always delivers amazing value to discerning clients. So, away with verbose instruments that only lead to info-obesity!”
Sarah Maloy, Vice President, Global Insights Strategy, PepsiCo, United States
“Research is ‘stuffed’ with the practice of gathering qualitative data at the end of a project to ‘bring the findings to life.’ Instead, qualitative data should be systematically gathered and analyzed at the early exploratory stages. Only then does it have the potential to be a source of real insight, as it focuses on truly feeling and understanding the problem or need. Solutions sparked by this type of insight from the start rarely need talking heads to sell them at the end.”
Ben Leet, CEO, Delineate, United Kingdom
“We are too stuffed with people doing the jobs that machines can now do when processes such as project management, data processing, charting, and data delivery can now all be automated. If we ditch that stuffing, then our amazing humans can add far more value in thinking and consulting – that’s the tasty turkey that our clients really want to feast on!”
Alex Hunt, CEO, Behaviorally, United States
“Insights is stuffed full of, well, human insight! The human behaviors we are tasked to understand are complicated, irrational, and always changeable. That’s what stuffs the industry full of genuinely new and useful insight. But to uncover new and useful insight into human behavior we must recognise in all our approaches the irrationality of our subjects, as well as allow this to drive change in how we collect and create data and communicate the story it tells. But I'm thankful in 2022, we've finally started to see the industry do just this!”
Jean-Marc Léger, Président / PDG, Léger, Canada
“There are those who prefer to serve market research as a dessert. Post-tests can have a certain interest but are often too late. A good market research dish is served as an appetizer. This is how we taste-test guests, expectations, attitudes, and behaviors. This helps establish a better strategy and often reduces the margin of error and…the bill.”
Paul Baines, Professor of Political Marketing, University of Leicester
“Most market research is incredibly useful but less useful when it’s stuffed chock-a-block with information that does not correspond with the client’s needs, or could correspond with the client’s needs, but is then not properly embedded in the client organization’s decision-making structures. This leads to research left on the shelf or in a computer folder. Similarly, too much research is ad-hoc when continuous research could be more useful in identifying trends.”
Nikki Lavoie, ESOMAR President-Elect, France
“If we’re looking to trim the fat as an industry and leave more room for the most impactful players in insight, a good place to start would be low-quality samples. It’s easy to dish out blame (it’s the participants! It’s the panel/recruiters! It’s the survey design!), but hard to agree on paths to improvement. Collaboration to find solutions to this industry-wide problem could lead to more room for insight staples, like strong ROI and strategic growth for agencies and clients alike.”
Pete Markey, Chief Marketing Officer, Boots, United Kingdom
“Despite being a huge fan of market research, there are moments when “stuffing” can appear! My main bugbears are firstly testing for testing's sake, often to sort of an internal debate or “just because we’ve always tested it” – research is not a crutch and should not be treated as such. This links to my second bugbear, which is researching without a clear plan afterward – research is to enrich a piece of work and to move work forwards, it's never just to provide some interesting insights. In short, research is there to drive action, not inaction!”
Christian Doessel, Senior Vice President, Behaviorally, Germany
“I feel the market research industry is still too stuffed with thinking and acting in silos.
In an industry …
- where there are discussions of “qualitative” or “quantitative” instead of “both.”
- where AI is more seen as a threat to become a job killer rather than an enabler for further growth.
- where big data is for data scientists and surveys are for researchers.
- where behavioral science is reduced to the basic differences between System 1 and System 2.
Our lovely industry deserves more “blended approaches,” more “embracing of the new,” more “working together” to get the best possible outcomes to our clients.”
Caroline Frankum, Global CEO, Profiles Division, Kantar, United Kingdom
“Good quality data has always been an imperative of impactful market research. But, in today’s always-on world, where time is increasingly pressured, and our ever-expanding digital fingerprints are stuffing more and more data into an already data-obese world, ensuring we can seamlessly and compliantly connect behavioural and attitudinal data in ways that enable us to know more by asking less has never been more important. So, at Kantar, we are focusing on amplifying our indispensable brand expertise with tech that powers us to go further, faster, and better to deliver the most meaningful human data at the speed of life.”
Lisa Wilding-Brown, CEO, InnovateMR, United States
“As an industry, we are too stuffed with homogeneity (meaning we need more diversity in talent/people). As we look forward to the future of the MRX industry, we always think of the survey methods or the technology, but not enough about the young researchers who will one day run the show! Our industry attracts some of the best and brightest, and I am deeply passionate about providing them with connections, education, and opportunities through initiatives like the Young ESOMAR Society (YES). It’s now more important than ever that we not only help insights professionals create a more fair and equitable research strategy for greater representation but also on how to source the talent to run that research.”
Joaquim Bretcha, International Director, Netquest & ESOMAR President ex Officio, Spain
"The price race to the bottom in digital fieldworks and/or datasets. In a time where people's willingness to participate declines, it is crucial to properly treat and reward participants (being active or passive) as their participation is the key raw material for the whole insights food chain."
Babita Earle, EVP Global Enterprise Partnerships at Zappi, United Kingdom
“There is too much wheel reinvention with projects/studies being constantly commissioned to answer questions for which data that already exists or 'thrown away.’ Not enough of trusting our gut and providing business partnerships based on years worth of market/functional experience.
Firstly, I like to focus our industry descriptor on our end game – ‘insights’ – rather than what we do – ‘market research.’ The latter is too stuffed with a legacy of clipboards and service provision rather than a strategic business partnership. Secondly, the idea that data science is the answer to everything and we don’t need research that answers the ‘why’ is outdated – we need to bring together big data and thick data to uncover insights that explain consumers behaviour enabling us to anticipate and shape the future and uncover business growth opportunities.”
Urpi Torrado, CEO, Datum Internacional, Peru
“The problem is not the stuff, but how you integrate them to expand knowledge. Technology has dazzled some players in the market, but we must keep in mind that it is just an enabler. To get the most of it, more qualitative and observational research is needed. The ingredients are out there, massive access to data, but still not enough to understand the whys.”
Zoë Ruffels, VP Consumer & Business Insights & Analytics at Mars, United Kingdom
“Firstly, I like to focus our industry descriptor on our end game – ‘insights’ – rather than what we do – ‘market research.’ The latter is too stuffed with a legacy of clipboards and service provision rather than a strategic business partnership. Secondly, the idea that data science is the answer to everything and we don’t need research that answers the ‘why’ is outdated – we need to bring together big data and thick data to uncover insights that explain consumers behaviour enabling us to anticipate and shape the future and uncover business growth opportunities.”
Mariela Mociulsky, CEO, Trendsity, Argentina
“Nowadays, we have data available everywhere. It is necessary to select from that data (data is not insight) to turn all that information into knowledge and applicable insights. Every source of information that abounds today, as well as technologies and artificial intelligence, is welcome to help. But they do not replace the journey and technical development of researchers that enabled us to comprehend the value of each data piece and the validity, ethics, and bias it needs. So that there is no surplus nor waste, it´s key to know which are the necessary ingredients for a final product of excellency. A product that helps the decision-making of a business in an unknown context, which needs us to be the light that leads the way to the future.
“We are experts in interpreting every piece of information, which includes the activities and ethnographic approach, to be present in people's lives beyond their relationship with consumption. Placing people in the center is understanding the wide variety of dimensions in their lives; their tensions, aches, and desires. With all our abilities and tools, our industry can design a fair product and differentiate what matters and what is left over. “It´s of key importance as well to have diversity in our teams (age, career, disciplines, and regional wise, among others). “To have a full table, we need to know which meals to make. And to fulfill our job, we need to know what is valuable to convey in our products, ensuring we always deliver the best meal.”
Joaquim Bretcha, International Director, Netquest | ESOMAR President ex Officio SPAIN
"The price race to the bottom in digital fieldworks and/or datasets. In a time where people's willingness to participate declines, it is crucial to properly treat and reward participants (being active or passive) as their participation is the key raw material for the whole insights food chain"
Shirley K Ng, Research Principal, Asia Pacific, Europe & Russia at General Motors - CHINA
"Over expectations of a conjoint study or analysis: In product development, researchers often use conjoint models to evaluate multiple features as attributes, and review how these utilities might impact respondents’ selections (or decision making). I often hear the term “market share” being used in the outcome of these conjoint results, and managers use these percent shares changes to align with their product forecasting mission. Over expectations of conjoint results are similar to stuffing in a Thanksgiving turkey. Over used or unrealistic expectations of conjoint results can cause “digestion problems” (skewed results). Below is a short list of what I see is the limitation of a conjoint study from my top of mind.
1. Conjoint attributes are just a short list of what can be involved in product-selection process, it does not truly represent how customers would make their final purchase decision.
2. Conjoint research forces respondents to look at and review attributes they otherwise would not care about at the time when they are shopping. So, these conjoint results are somewhat biased one can say.
3. A conjoint exercise is a “perfect world”. It does not represent emotional factors such as purchase decisions impacted by marketing campaigns, ad promotions, competitive responses such as flash sales discounts to draw additional sales etc.
4. A common mistake is to include too many attributes in the research and the sample size captures cannot support adequate analysis back end."
I hope you’ve enjoyed hearing from these 30 leaders and innovators in our profession. This article is truly stuffed full of top tips and wise counsel. Thank you to everyone who took part and once again, I wish all of you who celebrate, a happy Thanksgiving and hope these perspectives from around the world prove insightful and helpful. Happy Thanksgiving.
ABOUT THE AUTHOR
Crispin Beale is a marketing, data and customer experience expert. Crispin spent over a decade on the Executive Management Board of Chime Communications as Group CEO of leading brands such as Opinion Leader, Brand Democracy, Facts International and Watermelon. Prior to this Crispin held senior marketing and insight roles at BT, Royal Mail Group and Dixons. Crispin originally qualified as a chartered accountant and moved into management consultancy with Coopers & Lybrand (PwC). Crispin has been a Board Director (and Chairman) of the MRS for c15 years and UK ESOMAR Representative for c10 years. As well as being CEO of Insight250, Crispin is currently Group President of Behaviorally with responsibility for the client and commercial teams globally and the Senior Strategic Advisor at mTab.