Today’s social media channels provide an unlimited number of virtual soapboxes for people to express whatever they may be thinking, feeling, seeing or doing. This vast resource of expression is a disorganized world proving far more challenging to navigate than current analytics may lead us to believe. Charts and graphs quantify collective behavior, but what do all these click counts and sentiment scores really mean?

While elevator definitions may vary, social media research boils down to the act of observing people’s behavior online. Quantitative researchers harvest this data field with large plows, aggregating and calculating every comment, post and blog available. Qualitative researchers, on the other hand, are carefully assessing the landscape, sorting out the weeds and selectively plucking rich insights in advance of their studies.

We tracked down nine research pros who effectively listen and derive keen insights from this vast garden of conversation, to learn how they leverage freely available data for real-time discovery and understanding.

How Does This Garden Grow?
First, the good news: Despite grim predictions, social media research does not replace traditional qualitative research. The pioneers with whom we spoke all listen to social media as a complement to, not a replacement for, familiar qualitative methods. Not one envisioned social media listening as a stand-alone methodology.

Kathryn Korostoff, founder & president of Research Rockstar LLC, explains one reason social media listening is getting so much attention: “There are a lot of challenges today in traditional research that are making social media research more attractive... Online survey response rates are horrible, behavior economics is pointing out the limitations of self-reporting... there’s a lot going on in the conventional survey research world in particular that’s making researchers feel a little bit dejected.”

The signature advantage of social media listening is the ability to observe spontaneous conversations in a natural environment where people feel comfortable expressing themselves. Free-flowing conversations, opinions, and even rants replace the orderly question-and-answer structure of traditional market research. For Kathy Doyle, president of Doyle Research Associates, the social media conversation is “a qualitative conversation with one difference – we don’t get to ask the questions, but we do get to listen to the responses.”

Adriana Rocha, co-founder & CEO of eCGlobal Solutions, values social media as “a new ecosystem… of social and mobile technology platforms allowing people to be connected, to socialize and exchange information.”

Renee Murphy, owner of Hello There Research, calls her online observation efforts “netnography,” an emerging method based in anthropology that mines social media data for cultural insights. “What I do is not specific to social media; I could apply this to any other sort of artifact. This is just a process that I apply to unorganized data.” Murphy finds listening particularly useful for inspiring innovations and insight for her clients. “With social media, the artifacts are already there. I don’t have to ask the question and disturb the fish bowl. I didn’t have to change the behavior to get it.”

Ricardo Lopez, president of Hispanic Research Inc., also considers social media a tool, not a new methodology, akin to observational research that has been in practice for decades. “While I may be doing observation research in a supermarket and watching what people select on the shelf, I can do the same type of observation research by going to an online forum and reading comments.”

While Lopez refers to his social media listening as “online observation research,” Ellen Fenwick and Jayne Dow, directors at Firefly Millward Brown, describe their efforts to gain a micro view as the natural complement to validating core metrics and “a critical role for qualitative research… we think of it as social life streaming; it’s about reframing the idea of listening.”

What shifts social media listening to a different dimension is the fact that millions of people across the globe are engaged, spanning thousands of websites and blogs. Doyle sums it up by saying, “It’s qualitative, [but] on a massive scale, like we’ve never seen before.”

That “massive scale” is exactly what creates some of the confusion and controversy surrounding social media listening. With millions of posts and photos, friends and likes, isn’t all that data quantitative? Maybe not. Korostoff explains that, “without rich demographic data, I can’t tell if the comments in social media are representative of my target market… the results are never directly actionable without additional research.”

Annie Pettit, chief research officer of Peanut Labs, says the distinction lies in how the information is used, explaining that while the analysis can be quantitative (e.g., tracking brand sentiment or comparing brand image), it can also be qualitative (when it is used to gain an in-depth understanding of groups of people and how they communicate). The other researchers we spoke with agreed: Social media listening perfectly fits the qualitative researcher’s listening and synthesizing skill set.

Mary Aviles, principal consultant at Connect 4 Marketing, has recently added social media research as a standard offering to clients. “Social media gives me a whole new way of listening to my client’s competition… We believe social media is a useful source of information, so we offer clients an analysis of online messaging.”

Qualitative researchers also appreciate the speed and efficiency with which social media listening projects can be initiated and completed. There’s no delay while participants are recruited, and the conversations are continuous and always up-to-date. Social media listening is a fast, cost-effective way to gather information from a large number of consumers.

“We use social media when we don’t have time to do other methods,” explains Korostoff. Murphy further highlights the speedy advantage when she asks rhetorically in “what other research method can you get shopper insights in four days?”

Gathering Rosebuds While We May
While online searches can be ad infinitum, the key to successful social media listening is knowing both where to start and when to stop. “I’m not exhausting the world of possibilities, I’m looking for recurring themes among a fixed number of preselected comments,” explains Aviles.

Both Aviles and Murphy prefer to start with a hypothesis – the client’s understanding of the problem – then seek out real data online to prove or squash the idea. Aviles explains, “I try to show a pattern that might suggest where the tide is turning, what consumers are experiencing.”

While only two of the experts we interviewed actually interact with customers on social media, all of them go online for their preliminary phase of research, as the forerunner to a traditional qualitative study or an online survey. These applications typically include:

  • Discovering more about an issue, trend, category, industry, company, brand or competitor;
  • Monitoring what consumers are saying about a brand, product, company, or event;
  • Researching and comparing positioning and experience against the competition;
  • Tracking sentiment toward brands, companies, events and personalities over time;
  • Looking for white space to stimulate product innovation;
  • Developing further hypotheses for subsequent qualitative investigations;
  • Driving questionnaire development and helping reduce survey length;
  • Understanding semantics;
  • Generating quotes to support other research phases (both qualitative and quantitative); and
  • Recruiting participants and obtaining consent to observe their private behavior on social media.

Both Murphy and Aviles use their findings in social media as a jumping-off point for competitive analysis and driving questionnaire development. They scour the Web, including popular social media channels, public forums and blogs, to find a slew of available conversations on the topic. Aviles explains the detailed process she took in a recent project, using TweetDeck to conduct initial Twitter hashtag and @[competition] searches to reveal alternate data sources, then plugging sources into SocialMention and TweetReach to locate influencers in the field. On a typical study, Aviles will look through hundreds or thousands of comments to identify trends and themes, which she ultimately separates and codes for further analysis.

Social media listening is just one piece of the qualitative research puzzle. It is limited to recording what people say or do, not what they think or feel, or even don’t do. Fenwick explains that “we still have to balance what we observe with additional exploration and insight. When we are observing interactions with social media, we’re able to see what they actually do but we’re not necessarily able to see what they thought about doing, or what they considered but didn’t do.”

With the client’s research objective top-of-mind, Murphy goes online in search of “unmet needs, pain points, category trends, target or segment identification, humanizations, articulations, brand positioning, what people are talking about, how consumers are experiencing and using the product… I always have specific questions in mind.”

Some researchers are blending social media listening with existing online methods to create new ways to interact and observe consumers’ behavior online. Firefly Millward Brown recruits social media users for custom studies and observes their behavior across multiple social media channels, using an aggregator to condense what they describe as the “raging river” of conversation to a “more manageable stream.” Lopez dons a research hat and actively dives into topical forums in both English and Spanish to engage consumers, noting that “this piece is closer to what we refer to as ‘online research.’”

Miami-based eCGlobal Solutions also focuses on Latin America and the U.S.-hispanic market, combining listening in the natural social media environment with custom online insight communities. Rocha explains, “When we create an environment that is social – an online community, for example – we can listen. That’s where the greatest and deepest insights come, when we just let people talk and we listen.”

A Good Garden May Have Some Weeds
As exciting and useful as social media listening can be, our nine pros pointed out some key challenges and limitations.

It’s easy to get lost

The experts we spoke with all stressed the importance of having a plan for social media listening and anticipating what you want to find. There’s so much conversation going on, in so many places, that aimless searching can easily lead you down a time-consuming rabbit hole. There are several useful tools available to help categorize, aggregate and compile social media conversations [see inset]. Finding the ones that work for you and your study can be challenging, as they vary in purpose, features and price.

Too much information

The flood of data online is never-ending. “I caution clients about putting themselves in the type of situation where it is information overload,” explains Korostoff. “If nobody has the time to actually read it and assess it, then [listening efforts are] really being wasted.”

If it’s not there you won’t find it

Social media listening is most fruitful for brands, issues, companies, events and other entities with higher awareness and greater presence online. If people aren’t talking about it, or only a few people are talking about it, chances are you won’t find enough conversation to make the search worthwhile.

Automation has its limits

While coding and sentiment analysis tools are constantly improving, there are no set standards, and according to Pettit, these automated tools agree with a human’s interpretation of a series of posts less than 60 to 70 percent of the time. This is where the skills of qualitative researchers are invaluable – reading and synthesizing summaries of social media conversations within the context of consumer behavior, as well as the client’s business issues. Murphy understands the approach can be laborious, noting that “there’s always a refinement process… the first time you will always see some stuff that shouldn’t be there… there is a lot of garbage out there.” Through an arduous process of careful selection, our pros are all looking to find the right stuff to harvest from the start.

Extremes of opinion

Social media offers consumers an open platform. While some comments have no emotion, others will reflect extremes of opinion. “The sentiment that you get does seem to be extreme… the middle ground is missing,” explains Korostoff. “The super brand ravers or the complainers, we’re missing the middle. We’re getting these extremes of what is being harvested, the neutrals are just not out there.” That said, she’s also quick to point out that today, “it’s our best opportunity to understand online word-of-mouth, and given the volume, it does have a lot of value.”

There are no demographics

We can infer some things from what individuals say and do online, but there’s no current way to confirm basic demographics like age, income, household composition, or brand usage. So although social media listening tools can group consumers by the attitudes they express online or the behaviors they display, it’s currently not possible to correlate these segments with demographics, which limits how actionable the findings can be.

Few people in the industry understand it fully

There’s a tendency in the market research industry to equate social media with Big Data and view it as quantitative, but social media listening is just that – listening to conversations that are going on around us and using the skills of the qualitative researcher to look for trends, behavior, attitudes, opportunities and insights. “Big Data talks about the buzz,” Doyle notes, “but natural language processing is still not perfect, so when the analysis is automated sometimes the true meaning gets lost in translation. What we are doing is traditional qualitative analysis and discovery on a large scale.”

As a result, ownership of social media listening varies; Korostoff estimates less than half of her clients have a social media listening function within the research department, instead placing the responsibility in the marketing, PR or customer service areas.

Privacy is a growing concern

This is a great unknown in the world of social media listening. Some of the researchers we spoke with worry that this valuable tool might become impossible in the future if privacy regulations affecting social media conversations become too strict. Others are more optimistic, anticipating a future of better-informed social media users who monitor their own privacy settings and websites that facilitate privacy controls.

Currently, it’s the researcher’s ethical responsibility to protect the identity of consumers they quote or otherwise attribute when reporting on social media listening. Depending on how social media users are sourced, these protections include getting opt-in agreements from participants, stripping out all identifying information when reporting, or even avoiding highly sensitive topics, such as health care.

The Outlook is Sunny

While the pros acknowledge the challenges, all envision a bright future for the art of social media listening. People are not only online, they are active and comfortable. Among our experts, however, there was some difference in opinions as to where on the horizon the sun would be rising tomorrow.

While Rocha feels the science is in its early stages, she believes more researchers are coming aboard and anticipates an increase in overall understanding and acceptance. She believes the tools also will evolve and prices will drop.

Murphy believes future tools will be customizable and enable more in-depth analysis.

Noting the rapid evolution thus far, Aviles envisions a world with more purposeful tools designed specifically with the needs of qualitative researchers in mind. She also sees a bright future for more B2B listening.

Pettit, who’s excited about using mobile for self-ethnography, points out that future improvements will be motivated by the academic field, which she carefully monitors for new advances. She also foresees freely available platforms merging and offering more fee-based services.

While Doyle continues to educate her clients about social media listening, she expects many will manage most of these services in-house in the future. She also believes privacy will be a greater concern as the industry evolves.

Korostoff points to the growing limitations of online quantitative research to fuel a greater increase in demand for social media research and sees paid tools becoming more affordable.

Dow believes social media research will continue to evolve as consumers change the ways they use social media and wearable technology allows them to capture and share their own personal data with others through online channels.

Lopez sees privacy and international laws getting sticky, yet eagerly anticipates the future when an open-source code will hit the scene and reveal even greater opportunities for better tools and approaches.

The social media research field is vast and growing. Fenwick sums up the value of this new data source definitively by saying, “There’s no debate that this is where people are. There’s no question that social media is a part of people’s lives today. This is where the consumer is. We should be there too.”

 

Interviewees:

 

Mary Aviles, principal consultant at Connect 4 Marketing
Kathy Doyle of Doyle Research Associates, Inc.
Ellen Fenwick, managing director and Jayne Dow, director of qualitative insights and digital innovations of Firefly Millward Brown
Kathryn Korostoff, founder & president of Research Rockstar LLC
Ricardo Lopez, president of Hispanic Research LLC
Renee Murphy, owner of Hello There Research
Annie Pettit, chief research officer at Peanut Labs
Adriana Rocha, co-founder & CEO of eCGlobal Solutions

Free SML tools used

  • Twitter’s search engine
  • Tweetdeck
  • LIttlebird
  • Tweetreach
  • Wordle
  • Twitonomy
  • Followerwonk
  • Revelation (free tool for Word trees)
  • Pinterest
  • Google+

Where the data comes from

  • Proprietary aggregators for sentiment analysis
  • Proprietary listening tools
  • Proprietary data analytics
  • Boolean search on Google, etc.
  • Twitter
  • Facebook
  • Blogs
  • Google tools (e.g., analytics, trends, search)
  • Instagram
  • Pinterest
  • LinkedIn
  • YouTube
  • Flickr
  • Reddit
  • Vine
  • Snapchat
  • Scoop.it
  • Whatsapp (this private site cannot be sourced, but is a good place to watch for trends)

Social Media Listening Tips

  • Customize approaches for each client
  • Hypothesize before beginning
    the search
  • Set boundaries for time
  • Pull relevant comments and
    identify themes
  • Use Evernote to organize the data