By Alex Hunt with contributions by DeWayne Ray. Alex is the CEO of Behaviorally and DeWayne is VP of Quality at Burke, Inc. Both are members of the IA Standards Committee.
Utilization and application of artificial intelligence (AI) tools in research are expanding and already supporting exciting improvements in the speed, scale, depth, and predictability of insights the industry can deliver to end users. However, the impact AI can have on research raises familiar, new, and evolving considerations for practitioners. Key mandatories to ensure research integrity and quality when using AI tools are outlined in the Insights Association Code of Standards and Ethics.
Heightened Risks to Participant Confidentiality: A Focus Area
The Duty of Care the industry owes to research participants is outlined in Section 1 of the IA Code, and it is a requirement that data obtained for research purposes should never reveal the identity of its subjects without consent. Use of AI tools both expands the research data lifecycle and intensifies the risk that, without proper management, personal data of participants is reverse engineered by AI inference.
Researchers must actively ensure personal data is protected when any methodology or technology is used. Protecting personal data when AI is used is no exception and has multiple implications for practitioners. If participants have not approved the use of their data, safeguards must be in place to prevent personal data from being accessed by AI tools. If participants have consented to the use of their data, the application of their data must remain consistent with its intended use.
The capability to integrate ever larger and differently structured datasets for deeper insight is a transformative advantage AI presents the industry. As such, this is a particular area where practitioners must proactively assume responsibility for adhering to approved uses of each dataset. As analysis moves further from first order objectives and the data lifecycle extends to facilitate integrated analysis, the researcher has a responsibility to ensure foundational standards are not forgotten.
Methodological Transparency: No Black Boxes
Section 2 of the IA Code highlights the importance of transparency with research methods that are deployed. This is especially important when working with new and emerging technologies, such as those leveraging AI, as these are at least for now inherently less familiar to both research practitioners, stakeholders, and participants.
First and foremost, researchers must disclose to stakeholders the intention to use and actual usage of AI in the creation of deliverables, including AI’s purpose and role, and clearly indicate whether AI is used as assistance (supporting human-led outcomes), embedded within broader methodologies, or leading reporting outcomes. Essential subsequent disclosures include the AI technique deployed (are you using data already in existence to make predictions based on past patterns or trends, or are you generating new synthetic data from which learning is derived?), model-type selected (publicly available versus open-sourced or proprietary and closed), as well as accuracy of any prediction being made (standard metrics such as normalized mean absolute error or root mean square error being most appropriate to benchmark and interpret model accuracy ahead of decision-making). The creation and use of databases to compare current results with past normative scores are accepted, established approaches to evaluating new content. Inputs to AI models should be clearly disclosed along with any known, go-forward uses.
Similarly, participants must be made aware of the use of AI technology and how their data will be used. This requirement for transparency applies not only to the use of participants’ data but also to how the data is collected. If AI technology is used to capture responses from participants, such as in qual-at-scale AI moderation or through AI-enabled survey chatbots, participants must be made aware and consent to their use. Transparency is vital when research tools in question are new and complex.
Human Oversight: Cannot be Substituted
Application of AI to research does not render the human researcher obsolete nor redundant. To the contrary, no AI system should operate without human judgment embedded in its design, implementation, validation, or application. Any AI model deployed against any research objective must be audited regularly and examined to ensure the intended purpose is met and that output is not falling victim to any bias or flaws inherent in either source data or design. This is particularly important when considering the role demographic or cultural variables might play: as with any research, flawed data leads to flawed insights and decision-making. These risks are only intensified by AI. When utilizing AI tools, the human researcher must maintain a skeptical view, regularly interrogating outputs gleaned to ensure integrity, utility, and the absence of bias.
An Evolving Technology: An Obligation to Stay Informed
The insights industry continues in its adoption and application of AI. Like new technologies and ideas that have come before, from digital data collection through to behavioral sciences, capability will evolve and do so often ahead of true, full, or final consensus on standards and norms for use. In a world of change, the research practitioner has an obligation to adopt a learning mindset and stay up to date on the latest tools, trends, and risks in applying AI to research. Communication of its use and transparency about application are paramount. It’s always possible to innovate in research, but equally essential to do so in keeping with the spirit and letter of the IA Code.
ABOUT THIS SERIES: The Insights Association Code of Standards & Ethics sets the principles that guide ethical and professional market research, insights, and analytics. But how do those standards apply in everyday practice? In this series, members of IA’s Standards Committee bring the Code to life through practical examples, showing how it guides responsible research and decision-making across the industry.
About the Author
Based in Washington, DC, Howard is the Insights Association's lobbyist for the marketing research and data analytics industry, focusing primarily on consumer privacy and data security, the Telephone Consumer Protection Act (TCPA), tort reform, and the funding and integrity of the decennial Census and the American Community Survey (ACS).
Howard has more than two decades of public policy experience. Before the Insights Association, he worked in Congress as senior legislative staffer for then-Representatives Christopher Cox (CA-48) and Cliff Stearns (FL-06). He also served more than four years with a science policy think tank, working to improve the understanding of scientific and social research and methodology among journalists and policymakers.
Howard is also co-director of The Census Project, a 900+ member coalition in support of a fair and accurate Census and ACS.
He has also served previously on the Board of Directors for the National Institute for Lobbying and Ethics and and the Association of Government Relations Professionals.
Howard has an MA International Relations from the University of Essex in England and a BA Honors Political Studies from Trent University in Canada, and has obtained the Certified Association Executive (CAE), Professional Lobbying Certificate (PLC) and the Public Policy Certificate (PPC).
When not running advocacy for the Insights Association, Howard enjoys hockey, NFL football, sci-fi and horror movies, playing with his dog, and spending time with family and friends.