the White House’s effort is obviously not a scientific survey
The Marketing Research Association (MRA) is pleased that the White House grasps some of the value of survey, opinion and marketing research, based on their launch of this online survey effort.
However, questionnaire design is crucial to the quality of the insights you can garner from any research, and these questions are lacking. Questions about feelings toward data collection, etc., are devoid of context. Purpose matters: if someone is collecting your data for research, which aggregates the information to learn about groups, that is very different from collecting your data for purposes of spying on you, the individual, for national security or law enforcement. In addition to the different purposes of data collection and use, no mention is made of the costs and benefits involved for consumers and citizens.
Not only does the White House appear to ignore the importance of how data is used, it also appears to ignore the difference between the uses of data by different government agencies. The purposes of IRS data collection – normally, for help in collecting taxes owed by individuals – are extremely different from the Census Bureau, which needs to collect individuals’ data in order to learn about groups of Americans. Moreover, while some government agencies regularly leak data like a sieve, the Census Bureau is the only government agency we know of that has never suffered a data breach, is a world leader in data confidentiality, and is statutorily forbidden from sharing personally identifiable information with anyone else.
The White House website prefaces the survey by saying: “This is a complicated issue that affects every American — and we want to hear your feedback.” However, the White House appears not to care about whether the input they are gathering actually comes from Americans or American residents, versus anyone in the rest of the world, since there is no screening process, or even questions about residency or citizenship.
Of course, the White House’s effort is obviously not a scientific survey, since no effort is being made to sample respondents. Any agency that tried to run data collection this way would be unlikely to ever receive approval from the Office of Management and Budget (OMB) because of flawed design, but OMB answers to the White House and not vice versa. Serious research requires industry-standard design and controls to ensure quality. Without these in place, the results are guaranteed to be inaccurate and of little to no value. The White House could instead utilize in-house Census Bureau talent, and a wide range of talented research professionals in the private sector, to make this effort worthwhile and in the best interests of all concerned, including the national interest.
- Research magazine's Bronwen Morgan covered MRA's response, noting, "The White House has become the latest political target of criticism from the market research industry, after posting a flawed online poll on big data and privacy. Does this raise questions about politicians’ ability to legislate on data?"
- AdWeek covered MRA's response as well: "White House Privacy Survey Falls Short of Its Purpose: Research experts say questions are overly broad."
- Also, the International Association of Privacy Professionals (IAPP) reprinted MRA's views in their Privacy Perspectives blog.