Articles

06May

Ask U.S. Panel: Census Bureau Project Under Greater Scrutiny

The Insights Association (IA) continues to raise concerns about the Census Bureau's Ask U.S. Panel project, which would subsidize a private entity to develop a probability-based online research panel instead of purchasing the service from existing providers that already offer the service on the open market.

The Census Bureau first solicited public comments on its proposed online panel on December 7, 2021;  IA provided comments on it on February 7, 2022. The second Federal Register Notice (FRN) was a drilling down on the pilot stage; IA provided comments on that on March 31, 2022. The latest FRN, posted on April 4, 2022, was a slight revision of the prior one..

As IA told the Office of Management and Budget (OMB) on May 4, 2022, "The Bureau intends to spend at least the first two years of the project on this pilot before trying to make their panel probability-based in the third year. This is particularly galling since numerous private sector insights companies and organizations already provide well-established high-quality probability-based panels to the federal government and other customers without needing federal subsidy and multiple years of development time. At best, the proposed plan for the Census Bureau to develop a probability-based research panel is duplicative. The Bureau’s plan is also anti-competitive, given these existing panels and the Bureau’s intent to fund an additional insights firm to spend years building one whose intellectual property and technology that firm would get to keep for its own purposes."

Interestingly, we learned this week that the ingishts industry "is not the only interest raising concerns about the Ask U.S. Panel. The Department of Commerce’s Office of the Inspector General (OIG) notified the Census Bureau Director on January 14, 2022 that it had initiated “an evaluation” of the “award and use of a cooperative agreement to participate in a joint statistical project with Research Triangle Institute, an independent nonprofit institution.” The OIG’s “objective is to determine whether the cooperative agreement was properly authorized, executed, and administered in accordance with relevant laws and regulations.

The Insights Association's most recent comments focused on the following areas:

  1. Federal agencies can (and already do) purchase such services from the private sector;
  2. Developing a duplicative online panel presents an unnecessarily high respondent burden;
  3. Developing a duplicative online panel puts an unnecessary financial burden on federal taxpayers;
  4. The federal government should not compete against the private sector;
  5. Government should not subsidize a private entity to develop (and keep for its own use) duplicative intellectual property; and
  6. The Census Bureau’s supporting statements do not reflect reality.

IA will continue to advocate for greater transparency from the Census Bureau about this project and its origins and objectives, and also why the Bureau is pursuing it instead of purchasing the same services from experienced companies and nonprofits in the insights industry that already provide these services, for a tiny fraction of the cost and deliverable in a short time-frame instead of a matter of years.

Read IA's May 4, 2022 comments to OMB in PDF form, or below:

On behalf of the Insights Association (IA), the leading nonprofit trade association for the market research and data analytics industry, I am respectfully submitting this third round of comments on the U.S. Census Bureau’s planned Ask U.S. Panel, now specifically focused on the first questionnaire in a multi-year pilot project.

These comments focus on the following areas:

  1. Federal agencies can (and already do) purchase such services from the private sector;
  2. Developing a duplicative online panel presents an unnecessarily high respondent burden;
  3. Developing a duplicative online panel puts an unnecessary financial burden on federal taxpayers;
  4. The federal government should not compete against the private sector;
  5. Government should not subsidize a private entity to develop (and keep for its own use) duplicative intellectual property; and
  6. The Census Bureau’s supporting statements do not reflect reality.

IA defends and promotes the indisputable role of insights in driving positive impacts on society and consumers. Our more than 7,000 company and individual members are the world’s leading producers of intelligence, analytics and insights defining the needs, attitudes and behaviors of consumers, organizations and their employees, students and citizens. With that essential understanding, leaders can make intelligent decisions and deploy strategies and tactics to build trust, inspire innovation, realize the full potential of individuals and teams, and successfully create and promote products, services and ideas.

The “Ask U.S. Panel” is being developed by the Bureau through a cooperative agreement to create a new “nationally representative survey panel for tracking public opinion on a variety of topics of interest to numerous federal agencies and their partners, and for conducting experimentation on alternative question wording and methodological approaches.”

The Census Bureau first solicited public comments on its proposed online panel on December 7, 2021; IA provided comments on it on February 7, 2022. The second Federal Register Notice (FRN) was a drilling down on the pilot stage; IA provided comments on that on March 31, 2022. The latest FRN, posted on April 4, 2022, was a slight revision of the prior one. Thus, this is the second 30-day comment period, on top of the original 60-day comment period.

According to the latest FRN, the “Pilot will recruit a probability-based nationwide nationally-representative survey panel to test the methods to track public opinion on a variety of topics of interest, and for conducting experimentation on alternative question wording and methodological approaches. A large-scale field Pilot Test will be conducted to recruit members for the panel, based on a probability sample of U.S. adults. Once Pilot Panel members are recruited, they will receive a topical survey to complete. The current notice announces the content of the topical survey for both the general population and the Department of Defense samples.”

The Bureau intends to spend at least the first two years of the project on this pilot before trying to make their panel probability-based in the third year.

This is particularly galling since numerous private sector insights companies and organizations already provide well-established high-quality probability-based panels to the federal government and other customers without needing federal subsidy and multiple years of development time. At best, the proposed plan for the Census Bureau to develop a probability-based research panel is duplicative. The Bureau’s plan is also anti-competitive, given these existing panels and the Bureau’s intent to fund an additional insights firm to spend years building one whose intellectual property and technology that firm would get to keep for its own purposes.

Unfortunately, the Census Bureau continues to dismiss the comments and concerns of the insights industry. The Bureau's April 4, 2022 supporting statement to OMB notes that, "commenters disagreed with OMB’s statement that existing panels do not meet OMB’s standards for transparency. Results from the pilot study described herein will be presented to the OMB and any future plans for pursuance of a full panel will be discussed with OMB at that point. Two commenters ask questions and pose additional concerns about planned procedures for the future of the panel described in this package. The current PRA request is for the pilot only. Any future uses of this panel or program will be submitted to the OMB and the public in future 30-day notices."

It appears that the Bureau seeks to avoid any criticism or questions of its plan and approach by preemptively dismissing it, which undermines the whole point of the Paperwork Reduction Act (PRA) and the public review and comment process in which the Bureau and OMB are engaged.

Our industry is not the only interest raising concerns about the Ask U.S. Panel. The Department of Commerce’s Office of the Inspector General (OIG) notified the Census Bureau Director on January 14, 2022 that it had initiated “an evaluation” of the “award and use of a cooperative agreement to participate in a joint statistical project with Research Triangle Institute, an independent nonprofit institution.” The OIG’s “objective is to determine whether the cooperative agreement was properly authorized, executed, and administered in accordance with relevant laws and regulations.”

Recent Congressional inquiries to the Bureau have been met with reference to this OIG evaluation as the reason for their inability to answer questions. However, if the OIG evaluation somehow prevents the Census Bureau from discussing the Ask U.S. Panel, should it not also prevent the continued pursuit of the project?

Here are more details on IA’s concerns about the Ask U.S. Panel:

  1. Federal agencies can (and already do) purchase such services from the private sector

The ultimate goal of the proposal -- to create a probability-based nationwide representative survey panel for tracking public opinion -- is already being fulfilled utilizing numerous non-governmental sources. Insights providers such as Ipsos, NORC at the University of Chicago, Dynata, SSRS, Gallup and others maintain probability-based research panels that could meet any needs of the Bureau or other federal agencies. Most of them already successfully provide such services to federal agency clients, including the Bureau itself. Plenty of other insights companies and organizations with panels could also adapt to provide probability-based panels if requested.

Since these insights providers offer their services commercially on the open market, the Census Bureau could acquire such panel research services with full and open competitions. So why does the Bureau feel the need to disregard the availability of ready commercial alternatives and develop its own panel?

  1. Developing a duplicative online panel presents an unnecessarily high respondent burden

The respondent burden from the Bureau proposal will be unnecessarily high. As already stated, there are numerous non-governmental alternatives available in the market today, engaging with research subjects across the country in support of private and public-sector clients. A duplicative research panel created by the Census Bureau would impose undue burdens on the American public when they are already participating in private-sector panels.

Moreover, the several years of pilot testing to develop the Ask U.S. Panel suggests that the Census Bureau thinks it is better to potentially waste respondents’ time on experimental piloting of a panel over several years, instead of taking advantage of an existing panel provider’s expertise and technology (and their ability to quickly produce insights on methodology and content).

  1. Developing a duplicative online panel puts an unnecessary financial burden on federal taxpayers

The development of a research tool by the federal government to meet needs that can and are already being satisfied by insights providers outside of the federal government presents an unnecessary financial cost to American taxpayers.

Besides just the cost and expertise involved in establishing this duplicative service, the Census Bureau has not considered the immense expertise in data quality, incentive management and delivery, fraud detection, and privacy and permissions management required to successfully maintain this kind of panel. The Bureau mentioned no planned procedures to monitor and mitigate attrition of panelists and how it would refresh the pool of available respondents. The multi-year pilot plan suggests they just hope to learn on the fly, which will be an expensive gamble.

Why should taxpayers fund the lengthy creation and complicated maintenance of such a duplicative service when it could be simply purchased for a tiny fraction of the cost in the open market?

  1. The federal government should not compete against the private sector

According to the original Notice of Federal Funding, the Ask U.S. Panel would be “open to government and other non-profit researchers and policy makers," meaning that the Bureau’s proposed panel itself could compete directly with private sector insights providers. The Bureau dismissed this problem in its supporting statement as something to worry about later: “The instant request is only for the pilot study to develop a proof of concept and refine methods. Approval for the build out and use of the full panel will be the subject of subsequent 30-day notices.

Since 1955, federal agencies have been charged with avoiding "activities conducted by the Government that provide services or products for its own use which could be procured from private enterprise through ordinary business channels”. The policy required the head of an agency to make any exception to such restrictions "only where it is clearly demonstrated... that it is not in the public interest to procure such product or service from private enterprise." This policy was reiterated by every Administration following, including in OMB Circular A-76 and other policies specifically requiring competitive sourcing.

The Census Bureau implies in the supporting statement that it has conducted a competitive sourcing analysis, but in fact has only checked to see if the data to be collected in the pilot project stage is duplicative of other Federal government agencies: “This research does not duplicate any other data collection or research being done by the Census Bureau or other Federal agencies. The purpose of this clearance is to stimulate research, which would not be done under other circumstances due to time constraints.

That is no substitute for an actual competitive sourcing analysis.

  1. Government should not subsidize a private entity to develop (and keep for its own use) duplicative intellectual property

By using a cooperative agreement, under which the intellectual property (IP) developed is owned not by the federal government but by the awardee, the Census Bureau is using taxpayer funds to establish a panel that is free to be used by a private entity for its own work long after the contracted work is complete. Per the Department of Commerce Financial Assistance Standard Terms and Conditions (December 26, 2014, § D.03.a), the awardee “owns any work produced or purchased under a Federal award.”

How could the best or most cost-effective way of pursuing the Census Bureau’s research goals involve directly subsidizing a private entity to spend years developing a service duplicative of existing offerings from other private entities?

  1. The Census Bureau’s supporting statements do not reflect reality

The Bureau’s supporting statement claims that the broader “statistical community… has been struggling with near-real-time measurement of key areas, including: 1) Privacy and confidentiality opinions and preferences; 2) Public attitudes towards data collection and use of administrative records; 3) Methodological choices regarding online instrument design decisions; 4) Survey design choices regarding wording and contact timing; 5) Messaging strategies to increase response rates; and 6) Novel data collection needs due to emerging national events, like the COVID-19 pandemic.”

In fact, if the “statistical community at large” has been struggling with such measurement, it suggests that they have either not looked very far for fulfilment options (since many such topics are regularly covered in publicly-available opinion polling), or they are simply seeking federal taxpayer subsidy for data collection that is already available for purchase in the private sector.

The supporting statement also insists that “[e]xisting commercial online panel alternatives typically fail to meet OMB’s standards for transparency, which require sufficient detail on data collection and estimation methods to allow reproducibility, and sufficient detail on data quality and representativeness to enable OMB to evaluate the fitness for purpose.

In reality:

  1. Plenty of online panel research meets OMB standards and is successfully procured by dozens of federal agencies and departments; and
  2. To the extent that OMB’s standards pose a challenge to research approval, the Bureau should be focused on helping OMB update the standards to reflect modern online panels (last overhauled in 2006, notwithstanding an updated Q&A from 2016) and helping federal agencies absorb and understand them, a venture with which the Insights Association and our industry would be thrilled to participate.

Finally, the Bureau’s supporting statement estimates that the pilot will cost a mere $3.5 million: “The Cooperative Agreement under which the Pilot data is being collected will cost the government a total of $3.5 million from inception (September 2020) through completion of the Pilot (September 2022). Census Bureau annual staff time is estimated at approximately $200,000 (or one FTE).

However, this hides the true cost of the Bureau’s plan, since actually building the panel, which the Bureau doesn’t propose to even do until at least year three, would cost a massively greater amount. NORC estimates it would cost at least $25 million to build the kind of online panel sought by the Bureau, with annual maintenance being quite expensive (as much as $2 million per year). Meanwhile, NORC estimates that studies from pre-existing probability-based panel providers could “be purchased for as little as $100,000.”

Conclusion

We applaud the Census Bureau for their ongoing innovation and dedication to serve as the leading source of the highest quality and most representative data for America's people and economy. The Insights Association dedicates much of our daily advocacy to supporting the decennial census and the American Community Survey (ACS), the two essential federal data sources underpinning statistical sampling and representativeness in almost all U.S. research studies.

The insights industry is no stranger to the importance of the Bureau and its core work; we would simply prefer that the Bureau stick to that core work (and doing it well).

IA remains gravely concerned about the shaky rationale and lack of need for the Ask U.S. Panel given numerous commercially-viable alternatives. The Census Bureau should be using the competitive marketplace of available insights services to acquire pre-existing research services on an as-needed basis, which would reduce public burden, save years of development time, and ultimately cost taxpayers a lot less money.

 

Page 10. Supporting Statement Part A Ask US Pilot_4_4_22.docx https://www.reginfo.gov/public/do/PRAViewDocument?ref_nbr=202202-0607-008 

Evaluation of the U.S. Census Bureau’s Award and Use of a Cooperative Agreement (#2022-420) https://www.oig.doc.gov/OIGPublications/Evaluation-of-Census-Cooperative-Agreement.pdf

CENSUS-ADR-ADRM-2020-2006579. U.S. Census Bureau Research and Methodology Directorate Cooperative Agreements. Department of Commerce. Page 10. https://www.grants.gov/web/grants/search-grants.html?keywords=CENSUS-ADR-ADRM-2020-2006579

Page 5. Supporting Statement Part A Ask US Pilot_4_4_22.docx https://www.reginfo.gov/public/do/PRAViewDocument?ref_nbr=202202-0607-008  

Pages 2-3. Supporting Statement Part A Ask US Pilot_4_4_22.docx https://www.reginfo.gov/public/do/PRAViewDocument?ref_nbr=202202-0607-008

Page 3. Supporting Statement Part A Ask US Pilot_4_4_22.docx https://www.reginfo.gov/public/do/PRAViewDocument?ref_nbr=202202-0607-008

OMB Standards and Guidelines for Statistical Surveys (September 2006) https://www.whitehouse.gov/wp-content/uploads/2021/04/standards_stat_surveys.pdf

QUESTIONS AND ANSWERS WHEN DESIGNING SURVEYS FOR INFORMATION COLLECTIONS. OMB. October 2016. https://obamawhitehouse.archives.gov/sites/default/files/omb/inforeg/pmc_survey_guidance_2006.pdf

Page 17. Supporting Statement Part A Ask US Pilot_4_4_22.docx https://www.reginfo.gov/public/do/PRAViewDocument?ref_nbr=202202-0607-008

NORC comments. February 22, 2022. Page 3. https://www.regulations.gov/comment/USBC-2021-0024-0004

NORC comments. February 22, 2022. Page 2. https://www.regulations.gov/comment/USBC-2021-0024-0004

About the Author

Howard Fienberg

Howard Fienberg

Based in Washington, DC, Howard is the Insights Association's lobbyist for the marketing research and data analytics industry, focusing primarily on consumer privacy and data security, the Telephone Consumer Protection Act (TCPA), tort reform, and the funding and integrity of the decennial Census and the American Community Survey (ACS). Howard has more than two decades of public policy experience. Before the Insights Association, he worked in Congress as senior legislative staffer for then-Representatives Christopher Cox (CA-48) and Cliff Stearns (FL-06). He also served more than four years with a science policy think tank, working to improve the understanding of scientific and social research and methodology among journalists and policymakers. Howard is also co-director of The Census Project, a 900+ member coalition in support of a fair and accurate Census and ACS. He has also served previously on the Board of Directors for the National Institute for Lobbying and Ethics and and the Association of Government Relations Professionals. Howard has an MA International Relations from the University of Essex in England and a BA Honors Political Studies from Trent University in Canada, and has obtained the Certified Association Executive (CAE), Professional Lobbying Certificate (PLC) and the Public Policy Certificate (PPC). When not running advocacy for the Insights Association, Howard enjoys hockey, NFL football, sci-fi and horror movies, playing with his dog, and spending time with family and friends.

Related

Fighting for You: September 2022 Legislative and Regulatory Update

Fighting for You: September 2022 Legislative and Regulatory Update

Temperatures may be cooling as the calendar turns to autumn, but legislative issues are heating up a...

Read More >
IA Again Asks Congress for More Census Bureau Resources in FY23

IA Again Asks Congress for More Census Bureau Resources in FY23

The Insights Association, in concert with Census Project coalition allies, is urging Congress to hel...

Read More >
Fighting for You: August 2022 Legislative and Regulatory Update

Fighting for You: August 2022 Legislative and Regulatory Update

The Insights Association has worked throughout July and August to improve a comprehensive federal pr...

Read More >
Senate Appropriators Demand Census Bureau Transparency on Ask U.S. Panel

Senate Appropriators Demand Census Bureau Transparency on Ask U.S. Panel

The Senate Appropriations Committee will require a detailed report from the U.S. Census Bureau about...

Read More >
Ensuring a Fair and Accurate Census Act - H.R. 8326 - Passes House Committee

Ensuring a Fair and Accurate Census Act - H.R. 8326 - Passes House Committee

The House Oversight & Reform Committee recently passed the Ensuring a Fair and Accurate Census Act (...

Read More >
Fighting for You: June 2022 Legislative and Regulatory Update

Fighting for You: June 2022 Legislative and Regulatory Update

To our pleasant surprise, Congress has actually been cutting compromises on federal privacy legislat...

Read More >
Members only Article - Please login to view