By Amber Wichowsky

One of the big issues facing the State Legislature this session is how to proceed on controversial mining legislation designed to make it easier for Gogebic Taconite to open an iron-ore mine in northwestern Wisconsin.

The Marquette Law Poll recently surveyed Wisconsinites on their opinions about the mine. But so did another polling outfit during the same timeframe and they reached very different conclusions. How could that be?

Full disclosure: I am not a fan of the standard “environment versus jobs” questions that are often asked on public opinion surveys.  Such questions are typically a variant of the following:

Some people think it is important to protect the environment even if it costs some jobs or otherwise reduces our standard of living. Other people think that protecting the environment is not as important as maintaining jobs and our standard of living. Which is closer to the way you feel, or haven’t you thought much about this?

There are certainly cases where this question offers an illusory dichotomization of this issue. Earlier this year President Obama told the Environmental Protection Agency: “I do not buy the notion that we have to make a choice between having clean air and clean water and growing this economy in a robust way.  I think that is a false debate.” The President went on to note the domestic jobs created in the clean energy sector and the tourism jobs supported by having clean waterways.

But in other instances such trade-offs become more pronounced, particularly in political discourse. Consider the proposed iron-ore mine in northwestern Wisconsin. Last month the Assembly passed legislation that would streamline environmental regulations and speed up the mining permitting process in order to avoid protracted regulatory delays. Democrats and environmental groups criticized the bill for weakening environmental regulations, endangering wetlands and risking the contamination of drinking water. The issue is now being debated in the Senate, where some Democrats and Republicans are trying to find a compromise.

In the latest Marquette Law School Poll, Wisconsinites were asked about whether they support developing the mine. The poll interviewed 716 registered Wisconsin voters between February 16 and 19. Interestingly, Public Policy Polling (in consultation with the Wisconsin League of Conservation Voters) also conducted a poll on the same issue at the same time (Feb. 17 and 18).

The surveys revealed a stark contrast of opinions. Whereas the Marquette Law Poll found that 52% of respondents were in support of developing the mine, the PPP/League of Conservation Voters poll found just the opposite, with nearly half of respondents opposed to the mining legislation.

As Prof. Charles Franklin, director of the Marquette poll, noted in his most recent On the Issues appearance (Feb. 22), these two polls offered a “teaching moment about how the framing of issues is so terribly important.”

I decided to take Charles up on his suggestion to make this a “teachable moment” in my undergraduate class on Elections and Public Opinion. I began class by showing students the exact question wording from the two polls. We then compared the survey items and discussed how question wording affects survey responses. Finally, we tried to come up with a new item that would address the criticisms students had of each question.

First, the questions. (I have added the actual survey responses in parentheses.) The Marquette Law Poll asked:

There is a proposal to develop an iron-ore mine in northwestern Wisconsin. Supporters argue that the mine will create 700 jobs and long-term economic benefits. Opponents argue that not enough environmental protections are in place to preserve water and air quality. Do you support (52%) or oppose (33%) developing the mine? (15% of respondents said either “not heard enough” or “don’t know”)

The PPP/Wisconsin League of Conservation Voters poll asked:

As you may know, the Wisconsin State Senate is considering an open-pit mining bill. Supporters of the bill say that Wisconsin should streamline its environmental regulations in order to create more open-pit mining jobs in Northern Wisconsin. Opponents say that the existing water protections should not be weakened to allow out-of-state mining companies to expose Wisconsin families to chemicals such as mercury, lead, and arsenic. Which comes closer to your point of view? Environmental protections should be streamlined (34%) or environmental regulations should not be weakened (49%)? (16% of respondents said “not sure”)

Students felt that the Marquette Law Poll did not state the environmental risks strongly enough. They argued that putting in an exact number of jobs (700) focused respondents’ attention on the potential economic benefits of the mine, while the opposition statement was vague and open to significant interpretation. (What exactly does “quality” mean anyway?) The students noted that they didn’t feel informed enough to weigh the risks against the rewards.

Students were equally critical of the question wording in the PPP/Wisconsin League of Conservation Voters poll. Whereas a discrete number focused attention on “jobs” in the previous item, many of the students said they focused their attention on the particular contaminants highlighted in the PPP question (mercury, lead and arsenic). Other students said that “out-of-state” signaled something nefarious.

The students then were not surprised to see the differences in opinion between the two surveys.

But could we do any better? It was clear that what the students wanted was more information: What specific jobs might be created? What collateral economic benefits might the mine yield? What are the environmental risks associated with iron-ore mining? What are the health risks associated with ground water pollution? How does the legislation affect existing protections for the state’s wetlands? Why are wetlands important in the first place? By the end of our conversation, it was evident that what started out as a “simple question” was becoming a more “complex discussion” of a complicated issue. Surely, the time constraints facing pollsters would preclude any such item from ever making its way onto a survey.

But what was also clear was that question wording makes a difference. The class didn’t necessarily feel that one poll was better than the other (both had their limitations). Rather, students were left with a number of questions about public opinion surveys. Should pollsters try to maintain neutrality when discussing hot-button policy debates (as the Marquette Law Poll tried to do)? Or should pollsters seek to mimic the way that political actors frame issues? As Professor  Franklin rightfully noted: “Political debate is as much about deciding the terms of engagement as it is about the details of policy and the [associated] benefits and costs.”  Yet, even if pollsters craft questions that reflect how politicians and the media frame issues, how do those who write the questions decide which frames to highlight?  Indeed, given today’s segmented media market, voters are often presented with multiple arguments about public policy debates.

Of course, one could also have asked both questions on the same survey.  Looking at the items again, the results are not necessarily inconsistent between the two. The PPP/Wisconsin League of Conservation Voters poll asked survey respondents about whether they support regulatory changes to allow more mining jobs in northwestern Wisconsin. The Marquette Law Poll asked whether survey respondents support developing the mine.  One could imagine supporting the mine on the grounds that it would create jobs, but also favoring the environmental protections provided in the existing regulatory regime. Indeed, this position is quite similar to the compromise proposal recently introduced by Sen. Dale Schultz (R-Richland Center) and Sen. Bob Jauch (D-Poplar).

So what did we learn from the two questions on mining? Clearly these questions allowed us to more deeply examine the effects of word choice on survey responses. However, comparing these questions also opened up a broader discussion about what one can take away from poll results. Yes, framing matters. But framing also includes how the media report on particular survey questions and the context in which poll results are discussed. Polls serve a number of purposes. They can be great for taking the temperature of the electorate (is the country heading in the right direction?) and for getting a sense of public support for candidates and their positions. But if there was one conclusion my students unanimously drew last week it was that we should not infer the complicated preferences of the public on the basis of just one question.

But what about the nature of public opinion more broadly? Is there such a thing as “popular will”? That discussion will come later on this semester.

Amber Wichowsky is an Assistant Professor of Political Science at Marquette University.