|
Transcript from a portion of the conference, Communicating the Future: Best Practices for Communication of Science to the Public, March 6-8, 2002. Panel
Discussion: Case Studies in Successful Evaluation of Communications Programs [Due to a technical problem with the recording of the conference, this transcript begins after brief presentations had been made by Kathy Geiger, Brookhaven National Laboratory, "Restoring Public Trust: Community Affairs Evaluation"Chris Gade, Mayo Clinic, "Medical Edge: Distributing Health News Through Local Television," Cindy Ferch, Orange County Water District, "Groundwater Replenishment Community Outreach Project", and Terry Devitt, University of Wisconsin, Madison, "The Why Files."] Jon Miller: As a team, we're managing the high technology. Let me just take one minute and say I think that one of the things I hope you'll get from this set of four presentations, which were not selected randomly, is that science communication is, in fact, a very diverse activity that involves knowing things from using the Internet to using local television to using community advisory groups, but often times it involves very specialized audiences; people who are potential patients to Mayo, people in a particular water district; people who have particular or other kinds of interests. And sometimes we speak in broad generalities, we speak of "the public," which by this time I hope everybody agrees there isn't such a thing as "the public," but there are many publics. I think that one of the reasons that not only the four cases are highlighted here today, but all of the cases are demonstrated out in the poster sessions, illustrate to the selection committee a very healthy and vigorous diversity in ways of communicating science, and the ways of conceptualizing audience. And I hope as you look at these things, you'll look at that aspect to it, also, because I think that part of the richness of this Best Practices conference is that it illustrates the breadth, if you will, and the scope of science communication. So with that brief observation, let me now invite you to ask questions of our panelists, and we will - also, in order to have these tape recorded in the videotape, if you would raise your hand and let Gail hand you a microphone, that also will help making sure your words are recorded for posterity. Over here first, I guess. Gail Porter: I feel like Oprah. Miller: Yes. Question: I have a question for Cindy Ferch. I wanted to know what the most interesting lessons your project learned were from the experiences in Los Angeles and San Diego. Was there anything interesting or unexpected that you benefited from? Cindy Ferch: Good question. We feel that one big reason that both projects failed, or were delayed - the project in Los Angeles was delayed, it is now finally going forward, was because politicians used the project as free advertising for their political campaigns. So, that's one of the things that we went to first, and we talked to all the political leaders in Orange County, and we said, "This is the project. This is why we need it." And that's continuing with our PR program - is to make it unpopular for somebody to be against the project, so that if some politician does decide to use it as a football, we have other politicians who are ready to say, "No, don't do that. That's not worth - this is a good project. Orange County needs this project." So, that's probably the one biggest issue. The second issue is that there was a perception of environmental injustice. The wastewater was coming from a rich neighborhood. The purified water was going to be served in a poorer neighborhood. Those were perceptions. They weren't completely true, but perception is reality. Miller: Ah, here first. Yes. Question: Just following up on that, the opposition, what was the gist of it, mostly? Ferch: You can't make me drink sewer water. Yuck. You know, that's gross. Question: So, it wasn't anything technical? I mean, they didn't say that mercury if present or - Ferch: Some of the other issues were related to, "You haven't told us," you know, "You're sneaking up on us." Even though in particular - I guess in both instances, the public officials who did take this position had been fully briefed and had voted to support the project. So, it's not based in rationality - you know, is it ever based rationally? But they used it, and they got people who are, perhaps, not as well educated to say, "Yeah, that's gross. That's unfair. We don't need it. We can do something else. We can spent money on desal [desalinization] even though it's three times the cost. You know, it comes down to "yuck." That's our biggest challenge, "I don't like that idea." Question: Okay. Just one little question about the Mayo tapes: How often does it happen that you're unhappy with the way that one of your productions is clipped? Because, for example, in the hiccup tape, which I'm sure was excellent, the 90-second version, if you clip it, people may get the impression that the tumor as a cause is much more prevalent than what it really is, because that makes a sensational story, and that's what they want to promote. Chris Gade: Actually, there's only been one specific occurrence, to be honest, where we've experienced real distaste with how they chose to portray it. There was a story about a patient from Lebanon who had a brain tumor and they, to make the story more effective, said that he had been shot in his head in Beirut, which actually was wrong. It actually happens fairly infrequently, and the majority of stories -and again, what you saw was an edited version of the air check. The vast majority of the stories are running almost as we script them, which you can talk about from a journalistic perspective whether or not it's appropriate or not, but the truth is their almost, their just inserting their talent over our script in almost every case. Miller: Over here, first. Question: Hi. One of the panelists, I forgot who it was, but somebody had talked about how credibility can be lost if funding or if the research looks like it's coming from a vested interest, and I was just wondering, what if an industry, such as maybe the beverage industry, wants to do a study showing that there's nothing wrong with drinking carbonated drinks or drinks with these chemicals in them, or any industry at all wants to fund something like that, is that something that they should just stay away from or they should hide or they should come out forward and say, "We are involved in this study, and it's still good scientific research."? Anybody would like to answer that? Gade: From my perspective, it's just essential that however that study's funded has to be fully disclosed when the findings are released. I wouldn't encourage anyone to not fund a study because they have a special interest in it, but the organization has to fully disclose what the funding source was. Miller: Over here, next. Question: Yeah, I was curious, is the Mayo Clinic for profit or not for profit? And I was also wondering if you could comment a little bit on the nature of the agreement that you have with these different broadcast news organizations and whether any money is exchanged during this contract? Gade: Mayo is a not-for-profit organization. The funds we generate through our clinical practice are plowed back into education research. The agreement is simply a contract, there is no exchange of money or sponsorship at all. We would never go in that direction. It's just - what it is is basically an agreement that they agree to regularly -that's all we say-- use what we send them. Miller: Right here. Yes. Question: Cindy, I have two questions. First one is, as I recall, you said you were going to flip the switch in 2006, so how valid do you think that data is, now, considering the fact that it's somewhat in the future. And the second one is, it seems to me that if the water tastes bad, all the PR in the world isn't going to help you, and my question is, how does the water taste? Ferch: We think that the data that we have is valid. We feel that people are - and that's based on both our telephone surveys and the focus groups, because when we talk people through the project and the focus groups, most of them come out accepting it at the end. Going forward, because we're going to be stepping up the PR program, we think education is key, because as I mentioned, in the focus groups or when I'm talking to a group from the Rotary or whatever, I say, "Wastewater to drinking water," everybody goes, "Yuck." I take twenty minutes to explain the scientific process, everybody's shaking their head like, "Yeah, that makes sense. That sounds good. It's better quality water than what we're getting now. Let's do it." So we feel that as we step up our education program, and as we educate the general public in Orange County, it's going to get better. And we want to raise awareness, so our goal is to raise awareness and to increase support. You're second question, "How's the water taste;" that really varies throughout the city. I will tell you - throughout the county. There are several cities, Huntington Beach and Fountain Valley, I believe, who have won national taste tests, but it does vary because different cities use different ratios of ground water to imported water. They use different amounts of chlorine; so obviously, chlorine has a lot to do with it. So, a lot of people don't like the taste, I will admit that. And it's very sheik to use bottled water, and now people are developing that as a habit, where they just drink bottled water. So, those are some of the challenges that we have to educate, you know - as a whole other subject of educating on the safety of tap water. So, those are challenges that we have. Miller: Back here. Question: Ah, yes, I'm from the American Institute of Physics, and we also have a syndicated television program of 90-second reports. About one-third of them are related to physical sciences, a third related to earth sciences, and a third related to medical and life sciences. We do sell the program, as of about two years ago, to TV stations throughout the United States. There's 211 markets, and we sell, right now, to one, exclusively, in each market, and we're in 80 of those markets. Now, our goal is for it to be self-sustaining at some point. My question is in reference to audience impact, and our board and our advisory committee is always asking, "Well, how do we measure audience impact?" And in fact, our goal is to increase the public's appreciation and awareness of the role that science and scientists play in their daily lives. We certainly can measure the - whether the stations are airing the programs, but the question is, are we indeed having an impact? We do know that the majority of the audience cite TV, local TV news, as their primary source of information, and if you want to have a chance at reaching that audience, you have to have a presence in that medium. But then again, how do you measure whether or not we are increasing public awareness and appreciation? Porter: I think that's for you, Chris. Gade: Was - I'm not sure what the question is. What - is there a specific question you have, or - ? Sorry. Question: Yeah, I'm asking, I saw that you were a - you developed an evaluation for your program, and you basically are evaluating it based on the Nielson Ratings and whether the stations are airing it. And you also did some surveys, but it was more - your goal was to determine if people were more aware of the Mayo Clinic. Okay. I'm saying that our goal is not an institution, but it's to increase awareness and appreciation of science. So, how then would you - I'm asking, whether it's you or other people on the panel, for ideas on how you measure the impact of this on the audience. Gade: I think measuring the impact is incredibly hard to do. The pre and post test is about the best you can do, perhaps, in those markets that you're in. Your challenge is even more difficult, because you're just asking for general awareness. I have no specific, you know, nugget of information that would help you. I think just doing some sort of pre and post-test is your best bet. Porter: John, do you want to add anything to that from all your many years of public opinion survey and public and knowledge surveys? Miller: Yes. We've been involved in evaluating projects like that, and I think that what you have to do is get a baseline, ideally before you begin to putting it to the evening news, and then you have to monitor that over a period of time. I think every learning model that I'm comfortable with says it's a cumulative process. It's very hard to take a single segment, or even a week of segments or whatever, and get a reading out of that. The meter just doesn't move for small increments. But over a period of time, if you are adding information of that kind, if your objective is appreciation and awareness, I think that you can see something, but it would take at least six months, if not a year, for you to see something that's - the critical point is to have a good baseline before you start. It's also very useful to do it in other communities where you're not running it, because if the whole country goes up, which sometimes happens, then you can delude yourself into thinking that because you've got a three-point or a five-point gain in one community, that it is entirely due to your commercials or to your inserts. And you want to also make sure that you're monitoring in markets that you're not in to see if there's a more generic change that's occurring at the same time. Question: Right, what's the cost of that research? What would be the cost of that research for an organization? Miller: Oh, it would be probably in the order of $200,000- $300, 0000 if you did it over a year; something in that ballpark. Pardon? About two or three markets, yeah. Okay. Back here, I guess, is the next hand I see. All right. Two more questions, then we're going to go eat chemicals. Question: Brief question in regards to the ground water study: In Europe, it's not unusual for a community downstream to be drinking water which has been recycled two or three times. Have you involved external experts from other places, either in North America or Europe, in explaining your issues, and if so, have they been accepted? Ferch: Yeah,
that's a good question. Actually, in Orange County, we're replenishing
the groundwater basin with tertiary-treated wastewater from upstream communities,
so we're doing that in Orange County already, but a lot of people aren't
aware of it, you know, so we're - we mention that, as well, when we go
out. We're already doing it. This is better quality water than what we've
been replenishing the groundwater basin with. As far as experts, we do
have advisory - scientific advisory panel that is made up of scientists
from around the world: toxicologists, epidemiologists - my mind goes blank,
but yeah, lots of water-quality people that have studied this water, compared
it to the other water sources, and say that this is as good as or better
than any other water source that we have. We recently developed a white
paper on that water quality issue, so we'll be using that to distribute.
And we're - yes, we do have some who are working as spokespersons, but
we're still working to develop that so that we actually have those scientists
speaking out as spokespersons. Yes. That was one of the messages that we tested in our focus groups was, "Who would be the messengers that people would accept?" And scientists and health officials, especially, we're really working at reaching the doctors in Orange County to get them onboard and try to get a couple doctors as messengers, as well. Miller: Yes. I guess the last question goes over here if we have a microphone? I'm sorry - Question: I've got the microphone. [laughter] Miller: Ownership of the microphone is very important. Yes. Question: Chris, you mentioned that it didn't necessarily need to be the M.D. who was the expert in your insert. Can you comment on the frequency that you used Ph.D.s, for example, as the expert, and did you evaluate if they were as effective? Gade: Actually, the focus of that comment is, "Does the person who is the talent onscreen need to be an M.D. versus not? Versus, should there be an M.D. in the story versus a Ph.D.?" What we've found in the focus group research is that they didn't necessarily need a physician host of the segments. I can tell you that in our segments themselves, the vast majority, because we're primarily a clinical practice as opposed to a scientific research institution, is, I would say, probably 60-65 percent physician-M.D.s in the segments. Miller: Okay. I think with this we're going to stop to keep roughly on schedule. Let me just remind you that lunch will be served in the same place that you had coffee at the coffee break, which is the West Court of the cafeteria. It is cafeteria-style, and so, proceed west.
Created:
7/14/02 |