"Because I come from what might be called the practitioner side of things, I don't make a lot of distinction between evaluation and research. That doesn't matter to me, even though it seems to matter to researchers."
"The mantra that I throw on the table for my folks is: 'Open early and open often.'"
"Being contentious isn't necessarily a bad thing because it leads people to being critical about what value the goals have and for whom."
"I would posit that the majority of folks are informal learners and are not trained in that sort of methodology or formal interpretation. So it's a foreign language to the very practitioners who should be and could be using the knowledge and insights from research."
"One priority should be to be the best repository for the research and evaluation work that you can be. Another priority should be to make research and evaluation as accessible as possible to those who are not the ones who put it there in the first place."
Interview by Marti Louw :: Spring 2008
Louw: The first question is just talking about how research and evaluation—as distinct activities—can better support the design process in a museum?
Martin: In museums, where research and evaluation have had a lot of impact, one way is to include all of the folks who are involved in making exhibits in the process of evaluation. That way, everyone is involved in figuring out the issues and has a say in which issues evaluation should address. Everyone can actually do some data collection or observational things and be part of the process of sorting out and analyzing the results. Because I come from what might be called the practitioner side of things, I don't make a lot of distinction between evaluation and research. That doesn't matter to me, even though it seems to matter to researchers.
Louw: The Science Museum of Minnesota seems to have found a successful way of integrating research and evaluation into its exhibit–design process. Can you talk about lessons learned?
Martin: We have the advantage of being able to do what many organizations cannot do—which is to do everything from concept and proposal development to exhibit development, exhibit design, prototyping, production, all of the associated evaluations, remediation, and distribution. All of those things tend to either overlap or to inform each other in our process. So rather than having a process where somebody hands something off to somebody else who then takes it and hands it off to someone else, our process is more integrated.
A big part of our design process is engaging visitors in front–end evaluation and formative evaluation and testing the process throughout all of the stages of exhibit development. The mantra that I throw on the table for my folks is: "Open early and open often." So before getting locked into an idea or when there is still time and money to make adjustments, we test stuff fairly early with visitors and in a variety of ways. Rather than leaning heavily on what has already been done or our own assumptions, we get something into the hands of the public and see how it actually works. This iterative process means that there's more insight earlier on.
Louw: Learning goals seem to be something that can be rather contentious. How do you come to some agreement on a set of learning goals that are both inspirational and specific enough to be useful guides to exhibit designers, educators and evaluators? How do you define and use learning goals in your exhibition process?
Martin: Being contentious isn't necessarily a bad thing because it leads people to being critical about what value the goals have and for whom. The notion of what you're trying to accomplish needs to be informed by what you're trying to do, and whether it actually works for the people you're trying to get to achieve these goals. Sometimes the learning goals are for us: Here's what we want to learn from this project.
Often, however, the learning goals are for the ultimate users of the exhibits. What do we hope that visitors get out of this? We don't control that. One thing that I would posit is that an exhibit's ultimate authority is not the developers, or the curators, or the scientists. An exhibit's ultimate authority is the visitor. They're the ones who choose what to access and what not to access. The role of providing those points of access is very important and we have to use all of our insights, expertise, and passion to achieve that. But, if somebody doesn't engage with the exhibit, it doesn't matter what we think.
Louw: Tell us about one of your favorite exhibits?
Martin: Personally, one of my greatest exhibit experiences has been at the Butchart Gardens on Victoria Island. The thing that I really liked about it—as an informal learner—was that every time I didn't know something and I wanted to know it, something was there to help me understand what I wanted to know. So, if I looked at a plant and I didn't know its name, there was a label. But when I looked at plants that I didn't want to know their names, there wasn't a label. It was bizarre. I had this wonderful flow experience, where the environment and the story surrounding it and the interpretation provided were all appropriate. And it had nice food and places to hang out.
Louw: We talk a lot about research–practice cycles of influence and how research can help practice and how the questions of practitioners should influence researchers. There's a certain irony that the research community isn't necessarily designing its products for practitioners, who might be consider a kind of informal learning audience. Could you talk a little bit about that?
Martin: A lot of the research and evaluation findings come out of a formal education tradition and out of the discipline of writing and researching and reporting that is learned through formal education. I would posit that the majority of folks are informal learners and are not trained in that sort of methodology or formal interpretation. So it's a foreign language to the very practitioners who should be and could be using the knowledge and insights from research. Someone like me, who's slightly dyslexic and a slow reader, finds it excruciating to plow through that stuff.
However, I find the insights that are in research extremely valuable and have been important to my work. There are a lot of really smart, insightful people who have come up with it. In my experience, researchers generally talk about research in much better terms than they write about it. If research is also about facilitating informal learning, then you have to do it in ways that works for informal learners right out of the gate. If you have the conversation amongst folks who are already in the realm of research, that's fine, but you probably don't get any "knowledge transfer" outside that realm. Knowledge transfer is kind of a silly concept to me because it's unfamiliar to me as a terminology or as a way of thinking. In the informal world, knowledge is constructed on the spot by building on stuff that you already knew, were engaged in, or are emotional about. In the informal world, there's much more engagement through emotion as the starting point than there is through cognition.
Louw: You mentioned that you have gleaned insights from research. Can you recall any insights that matter to you or which may have affected a decision you made or your work generally?
Martin: For me, personally, they've tended to be pretty big things. We did a lot of research when I worked at the Minnesota Historical Society. We were going from being a very small museum within a big historical organization to a pretty big museum within a bigger historical organization. We needed to reach a much broader public than we had done before. To do that, we conducted a lot of visitor research about how the public thinks about history in contrast to how historians think about history. For historians, history is a seamless continuum: There's everything that's ever happened and there's everything that's going to happen. Most of us who are not historians think about history as everything before my time, everything during my time, and everything after my time. Our lives tend to be the reference points for history. That's an interesting way to think about how you have to access history differently for people who are not historians. It's a much more emotional, personal kind of place.
When the Science Museum of Minnesota was in the midst of planning a new building to open in 1999, we did a summative evaluation of our old museum while also doing a front-end study of what we planned to do in the new museum. The staff's values about having fun and learning were embedded in each process. As a staff, we were arguing about what we would do for the people who came to have fun and what we would do for the people who were coming to learn something. Could we segment the audience by those who come for fun and those who come to learn? If so, we figured we could make sense out of those differences and plan appropriately. However, the interesting thing we found out was that visitors—whether they were young kids, middle aged adults, young adults, family groups, or visiting on their own—all came to have fun and to learn. Visitors didn't pick apart these things, although individuals have different ideas about what's fun and how they learn. Rather than a one–size fits all approach, the research showed the complexity of why visitors come to the museum—it's an enjoyable experience and they expect to learn while they enjoy themselves. Sometimes enjoyment can be very focused and serious, so fun is not all fun and games. Once the staff understood this, we no longer had that argument. That was very important at that time in our planning.
The second thing we found out was also pretty fundamental: There seems to be a direct correlation between understanding and enjoyment. Folks who understand what an exhibition is doing, enjoy an exhibit more than those that they cannot figure out an exhibit's purpose, or at the very least, how an exhibit's components work.
The third insight was that visitors value the things that they cannot see anywhere else, things that they are able to learn by doing, and things that are relevant to them and help them make sense out of their own lives. These became wonderful guiding principles that the staff worked out in conversation with the evaluator. These things rise to the top and are the things that you remember and you use.
Louw: What do you think the priorities should be for InformalScience.org going forward?
Martin: One priority should be to be the best repository for the research and evaluation work that you can be. Another priority should be to make research and evaluation as accessible as possible to those who are not the ones who put it there in the first place.
Louw: In what kinds of forms and channels would you like to see this research knowledge made available so that it would be more palatable and useful to the people you work with?
Martin: Creating synthesized versions of research findings—a sort of CliffNotes—would be invaluable. Adding the next layer of mediation would be very helpful.
I also think that more sophisticated ways to search the database would make it more useful for folks who aren't researchers. Right now, the filters used to search the database probably work great if you're part of the research community and you already know the names of other researchers. But if you're a designer—who is loathe to use any kind of audience research anyway—it isn't as useful. For example, dates don't mean anything and authors don't generally mean anything to designers. If you want to direct designers to the site, then it has to be usable in their terms. Key words searches around projects, topics, or even disciplines would be where people are coming from. We need other criteria for practitioners to get much out of it. I guess I'm saying that you want the search to be more "googlish." That should be really just easy to do, huh? (Laughs)
Louw: How will informal science projects look different five years from now?
Martin: Frankly? They probably won't look that much different than they do now. We can push on some things, but exhibits look similar to a certain degree because exhibits build on each other. As active as I am in the field, I don't know that I have a sense of some standard for "different." Each exhibit is always a little different. Sometimes it's "different good" and sometimes it's "different bad." That's a fairly subjective measurement on the part of those of us in the field and what we understand about how visitors interact with exhibits. If I had a wish, it would be that we have a stronger connection to visitors and how they actually engage and use and value what we produce.
Louw: What do you see as being the major challenges for informal science in the next five years as a field?
Martin: Speaking from the perspective of the science center field in particular, a major challenge is that most of the investment in the past twenty years has been in new buildings, and to some degree, the exhibitions that go into new buildings. That era is pretty much over. Science centers that created huge new buildings now need to do something different with what's inside those buildings. We've already played out a lot of the value in what was created. What's next is to create new value so that science centers continue to be valued by the public.
Major investment is needed in a fairly challenging funding climate, but this could be a really cool opportunity to create new models. Basically, the Ontario Science Center has just done that as part of a major initiative. They took a building that was done 30 or more years ago and rolled over 30–40% of what visitors can do. First, they fundamentally changed the audience they serve by redesigning for families. And second, they reinvented the experience so that visitors can do more engineering and do more problem–solving on the floor. They did all of this with an eye towards changing the relationship between the science center and the folks who already use the science center and those folks who aren't using it yet.
Here at the Science Museum of Minnesota we use a lot of different strategies for making sure that the physical place has value for different user needs—from family science nights to activities with diverse communities to blockbuster exhibits that people cannot see elsewhere. We recently developed and premiered an exhibit on race with the American Anthropological Association. Not only was this exhibit a unique experience for our visitors, but it also provided a way of re-conceiving exhibitions and museums as places where sensitive social issues and science can be confronted safely. It’s a matter of matching the mission with the needs and values of people, but of course this means that we have to be in really close touch with the people we want to connect.