Means to an End
Means to an End
Abstract and Keywords
Our last principle of social change, covered in Chapter 5, involves measuring success in terms of what has been accomplished and what is still to be achieved. Multiple methods are needed to analyze both the nature of a social problem and its possible solutions. These methods include documenting changes in policy, observing changes in practice, and gathering self-reported data from people within the community. Chapter 5 also examines how these types of methods have been successfully used on an array of policy issues, including litter abatement, handicapped parking, and media-based studies. No matter what the different strategies, types of measurements must be developed to define and detect when, and if, change has occurred. In determining how and what to measure, we go straight to the source, working with community members who are directly affected by the environments being altered.
We have seen how second-order activism can improve people’s lives and bring new resources and opportunities to a community by identifying the power holders, overcoming the obstacles they create, and garnering the support of like-minded others. Our next principle, evaluating short- and long-term strategies, may sound dry. Although it lacks the glamour of hands-on social activism, it is essential to long-lasting success. Many social activists, groups, and organizations mount social justice efforts, but the activists who are effective often evaluate their methods along the way.
In some cases, the results of activism are obvious, such as the ousting of a dictator. However, the effects of social action are often not immediate or easily identifiable. Take, for instance, Rachel Carson, a scientist who compiled findings from diverse areas to bring an extremely important issue to the public’s attention, leading to second-order change. After World War II, deadly pesticides were being used everywhere—in homes, in gardens, and even on commercial crops that ended up on family dinner tables—but no one was aware of their danger, least of all the public.
Carson wrote a groundbreaking book in 1962, Silent Spring, which detailed the widespread use and serious dangers of these poisons. A writer, scientist, and ecologist, Carson synthesized the results of hundreds of disparate scientific research articles to eloquently illustrate our society’s predicament. Silent Spring portrayed, in graphic detail, the toll that dangerous pesticides were taking on animals, the environment, and people. Although insulated in concrete cities, we were slowly destroying the natural world with toxins. In particular, DDT, a commonly used pesticide at the time, was destroying all types of wildlife. Immediately following the publication of Silent Spring, Carson was viciously attacked by the chemical and pesticide industries for her outspoken views. Despite her personal struggles with cancer, her reverence for nature and determination to protect it for future generations kept her dedicated to her work. Her book moved President Kennedy to create a special panel to study the effects of pesticides. Before Carson died of cancer, she saw her theories vindicated. She (p.110) testified at congressional hearings, and ultimately, her work led to the banning of DDT and the enactment of the Clean Water and the Clean Air Acts.
Carson’s 4-year social change effort was a success; her work documenting abuses in the pesticide and chemical industries led to the enactment of federal policy laws. She is now hailed as one of the first environmentalists, and her book is regarded as a watershed event in the pursuit of safeguarding the environment. However, many social changes do not reach the level of national legislation—but even smaller victories can be measured when compared to both their short- and long-term goals. Activists can gauge a program’s success and reassess strategies by identifying where progress is, or is not, being made.
The different principles of social change can occur concurrently or in varying orders. Vulnerabilities in any stage of the principles can put a social movement at risk. For example, if an activist has a clear vision of his or her desired social outcome but does not take into account the aspects of power that are resisting the change, he or she may fail. Failure can also occur if an activist organization has a clear idea of its desired second-order change and recognition of opposing power structures but does not have the capacity to organize and work with communities. An activist must be aware and informed of each step of the movement. While the dynamic nature of social activism campaigns may make evaluation challenging, measuring progress is vital and lies at the heart of each community strategy. Evaluations capture and help redefine the essence of each principle of social change. Program evaluations can also reveal weaknesses, identify where the weaknesses stem from, and help focus efforts on critical stages of systemic change.
Given the fluid nature of a campaign for social change, an activist may not know the best way to measure changes, or even what changes to measure. For example, a nonprofit organization might design and implement educational programs for inner-city preschoolers. Many of these types of programs frequently only focus on gains that measure academic achievement; but in fact, the real second-order changes may be the creation of new jobs for the mothers who participate as teachers in these programs. In this situation, effective evaluations can open up new realms of possibilities and can help an organization redefine itself from a first-order service provider to one capable of achieving second-order social change. The change agent must know to look out for shifts in results and be flexible with predetermined goals.
Collecting data to reveal organizational problems is a critical step toward enacting policy changes. In the fight against Big Tobacco, we sent young people into stores in Woodbridge, Illinois, to see if they would be allowed to buy cigarettes; their purpose was to gather data that would be used to gauge the rate of illegal sales to minors. This research backed the passage of the federal Synar Amendment in the 1990s that required all states to reduce illegal sales of tobacco. Before passing that milestone legislation, however, we had to first (p.111) achieve smaller gains. Initially, Officer Talbot assumed that by sending storeowners in Woodbridge a written notice, tobacco sales to minors would no longer be a problem. After our data collection, we learned that the notice strategy had not worked—storeowners and their employees were still selling to minors. This information led to us creating legislation that gave police more authority to fine owners, as well as to give tickets to minors if they were smoking in public. If the Woodridge Police Department had not partnered with us to determine the extent of the problem, they would not have had a realistic picture of the situation.
After several months of fining storeowners, Officer Talbot was pleased to find that the number of illegal sales had dropped significantly and thought that enforcement was no longer needed. I, however, believed that illegal sales would increase again if the police stopped their efforts and recommended that they continue monitoring stores and fining those that continued to illegally sell tobacco to minors. After another 6 months, rates dropped to almost nothing and, because of sustained enforcement, they have remained that way for over 20 years. Data was critical in guiding the enforcement effort over time. With persistent monitoring and enforcement, we were able to set an important precedent for the entire nation regarding tobacco sales to minors and help reduce a serious public health problem.
Without consistent and widespread assessments, most of the victories described in prior chapters would not have been possible. As we discussed in chapter 2, little attention was being paid to the devastating illness of myalgic encephalomyelitis (ME)/chronic fatigue syndrome (CFS) in the early 1990s. Community-based research into the prevalence of the illness allowed us to confront the myth that it was a rare disorder. I worked with epidemiologists, research methodologists, statisticians, physicians, psychiatrists, and psychologists to challenge the Centers for Disease Control and Prevention’s (CDC) unrealistic low-prevalence estimates. Our evaluations later opposed the grossly inflated prevalence estimates when the CDC broadened the definition of ME/CFS and counted people who did not have the illness. Our work benefited a coalition of patient groups and research organizations and contributed to a change in the CDC’s ME/CFS leadership.
Evaluations also help to clarify accountability within organizations and identify new avenues for second-order change. In the case of Oxford House, we were able to directly assess the difference between first-order change (sending people to the same communities in which they had lived before addiction treatment) and second-order change (placing people in Oxford Houses following treatment). We learned that Oxford Houses allowed people to develop abstinent friendship networks and observe role models who successfully held jobs. In our studies, a random half of a group of people leaving treatment centers for alcohol or other drug problems were placed in Oxford Houses in order to determine (p.112) whether or not the Oxford House experience would help people stay abstinent, employed, and out of jail. What we found was just that: The people who were provided an opportunity to live in an Oxford House were twice as likely to remain abstinent over the next 2 years. This second-order intervention had a powerful influence on the lives of people who were offered this empowering setting. These results garnered much attention and helped to legitimize the Oxford House model for public policy officials. More significantly, it provided support for the expansion of the model throughout the country. Based on our work with this organization, the federal government listed Oxford Houses within the National Registry of Evidenced-Based Programs and Practices (http://nrepp.samhsa.gov/ViewIntervention.aspx?id=223).
The Many Roles of Research
Participants within program communities have always played a large and critical role in my efforts for change over the years. Active participation of community members in the planning, execution, and evaluation of research is vital for any social change. Referred to as participatory approaches (Jason, Keys, et al., 2004), these markedly differ from traditional models of research in which scientists generate their own ideas about the research questions to ask or the services clients need. In other words, in traditional research, community members are often excluded from the entire process. In contrast, participatory approaches allow community members to shape the research questions and inform researchers of what services are needed by identifying the criteria or standards against which effective service should be judged. This method promotes not only collective, reciprocal, and egalitarian relationships but also a common language (Jason, Keys, et al., 2004). Participants enrich and expand the goals of the project and offer perspectives that are often overlooked. They have a unique understanding and knowledge of the issues we’re trying to change; in fact, many of the best ideas come from them.
At a community intervention level, these kinds of participatory approaches often begin with a community needs assessment. For an activist planning to implement new service programs in a community, such as a health promotion or prevention program, the first step is often to conduct a community needs assessment. In this way, instead of bringing a preset program agenda to a given community, the activist allows the community to set the agenda based on its own needs. Through assessment strategies such as holding community forums or meetings with a selected group of community leaders, an activist or agency can understand the most pressing unmet needs in a community while developing rapport and collaborative relations with community members. Furthermore, with a high degree of collaboration, the program will be more consistent with (p.113) the characteristics, practices, and values of the community and, thus, more likely to be adopted by community members (Rhodes & Jason, 1991).
Participatory approaches are also needed in larger social activism efforts. There are many ways to document the success rate of a change effort and just as many ways to chart its progress. When trying to analyze the nature of a social problem or find a solution for it, surveying changes in attitudes, behavior, or policies is critically important. Simple self-report methods that use rating scales and questionnaires can track these changes, such as asking open-ended questions regarding how community members feel about the social issue you hope to address. By measuring public opinion and knowledge through polls or surveys, you can identify the roots of any public opposition to your cause and develop education or awareness initiatives that target those beliefs. Likewise, you could discover that the majority of people surveyed already support the change you seek, and you could use this as powerful support to motivate lawmakers to act. When Dr. Pick hoped to introduce sexuality education in Mexican public schools on a national level, the Minister of Education asked her to provide evidence of public support for the issue. She was able to have some sexuality education questions included in a national Gallup poll, and results showed that 95% of Mexicans believed sexuality education should begin in the first grade (IMIFAP, 1993). The poll results provided Dr. Pick with the proof of public support that she needed to sway powerful opponents to her side.
On a smaller scale, it is also important to gauge how community partners feel about their involvement in the program or research project. In one instance, we assessed the attitudes of community members who worked as participant recruiters for our Oxford House research (Jason, Davis, Olson, Ferrari, & Alvarez, 2006). As the quotes below indicate, it was clear that their involvement with us had been a positive experience.
The group of us, I would say, definitely formed into a family, similar to that of Oxford House.
I felt that my ideas and experience were really important and valued in the shaping of this study….It gave me self-confidence and made me realize that I had something, a lot to offer….[B]eing part of such an open-minded, nurturing research team allowed me the opportunity to really open my mind and learn a lot of new things and different ways to look at things.
I am now attending college full-time at age 45….[M]y association with the DePaul students and staff has motivated me to want to earn a degree. My involvement with the Oxford Study probably will be one of the most significant milestones in my life.
(p.114) Although self-reported data are essential in measuring the effects of our change efforts, it is equally as important to look for actual changes in people and their environments. Prior to analyzing these changes, however, information and feedback must be gathered. The feedback-gathering process once happened very organically for me at the end of a presentation I gave at an annual Oxford House convention. An Oxford House member approached me and suggested that we include the issue of tolerance in our future research. He explained that prior to living in an Oxford House, he had been very prejudiced against people who were different from him, such as people who were HIV positive. But while in Oxford House, he met a woman who was HIV positive; she was now his girlfriend. He wanted to point out that, for him, living in Oxford House went beyond staying clean and sober—it made him a more accepting and generous person. After hearing this information, one of our team members began a study on the development of tolerance among Oxford House members. What we learned confirmed this man’s experience: Tolerance increased by living in Oxford Houses (Olson, Jason, Davidson, & Ferrari, 2009). Had we not listened to this member’s feedback, we would not have known about the opportunity to study this additional, important benefit of Oxford House.
Before describing other examples of the ways in which the fifth principle contributes to the goals of social activism, a distinction should be made between the types of research and evaluation involved in this process. At times, the issue at hand requires rigorous research to uncover and expose injustices that demand public attention and correction. Like Rachel Carson’s years of research and writing about DDT and other harmful chemicals, the process of conducting this research can be the primary action of the activism, and the wide dissemination of the research itself leads to second-order change. A similar process occurred with the ME/CFS research on prevalence rates and later investigation of the CDC’s use of ME/CFS research funding—the research, along with the advocacy efforts that it supported, spurred change.
In other cases, research takes a supporting role to social activism. Formative research provides guidance to inform action and design and develop initiatives for change. If your aim is to improve education with a new curriculum for elementary school children, you may first do some research to determine what areas of learning your program should target. Then, once you have developed the curriculum that targets those areas, you could test the educational materials with a small group of students to see which sections to improve before introducing the program in schools. Both are examples of formative research. Formative research helps initiatives succeed by taking an informed, comprehensive approach to solve the issue before taking large-scale action.
On the other hand, summative research allows activists to evaluate the success of their actions after they have been implemented. For example, working with the Woodridge Police Department, our summative evaluations of the (p.115) different approaches we took to reduce illegal tobacco sales to minors let us know which interventions worked and which needed to be altered. With each cycle of evaluation, adjustments fine-tune the interventions while providing the information needed to create fresh ways to tackle major social problems (Bogat & Jason, 2000).
Formative and summative research are conducted by academics, activists, nonprofit organizations, governments, international agencies, and others who share the common goal of developing and evaluating successful interventions. Although most of the following examples focus on research and program evaluations that I conducted as an academic researcher at DePaul University, the fifth principle of employing research techniques to measure success applies to scientists and other social activists alike. Contrary to popular belief, collecting data does not have to be boring or repetitive, nor should it be feared. Parts of the evaluation process are extremely straightforward, and community members and partner organizations can help streamline it by sharing their priorities with the appropriate people.
An Unlikely Research Topic
In the late 1970s, I once invited a representative from Martin Oberman’s office, a Chicago alderman, to speak to the students of my community psychology course. After his presentation, I asked the representative which problem generated the most community dissatisfaction. I had privately pledged to work on whichever problem he mentioned. When he answered “uncollected dog feces,” my jaw literally dropped.
Who was resisting change here? Not corporations or politicians. It was the citizens themselves—or rather, citizens who owned dogs. Not picking up after a dog was the most offensive and most frequently complained about type of litter in urban, residential areas, and a less-than-desirable research topic. However, I’d made a promise to the representative and agreed to collect some data on this problem. I surveyed a group of university undergraduates about their opinions on dog droppings. The students also ranked dog droppings as the most offensive type of urban litter. More importantly, dog feces were linked to several well-documented, widespread health hazards in Chicago, including the transmission of infections and parasites. And, of course, dog droppings defaced a community’s beauty and attractiveness (see Figure 5.1)—not to mention the extreme aggravation experienced after stepping in them.
I needed to address this problem with a systematic approach. At that time, I had a new clinical psychology graduate student working with me. I knew he probably wouldn’t be thrilled that he would be studying dog litter for his first assignment in graduate school. I wasn’t sure how to introduce the idea, but I carefully (p.116)
For the next few months, Francis and I worked on a study to find ways to deter dog litter. To get an initial measurement of the problem, we counted all fresh defecations in an eight-by-five-block area around DePaul and found 1,147 droppings. This large number suggested that dog litter was a serious problem within this community. We next picked a target block and, for 5 hours each day, recorded the number of dogs, the number of dogs who defecated, and the number and weight of dog defecations picked up by their owners. Unfortunately, to accomplish this, we had to pick up and weigh all defecations each morning.
We discovered that there were a few unforeseen, unpleasant consequences of this research, besides actually counting, picking up, and weighing every dog dropping. For instance, even after I left the office, I found myself counting dog droppings on walks everywhere I went. To add insult to injury, my prolonged exposure to waste and the pests that they attract caused my apartment to become infested with fleas. Several of my research assistants mentioned that they had recurring nightmares about picking up dog feces.
Our evaluation used a simple research structure: We first collected data on dog litter pickups and then introduced an action—posting anti-dog-litter signs—to see if the dog litter pickups increased (Jason, Zolik, & Matese, 1979). For activists, analyzing a behavior pre- and postintervention is a good way to determine if your action or intervention is effective; if behaviors change after the intervention, you can assume that your approach is effective. During the initial week of observations at the target block, we saw that very few people picked up after their dogs, resulting in over 19 pounds of dog defecations left in the area. Next, we posted the anti-dog-litter signs, hoping to change owners’ behavior. Unfortunately, these signs were ineffective and did not change owners’ behavior or reduce the amount of dog feces deposited on the block. Yet, when all dog owners were given instructions and a demonstration on how to use a plastic bag to pick up dog feces, 82% of the dog owners picked up after their dogs. Although changes in observed behavior are important to document, comments from local residents can also be valuable in data collection; we recorded some interesting reactions from dog owners:
“Dog shit isn’t important. It dissolves. Why don’t you do something about litter?”
“You’re transient and what you are doing is kind of pissing in the wind.”
“If you are such a concerned citizen, follow behind and pick up.”
“I’m enthusiastic with what you are doing and am going to try to get a dog barrel on this street.”
“Leaving defecations on lawns is disgusting and I am encouraging others to carry bags.”
We provided the Chicago alderman with details of the results of the study. He later invited me to testify at city hall to support an ordinance that would require dog owners to pick up after their dogs. At a televised hearing, I shared the findings of our “prompting” intervention, which effectively motivated dog owners to dispose of their dogs’ waste. After this testimony, a journalist for the Chicago Daily News (July 12, 1977) wrote: “In what surely must be one of the most bizarre academic studies in the nation ….” Using data from our study, this reporter estimated that 382,000 pounds of dog excrement was deposited daily on city streets. When I later contacted journalist Ellen Warren regarding a few inaccuracies, she replied that “the story generated great readership interest.” This story was followed by an editorial in the Sun Times that read:
We’re not sure what contribution this study makes to the discipline of psychology, but if it persuades the city council to pass a stronger ordinance to discipline dog owners, it will have been more than worth the effort….The pursuit of knowledge aside, it did manage to clean up one neighborhood.
The study had hit a nerve—so much, in fact, that the police in one district began dispensing plastic bags to dog owners, and there was even a photo opinion asking residents: “Should owners clean up after their dogs?” A cartoon in the Chicago Daily News of July 22, 1977 (page 11), portrayed a dog wearing a diaper with the title: “Solution to a problem.” I received lots of fan mail, including one that said: “This is a lost cause. The only solution is to hire out-of-work people who can clean the parks and parkways.”
Following the completion of our study, several community groups contacted me about setting up their own dog-litter-removal campaigns. One group in particular eagerly wanted to work with my team in identifying which streets had the greatest accumulations of dog litter. They set up a 1-day action initiative during which community members approached dog owners to inform them of the recently passed ordinance and also offered them newspapers or plastic bags to pick up after their dogs (Jason, McCoy, Blanco, & Zolik, 1980).
For the first few days after the campaign, we learned that rates of dog litter had declined by about two pounds per day. But over the next few weeks, the dog droppings increased. The 1-day intervention was not as successful as we (p.119) had anticipated. A big reason, we assumed, was the timing of the initiative: We staged the activity over a weekend when owners may have been out of town; thus, a sizeable proportion of dog owners were not reached. Furthermore, we realized that a single day was not enough to create new behavior patterns. Not every change effort will work every time, but failure is also valuable: Community members can better understand how difficult it is to change behaviors.
Rather than becoming discouraged, the locals reconvened with us to learn from the data. After reviewing it, they had renewed confidence to tenaciously launch a longer “prompting” intervention, set over a period of several weeks. Their persistence paid off—pickup rates increased nearly 90%. One unexpected benefit from the more intensive intervention was a positive ripple effect that was triggered from the target block of our intervention to the surrounding areas at a 13-month follow-up, as shown in Figure 5.2. We assumed that this ripple effect occurred because dog owners frequently walked their dogs in a wider area than just their own neighborhoods. When the pickup behaviors among owners were strengthened in the target areas, they were likely going to continue this behavior in the adjacent, nontarget blocks.
As we saw in our second attempt to educate the area about the new ordinance, community groups certainly have the power to take charge of data collection. They can then apply more refined techniques, tailored to their particular needs, to achieve documentable success. Residents first took the initiative to identify
The successes from these two interventions, along with my testimony at the city council hearing, helped make Chicago one of the first cities in the country to pass a “pooper-scooper” ordinance. The Chicago alderman later wrote to tell me that our project helped change politicians’ attitudes toward the issue: “In the past, this problem has often been scoffed at and not taken seriously. Your comments regarding the dog defecation problem altered that perception greatly.” The initiative spread to towns all over the country, and similar ordinances were adapted from the Chicago legislation.
After all of these successes, I attended a psychology convention and was eager to meet new colleagues. As I entered a room of about 100 other psychologists, Emory Cowen, my former mentor, saw me and shouted, “There’s the dog shit man!” Although he was joking, it was still early in my career, and I certainly didn’t want to be known as the “dog shit man” forever. It was time to pursue other areas of research and social change. Nevertheless, I still continue to receive requests from communities asking for my help in uncollected dog litter. Thirty years later, people still send me cartoons regarding the defecation habits of cats, ducks, and other animals.
I was more than ready to move on from studying dog excrement, but in 1981, a DePaul graduate student conducted a study involving different types of litter in small parks in Chicago. This project sought to improve the conditions of neighborhood parks that were sullied by trash or in need of repair. She collaborated with the Chicago Parks District, the organization Friends of the Parks, and most importantly, the Boy Scouts of America, whose young scouts played an integral role in cleaning up the parks. The graduate student’s evaluation of the project showed measurable improvements in the appearance of the parks, as well as increases in the participating scouts’ problem-solving skills and psychological sense of community (Bogat, 1983). The research showed that if individuals work collaboratively, even on seemingly trivial problems such as litter, they can also become more invested in their communities and more civically engaged.
Seatbelts and Child Safety
The litter projects had large effects on the aesthetics and sanitation of public spaces—bringing communities together—but other research can influence (p.121) legislation that saves lives. Before states began to pass child safety restraint laws in the late 1970s, thousands of children were either injured or killed in car accidents each year, and improper restraint in a car was once the leading cause of death for children under 1 year of age. In the early 1980s, Illinois legislators considered legislation that would require infants and children to wear a seatbelt or to be placed in an equivalent type of restraint while in cars. Given the gravity of the problem, our research team wondered if legislation would effectively change citizens’ behavior and enforce child restraints. We first needed to spend several months gathering relevant information. We looked inside cars to see whether or not infants and children were placed in car restraints. Then, we used telephone surveys to collect information about the public’s attitude toward the child restraint bill. The goal was to use both data collection on attitudes and first-hand observations to build a more convincing case when trying to influence policy officials.
Although we were also working closely with an Illinois organization that was advocating for the passage of this bill, we were unsure if our data would persuade legislators. To see if we would achieve the desired impact, we turned to a simple technique used by many in scientific fields: We provided the information to only a random half of the legislators and then measured the difference between the informed and uninformed groups. We sent the information to only half of the Illinois state legislators prior to a vote on the child passenger restraint bill. In that letter, the senators were informed that 140 children in Illinois were killed and 25,828 injured in automobile accidents between 1975 and 1981. We also pointed out that, through our observations, 93% of Illinois children were not in adequate restraints while riding in cars. We also provided the legislators with the results of our survey, in which 78% of adults supported the child passenger restraint bill. By sending this critical information to half of the legislators, we were able to see the effect of our letter. Seventy-nine percent of senators who received the information voted for passage of the bill, whereas only 53% of senators who did not receive the letter voted for the bill. In either case, we were gratified that the majority of senators in both groups voted for the passage of legislation that would help protect children (Jason & Rose, 1984).
Based on your knowledge of your target group—in this case, senators—you may need to consider several factors to generate a positive reception. For example, we made sure that our letter was received 1 week before the vote, and that the information was clear and concise. In this way, we increased our chances that the letter would be both read and remembered. We also gave senators specific data, particularly on low rates of safety seat usage and citizen enthusiasm for the proposed legislation. Lastly, we gathered information regarding the state’s share of medical care costs following debilitating car accidents. Cost-saving policies are always effective with legislators. Our intervention was a great success. Even (p.122) the governor requested a copy of our findings before finally signing the legislation. As of July 1, 1983, the Illinois Child Passenger Restraint Law required children under the age of 4 to be placed in an approved child or infant car seat. Also, children aged 4 to 6 must be placed in either an approved restraint system or a secure seatbelt.
Measuring your results doesn’t have to be complex; you can simply compare the data before and after your action. In the case of the new child seatbelt law, we compared our results from periods of time before and after July 1983 (the month the law went into effect). We found that the bill worked. With passage of the legislation, use of appropriate restraints increased from 49% to 74% for infants younger than 1 year of age. For children between the ages of 1 and 4, use increased from 13% to 42%. Overall, children’s deaths caused by traffic accidents decreased by a startling 53% when compared to a period 2 years before the law was enacted (see Figure 5.3). This is a great example of how working collaboratively with community-based organizations can help foster solutions to serious and significant problems within a community. Our data influenced policy officials to support laws that contributed to second-order change. As a result, our children are much safer in cars today.
Much like our child restraint efforts, various campaigns and accompanying evaluations have significantly helped to reduce smoking. Smoking once dominated American culture but has now been relegated to sidewalks thanks to decades of action, evaluation, and policy. Early on in our work, we developed methods to evaluate the success of creating a nonsmoking section in a student cafeteria before there were laws restricting use (Jason & Liotta, 1982). At first, we picked a time each day to count the number of smokers in a particular section of the cafeteria. When we posted no-smoking signs, the number of smokers in the
Throughout the course of our study, people were encouraged to continue asking others not to smoke in the cafeteria after seeing beneficial changes; coalitions that shared our goal also maintained the needed changes. For example, within the permanent nonsmoking section that we had established in the cafeteria, an individual who collected unreturned trays was assigned by the university management to continue politely asking smokers not to smoke. Data collected 3 months after the end of the nonsmoking section research project showed that customers had started to do the same. At one of the follow-up sessions, a student, without any prompting from us, was observed approaching smokers and asking them not to smoke in the nonsmoking section. It seemed that from the creation of this designated section, new social norms were being established; nonsmokers were becoming more comfortable and began to assert their voices. Both management and customers began to change their behaviors.
Affecting Handicapped Parking Regulations
Evaluating action in a campaign for second-order change is necessary for a number of reasons. Activists need concrete data to persuade both members of the community and policy makers to support their causes. A more subtle approach to influencing social policy concerns the ways that laws are actually implemented. Many years ago, I read in the news that handicapped parking regulations were about to be changed. The Illinois code originally required official vertical signs and pavement markings to indicate handicapped parking spaces. However, a task force recommended that the vertical signs could be optional and that ground markings would suffice.
I believed that this change was ill-advised. Without the vertical signs, nonhandicapped drivers were less likely to feel publicly shamed if they were able to obscure the ground markers by parking their cars right over them. As occurs all too frequently in policy decisions, nobody had taken the time to investigate whether the change would lead to more violations. Worse, no data had been collected to evaluate ways to keep nonhandicapped drivers from parking in reserved spaces. Many community organizations may relate to the importance of the need for this type of evaluation. Policy changes are certainly not immune to the law of unintended consequences and can inadvertently worsen conditions for a community. In this case, such a seemingly innocuous change would (p.124) probably create fewer available handicapped parking spaces, making accessibility even harder for handicapped people.
To keep this obvious reality from being overlooked by the policy makers, I quickly organized a study that monitored the habits of drivers in a parking spot painted with a handicapped symbol (Jason & Jung, 1984). Often, the best way to find out whether a method is effective is to implement it for a period of time and then alternate between removing and replacing the method in the same area. By switching back and forth, you can gather data and compare the effects. In this case, we observed violations occurring 53% of the time over a 10-day period in handicapped spaces with only ground markings. When a prominent vertical sign was visible along with the ground markings, there were no violations for the next 4 days. The sign’s effectiveness was very apparent: When it was taken down for several days, violations shot up to 47%. Not surprisingly, when we reposted the vertical sign for the next 5 days, illegal parking dropped back to 7%. The last time we removed the vertical signs, violations increased to 59%, and when the vertical sign was reposted, no violation observations occurred for the next 6 weeks.
This simple experiment provided us with the results I suspected—tangible evidence proving that ground markings alone were not very effective. Fortunately, before the Illinois Vehicle Code was amended, the task force chairperson told me that our study had been critical in convincing members of the need to retain vertical signs. The existing Illinois Vehicle Code was maintained. Policy evaluations do not have to be complicated, labor intensive, time consuming, or costly in order to influence regulations at the legislative level.
Nutrition and Supermarkets
Public health officials are warning us that the typical American diet has too much saturated fat, sugar, and salt. This diet has produced a generation of overweight adults and children who are at an alarming risk for serious chronic illnesses such as diabetes. Rather than waiting to implement evaluation techniques after problems have already occurred, we should be assessing what we can do to prevent them. One approach is to go to the source of people’s food in the first place—the supermarkets. Despite the monopolistic features of these large stores, customers are the power holders; they have the power to influence these stores’ financial bottom lines, as well as the products they carry. However, as we have seen, power holders are often reluctant to change their behaviors; if it’s working, why tamper with success? Unless, of course, consumers alter that logic. A number of years ago, I worked on a study that tried to influence managers of supermarkets to stock healthier foods while simultaneously convincing consumers to buy it (Jason & Frasure, 1980).
(p.125) Our initiative began with a team of DePaul researchers who went to four Chicago supermarkets. They found that, as expected, these stores did not stock the type of nutritional foods that were offered at health food stores. Once a week, we walked into the stores and talked to the store manager about carrying items such as brown rice. A separate target item was then prompted in each store for a 4-week period (using what is called a multiple baseline design). Our effort to increase the number of healthy foods that were stocked was successful in one store, which began stocking one of our suggested items. With that small success, we were convinced that it was possible to persuade at least some supermarkets to change what they carried.
Our next goal involved encouraging customers to buy healthier products. Much like the research on spaces for drivers who are handicapped, we counted the amount of products sold during a baseline period, posted a sign in one of the aisles, removed it, and then replaced it. The intervention involved posting an 18×14-inch sign in both English and Spanish next to the rice section, and we were able to double the purchase of the more nutritious product. The sign read:
Buy BROWN RICE instead. It’s More Nutritious.
In fact, Brown Rice has more CALCIUM, POTASSIUM, IRON, SULFUR, and IODINE. And most importantly, Brown Rice contains a bran layer, which is extracted from white rice in milling processes. This bran layer is packed with protein and thiamine. And it’s even easier to digest.
Brown Rice has more of everything that’s good for you.
These results were promising: Our in-store advertisement for brown rice was effective, and the store continued to stock the item. Rather than waiting for people to suffer weight or health problems, we can persuade supermarket managers to offer more high-quality products and encourage customers to select healthier food. Even with our recent push for fresh fruits and vegetables in inner-city convenient stores, the food desert—the availability of only processed foods—is still a fact of life for many areas of Chicago and other urban areas. If we want to protect the best interests of people in inner cities by enhancing their health and quality of life and save tax dollars spent on health care, more work needs to be done to provide fresh, unprocessed, and locally grown food to people in urban and suburban areas. Michelle Obama has been an energetic promoter in past years to change these types of lifestyle factors.
Media-Based Health Promotion Efforts
Self-report measures can also be used to evaluate campaigns that include large segments of the population who use media. About 30 years ago, I became (p.126) interested in trying to decrease excessive television viewing (Jason, 1985), as well as negative advertising (Jason & Klich, 1982). In the latter study, we documented ageism, racism, sexism, and cigarette and alcohol advertisements in 24 print media sources over a 3-year period. We sent the findings to several activist organizations that used them to successfully expose these abuses to larger audiences. A key aspect of effecting social change is providing relevant details that can be presented to the public arena. Unfortunately, there was some fallout from this study that nearly affected my career. One of the media sources, the Chicago Sun-Times, published the results prominently on a full page. In an unfortunate coincidence, this happened on the same day that I was being evaluated for tenure at the university. It was a day that I would never forget. During my interview with members of the university tenure committee, the president of the university asked me what this study had to do with my discipline of psychology. Although I had inadvertently embarrassed a number of companies that contributed to the university, my strong publication record and endorsement from peers helped me ride out this controversy. Nevertheless, this experience served as a potent example of how even the best intentions and efforts can be risky endeavors for activists.
I next decided to design and launch a series of large-scale, community-based, preventive media interventions that targeted specific problems, such as addictive behaviors. Radio, television, social networking sites, and newspapers are the best ways to connect with the community; they can very effectively get people to participate in positive initiatives. Media outlets can reinforce the concepts and objectives of a campaign and encourage involvement. By advertising free health promotion pamphlets and materials through the media, individuals are given easy access to tools to educate themselves even further.
Health promotion interventions often use social marketing strategies to promote changes in behaviors and attitudes. Unlike commercials that bombard us with unnecessary products or socially irrelevant messages, we can educate people through the media by using the Four Ps of marketing: a positive product (a change in behavior that improves one’s health, such as giving up smoking); price (the costs and benefits of taking on this new behavior); place (the availability, or accessibility, of the new health product); and promotion (the positioning of the product in the community).
For a number of interventions, ranging from weight (Jason, Greiner, Naylor, Johnson, & Van Egeren, 1991) and stress management (Jason, Curran, Goodman, & Smith, 1989) to HIV prevention in families (Crawford et al., 1990), funding came entirely from partner organizations in Chicago. This demonstrated the power and effectiveness of community groups’ resourcefulness and independence and proved that one does not have to rely on state or federal funds for assistance. These interventions can also be evaluated to ensure that the target audiences are being reached in meaningful ways. The interventions I helped to (p.127) launch in the 1980s and ’90s may serve as examples to guide organizers to create tailored, media-based interventions today.
Smoking Cessation Interventions
In the 1980s, after working to help establish nonsmoking sections in various settings in Chicago, I had the opportunity to begin work on creating media programs for tobacco cessation. As a member of the Smoking and Health Committee of the Chicago Lung Association, I noticed that only a minuscule number of smokers were participating in smoking cessation clinics led by the association throughout Chicago. It was obvious that the program was not serving its purpose and that a new direction was needed.
To reduce barriers of participation and reach large segments of the population, social psychologist Larry Gruder and I proposed that we begin a media-based effort that would make the program accessible to a wider audience. We launched a large-scale smoking cessation campaign on Channel 5, the NBC affiliate in Chicago, called “Freedom from Smoking in 20 Days.” This self-help program aired twice in the evening for 3 weeks; its purpose was to encourage half a million viewers to pick up one of the 50,000 self-help manuals distributed in over 300 True Value Hardware stores. Biweekly support group meetings (advertised on the program) held throughout Chicago provided educational and social support. At the end of the program, 41% of those who watched the media program, received a manual, and attended self-help group meetings were able to quit smoking, compared to 21% of those who only watched the television broadcast and picked up manuals (Jason et al., 1987).
Using the momentum from our first successful effort, which put an effective structure in place, we sponsored another 20-day smoking cessation program in November of 1985. This time, we partnered with WGN-Channel 9, which allowed us to reach between approximately 286,000 and 583,000 people (about one-third of these viewers were likely to be smokers) in the Chicago area. This time, 100,000 free manuals were distributed (Jason, Tait, Goodman, Buckenberger, & Gruder, 1988).
Both of the 1985 health promotion programs were funded exclusively by the local community and potentially saved the successful quitters hundreds of millions of dollars. At the time, an estimated 150,000 smokers were involved in the media campaigns. If only 10% (a very conservative estimate) continued to abstain from cigarettes for the rest of their lives, then our programs would have helped 15,000 people quit smoking. The public health implications of these findings are profound; the lifetime health savings in preventing chronic disease, such as cancer, for each middle-aged adult who quits smoking is conservatively estimated to be $40,000. Our programs possibly saved (p.128) $600,000,000 in health care costs. This kind of cost–benefit analysis can be one of the most illustrative and compelling types of summative program evaluation.
Throughout the planning stages of each social action campaign, you gain valuable experience that prepares you for larger undertakings. With additional support from partners, you will also learn to successfully navigate through sensitive situations. To illustrate, several members of our planning group wanted to partner with McDonald’s fast food restaurants to distribute our self-help manuals. Although McDonald’s was willing to pay a lot of money for the increased traffic and publicity, I strongly objected, as these restaurants served fast foods with formidable health risks of their own. I believed it was important to remain true to our own values in every step of the social change process.
Tobacco and Addictions Prevention
Aside from helping smokers quit, I was also interested in preventing youth from picking up the habit in the first place (Jason, 1979; Jason, Mollica, & Ferone, 1982). My next media project in 1989 aimed specifically to change the type of exposure that African American youth received about tobacco in the media. In order to do this, we modified our media intervention approach for smoking cessation: Rather than using stores to distribute materials advertised on television, we delivered them right to participants’ homes through local newspapers and popular shows.
We designed and implemented a prevention program at 472 elementary schools that combined the school-based curriculum, “Smoking Deserves a Smart Answer,” from the American Lung Association and incorporated the use of media (Kaufman, Jason, Sawlski, & Halpert, 1994). Three popular media outlets partnered with us: WGCI ran eight smoking prevention public service announcements on the radio, created a call-in talk show that empowered parents to help their children resist smoking, and ran a smoking prevention rap contest, airing the winning entries. In addition, the Chicago Defender printed a smoking prevention curriculum on their weekly children’s page, and a billboard company also promoted a prevention contest. Participants created posters, and the winning submissions were displayed on billboards in the Chicago area. These contests that promote creativity and encourage kids to express themselves through their own language and style can give the antismoking message a personal resonance for children. Our work paid off: Not only did students decrease their use of tobacco, but also they reported lower family use of cigarettes, alcohol, and marijuana.
In the spring of 1990, I helped to develop another smoking and drug prevention program for all school-age children along with the Illinois Department of Alcohol and Substance Abuse, and with the support of the Chicago Tribune (p.129) and WGN News. We launched a statewide campaign aimed at children and parents called Kids InTouch (Jason, Pokorny, Kohner, & Bennetto, 1994). In this cross-platform approach, our InTouch daily supplement in the Chicago Tribune was published along with WGN news telecasts on the topics covered. Newspaper supplements provided information such as phone numbers for parent workshop sites, Chicago-area students’ antidrug drawings and messages, a statewide resource guide to addiction prevention activities, and a referral for services on a national level.
By including facts about an issue and its effects on the community in your media intervention, you can build awareness, increase sensitivity to issues, and publicize support and training groups throughout the area. Our six-part news series explored topics like first-time drug use, abuse, and children of parents with substance abuse disorders. We also included individualized profiles of Chicago families who had overcome these issues. Parental involvement in the program was a significant component to bringing drug awareness to the family. More importantly, the parent–child bond strengthened the entire family and helped develop parenting skills.
We also broadcast our intervention through the time-honored kid tradition of Saturday morning cartoons. Our message aired on Chicago’s Channel 32, as well as other stations on different days and times, for 6 consecutive weeks. The High Top Tower series was intended to prevent drug use and dealt with self-esteem, tactics for resisting drug use, problem solving, and alternatives to drugs. High Top Tower encouraged elementary-aged children to focus on family, health and safety concerns, and drug and alcohol awareness.
In these media efforts, we demonstrated how change agents can utilize local resources as they work to develop and implement large-scale prevention and health promotion campaigns. With the help of these resources, we strengthened the effectiveness of interventions by incorporating our efforts into participants’ various “worlds”—for example, family, school, work, and community. Strong partnerships can lead to developing a network of key organizations. When planning our first media programming, we were fortunate that Donna Stein at a progressive health maintenance organization (HMO) was simultaneously seeking to launch a similar project; this helped us gain access to a number of for-profit businesses and television stations. The success of our first program was also a stepping stone to access to other media outlets and organizations.
Taking a step back from examples of program evaluations from my own career, the best case for the utility of measuring success is that of microcredit, a term (p.130) that refers to a myriad of types of microloans and services that are provided to poor people who would otherwise be excluded from formal financial services. The microcredit model began with Grameen Bank in Bangladesh in 1976 and has been replicated around the globe, causing a revolution in the field of international development and shifting development funding priorities heavily in favor of microcredit programs. By 2009, nearly 200 million borrowers of microcredit have been served by microfinance institutions like Grameen Bank (Reed, 2011).
Although microcredit has been modified and reinvented from the Grameen framework, any variation of the model is easily evaluated. The success of microcredit programs can be measured in a number of tangible ways: microcredits granted, number of individuals receiving microcredits, repayment rates, savings or profits accrued by community bank lenders, and microcredit effectiveness indicators such as the number of houses built or small businesses started, among others. Collecting testimonies from the many beneficiaries of microcredit programs is also a compelling way to personalize the measures of success. The measurability of microcredit programs partly explains international donors’ preference for these programs over other development initiatives with results that are harder to quantify. By measuring the outcomes of microcredit programs, microfinance institutions and nonprofit organizations have popularized and legitimized this model of development and are thereby able to capitalize on international funding support to expand and improve their microcredit programs.
Evaluating Unintended Consequences
If evaluations are poorly conducted or reported on, they can end up promoting unsuccessful programs. Interventions, such as youth mentoring programs, which people intuitively believe should have a positive effect, actually can have a negligible or even harmful impact depending on how they are implemented. These programs have become extremely popular in recent years—funding (primarily federal funding) has spurred a sixfold increase in the number of these kinds of programs over the past decade.
However, when the large-scale evaluation of the effectiveness of these programs was conducted, researchers found that only a handful of these programs actually had a meaningful impact on the lives of the vulnerable youth who were partnered with adult mentors (Rhodes, 2008). In research, the “effect size” is a concept that quantifies the real-world impact of an intervention; if the effect size is small, your program may have had an impact, but the “size” of this impact could be negligible. Therefore, poorly reported program evaluations will tout the fact that a statistically significant difference exists between groups of youth who (p.131) participated in a mentoring program and those who did not, but will fail to mention that a very small effect size was found. In the large-scale evaluation of mentoring programs, small effect sizes across most programs were found; some programs showed negative or no effects, and the positive effects that were discovered often faded away within just a few months after the “successful” program ended (Rhodes, 2008).
As Rhodes and DuBois (2006, p. 13) point out:
Modest findings from the evaluation of an intensive community-based approach to mentoring helped to galvanize a movement and stimulate aggressive growth goals. These goals necessitated that mentoring be delivered more efficiently, which, in turn, changed the intervention to something that bears decreasing resemblance to its inspiration.
This is not to say that all mentoring programs fail—and, in fact, the research shows that some have quite positive impacts. However, poor research, reporting, and understanding of evaluation results have resulted in millions of wasted federal funds and a push for quantity over quality of mentoring programs. It illustrates the importance of rigorous evaluation in developing new or expanding existing programs (Rhodes, 2008); researchers must even be skeptical and impartial toward programs that intuitively seem beneficial—the reality may surprise you.
In this chapter, I explained the importance of evaluating change efforts. Few initiatives, no matter how positive, are successful without some type of documentation to support your efforts. To this end, community activists, who may not be trained to design and evaluate a means of tracking a project’s progress, may be best served by joining forces with researchers. Cooperation and flexibility among the activist and the researchers are critical, as both parties venture into unfamiliar areas of expertise. For this two-way relationship to thrive, everyone involved should respect the dignity, opinions, and needs of all of the partners. This respect will lead to trust among researchers and the community members, which will help facilitate an understanding of how each member’s strengths or skills can best be utilized to achieve the mutual goal. In addition, each party must recognize and appreciate the fact that roles can evolve over time.
When trying to bring about second-order change, activists often must deal with powerful individuals and institutions. When this happens, coalitions can help challenge those who want to maintain the status quo. Because this process (p.132) usually takes place over long periods of time, activists must develop the patience and persistence to see the task through. Our final principle of social change, measurement, provides the opportunity to consider the tangible short- and long-term results of our activities. Using evaluation techniques such as self-reported data and observed changes, we can determine whether or not a particular intervention or social change strategy has achieved its goals. Far from dry or lifeless, evaluation is a tremendously powerful tool that helps us discard ineffective strategies and replace them with more successful methods.
For many scholars, traditional academic practice involves publishing one’s findings and then expecting the results to have a voice and influence in the public forum. I once published a study, in an obscure journal, exploring psychological damage to women who had been harassed by men (Jason, Reichler, Easton, Neal, & Wilson, 1984). After the study was published, I was contacted by a lawyer in Philadelphia who informed me that he was using the findings of my study to support his client in court against a harasser. Was this social action research? Since the findings were used in the public domain, in a sense, it was. However, from a community perspective, if publication of the findings does not lead to some second-order outcome, then the activists and researchers should try other methods to shed light on the findings and attempt to influence social policy.
The examples of evaluation that were presented in this chapter informed me that many of my interventions were ultimately successful, even if adjustments needed to be made from failures along the way. The successful results that we were able to quantify were helpful in justifying and garnering support for future interventions. The purpose of evaluation, however, is not only to point out our successes; evaluation must be an unbiased judge—able to highlight where programs have failed to produce meaningful effects.
One aspect of producing quality research is accounting for and minimizing confounding variables. For example, when we were trying to determine the effects of “no smoking” signs on smokers within a neighborhood grocery store, a security guard saw the signs and tried to help us by telling customers to “kill that cigarette.” Ironically, since our research aimed to assess the effects of the signs alone in encouraging people to stop smoking in this public setting, we immediately asked him to discontinue his intervention. Counterintuitive to the overall goal of our activism, it was important to be able to accurately assess the effects of the signs alone to know if and how to alter future interventions. During our community efforts to encourage dog owners to pick up after their dogs, a girl scout troop heard of our efforts and began distributing flyers to the community that described dog owners who did not pick up after their dogs in the most unflattering ways. Although comical, the girl scouts’ actions also could have altered the results of our research.
Just like these unintended confounds, researchers must also keep track of the unintended consequences that occur from our interventions (Trickett & (p.133) Mitchell, 1993); sometimes, our second-order efforts are altered in ways we do not expect. For example, a state official once asked me for advice about funding a community-based organization with which I was very familiar. Since I knew the organization’s leaders and its mission, I felt confident in my recommendations. It was a relatively small and seemingly harmless intervention on my part, but when the state began funding this community organization, the push for a statewide program expansion distorted the mission and it no longer met the needs of its members. Instead, it became driven by economics; the organization’s generous volunteer services were replaced with a mandated, impersonal program.
Over the years, I’ve seen far too many cases like this one: When state or federal funding becomes involved, it can hamper self-help groups and organizations from voluntarily contributing to community-based programs. Unforeseen issues, such as funding, can complicate the delivery of second-order types of programming and, in the process, contribute to more first-order interventions that become part of the problem. Evaluating these important types of well-intentioned failures can help guide organizations toward more successful second-order change strategies.
Unfortunately, sometimes unforeseen effects can have much more dire consequences. At the far extreme of such consequences, consider the scientists who researched the structure and properties of the atom at the beginning of the 20th century, unaware that their scientific breakthroughs eventually would lead to the creation of the atomic bomb and the unforgettable devastation at the end of World War II. While the United States’ decision to drop the bomb on major Japanese cities led to Japan’s surrender, the ethics of using atomic weapons on civilians was not properly examined by our government or by the scientists who developed the bomb. The discovery of these weapons also caused the ensuing 50-year atomic arms race with the Soviet Union, which has created, despite later disarmament efforts, present-day nuclear arsenals with enough power to destroy the world several times over.
Other times, the research itself, rather than its implications, goes against basic ethical and moral codes. For over 40 years (between 1932 and 1972), the U.S. Public Health Service conducted an experiment on African American men with syphilis to investigate the effects of untreated syphilis on the body. These men were never told they were suffering from syphilis and were not treated, even though by the 1940s, penicillin had been discovered as an effective treatment (Jones, 1993). Even the Surgeon General participated in the experiment by convincing the men to remain in the program. In 1972, the story finally broke in the Washington Star; the government was forced to end its experiment and provide the men with effective treatment, but by then, over 100 men had died and dozens of women and children had been infected.
Although this type of medical research falls into a different category than the evaluative research that is key to measuring the success of social activism, it is (p.134) important that researchers and activists alike are aware of the powerful consequences that research can have when ethical standards are not upheld. When working for social change, we must pay close attention to the moral issues behind our work and consider both its possible intended and possible unintended consequences in order to strive to achieve the most beneficial outcomes of our actions.
For more information on the American Lung Association’s “Freedom from Smoking” program, visit this site: http://www.ffsonline.org/program-overview.html
This link includes a video clip on the dynamics of smoking, and it shows networks from 1971 when smoking was prevalent to many years later as smoking was reduced in the general population: http://www.nejm.org/doi/full/10.1056/NEJMsa0706154
Chris Corbett (2011) has developed excellent materials for working with legislators. He can be contacted at firstname.lastname@example.org
Steven R. Howe (2007) also has done exemplary policy work related to housing and the transition to self-sufficiency for low-income populations.
Steve Fawcett (2011) and an international team of health care workers has developed the Community Tool Box, which has more than 7,000 pages of free information on skills for building healthy communities. It can be found at this site: http://ctb.ku.edu
For those with interests in qualitative analysis methods, there are a number of excellent online websites (e.g., http://onlineqda.hud.ac.uk/Introduction/index.php) that provide a collection of learning tools for conducting these types of analyses. Mixed-methods research involving both quantitative and qualitative methods is the topic of Jason and Glenwick’s (2012) book Methodological Approaches to Community-Based Research. This book provides more advanced research strategies for understanding cutting-edge research methodologies that analyze data in special groupings, over time, or within various contexts, which are holistic and culturally valid and support contextually grounded community interventions.
To read a New York Times article that identified evidence-based programs that are happening at various government levels, visit this link. It gives an idea of how we can help to create policy change: http://opinionator.blogs.nytimes.com/2012/05/30/worthy-of-government-funding-prove-it/