POLL-ICY PERCEPTIONS:
Designing Polls to Measure Public Opinion about Policy Issues

Grades:  6-8, 9-12

Subjects:  Civics, Current Events, Language Arts, Social Studies

Related New York Times Article
"Poll Shows Drop in Confidence on Bush Skill in Handling Crises, By TODD S. PURDUM and JANET ELDER", October 6, 2003

Overview of Lesson Plan:: In this lesson, students choose sample methods, survey instruments, and question formats to create their own opinion polls about local policy issues.

SUGGESTED TIME ALLOWANCE:
1 hour- 90 minutes

OBJECTIVES:
Students will:

1. Respond to two polls - one silent and the other public - questioning the ability of a school authority figure; evaluate how anonymity may affect responses.
2. Learn about a New York Times/CBS News poll that evaluates public opinion of the president by reading and discussing the article "Poll Shows Drop in Confidence on Bush Skill in Handling Crises."
3. In groups, experience the process of developing polls, focusing on student-related topics.
4. Write "methodology statements" that describe the steps taken to ensure accurate poll results; explore how the results of polls could affect policymaking; administer and evaluate the polls developed in class.

RESOURCES / MATERIALS:

• student journals
• pens/pencils
• paper
• three large pieces of paper, each with one of the following statements written in large letters: "Approve/Good Idea," "Disapprove/Bad Idea" and "Don't Know"
• small slips of yellow paper (one slip for each student in the class)
• small slips of blue paper (one slip for each student in the class)
• classroom blackboard
• copies of "Poll Shows Drop in Confidence on Bush Skill in Handling Crises" (one per student)
• four large, blank index cards and marker to record and distribute poll topics
• handout of poll design procedure copied from Activities/Procedures 3. (one per group)

ACTIVITIES / PROCEDURES:

1. WARM-UP/DO-NOW: In this initial activity, students will respond to two polls - one silent and one public. The silent poll will involve students writing on pieces of paper and passing them forward. For the public poll, students will move to three sides of the room, standing near posters with statements that reflect their view on the question. Before students arrive, hang these three "statement" posters on three walls of the room ("Approve/Good Idea," "Disapprove/Bad Idea" and "Don't Know".) For the silent poll, place one yellow and one blue slip of colored paper on each desk. The polls ask students to assess the abilities of a principal, vice-principal, teacher (perhaps yourself) or another authority figure whom you feel comfortable choosing for the activity.
   Upon entering class, students respond to the following (written on the board prior to class): "On your desk are two slips of paper to be used as your ballots in a silent poll. On the yellow slip of paper, answer this question: Do you approve or disapprove of the way [name of principal, vice principal, co-teacher, etc.] is handling [his/her] job? (Respond with Approve, Disapprove or Don't Know.) On the blue slip of paper, answer this question: Do you have confidence in [name of authority]'s ability to deal wisely with a school crisis, or are you uneasy about [his/her] approach? (Respond with Confident, Uneasy, or No Opinion.)" Collect the papers in two piles, one for each question, and clearly tally the results for each on the board.
   Next, explain to students that they will be participating in a public poll. Point out the posters hanging on the three walls of the room. Read the first question on the board aloud, and ask students to move to the sides of the room that best reflect their views. Tally the results on the board. Then read the second question aloud, and ask students again to move to the sides that best reflect their views. Again tally the results, and allow the students to return to their seats.
   Discuss the following: Are the results for each poll identical? As a participant in the poll, was one poll easier to respond to than another? Why or why not? Did you feel pressure to respond in a different way when the poll was public as compared to when the response was anonymous? Was it easier or harder to come to an opinion when the responses were anonymous? Why or why not?
2. As a class, read and discuss the article "Poll Shows Drop in Confidence on Bush Skill in Handling Crises," focusing on the following questions:
   1. How has the public's confidence in President Bush's handling of international crisis changed in the last five months?
   2. According to the poll cited in the article, how does a majority of the public now feel about his ability to make good economic decisions?
   3. What conclusions do the reporters draw from the survey with regards to the public's opinion of the Democratic presidential candidates as the potential future leadership of the country?
   4. Which poll results surprise you? Which do not surprise you?
3. Explain to students that they will be acting as teams of news researchers to devise a methodology for conducting a poll that will measure public opinion about school or community policymaking.
   DETERMINING A POLL TOPIC
   Begin by asking the class as a whole to brainstorm non-academic issues which affect the student body or pending local government issues that directly affect students. What are issues that students feel arouse concern, create conflict, or deserve attention from the school board, student council, city council, mayor's office, or other public governing bodies? Examples might include dress codes, use of cell phones on school grounds, security lockdown procedures, cafeteria menu selection, athletic programming, school dance conduct, local traffic or parking policies, mall or retail shopping developments, police procedures, etc. (Some teacher discretion may be required; be prepared to qualify this brainstorming session by saying that certain policies, such as those relating to drug use and sex among minors are off limits in this forum, if you wish.) Write the topics on the board as students suggest them, seeding them with ideas culled from local information sources. Then choose four of the strongest topics, and write them on index cards (one topic per card). Divide the class into four groups, and give each group one of the index cards.
   Next, explain to students that each group will be designing a poll that will accurately measure students' opinions about their topic, using the following guidelines (copied into a handout for easier student access):
   POLL DESIGN PROCEDURE
   [adapted from "Conducting Surveys")]
   I. Define Poll Function
   • What is the purpose of your poll? What do you hope to learn by polling the local population? State your purpose in one sentence.
   
   II. Choose a Poll Instrument
   • How will you conduct your poll -- in writing or by voice? Before answering, consider the following:
     Written polls can be given to respondents to answer at their leisure.
     Advantage: Respondents remain anonymous, so polltakers have greater confidence that respondents will answer the poll honestly.
     Disadvantage: Poor response rate. It's hard to control how many respondents will actually fill out the poll.
   • Interviews are administered face-to-face or over the telephone.
     Advantage: Response rate will be higher.
     Disadvantage: Respondents may not feel completely anonymous and will alter their answers.
   • Discuss:
     • Do you think you could get a higher response rate from e-mail than from a written ballot? Would respondents be likely to answer truthfully by e-mail? Why or why not?
     • How do telephone surveys help to maintain anonymity? Would having to answer directly to a caller still alter a person's response?
     • Would instant messaging be an effective method? Why or why not?
   • To do:
     Evaluate the response rates you can expect from each of the five instruments available (written ballot, e-mail ballot, face-to-face interview, telephone survey, and instant-messenger survey). Then weigh these against the need for anonymity, and choose the best instrument for administering your survey.
   
   III. Define Poll Questions
   1. "Rough out" your questions:
      • What are the burning questions that the public wants to know about your topic? Make a list of all the important questions you can think of relating to this topic.
      • What questions are people not likely to answer truthfully or will require an overly complex response? If a question is too hard to answer, the response rate will suffer or the response will be too difficult to equate among the population you are surveying. Keep it simple. Circle at least three questions that ask for (or could be rephrased to ask for) a simple check mark in a list of alternative answers, a simple yes or no, or a one-word or single phrase response.
   2. Fine-tune your wording:
      Polltakers generally ask two types of questions - close-ended or open-ended. Close-ended questions (yes/no, multiple-choice, on a scale of 1 to 10, etc.) are easier to tally because they provide a definite answer, but they may not give you much insight into people's true opinions. Open-ended questions ask for the respondent to provide his or her own answer, but responses are hard to tally.
      Discuss:
      • Would a presentation of the results be more influential to policymakers if it included answers to open-ended questions?
      • What format would be the most appropriate for your topic?
      
      To do:
      • Refine the question format for each of the questions you circled, and rewrite them as either a closed- or open-ended question.
   IV. Select a Sampling Method
   • How will you objectively pick respondents? Before answering, consider the following:
     Polltakers rely on at least four ways to objectively choose the group of participants they will survey:
     1. Random Sampling: Each person in the population has an equal chance of being selected. For example, your group might choose a random sample of 50 students from your school by writing the names of every student in the school on a card, mixing up all the cards, and randomly choosing 50 cards.
     2. Systematic Sampling: Respondents are chosen from a list using a fixed interval. For example, the names of the student body can be arranged in alphabetical order, and polltakers would choose every tenth person.
     3. Stratified Sampling: The school population is sorted into relevant subgroups, and people are randomly selected from each. For example, the population being polled might be sorted into gender or ethnic groups, then individuals are randomly selected from each group.
     4. Cluster Sampling: In this sample method, groups, rather than individuals, are randomly selected. For example, the polltakers might randomly select classes in a school and interview all people in those classes.
     
     Discuss:
     • Which sampling method best suits this topic?
     • Which sampling would be easiest to obtain?
     
     To do:
     • Be prepared to "bid" for a specific sampling method by presenting arguments on why a particular instrument would work for a poll on your topic. The teacher will listen to all the groups and then assign each group one sampling method.
4. WRAP-UP/HOMEWORK: Individually, students respond in writing to the following prompt (written on the board for students to copy before leaving class): "Prepare a 'methodology statement' for the poll you designed in class. This statement should summarize the poll's function and reasons why your group selected a particular instrument and sample size. When describing the poll function, expand on why the results of this survey of public opinion should be considered by policy makers weighing the issue in question. In addition, explain how the questions are worded to avoid bias and to achieve accurate, standardized results." Polls should be administered and evaluated in the future, and each group may write an article to be submitted to the school newspaper that shares their results.

DISCUSSION QUESTIONS:

• How does sample size affect the margin of error of polls?
• Why is the process of random selection important in polling a large group of people, such as the American public, with a small sampling?
• What responsibility do journalists have to report accurate statistics in news?
• Can statistical findings be reported accurately and still reflect a bias? Why or why not?

EVALUATION / ASSESSMENT:
Students will be evaluated on their participation in class polls and class discussions, participation in group design process of a poll to measure public opinion, thoughtful completion of a poll methodology statement that explores how the results of their polls could affect policymaking, and administration and evaluation of their group polls.

VOCABULARY:
critical, domestic, incumbents, integrity, contenders, sought, margin, sampling, intensified, integrity, enacted, partisan, mere, dynamic, laissez-faire

EXTENSION ACTIVITIES:

1. Conduct the poll that your group designed in class, and write an article summarizing the results for your school newspaper. How will you best communicate the findings to your readership? Begin by tallying each result in four different ways - as a proportion (for example, 3 in 10 high-school seniors approved of the school dress code), as a percentage (30% of high-school seniors), and as a fraction (nearly, or less than, one third of high-school seniors). In addition, express the inverse quantity (7 in 10, 70% or nearly two-thirds), and state the opposite conclusion (did not approve of the school dress code). Which expression will reveal the most interesting result? Which expression is the fairest to your subject? Are there ways to emphasize, or even exaggerate, the results by inserting modifiers before each expression (such as nearly, less than, most, etc.)? How does this suggest ways you might use the results to affect public opinion with your article?
2. Examine the results of The New York Times/CBS News Poll referenced in the article read in class. Create a series of graphs that show the change of public opinion about President Bush's abilities from February 8, 2001 to October 1, 2003. Include annotations at points of abrupt changes in the graphs that identify events and policies that affected these changes in public opinion.
3. Examine the results of questions about the ten Democratic candidates in The New York Times/CBS News Poll. Graph the results so there is a clear, visual comparison showing how these candidates rank to each other.
4. Examine the table in question #3 of The New York Times/CBS News Poll. This table tallies the number of respondents surveyed who identified one issue as the most important problem facing the country. Reorder the list to reflect the order of importance determined by the respondents. Then, using the results, write a briefing paper for President Bush outlining the issues on which he should focus, and suggest specific policies that he should consider in order to address these public concerns.
5. Research the difference between quantitative and qualitative research. What aspects of The New York Times/CBS News poll are qualitative, and which aspects are quantitative? What is the poll's function? What does it validly measuring? Do you think the questions were worded fairly? Was the selection of questions balanced or biased? Write a letter to the editor discussing your responses.

INTERDISCIPLINARY CONNECTIONS:
Economics- How do political polls that evaluate public voting potentials compare to marketing surveys that evaluate consumers' buying habits? Design a poll that measures the buying habits of specific products for your peers, and write a poll proposal that explains how the results can be used to enhance the marketing strategies of companies selling those products. Include an estimate of the labor and resources required to conduct the poll, and demonstrate how the costs of these are justified by the information that the poll will capture.

Media Studies/Technology- Analyze the Internet as a medium for gathering public opinion. What different types of survey instruments are possible on the Internet? How have television and radio broadcasters used simulcast Internet polling (examples include American Idol balloting and All-Star balloting for sports broadcasts)? What are some other ways that "cross-over" polling methods can be used to enhance programming? Invent an Internet-based poll that can be used to measure public opinion, and write a proposal that explains why this instrument would be more effective that written ballot or telephone surveys.

Copyright 2003
The New York Times Company