Research Methods for Everyone, Part 1: Why Methods Matter and How to Do Research Ethically

In this post, the first in a six-part series, I will work through Amanda M. Rosen's Great Courses series Effective Research Methods for Any Project.^[1] Rosen, a political scientist who has spent her career teaching research methods, makes a simple wager: methods are not just for scientists. Anyone who has to answer a question with evidence, whether the product is a dissertation, a business plan, a public health decision, or the choice of a safe car, is doing research. While methods talk can sound like a hoop for graduate students to jump through, I will argue, following Rosen, that method is better understood as a discipline against self-deception; how we discover information matters just as much as what we learn. This series distills the course's twenty-four lectures into six essays and adds some connective tissue for students and researchers in the humanities and social sciences.

The pump handle #

Rosen opens the course with the story every methods teacher loves, and for good reason. In August 1854, cholera broke out in Soho, London. More than five hundred people fell ill in a neighborhood just north of Piccadilly Circus, and cholera could kill within hours. The expert consensus held that the disease traveled through "bad air," or miasma. The same experts, Rosen notes dryly, treated most sickness by opening a vein and applying leeches.^[2]

John Snow, an obstetrician, believed cholera spread through water contaminated by sewage. What makes Snow the patron saint of research methods is not that he was right; plenty of people have been right for bad reasons. What makes him the patron saint is how he went about showing it. He mapped the cases and traced them to a single pump at the corner of Broad and Cambridge Streets. Then he did the thing most of us skip: he investigated the people who didn't get sick, and he found that they drew their water from other sources. He was not collecting evidence for his theory; he was trying to eliminate every rival explanation. Officials removed the pump handle, and the outbreak ended.

Two details in Rosen's telling deserve attention. First, the experts were not persuaded; twenty years later, many still blamed bad air. Acceptance came only after other researchers confirmed Snow's claims. Research, as Rosen puts it, "is incremental and cumulative and requires continual testing, retesting, and debate." Second, Snow's method is available to anyone. He asked a question, observed systematically, formed a hypothesis, tested it against both victims and non-victims, and communicated his results. His procedure is the scientific method in miniature, an approach with roots in Aristotle and refined by Francis Bacon and Isaac Newton. Rosen is careful to add that even the tidy six-step version taught in classrooms is "an idealized approach that rests on the assumption that there is an objective reality for us to observe, something that no one has really managed to demonstrate." She takes the philosophical challenge seriously, and the course returns to it when it reaches interpretivism; I will return to it in part five.

A vocabulary for the work #

Before choosing methods, the researcher needs names for the kinds of research on offer, and the course's opening lecture supplies a useful working vocabulary. Basic research aims at knowledge, understanding, and prediction: why some people marry, how dolphins communicate, why businesses fail. Applied research aims at direct practical application: curing a disease, designing a curriculum, writing a business plan. The two depend on each other, since good applied work draws on basic findings.

Cutting across that distinction, research can be 1) exploratory (getting oriented in a new area), 2) descriptive (answering who, what, when, and where), or 3) explanatory (answering why and how, the territory of cause and effect). It can be quantitative, analyzing large amounts of numerical data for breadth, or qualitative, sacrificing some breadth for depth in a smaller number of cases. And it can be deductive, starting with theories and testing them, or inductive, starting with observation and building theories from the results.

None of these labels is a value judgment, and in my estimation that point matters more than it looks. Rosen mentions the Perestroika movement of the early 2000s, a revolt within political science against the dominance of quantitative approaches.^[3] The lesson she draws is not that one side won but that the question is always one of fit: what does your research question actually require?

The five marks of good research #

Lecture two condenses decades of methods teaching into five characteristics. Good research is 1) systematic, 2) objective, 3) empirical, 4) cumulative, and 5) transparent.^[4]

Systematic means the work follows sound procedures rather than improvisation. Objective means the researcher stays neutral and tests hypotheses by trying to knock them down. "We never set out to prove ourselves correct," Rosen says. "If anything, our job is to try our hardest to disprove our ideas." Empirical means the evidence is grounded in observation, in what we can see rather than what we hope is true. Cumulative means each project builds on what others have found; Rosen's image is a jigsaw puzzle, your study one piece among many, and if you do not know what other pieces exist, you will not know where yours fits. Transparent means you show your work, disclose your conflicts, and share your data so others can replicate it.

Her running example makes the abstractions concrete. Weather forecasting earns our trust, even imperfect trust, because it has all five marks. During the brutal 2017 hurricane season, forecasters projected Hurricane Irma's path as a cone of possibilities covering both Florida coasts. Media coverage fixated on Miami, and when the storm tracked up the west coast instead, some called the forecast a failure. But both coasts sat inside the cone all along, and the east coast still took storm surge. The system worked; our reading of it did not.

The negative example is harsher, and it has the shape of a grift. In 2014, UCLA graduate student Michael LaCour published a study in Science with Columbia's Donald Green claiming that brief conversations with gay canvassers durably changed voters' minds on marriage equality. The finding drew wide coverage and earned LaCour a job offer from Princeton. In May 2015, two graduate students trying to build on the work discovered that the data had apparently never been collected at all. Science retracted the paper, and the job offer evaporated.^[5] The detection, I want to stress, is the point. Because research is cumulative and transparent, the community catches fabrication even when it fools the reviewers; the norms the huckster counterfeits are the very norms that undo him.

The same lecture offers a toolkit for everyday evaluation that I would happily assign to any first-year student: check whether a claim cites sources; vet the credentials of the experts; be suspicious of personal experience offered as evidence ("my parents spanked me and I turned out fine"); watch for charged language; and distrust anyone whose findings deal in always and never. Rosen also lingers on the lacuna between scientific consensus and public belief. A 2013 study of nearly twelve thousand peer-reviewed abstracts found 97 percent of those taking a position endorsed human-caused climate change, yet a 2016 Yale survey found only about half of Americans believed it was man-made, and over a quarter thought scientists were deeply divided.^[6] Her caveat is characteristic: the 97 percent figure counts actively publishing climate scientists, and "whom you count as an expert matters." Even when defending consensus, she models the habit of interrogating measurement.

Why ethics boards exist #

The third lecture is the one I would most want every graduate student to sit with, because it explains a bureaucracy that otherwise feels like pure friction. Research ethics, the codes and norms of accepted behavior for those who conduct research, exist because, in Rosen's blunt phrasing, "researchers without regulations can be very dangerous."

The history is grim, and I will state it plainly. Nazi physicians froze, burned, and poisoned concentration camp prisoners. The revelations at Nuremberg produced the Nuremberg Code of 1947, the first formal regulations for human subjects research, with voluntary consent as its first principle.^[7] The World Medical Association's Declaration of Helsinki followed in 1964, declaring that generating new knowledge "can never take precedence over the rights and interests of individual research subjects."^[8] And in the United States, the Tuskegee syphilis study ran for forty years; the U.S. Public Health Service observed African American men who were never told they had syphilis and who were denied penicillin once it became the standard cure. Many died; many infected spouses and children. When the press exposed the study in 1972, the outrage produced the National Research Act of 1974 and, eventually, the Belmont Report of 1979.^[9]

Belmont's three principles now govern American research through what regulators call the Common Rule. 1) Respect for persons treats individuals as autonomous agents and gives special protection to those with diminished capacity. 2) Beneficence (from the Latin for "doing good") requires maximizing societal benefit while minimizing individual harm, and Rosen's list of harms is usefully broad: physical, mental, psychological, social, economic, and legal. 3) Justice responds directly to Tuskegee's core obscenity, the long history of research that used the poor and disadvantaged as subjects while the benefits flowed to the privileged.

Institutional review boards (IRBs), required for federally funded human subjects research since 1981, are where these principles meet your project. Rosen walks through the three review levels, from exempt through expedited to full board, and delivers the line every researcher needs tattooed somewhere visible: you do not get to decide that your research is exempt; the IRB does. She is honest that the system has gaps. A 1990s whooping cough vaccine trial barred in the United States because of its placebo design simply moved to Italy and Sweden, and it passed review boards in both places. "Legal and ethical," she reminds us, "are not the same thing."

What keeps the lecture from being a compliance seminar is Rosen's insistence that ethics is also personal practice, even for those of us whose subjects are dead or made of paper. Do not fabricate, and design studies so that any result is interesting, which removes the temptation. Disclose funding. Attribute ideas, and cite thoroughly and often. She even describes her own interview data practices (stripping names from recordings, locking consent forms away) and jokes about envying historians, who mostly study dead people and thus rarely meet an IRB.

Why this course, and this series #

Thicker methods textbooks exist, but Rosen's course does something textbooks rarely manage: it makes methods feel like a form of intellectual honesty rather than a hoop. The through-line of these first three lectures, as I read them, is that method is the bulwark between us and our own wishful thinking. Snow beat the miasma theorists not because he was smarter but because his procedure forced him to look at disconfirming cases. The five characteristics of good research are, at bottom, disciplines against self-deception. And the ethics apparatus exists because curiosity without constraint has a body count; method and ethics are, in this way, flip sides of the same coin. "Ethical review boards," Rosen says, "are there to protect people from our curiosity."

In the next post, I will take up the passage from a topic you love to a question you can actually answer, the literature review as your entrance ticket to the scholarly conversation, and what makes a hypothesis worth testing.

Notes #

---

1. Amanda M. Rosen, Effective Research Methods for Any Project (Chantilly, VA: The Teaching Company/The Great Courses, 2019), 24 lectures. Rosen is Associate Professor of Politics and International Relations at Webster University and a recipient of the William T. Kemper Award for Teaching Excellence. This post draws on Lectures 1–3. ↩︎

2. Rosen, Effective Research Methods, Lecture 1, "Why Research Methods Matter." On Snow, see also Steven Johnson, The Ghost Map: The Story of London's Most Terrifying Epidemic (New York: Riverhead, 2006). ↩︎

3. On the Perestroika movement, see Kristen Renwick Monroe, ed., Perestroika! The Raucous Rebellion in Political Science (New Haven: Yale University Press, 2005). ↩︎

4. Rosen, Effective Research Methods, Lecture 2, "Characteristics of Good Research." ↩︎

5. The retracted paper is Michael J. LaCour and Donald P. Green, "When Contact Changes Minds: An Experiment on Transmission of Support for Gay Equality," Science 346, no. 6215 (2014): 1366–69; retraction, Science 348, no. 6239 (2015). The irregularities were documented in David Broockman, Joshua Kalla, and Peter Aronow, "Irregularities in LaCour (2014)," May 19, 2015. ↩︎

6. John Cook et al., "Quantifying the Consensus on Anthropogenic Global Warming in the Scientific Literature," Environmental Research Letters 8, no. 2 (2013); Yale Program on Climate Change Communication, Climate Change in the American Mind (2016), discussed in Rosen, Lecture 2. ↩︎

7. "The Nuremberg Code (1947)," in Trials of War Criminals before the Nuremberg Military Tribunals, vol. 2 (Washington, DC: U.S. Government Printing Office, 1949), 181–82. ↩︎

8. World Medical Association, Declaration of Helsinki: Ethical Principles for Medical Research Involving Human Subjects (1964, amended repeatedly since). ↩︎

9. National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, The Belmont Report (Washington, DC: U.S. Department of Health, Education, and Welfare, 1979). On Tuskegee, see James H. Jones, Bad Blood: The Tuskegee Syphilis Experiment, rev. ed. (New York: Free Press, 1993). All discussed in Rosen, Lecture 3, "Doing Research Ethically." ↩︎