Table Of Contents
In this post, the second in a six-part series on Amanda M. Rosen's Great Courses series Effective Research Methods for Any Project, I follow lectures four through six into the stage of research where most projects quietly die: the passage from a topic you find interesting to a question you can answer, a literature you know, and a hypothesis you can test.[1] Part one covered why methods matter and how research ethics developed. Here I will argue that this early stage is best understood as a series of conversions (a worry into a question, a curiosity into a position, a hunch into a testable claim), and that each conversion, though it costs something, is precisely what makes the eventual answer worth having.
A topic is not a question #
Rosen opens lecture four with a teaching trick. Imagine, she says, that this course is now graded, and your entire grade rests on one essay about your favorite hobby. Nothing more; that one sentence is the whole assignment. Where would you even begin? Her own hobby is knitting. Should she write about its history, its techniques, its economics, its communities? The paralysis you feel facing that prompt is the paralysis of a topic without a question. Now change the assignment to "What is the history of how your favorite hobby became a hobby?" and the fog lifts. You know where to start, how to proceed, and, just as important, when to stop.
"Most research starts with a topic," she says, and "starting with a topic is fine, but you shouldn't end with a topic." She testifies from experience: she spent two years stalled on a dissertation topic that bored her, making no progress at all, then switched to a question that gripped her and finished the PhD in two years.
Good research questions, in her account, share three characteristics. 1) They are unanswered. Research aims at producing new and original knowledge, so the answer should not be findable in a quick internet search, and the question should let you enter a debate, fill a lacuna, or solve a puzzle. 2) They are appropriate in scope, matched to the length of your product, your access to data, your time, your skills, and your funding. She tells a story about a graduate school colleague who proposed a comparison of Japan and Turkey; the group tried to talk him out of it as impractical until he mentioned that he was fluent in Japanese and Turkish. Objections withdrawn. Scope is always scope-for-you. 3) They are empirical rather than normative. Empirical questions ask how the world works; normative questions make judgments about how the world ought to work. "Should people adopt a cat?" is normative. "What is the impact of adopting a cat on people?" is empirical. The normative question can and often should motivate the empirical one, but data alone cannot answer it.
The empirical-normative line gets a small detour worth keeping. A norm, Rosen explains, is "just a shared code of behavior," usually an unwritten rule a group holds for interacting. You feel norms most sharply when someone violates them, as when a stranger sits directly beside you in an otherwise empty movie theater; the same seat choice in a packed theater violates nothing. Normative questions are questions about those oughts, and the researcher's move is not to abandon them but to convert them, mining the ought for the empirical questions buried inside it.
For finding a topic in the first place, she offers a brainstorming exercise I have started recommending to students. Make two columns, topics you know a lot about and topics you would like to learn more about, then look for productive collisions. Her own pairing constellates the theory of international relations with the television show Survivor, which is a better research site for theories of alliance behavior than it has any right to be (the show, I would add, is a primary source here, not a garnish). A colleague's pairing of the history of France with cooking yields the history of French cuisine. The collision is where sustained motivation lives, and motivation, as her two lost dissertation years testify, is a methodological variable.
The craft advice is equally concrete. Start questions with the five Ws and H, favoring why and how for causal work. Avoid yes/no questions. Do not presume the answer in the question itself, as in "Why are eggs a healthier breakfast option than cereal?" And do not overcomplicate: "it's better to answer a narrow question well than a large question poorly." Her worked example is climate skepticism. For most of us, "Is global warming real?" is a poor question because a scientific consensus already exists. But "Why is there continued skepticism of man-made climate change in the United States, given a scientific consensus?" is a live puzzle, the kind of mismatch between certainty and belief that fuels genuine research.
The literature review as entrance ticket #
Rosen builds lecture five around a scene anyone who has attended an academic conference will recognize. Imagine barging into a cocktail party circle of Shakespeare experts and announcing your pet theory that Shakespeare's plays were written by someone else. You do not know that the theory has been discussed to death, and you do not know that the polite strangers in the circle are the leading authorities on it. "So don't be a cocktail party bore," Rosen concludes. "Do the literature review."[2]
The definitions here are the ones I find students most often lack. The literature is the body of scholarly work on a given question, and a scholarly source is one written by an expert, written for other scholars, and peer reviewed. All three criteria must hold, which is why textbooks and serious journalism, whatever their virtues, do not count. The literature review is both a process (finding and evaluating that body of work) and a product (the section of your write-up that presents what you found). Her practical method is the snowball: find one or two recent, relevant articles, mine their bibliographies, then mine the bibliographies of what you find there. The sources that keep recurring are the core of the literature; Google Scholar citation counts help you triangulate.
The crucial distinction is between an annotated bibliography and a genuine review. Listing sources one by one, with a summary of each, is a first step, not a lit review. A real review synthesizes by theme, debate, gap, or method. "Make the authors talk to each other," Rosen says. Her cautionary tale is personal. In graduate school, she presented work at a conference having skipped part of the literature, and an audience member pointed out a highly relevant book she had missed. "It was mortifying, and I almost quit graduate school over it," she admits, but the work got stronger. The stakes scale up outside the academy too; a lawyer's literature review is case law, and skipping it is malpractice.
The subtler payoff is that the literature supplies standards. If you write your own survey questions on healthcare instead of using established national ones, you lose the ability to compare your results with anyone else's. If you study war, you inherit a debate about what counts as one: the Correlates of War project defines it as sustained combat between organized armed forces with at least a thousand battle-related fatalities in a twelve-month period.[3] Definitions like that one are the accumulated capital of a field. Ignoring them means starting from zero and being incomparable to everything.
Theory, hypothesis, and the forbidden word #
Lecture six begins with a confession of pedagogical theater. Rosen bans the word prove in her classes. Papers using it get a giant red no, and if a student says it twice aloud, she throws herself to the ground in mock death, to rise again only when the class explains why. Mathematics has proofs; courts prove guilt beyond a reasonable doubt; science does something different. "In science, we set out to disprove, not prove."[4]
The scaffolding underneath the theater is the relationship between theory and hypothesis. Theories are generalized explanations for what we observe in the world; hypotheses are testable statements that give the researcher's best guess at an answer to a research question, derived from theory. Her image for the relationship is hard to forget: "A hypothesis without theory is like a birthday cake without frosting, incomplete, unappetizing, and likely to end up in the trash."
A strong hypothesis has four features. 1) It is probabilistic, phrased in terms of more likely and less likely rather than always and never. 2) It is generalizable, about groups and concepts rather than about her cat Xander or her dog Obie. 3) It states a specific relationship between variables, and here the course introduces the vocabulary that runs through everything that follows: the independent variable, X, is the proposed cause, and the dependent variable, Y, is the outcome; relationships can be positive, with variables moving in the same direction, or negative, moving in opposite directions. 4) It is falsifiable, a criterion Rosen credits to the Austrian-born philosopher Karl Popper (1902–1994), for whom falsifiability distinguishes science from non-science.[5] Her example of the unfalsifiable is the Loch Ness monster. No search of the lake can disprove Nessie to a believer, which is precisely why, "despite no evidence beyond a blurry photograph, plenty of people still think Nessie is out there." The same structure explains why certain religious claims sit outside scientific testing, not because they are unimportant but because no observation could count against them.
Alongside your hypothesis stands its shadow, the null hypothesis, which states that no relationship exists between your variables. In a twist that surprises most students, statistical testing actually engages the null: we try to disprove the null rather than prove our own idea. That inversion is the operational form of scientific humility, and part five of this series will show how it cashes out in p-values and significance tests.
For actually generating hypotheses, Rosen offers a drafting method humble enough to use. Put your outcome, the dependent variable, on the right side of a page. Array every candidate cause you can think of on the left, connected by arrows, then let your reading of the literature thin the arrows down to the ones worth testing. She even supplies sentence templates: "Value one of X is more likely to lead to value one of Y than value two of X," or, for correlational claims, "There is a positive relationship between X and Y." The templates feel mechanical, and in my estimation their machinery is the point: they force every vague intuition to declare its variables, its direction, and its testability before it gets to call itself a hypothesis.
The lecture closes with the kind of candor that makes this course better than a textbook. Negative results are hard to publish, Rosen acknowledges, and researchers sometimes run models until something turns up significant, then "pretend that this was their hypothesis all along." She names the temptation precisely so you will recognize it from the inside, and she declines to bless it. The discipline of stating hypotheses before testing them is what separates research from rationalization; the alternative is a genteel form of grift.
What this stage is really for #
Reading these three lectures together, I return to my thesis: the early stage of research is a series of conversions. A normative worry becomes an empirical question; a private curiosity becomes a position in a scholarly conversation; a hunch becomes a falsifiable statement with named variables. Each conversion costs something. The question is narrower than the worry, the literature humbles the hunch, and the hypothesis exposes you to being wrong in public. But each conversion is also what makes the eventual answer worth having. An unfalsifiable claim can never be more than an opinion with footnotes.
For those of us in the humanities, treating this machinery as the property of someone else's discipline is tempting; Rosen's examples, drawn from politics, business, cooking, and pet ownership, suggest otherwise. A dissertation on nineteenth-century religious print culture still needs a question rather than a topic, still owes its readers a genuine synthesis of the field, and still benefits from stating what evidence would count against its argument. The vocabulary transfers even where the statistics do not.
In the next post, I will take up research designs from experiments to ethnography, how to measure concepts without fooling yourself, and why sampling is the difference between evidence and anecdote.
Notes #
Amanda M. Rosen, Effective Research Methods for Any Project (Chantilly, VA: The Teaching Company/The Great Courses, 2019). This post draws on Lecture 4, "From Topic of Interest to Research Question," Lecture 5, "What's Already Known: The Literature Review," and Lecture 6, "Generating Hypotheses and Theories." ↩︎
Rosen, Effective Research Methods, Lecture 5. ↩︎
On the Correlates of War definition, see Meredith Reid Sarkees and Frank Wayman, Resort to War: 1816–2007 (Washington, DC: CQ Press, 2010), and the project's documentation at correlatesofwar.org, discussed in Rosen, Lecture 5. ↩︎
Rosen, Effective Research Methods, Lecture 6. ↩︎
Karl Popper, The Logic of Scientific Discovery (1959; first published in German as Logik der Forschung, 1934). Popper's falsifiability criterion is discussed in Rosen, Lecture 6. ↩︎
Tags : research-methods methodology academia writing
Webmentions
No webmentions yet.