Session 5
Session #5
March 3, 2021
CREDIBILITY OF SCIENCE
In Session #3, two weeks ago, we discussed the benefits of science and why and how governments support it through grants, prizes, and other mechanisms. We focused on Dava Sobel’s short book about prizes for determining position at sea (Longitude) and Vannevar Bush’s blueprint for funding science through the US government (Science: The Endless Frontier).
What are the failures and uncertainties of science and their consequences?
Our objectives in this session are to discuss some of the weaknesses of the scientific method:
—-its inherent uncertainty and its fallibility, as well as
—-the undermining of science by misconduct and poor practices (such as those that produce work that is not reproducible).
Another way to think about these problems is to ask: Why do scientists get things wrong? The answers include misconduct (willful cheating); sloppiness; and the fundamental difficulties of deciphering how the world works (that is, science is hard and never-ending, and mistakes are inevitable).
We will consider what can be done about these things—for instance, to minimize inaccuracy, error, and fraud—and what the consequences of these properties of science might be for public policy, including:
— policies based on flawed science;
—public distrust of science; and
—the willful misuse or abuse of the nature of science to advance special interests.
INSTRUCTIONS:
1) Misconduct.
Everyone should watch this short talk on YouTube by Howard Schachman, who made a compelling case for a strict definition of what constitutes misconduct:
https://www.youtube.com/watch?v=k554m3CghYI
Are you satisfied with Howard’s definition? If not, how would you change it?
2) Reproducibility
Science has always been defended as sound because important conclusions, proposed by individual laboratories, are generally accepted only after scientific communities have repeatedly found or built on the evidence that supports the conclusions. But this claim has recently been challenged.
Everyone should read this brief account of the difficulty that pharmaceutical companies claim to have had in reproducing work that could affect what drug companies work on:
http://www.nature.com/nature/journal/v483/n7391/full/483531a.html
Several people have argued that we should have mechanisms for reproducing important or widely cited work. Think about the benefits and difficulties of doing this. At least three people should look at papers in the January 2017 issue of the open-access journal, eLIFE (https://elifesciences.org/archive/2017/01)
to see what is being done and what some people think about the efforts to demonstrate reproducibility.
At a minimum, those same three should look at the very short essay by Nosek and Errington: https://elifesciences.org/content/6/e23383
Some of you interested in cancer biology should look at an article about one of the five replication efforts (perhaps one about certain mutations in melanomas: https://elifesciences.org/content/6/e22662)
Another possible contributor to the replication problem is “publication bias”, a tendency to favor publication of so-called “positive results.” Some of you from the social sciences might find this interesting:
https://www.ncbi.nlm.nih.gov/pubmed/?term=nissen+gb%2C+magidson+t%2C+gross+k
- Failures of scientific methods can have important practical consequences.
One striking example, recently in the news, concerns the use of science in the judicial system. At least one person, preferably more, should read the executive summary of a recent report from the President’s Council of Advisors on Science and Technology (PCAST) on the imperfect accuracy of such methods:
Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods (PCAST, Executive Office of the President, September 2016)
- Abuse of statistical methods can lead scientists astray. One such practice—called P-hacking— was brought to my attention last year by a student from the MHC360 class of 2019. At least one of you should look at the recommended article and be prepared to summarize it. Here’s what the former 360-er had to say:
“Stats Experts Plead: Just Say No to P-Hacking” – The Wire) directly references food scientist Brain Wansink’s case in its discussion of how researchers can improve the usage of p-values in their studies – and cautions against misusing them to sensationalize results or to make data more publishable.”
- The inherent limitations of the scientific method. Philosophers of science have long worried about what is necessary for scientific observations to become the basis of accepted conclusions about how the world works. One of the best-known theses is this one: Karl Popper, The Logic of Scientific Discovery, 1935. At least three people should browse sufficiently in this book to summarize Popper’s widely cited point of view.
Excerpt: http://personal.lse.ac.uk/ROBERT49/teaching/ph103/2013-2014/pdf/Popper_LSD_Ch1.pdf
Squashed version: http://sqapo.com/popper.htm
A recent book that confronts the question of how Popper’s ideas are used in the practice of science—The Knowledge Machine: How Irrationality Created Modern Science by Michael Strevens (Norton, 2020)—- was entertainingly reviewed in an article I mentioned among references for Session #1: Joshua Rothman, “The rules of the game: How does science really work?” The New Yorker, Oct. 5, 2020. https://www.newyorker.com/magazine/2020/10/05/how-does-science-really-work I’d encourage a few of you to read Rothman’s essay and report about it.
Brief optional amusements:
- Sara Whitlock, “A new PhD student learns her first lesson: Certainty doesn’t exist in science,” STAT 12/30/16
https://www.statnews.com/2016/12/30/science-grad-student-lesson/
- PLOS ONE essay on the role of the media and institutions in promoting questionable conclusions:
https://www.statnews.com/2017/03/02/science-media-news/
4) The inevitability and virtues of scientific errors
The titles below refer to essays that tout the virtues of making mistakes in honorable efforts to solve big problems. None of this is required reading, but I’d be pleased if one or more of you wanted to follow up on Watson’s The Double Helix by reading the two chapters in Livio’s excellent book that describe how Linus Pauling (arguably the greatest chemist of the 20th century) got misled about the structure of DNA and lost out to Watson, Crick, and others.
Livio, Brilliant Blunders From Darwin to Einstein: Colossal Mistakes by Great Scientists That Changed Our Understanding of Life and the Universe, Simon and Schuster, 2013. (Chapters 6 and 7)
Schultz, Being Wrong: Adventures in the Margin of Error, HarperCollins, 2010.
Hannah Bloch, “Failure is an option,” National Geographic, September 2013.
5) Consequences of the inherent inaccuracy of science for the formulation of public policy.
We will think about this in relation to current debates about some highly sensitive topics: climate change, vaccination, evolution, and compensation for sports injuries, and we will also discuss these issues as they affected the control of tobacco use.
Can you recall moments during the Covid-19 pandemic in which doubts about the validity of science have affected public policy? I suspect that all of you have good examples.
Climate change
The debates about climate change can affect legislation, regulation, and budgets. Everyone should read Robert Proctor’s brief summary of the issue and its contemporary relevance:
For those who have not followed the science about climate change, here is a brief, useful consensus view:
http://www.nytimes.com/interactive/2015/11/28/science/what-is-climate-change.html
Sports injuries
There are complex medical issues here and a lot is at stake for team owners. You may notice a pattern in the way the issues are framed.
http://www.nytimes.com/2017/02/08/sports/hockey/nhl-chronic-traumatic-encephalopathy-cte-juliet-macur.html?smprod=nytcore-ipad&smid=nytcore-ipad-share The league’s onerous demand in a class-action suit on head injuries seems to show little regard for medical privacy and accepted scientific facts.
Other contentious issues (with different degrees of consensus)
Does immunization increase the risk of autism?
Is evolution of species still a theory or an established fact?
Does tobacco use increase the risk of cancers and other diseases?
I am making no specific reading assignments for these topics but our discussion will be informed by anything that anyone chooses to read from a vast literature.
Additional readings
To enlarge the discussion beyond science to politics and to think about how the political approaches are framed, here are a few relevant, recent books on the political uses of uncertainty:
- Oreskes and E.M. Conway: Merchants of Doubt: How a Handful of Scientists Obscured the Truth on Issues from Tobacco Smoke to Global Warming
(Bloomsbury Press, 2010). I will summarize the argument for our discussion.
- Nesbit: Poison Tea: How Big Oil and Big Tobacco Invented the Tea Party and Captured the GOP(St. Martin’s Press, 2016)
Shawn Otto, The War on Science (Milkweed, 2016)
