Updated: Sep 29, 2021
We spend a lot of time discussing how to interpret science at ClearerThinking, and for good reason. Science is the best system available for improving our understanding of the world, but like all human endeavors, it's flawed and susceptible to a variety of errors. It's important to understand these issues in order to get the most out of science-related news – especially when it comes to absorbing "novel" or "revolutionary" new findings with appropriately-sized grains of salt at your disposal.
To that end, we recommend reading Vox's recent lengthy feature on the 7 biggest problems currently facing science, according to scientists themselves. Compiled from 270 responses to a survey distributed to English-speaking biomedical and social scientists, the piece isolates a number of structural problems with the way that scientific funding, publishing, and self-regulation work. (Ironically, the survey itself was admittedly not thoroughly scientific, but it does a great job of diagnosing these issues anyway.) These problems can dilute the purity of knowledge that scientific rigor is designed to protect, miscommunicate the findings that research produces, or even drive talented people away from jobs in science.
Since the piece itself is quite long, we've summarized its 7 main findings in brief below. Before we get to that, though, it's worth reiterating that this discussion does not amount to an indictment of science as a whole. The scientific method has produced more reliable and useful information about how our world works than any alternative, and it's unique in that it has mechanisms for acknowledging its own imperfections. Identifying shortcomings in the way science works is itself an important part of science – conducting reliable research is a process that demands constant revision and refinement, and thinking carefully about what's not working now is the only way to improve that process in the future.
Issue #1: Academia has a huge money problem.
Simply put, science is expensive, and there's not enough money to go around. Universities don't have the resources to fund all the research their faculties conduct, and scientists face substantial pressure to regularly publish their work in order to keep their jobs. As a result, these academics frequently rely on outside grants to fund their research.
This necessity produces a lot of problematic incentives for scientists. Government grants, which are the most common type in the United States, have become increasingly rare and therefore increasingly competitive. This dynamic encourages scientists to produce more predictable – and less informative – experiments, because such projects are more likely to win funding.
Meanwhile, the most obvious alternative to public funding – private grants – are often set up by special interests, which encourage research that supports their agendas.
Many grants also expire after 3 years, which makes novel long-term research much more difficult to fund.
As a result of all these factors, scientists tend to focus on research that's safer, more conservative, and ultimately less useful than they otherwise might.
Issue #2: Too many studies are poorly designed.
Scientists face a lot of pressure to produce novel, groundbreaking discoveries. However, such discoveries simply don't happen very often.
In fact, studies which "fail" – that is, which don't verify the hypothesis they're designed to examine – are just as important for advancing understanding as studies which do. However, "successful" studies offer far more in the way of professional rewards for scientists.
As a result, it's tempting for scientists to make their findings look more significant by designing less-rigorous experiments, failing to report the level of statistical significance of their results, or even deliberately misreading their own data.
Some respondents to the Vox survey proposed alleviating this issue by changing the way scientific publishing works. One possible technique would involve publishing papers that demonstrate highly rigorous design, rather than statistically significant results.
Issue #3: Scientists rarely replicate experimental results.
As we have frequently mentioned on ClearerThinking, it takes multiple studies to really verify a scientific result, as both random chance and flawed experimental design can throw individual studies off the mark.
Unfortunately, there are very few professional incentives for scientists to even attempt to replicate results. Both funding agencies and scientific journals prefer papers that either produce new results or directly contradict older ones, and many studies are too vague about their own methods to be replicated.
This situation has produced a major deficiency in replication. For example, one 2015 study examined 83 frequently-cited papers and found that just 16 of them had been successfully replicated.
As a result, many findings that would be moderated or even contradicted by replication attempts pass into the realm of accepted truth.
Issue #4: Peer review doesn't work the way it's supposed to.
Many academic journals rely on peer review — a process in which scientists unaffiliated with a study submitted for publication check it carefully, providing constructive criticism along the way — to determine which papers are worth publishing.
However, this vetting process often fails to catch errors. Again, the underlying reasons have to do with incentives for scientists. Peer review typically isn't paid, which means that there's little reason for reviewers to devote precious time to reading the papers in their charge carefully.
Issue #5: Scientific journal paywalls make reading results expensive.
Many prestigious scientific journals are owned by for-profit publishers, which place their content behind expensive paywalls – some of these journals cost hundreds or even thousands of dollars per year for a subscription.
As a result, it's prohibitively expensive for many researchers whose home institutions don't pay for journal subscriptions to access all of the published research that's relevant to their fields. As a result, pirating research papers has become a common practice among less-fortunate scientists.
Issue #6: Science is often poorly communicated to the public
Bad science reporting is a subject we've discussed at length before. But it's not just reporters that have incentives to garble or exaggerate scientific findings.
University press offices often inflate the importance of findings too; even scientists sometimes attempt to raise their public profile – or secure more funding – by hyping up their results.
Scientists themselves also don't have many incentives to put their work in terms that the public can understand in the first place.
Issue #7: It's very hard to be a young scientist.
Science is usually considered a prestigious line of work, but many young scientists – especially graduate students and postdocs – work extremely long hours for scant pay, with little in the way of benefits or job security.
Many of these freshly-minted scientists, who are responsible for a great deal of the day-to-day labor that research requires, are treated as independent contractors on short-term contracts. This drives them to produce conservative research with fast turnaround times, just as 3-year grants do.
Universities also produce more new Ph.D.s than they do job openings for academics, which means that it's often difficult for young scientists to find employment at all.
These stress factors hurt morale and sometimes drive talented people to leave the field altogether. For example: one 2015 study at Berkeley found that a full 47% of the university's Ph.D. students showed signs of depression.