Sunday Note: I did my own research

Of course you did ...

When people use the word “research” as in “I did my own research,” it seems they are confusing the results of their personal process with “knowledge.” In a world where seemingly endless amounts of information available at our fingertips, there is clearly a desire by many to make sense of that information by personal exploration. However, to confuse information gathering (“research”) by an individual with “knowledge,” is a phenomenon that highlights a troubling trend for those of us in the “reality based community.”¹

Within the reality based community the word “research” has more to do with hypothesis testing in the context of scientific discovery than searching for data to confirm something you already believe or suspect. One of humanity’s greatest innovations has been outsourcing the process of knowledge generation to the masses (Rauch calls it the original social network). Karl Popper, the twentieth century philosopher of science, observed in The Logic of Scientific Discovery, that science may think of itself as looking for proof or verification that something is true when in reality the method of science that produces knowledge is falsification. We depend on the network to destroy bad ideas and elevate the good ones.

This is a key point. Sure, you and I can run off and try to find support for the things we believe in. In fact, thanks to the psychological phenomenon known as confirmation bias, it feels good to do so. Both amateur and professional scientists often do just that. The difference would be that they then subject their findings to disconfirmation by the network and ultimately agree to abide by the network’s conclusions. This process of knowledge generation is very egalitarian. Literally anyone can propose an idea and engage in research, it doesn’t mean the ensuing results can be counted on as knowledge.

Knowledge generation via science is a system that operates by subjecting all hypotheses to criticism. Completely unlike content posted by the average YouTuber, who is free to post anything they like, knowledge is arrived at by testing ideas experimentally, logically, and many other ways. Science shoots down thousands, even millions, of hypotheses every day. Contrast this with YouTube what-to-watch-next and Google search algorithms which are essentially giant confirmation bias engines seemingly designed to keep you engaged in their respective platforms.

The key point here is that knowledge generation is not something that can be done in solitude. Knowledge acquisition via science is an error seeking system, it is not, as so many seem to think, a process. Rather, science is a specialized social network where participants have agreed to play by two fundamental rules. The results produced by this social network’s participants form the basis of what we call knowledge and inform those of us who choose to live in the reality based community.

This network now transcends the sum of it's participants, it has become a hive intelligence, a social mind. Steven Sloman and Philip Fernbach describe this in their 2017 book, The Knowledge Illusion: Why We Never Think Alone,

People are like bees and society a beehive: our intelligence resides not in individual brains but in the collective mind. To function, individuals rely not only on knowledge stored within our skulls but also on knowledge stored elsewhere: in our bodies, in the environment, and especially in other people. When you put it all together, human thought is incredibly impressive. But it is a product of a community, not of any individual alone.

What are the two rules that have so propelled our hive mind and the state of knowledge for humanity? Jonathan Rauch has probably described them the most succinctly, so I shall use his version²

1. No one gets the final say.

2. No one has personal authority.

Both rules have deep social network implications. “No final say” requires that for something to be considered knowledge it must be subject to scrutiny. As you can infer from this, what we would call knowledge is always provisional. We only accept something as knowledge as long as it has not been undone by additional scrutiny.

It is our acknowledgement that anyone and everyone enjoys a possibility of being wrong. Claims of knowledge therefore can never be said to have been settled for good. In turn, this implies no authority or activist can legitimately shut down inquiry or debate. Should someone try to shut down inquiry or debate, or fix the outcome of an inquiry, they are not engaged in knowledge generation.

“No personal authority” deals with the question of what counts as scrutiny and what does not. This is where relying exclusively on results from a YouTuber comes into conflict with knowledge. Sure, we can engage in research to gather information, but what we gather is not necessarily knowledge. As Richard Feynman said in his 1974 Cal Tech commencement address³ “The first principle is that you must not fool yourself—and you are the easiest person to fool."

The rule requires that for a claim, no matter who makes it, to have the potential to be knowledge it must be subjected to critical analysis. We acknowledge this because none of us can be trusted to be the final arbiter due to our own tendency towards confirmation bias.

This brings us back to research. It’s fine that you’ve done your own research as long as you don’t consider it knowledge until it has been subjected to, and survived for the time being, intense scrutiny. An example of this scrutiny is is peer-review. Peer-review is the process in which research must pass the critical scrutiny of other experts before it’s published in a scientific journal.

When, through the process of peer-review, diverse,, independent, and repeatable lines of evidence converge onto a conclusion, the conclusion is considered strong, and experts generally accept it. This result in expert consensus, or the collective position of experts based on an evaluation of the body of evidence. Yet, as we noted earlier, it can never be the final say. Others can then build upon the foundation of knowledge that was gained to learn more about what they don’t know.

In the modern era it is quite impossible to be an expert in all areas of human knowledge. Instead, we generally trust experts and use the consensus as a short-cut for our decision-making. For example, if five randomly selected doctors told me that the recent mRNA vaccine was safe and effective against COVID-19, I would surely trust their opinion (as well as the body of research) to inform my opinion over some random ass YouTuber! Sadly those seeking to make this a political issue, like right wing radio and television media, know full well that the public trusts the expert consensus, which is why their strategy includes telling the public there isn’t one, using bulk fake experts to counter a real consensus, or even suggesting consensus isn’t a part of the process of science.

So, next time someone tells you “they’ve done their own research” give them the very skeptical look which such a claim deserves. This should be quickly followed by a look of sadness for them. Perhaps they will bother to ask why you are looking at them like that and you can invite them back into the reality based community and help them access actual knowledge via some peer-reviewed literature.

Don’t forget to tell them that what we “know” is subject to disconfirmation by new information developed in the future.

1. Rauch, Jonathan. The Constitution of Knowledge (p. 4). Brookings Institution Press. Kindle Edition.

2. Rauch, Jonathan. Kindly Inquisitors (p. 46). University of Chicago Press. Kindle Edition.

3. https://calteches.library.caltech.edu/51/2/CargoCult.htm