It may often seem like skeptics are simply down on whatever we perceive as "woo", sneering and dismissing such things out of hand.
It would be so much easier were that the case. The thing is, many skeptics would love for things like Bigfoot, telekinesis, and alien visitation to be real. Let's face it; those things would be
cool. Besides that, it would mean that there needed to be a complete re-write of the way we understand the cosmos. It would be thrilling and terrifying to thing that the most basic things that we thought we understood about the world to be wrong. Think about it. Piloted alien spacecraft doesn’t just imply “there’s someone else out there”; it means that physics doesn’t work the way we think it does. It means that faster-than-light travel is attainable, that power sources able to produce such travel are possible, that virtually everything we think we know about space, time, energy, and a whole host of other fields is wrong. That’s not bad to the mind of a skeptic. Utterly terrifying, perhaps, but not
bad. It opens up new fields of study, new opportunities, and a whole new paradigm of thought.
But, first, the skeptic wants you to prove it. Skeptics aren’t a bunch of Fox Mulders. We don’t want to
believe. We want evidence. Belief doesn’t cut it. I can believe my Great-Aunt Hattie takes regular jaunts to the moon and back. That doesn’t mean that I’m right about her lunar escapades, and that she enjoys tea on Tuesdays by the Apollo Rover. Frankly, my belief doesn’t even establish that I have a Great-Aunt Hattie. For that, I would need evidence.
Ah, evidence, the great bugaboo of woo-meisters everywhere. According to them, they have it. According to us, not so much. See, there are differing levels of evidence. I think
Steven Novella covers a lot of the basics very well at his various blogging locales. I find that, at a basic level, there are several layers, from anecdote to experiment and a whole mess of sliding scales of effectiveness for each.
***
“I know what I saw.”
“Y happened after X, so I know that Y is caused by X.”
“My Aunt Hattie told me it happened to her.”
Anecdotes, despite the
pithy skeptical saying, are evidence. They just aren’t
good evidence. They are extremely inadequate for establishing the reality of most anything, for a variety of reasons. Not the least of these is that people will lie for their own gain. We’ve got plenty of evidence of this without even looking at woo; politicians, criminals, small children…the list includes most everyone at one point or another, but others make a living from it. Aside from that argument, which some are quick to deny (“My healer/pastor/uncle/mechanic would never lie to me. He’s a good person.”), anecdote suffers from the fact that human memory is, in general, exceptionally poor. Numerous studies have shown that human beings start rewriting history in their heads almost as quickly as they experience things. Such rewriting is heavily influenced by personal biases, previous experience, and knowledge of conditions outside the referenced experience; in addition, humans tend to idealize or denigrate their own past beyond what actually occurred. These same biases also influence initial perceptions of an occurrence, which are then further biased by memory.
This is not to say that everything anyone has ever remembered is a lie. Rather, what it means is that the scale of evidence cannot be tipped very far by anecdotal evidence. It’s the featherweight of the evidentiary boxing contest; it’ll probably land a huge number of punches, but it’s not likely to knock the other side out of the ring. “But,” one might say, “what about anecdotes and statements from someone who knows what they are talking about? What about, for example, all the pilots and military people who have witnessed UFOs?”
Skepticism demands multiple answers to that question. First and foremost, what is it about a pilot that makes them more likely to understand a UFO? Is it simply that they spend a lot of time in the sky? Is it that they can handle complex equipment? Or is it more that we simply expect them to know more about what goes on up there; secrets that we regular folk don’t experience. Given that, why should we expect them to know more? Barring a grand conspiracy, a pilot may be just as clueless as a train engineer as to what a light in the sky may be. Proximity does not equal expertise.
As a counter-example, astronomers as a group have a
relatively low rate of UFO sightings. But key in that is what the term “UFO” actually means. People forget that it is an acronym, and that the first letter stands for “Unidentified”. Astronomers, by definition, spend a lot of time looking at the sky, sometimes orders of magnitude more than any other given individual. With that experience, they gain the ability to identify much more of what they see. An object that may be “unidentified” for a layperson could be something easily identified by an astronomer. That’s where a mismatch of knowledge and experience can make a difference. Getting back to the original question, assuming that someone knows what they are talking about, and them actually knowing what they are talking about are two different things. Once again, we are back to evidence.
Somewhat above anecdotal evidence is documentary evidence. Pictures, papers, letters, diplomas, copies, and anything else that has been written down starts to meet a better evidentiary standard. While these items can be forged, it takes more effort to do so. I recall a college professor of mine who gave a personal example of this (anecdote!). He was in a lawsuit with his previous landlord over damages to the property. His claim was that all the items were present when he moved in; his landlord claimed otherwise. Now, when this professor had moved in, he jotted down a quick list of the deficiencies so that he and the owner could go over them. When opposing counsel discovered that that this man had a seven-year-old handwritten list, they settled the case. Written words carry power. Given this power, one has the same concerns that come with anecdotal evidence. The source must be considered, both for reliability and subject knowledge. When the crazy hobo on the corner hands me a tract about UFO’s, I’m not likely to equate it to an article from
Phil Plait.
Documentary evidence is a good place to explore another factor that affects the strength of evidence: plausibility. In general, plausibility is a measure of how likely something is to be a certain way. If you come across a small child sprawled next to a bicycle crying, is it more plausible that he fell off while riding, or that he was attacked by gnomes who beat him with the bicycle (Fun fact: the phrase "
gnome attacks" gets 4500 hits on Google)? On first glance, you have no real evidence either way; however, the gnomes are not likely to be your leading hypothesis. Why is that? Most of us have previous experience with what a child falling off a bicycle looks like. We are unlikely to have had previous experience with gnome attacks. We’ve probably never even heard a report of gnome attacks. Perhaps gnomes are too extreme of an example. Perhaps the options that jump to your head are falling off, wolf attack, or hit by a car. Depending on the setting and visual evidence, any one of these items can be plausible. If you have knowledge that the child is a skilled rider, you are in a rural area with an established wolf problem, and there appears to be a bite on his leg, the wolf hypothesis may be most likely. Had you encountered a weaving vehicle just prior to seeing the child, being hit by a car has some evidence to back it up. With all the possibilities listed here, the truth is likely to be revealed with a little further investigation. When it comes to more bizarre events and claims, though, this is not always the case.
Documentary evidence usually plays a big role in the realm of conspiracy theories, and that’s where plausibility can be wielded to narrow the field possibility. Often, in the wake of a large event like the Kennedy assassination or 9/11, a flood of reports, investigations, documents, and theories emerge. Some of these are backed by large amounts of evidence; others rely on smaller pieces of information. When things sort themselves out, what often remains are official reports filled with reams of data, and the other guys pointing out inconsistencies in the official version. That phrasing may sound biased against “the other guys”; frankly, it is. The “other guy”, however, isn’t always wrong, as demonstrated by Richard Feynman in his
minority report on the Challenger disaster. Feynman, however, was both a genius and a dogged investigator with no love of authority or propriety; it made him the perfect person to take apart an institutionalized bureaucracy like NASA, and he generated real data to back up his report (more on that later).
9/11 is perhaps the “big dog” of modern conspiracy theory. The Moon Hoaxers are rapidly losing ground; no one thinks much about Kennedy getting shot these days; and, to be honest, David Icke isn’t worth discussing. If lizard people are on the agenda, I think we can stop the conversation now. 9/11, for people in the USA, is the type of thing that leaves a psychological dent that goes beyond the obvious initial trauma. It reveals a vulnerability and lack of control that most people don't consider in their daily lives. Sure, a person could be hit by a car any time they cross the street; but you never actually expect it to happen. 9/11 has the same sort of mental impact, on a larger scale. A defensive mechanism for dealing with that kind of vulnerability is to imagine that there's some person or group with a grander purpose is in control of the whole thing. Following that logic, such a cabal can be fought or exposed. The thing is, events like 9/11 are sometimes perpetrated by conspiracies. The 9/11 conspiracy was made up of members of a small terror group headquartered half a world away. That reveals some vulnerability that a lot of people don't wish to consider. The idea that a couple of dozen people got together, planned, and executed such a large-scale terror attack is somehow unimaginable. The idea that 9/11 was performed by a shadowy government conspiracy to false-flag another nation, destroy secret records, steal gold, or achieve some other goal is perhaps less frightening than the idea that a handful of determined men could kill thousands and destroy major landmarks just to make a point.
The weight of the documents, however, is squarely on the side of terrorists. There is nothing implausible about what they did. The physics works. The movements of the parties have been traced to an astonishing degree of accuracy. The event, as qualified experts have investigated it, happened pretty much as the government says it did. Any political abuses performed by the government afterward have no bearing on how the original event transpired.
Many wonder how the government or military could be “caught sleeping”, and decide that an “inside job” is the only way a terrorist attack of that magnitude could occur. These people have more faith in both their government (and in other people) than I would normally ascribe to. The history of any major government is rife with blunders of astounding magnitude. The rule of thumb called
Hanlon’s Razor applies: “Never attribute to malice that which can be adequately explained by stupidity.” Now, that guideline is a bit of shorthand referring to plausibility. What the Razor is saying is, when something bad occurs, it’s more likely that someone was not doing their job correctly than it is that they were actively planning for the bad to happen. While many conspiracies do exist, most are brought down because people can’t keep secrets. The bigger the conspiracy, the harder it is to keep quiet. In terms of secrecy alone, a conspiracy of dozens to perpetrate 9/11 is more plausible that a conspiracy of hundreds or thousands (which is what a government-sponsored cover-up would entail). Even aside from that, most conspiracies are of the mundane variety; they are criminal activity writ slightly larger than usual. When ideas about conspiracies veer from that track, it’s often into the bizarre or unreal ramblings of a mentally ill person (Mind beams, alien landings, shadowy world-ruling cabals, etc.). While some large government conspiracies are in evidence (Watergate, for example), what must be noted is that these conspiracies did fall. Sometimes it’s through the accident of a security guard being where he wasn’t expected; sometimes it’s because someone breaks ranks and spills his guts to a reporter in a parking garage. That’s a demonstration of human nature; you can’t herd too many people through a lie all at once; someone, for some reason, will change their mind, slip up, or just fail through plain coincidence. That is far more plausible than completely silent, eternally vigilant rulers in darkness.
A good deal beyond the weight of documentary or anecdotal evidence is evidence obtained by systematic observation. I place this as separate from experiment because, while experiments require detailed observation, detailed observation does not require an experiment.
Darwin’s notebooks are an excellent example of observational evidence; a detailed account of locations, forms, times, conditions, and patterns eventually gave rise to the theory of evolution through natural selection. Much of science is observational in nature; astronomy, as a field, is largely pursued through pure observation. It may take years of observing the same spot and comparing thousands of photos or radio recordings to note something new and exciting in the sky. Observational evidence is noting the fact that, while you were watching, a thousand people have stepped off the curb without falling into a hole. Not only that, but information such as that 75% of them were female, 30% were wearing blue, .3% only had one leg, and .1% got hit by a bus as they stepped off. Systematic observation produces reams and reams of data, which, under the right circumstances, can produce real discovery. However, as is often the case in the world of woo, the right circumstances can be hard to come by.
There are multitudes of ghost hunting shows on television these days. Observationally, pretty much all of them are crap, despite a mountain of cameras, EM field detectors, infrared imagers, and a whole host of other gizmos. Most of the “hunters” on these shows are reduced to jumping at vague noises, and generally looking like idiots, even with all the electronics. Why? Because they are making an assumption that has yet to be borne out by the observations. That assumption is that a “ghost”, should one exist, will be in anyway detectable by the junk they have brought along. Aside from this fact, they are wildly inconsistent in what their observations tell them. “It’s cold, so there is a ghost present.” “It’s hot, so there is a ghost present.” Certainly, after years and years of observation, they might have something; however, if the primary premise (that the equipment works) is flawed, the observations are flawed at their core. There are paranormal investigators out there that don’t make that mistake; one of the most famous is
Joe Nickell. He approaches ghost stories with a rational bent, without the fancy equipment. Where some of the more “TV-oriented” hunters are likely to find that a strange knocking sound and a “hot spot” indicate a ghost, Nickell is more likely to discover that there’s an old boiler in the adjoining townhouse. This helps demonstrate another rule of thumb:
Occam’s Razor. This idea is often mis-stated as “The simplest explanation is the best”. More accurately, it can be stated this way: “The explanation requiring the fewest assumptions is more likely to be correct.” This is called parsimony, and it makes a great deal of sense. If you have two explanations for an event, one that requires two assumptions that are not in evidence, and one that requires four assumptions, the explanation with less guesswork involved is likely more accurate. In the ghost investigation example above, the “TV hunters” are making the assumptions that ghosts exist; that they are detectable by the equipment available; and that they actually produce the phenomena sought. Nickell, however, needs posit nothing of the sort. It is well established that old boiler systems can produce knocking sounds, and produce heat that can be detected through a wall, especially in an older building. Finding the mundane instead of the fantastic, however, is assumed to not be as exciting, and people like Nickell don’t get a lot of airtime.
One often encounters arguments based on unsupported premises in the world of alternative medicine claims. A huge amount of alt-med stems from premises that are either unsupported, or flat-out false. Homeopathy relies on both the “memory of water” (never demonstrated to exist) and the “law of similars” (out-and-out wrong). Subluxation-based chiropractic is based on vitalism (false) and the existence of “subluxations”. Physical anomalies known as
subluxations do exist; I have them in both my knees. However, in the terms they are described by traditional chiropractors, they are not shown to exist. Rational chiropractors that reject the idea of subluxations do exist, and focus purely on musculoskeletal manipulation and therapy.
An examination of these kinds of claims can be referred to as “prior plausibility”. That’s another way of asking, “Does the claim make sense in light of what we already know?” When a claim is made that goes against mainstream theory, the first lens through which it must be examined is that of plausibility. The recently deceased
Hulda Clark made a career of promoting the idea that all cancer (and I do mean
all) is caused by a type of liver fluke. Claims like that, when compared to the weight of evidence behind the germ theory of disease and modern epidemiology, just fail at the start. From crystal “vibrations” to homeopathy to healing chants, skeptics often discount some things out-of-hand because the rest of what we know about doesn’t work that way. While that can be a dangerous attitude to take, leading to knee-jerk rejection of new ideas (some of which might turn out to be correct, if unlikely or counter-intuitive), an examination of prior plausibility should usually be the first step when considering a claim. In particular, one must consider if an alternative explanation is more plausible than the claim. Is that person speaking to the dead, thus rewriting everything we know about physics, death, and religion; or, alternatively, are they
cold reading? Can that person really move a compass with their mind, or do they have a
magnet on their thumb? While the former cases may be true, the latter are more likely. Again, Occam’s Razor applies; some of these possible answers require more assumptions than others.
The last level of evidence that I want to consider is generally the most powerful; experiment. A really good, scientific experiment is the systematic testing of ideas to answer specific questions while eliminating confounding factors. However, experiment is not necessarily that limited. Really, it’s just trying things out to see if they work or not, and then refining the idea based on the data. The
Mythbusters are a prime example of this. While the format is often an excuse to blow things up in an ever-expanding variety of ways, a lot of what leads up to that point is experiment. There’s a lot of, “Will X do Y?” involved. The design may be haphazard (though carefully calibrated for maximum TV fun), but most everything they do is a test of a simple premise. “Can I run across this viscous liquid without sinking?” Yes. “Can you use dynamite to clean out a cement truck?” Under certain conditions, yes. “Can an fMRI detect lies?” Not reliably. They don’t follow the science to the point of narrowing down all the influences, but they do often go in for repeatability and eliminating as much variability as the TV format will allow.
A further example of that level of experiment is, well, frankly, anything
Richard Feynman ever did. Aside from his actual work as a physicist, he performed experiments on everything from how ant trails are formed to how easy it was to break into the locked filing cabinets at the Manhattan Project. His experiment involving rocket booster O-ring material and ice water was the defining factor in determining what happened when a space shuttle exploded.
Those types of experiment are fine outside-the-lab examples, but are still subject to much of the bias that humans bring to the equation. We may mentally tweak conditions or data to fit our prejudices and preconceptions. A poorly designed lab experiment can have the same problems. A major example of this was the Jacques Benveniste publication on the memory of water. Though I won’t go into great detail here (it’s been
covered repeatedly elsewhere), Benveniste and his lab produced a paper showing evidence for the claim that water could have a “memory” of substances it had previously been in contact with. Upon further examination, however, it was found that because samples were unblinded, and affected cells in the samples counted by hand, it left open the possibility that the lab technicians may have been introducing their own desired results into the mix. Following a properly blinded experiment involving
James Randi, among others, the effect ceased to manifest. Randi, in his dogged pursuit of charlatans and con artists, has made a career out of applying principles to eliminate bias or cheating in claims of the extraordinary. While not a scientist himself, as a conjurer, Randi is familiar with the way the human mind can fool itself, and where many of the biases of observation lie. That is a magnificent skill to have when sniffing out errors in experimental design.
A scientific experiment is intended to do several things. First, it seeks to eliminate sources of bias; objective results far outweigh the subjective. Secondly, it should be designed to minimize confounding factors; for example, a drug trial has to account for effects that may be due to other medications a subject is taking. Finally, it should be designed to account for how the designer is wrong about the question. Being wrong is often more informative that being correct. Being wrong can reveal confounding factors; if it was a test for repeatability, it can indicate that something was wrong with another experiment. A negative result narrows the range of possibilities, bringing an investigator closer to a more accurate answer. This philosophy of falsification is what really sets science apart as a means of understanding the world. It’s also where one of the greatest misunderstandings of science originates. Scientists are often accused of having no doubt in their own work, and having an arrogance of knowledge. It is true that there are some aspects of science where you don’t hear a lot of professed doubt, such as the germ theory of disease. That’s because much of the doubt has been eliminated with repeated attempts at falsification; if thousands of people have spent decades actively seeking flaws in an idea, what’s left tends to be pretty accurate (even if the original intent was entirely different; see
phlogiston). That’s not to say any theory is perfect; scientists argue with each other constantly. Depending on the field, the arguments are often more likely to be about details of a theory than they are to be about the theory itself. It’s the “big parts” that scientists are often accused of being inflexible on; this may well be true, as those tend to be the most well-studied and tightly tested aspects of an idea. No serious evolutionary scientist questions that DNA and natural selection are major components of the theory; whether or not a particular gene expression will affect development might be an ongoing fight. In any case, the fact is that mainstream science is constantly trying to figure out reasons why they might be wrong. This makes it a stronger system than one that only relies on positive results.
With this list of possible levels of evidence (not complete, by any measure), I hope I’ve provided some glimpse as to why skeptics are skeptics. It’s not that we push away the fantastic, close our eyes and ears, and try to block away what doesn’t fit our worldview. All the skeptics I know would love for some form of the fantastic to be real, if for no other reason that it would open up a whole new world to explore. It would be new and potentially frightening; but first we require it be real. Until then, the fantastic is for entertainment, not everyday life.
Posts
