Tools of Thought: The Whole Enchilada
Do you like enchiladas? I do. I like the combination. It’s much better than just the corn tortilla. It’s better than just eating sauce by itself. And, although I like the fillings, even the fillings are better to me when they are part of the whole enchilada. Sometimes, things only “work” when all the ingredients are present.
Do you think it’s safe to drink and drive? What if I told you that 13% of the drivers involved in fatal traffic accidents had been drinking? Would that convince you that drinking and driving don’t mix?
It shouldn’t. Not by itself. Not only does that statistic, in and of itself, not show causality, it does not even show correlation!
What else would you need in order to show that there was at least a correlation between drinking and driving? You would need to know what proportion of people who were not involved in fatal traffic accidents had been drinking. What if that percentage were 50%? What if it were only 1%?
Let’s take another example. What is the cause of your back pain? Imagine that you went to the doctor and that you had an MRI and it showed that you had a bulging disc or some other back abnormality. Your doctor tells you that you need surgery to fix the bulging disc and that this will get rid of your pain. In fact, the doctor says that 50% of the people who have back pain have a bulging disc or other spine anomaly. What can you conclude from this about the relationship between spine abnormalities and back pain? Nothing. Not yet.
You cannot logically conclude anything from this statistic in and of itself. You have to know what percentage of people who do not have back pain also have a spinal abnormality. What if it’s exactly 50%?
What if it’s only 1%? If it’s only 1%, it would seem that having a spine anomaly might be a good indicator (though not necessarily the cause) of potential back pain.
What if 90% of the people without back pain have bulging discs? Then, it might seem that having a spine anomaly is a good thing!
Once again, before we can even establish that there is a correlation or association between back pain and spine abnormalities, we must know four numbers:
How many people have back pain AND a spine abnormality.
How many people have back pain and NO spine abnormality.
How many people have NO back pain AND a spine abnormality.
How many people have NO back pain AND NO spine abnormality.
Then, we can test these numbers of find out whether there’s a statistical association between back pain and spine abnormality. Even if we find that there is a statistical association between the two, it would not prove that the abnormality causes the back pain. For example, it might be that some other factor is responsible for both. For instance, it might be that people who regularly run, swim, walk, or otherwise exercise have far less back pain and far fewer spine abnormalities. Conversely, it might be that people who experience back pain exercise far less and this lack of exercise is what causes more spinal abnormalities.
Or, it might be that older people have both more spine abnormalities and more back pain. It might really be age that causes an increase in both.
But be careful. If you take a snapshot in time, it might be true that today’s old people have more spine abnormalities and more back pain than today’s young people. But does that prove that age itself is to blame? No. It might be that people who are old today were exposed to more pollutants than younger people. Or, it might be that people who are young today are exposed to many more artificial preservatives and that one of these actually helps prevent spine abnormalities and back pain.
The point is not that science is impossible. It isn’t. But neither is it trivial.
Most people who are scientists were good in science and math and they enjoyed it. While most people were struggling through science, some people, like me, loved it. In the first grade, we had a “science text book.” I found it fascinating. But I didn’t stop there. I went to our local public library on Arlington Street, and discovered that they had our textbook on the shelves — but right next to it, they also had the science text books for grades two through six. I read all of them in order and immediately the day I discovered them.
My grandfather subscribed to Sky and Telescope, Scientific American, and The Natural History Magazine. I began reading these at an early age. If I didn’t understand something, Grandpa would explain it to me. By the third grade, I was very familiar with much of the simple “nomenclature” of anatomy, paleontology, astronomy, biology. I read 2-4 science books every week during my childhood. Mr. Wizard was one of my favorite TV shows. He was the “Bill Nye” of the 1950’s. I got a chemistry set and joined another “club” that sent a different science experiment every month.
Most, but not all, people who are scientists today also loved science as kids. And, in high school, they — like me — continued to study and read about science. And most of them took further science and math courses in college. And most of the people who are “scientists” then went to graduate school and studied science for another four or more years. Graduate school for science is the intense study of science. The grad student doesn’t take physical education or literature courses or “Western Civilization” courses. If they are in a science Ph.D. program, that is what they are studying. Most then do some sort of “Post-Doc” meaning that they get a job, but continue to learn on the job under the tutelage of someone already expert in the field.
One of the things that I find fascinating about current American society is that a huge proportion of people no longer trust what scientists say. Most folks would never go to a forest ranger to have their teeth fixed. They would never go to a professional golfer to have their gall bladder removed. They would not rely on a car mechanic for open heart surgery. Yet, when it comes to climate change or the safety of vaccinations, they are willing to rely on very rich people with a vested interest to tell them what to think over a body of experts who have devoted their lives to learning about a particular field. And the “arguments” of those with vested interests are generally like the opening example. They do not tell the public all the facts or observations needed to make rational decisions.
Let’s now return to our alcohol example. When I was an undergraduate, one of my part-time jobs was as a projectionist. One week I worked all week during spring break for a week-long conference at Case-Western Reserve on the effects of alcohol on driving. It turns out that (as you might expect), there is an association between drinking and being in an accident. People who are scientists not only collected data on this but also studied driving and performance under a wide variety of circumstances.
This is also crucial to understand. Scientist do not typically just go work in a lab by themselves and come to a conclusion. They present and publish their work so that they get feedback from other experts. Of course, like other human beings, they would prefer that their work be “perfect” but at the same time they welcome feedback from these other experts. They don’t just keep asserting over and over and ever more loudly that they are “right.” A person who insists like that will soon find themselves shunned by the scientific community.
So these scientists studying the impact of alcohol on driving came together from all over the world in order to get feedback from each other. Each became a teacher and a student. People who are done with school and don’t want to be students or teachers any more should not really become scientists.
I am very good at finding other people’s typos. I am horrible at finding my own. That’s why I ask other people to proofread my work.
It’s the same with science. Despite having worked for years as a scientist, individuals do still make mistakes in logic. That’s why the consensus of a large group of scientists means more than the conclusion of one scientist who has come to a conclusion.
As for alcohol, some scientists at that conference at Case-Western measured the impact of alcohol on “reaction time” — how quickly a person reacted to a simple stimulus. They measured, for instance, how long it took people to push a button after a light came on. In more “realistic” tests, people sat in a driving simulator and scientists measured how quickly they put on the brakes when a red light came on. In those studies, it turned out that simple reaction time didn’t really become noticeably impacted until the person had three stiff drinks on an empty stomach!
Someone from the insurance industry who had studied accidents in the real world, however, found that even a half drink significantly increased the chances of being the driver in an accident! That’s quite a discrepancy!
But scientists do not throw up their hands at this point and say, “Oh, my! Science is hard! Let’s give up and go watch TV.” No. Scientists find this an interesting problem. Why is it that it takes three drinks to find a significant impairment in reaction time, but in the real world even having half a drink causes a significant increase in accidents? So, they begin to tease this apparent contradiction apart with more experiments.
Does it have to do with particular people? People who agree to participate in lab experiments are not a random group of people from the whole population. They tend to be younger, healthier, and better educated. Could that be part of the apparent discrepancy?
What about mood and motivation? When you come into a laboratory and people measure how quickly you can respond, you might tend to be in a “serious” mood and typically, in our competitive society, you are motivated to be as fast as possible. When you are out partying, you may be in a much different mood. Perhaps your motivation, especially if you are a teen-ager, is to “impress friends” that you are not “chicken.” So, maybe 1/2 drink might induce you to do stupid things that you wouldn’t do when you are sober. At the same time, if you’ve been drinking, chances are that your fellow passengers may have also been drinking so they may cheer you on in your choice to pass on a curve or engage in a high speed race or chase on the highway. Even if they don’t cheer you on, having friends there talking, laughing, and singing may distract you from your main task which is to keep everyone safe. Indeed, it does turn out that while simple reaction time takes three drinks to show a significant slowing, your ability to switch between tasks and to control your attention begins to suffer immediately under the influence of alcohol.
What about back surgery to cure your back abnormality? I am not an expert on backs. Here’s a nice summary of things to think about before having back surgery.
And I highly recommend the books by John Sarno. Reading this book completely cured my sciatica. Naturally, that doesn’t prove it will work for you, but you may want to try a $10 book before you go under the knife for $10,000, $100,000 or more!
Don’t drink and drive.
Science works. But it’s not trivial. Experts know more than you do. But the knowledge isn’t inaccessible. It’s not some secret hidden knowledge that scientists want to keep from you. You can become an expert in some field. But it will take time, effort, and an open mind. Or, you can take the word of experts. Or, you can decide that you’d really be okay with a forest ranger fixing your teeth or a car mechanic doing your open heart surgery. Or, you can decide to rely on people funded by fossil fuel companies to tell you whether climate change is real.
Scientists mainly work together to seek the truth. Their “culture” is not to lie, cheat and steal in order to get rich. They get their kicks in other ways.
If someone tries to imply a causal narrative about A causing X, you have to see the number of cases in FOUR categories before you even have evidence of an association. If they only tell you how many cases are in A & X, they are trying to convince you of something, not in having you see the truth. You need to find out how many:
A & X
A & ~X
~A & X
~A & ~X.
Even if A & X are statistically associated, it doesn’t prove causality. Proving causality depends on careful study using a variety of methods to converge on the truth.
If all your doctor tells you is that you have a back abnormality and you have pain, and that therefore you need back surgery, get a second opinion.
Of course, it’s a free country. So far. And, if you want to go to a Mexican restaurant, you’re free to order enchiladas and scrape all the sauce out and then scoop out all the ingredients and just eat the corn tortillas. You are missing out though. And if you then declare that Mexican food isn’t that great…? Well, you do the math. You really should have gone for the whole enchilada.
Introduction to A Pattern Language for Collaboration.
The Pros and Cons of Artificial Intelligence.
The Myths of the Veritas: Book One.
Pingback: Finding the Mustard | petersironwood
Being of a more creative bent, nonetheless I have the utmost respect for the endlessly broad field of natural law and process you call by the umbrella term ‘science’. I am myself a back surgery statistic and had the other statistics presented to me, as any responsible surgeon would. However, I’m also a great fan of empirical and experiential in addition to strictly linear data, and felt that the only consideration meaningful to me at the time was ‘would the procedure reduce my pain?’ If 50% of people experienced a reduction, irrespective of causation, then I too had a 50% chance of losing my pain, which seemed like a good deal. Decisions like this are intensely personal and cannot be based solely on figures representing other people, I think. You probably find my logic feeble and woolly, but it works for me. Perhaps I would find your creative output similarly wanting… 🙂
I totally agree that if something works, why not (provided the side effects aren’t too great). John Sarno actually talks about this too. I recall reading one study that showed the placebo effect is stronger if the cost of the placebo is higher! Surgery is a very “heavy” placebo.
Pingback: The Door without a Key | petersironwood