Have you ever seen one of those optical illusion paintings? There’s a famous one where some people see an old woman, while others swear it’s a young woman. Or maybe you’ve pondered over an M.C Escher print where the stairs seem to go on infinitely. It’s said that perception is reality. But things aren’t always what they seem to be…or are they? Fundamentally, art and science are both about observation and interpretation. But while it’s harmless to be on completely opposite ends interpreting a painting, it can be life-threatening to have the same bias in opinion in a scientific study of a prescription drug. And unfortunately misleading interpretation of data from randomized scientific trials is common.
Let’s say you’re reading an online article and it references a scientific study. You are in fact, reading the author’s interpretation of the research. Is the author a scientific professional? Does the author have the correct credentials and experience to break down and analyze a full scientific paper? Or are they simply reading the abstract as many physicians often admit to doing? A complete research paper in APA style that is reporting on experimental research will typically contain a Title page, Abstract, Introduction, Methods, Results, Discussion, and References sections. The Abstract is a concise summary of the experiment or research. But it’s very short, usually under 200 words. How much can you glean from 200 words? And finally, do you trust the author to interpret the data correctly, or in the way the scientific research was designed to answer the question.
Wait, did I just say the research was designed to answer a question? YES! “Design is the deliberately planned ‘arrangement of conditions for analysis and collection of data in a manner that aims to combine relevance to the research purpose’.”
Let’s look at this study for example: Click Here
From the first several words, it is clear to me that you should be wary of this research study. There were 139 people involved in this study. There is not enough statistical power to make a valid point, or to be a powerful study. By example, pharmaceutical companies conducting research on statin drugs enroll 10,000 people in their study. If you have a small study, you need an overwhelming amount of difference to prove your point. Therefore, by equipping such a small sample size, I’m thinking the authors of this particular study wanted to discredit information, i.e. “Background: The long-term efficacy and safety of time-restricted eating for weight loss are not clear.”
Even huge case studies of thousands of people, with double-blind methods, you have to read between the lines. Was the study designed with cherry-picked candidates, with specific chronic diseases, or characteristics that would make him attractive towards the “question” they want to answer?
The hazards of scientific studies are that it is agenda driven content. And its agenda is front and center in the headline and abstract of a study. Very few people read the full study. If you had read the full study you would have discovered that study results show that in one year the time restricted patient lost about 20 pounds, while the food restricted patients lost about 14 pounds. Clearly, the time restricted patients (intermittent fasting) lost more weight than the calorie restricted patients (starvation).
So I urge you the next time you hear someone reference a study, understand that you may be misinformed. You may be at the end of a long game of telephone. At one end, a scientist was designing a clinical study to answer a very specific question, an author interpreted real data information in a way that was most likely distorted, then eventually passed down to you.
References:
Selltiz C.S., Wrightsman L.S. and Cook S.W. 1981 Research Methods in Social Relations,
Holt, Rinehart & Winston, London, quoted in Jankowicz, A.D., Business Research Methods, Thomson Learning, p.190.)
