BPA is bad for you, we’ve been told.  It was banned from baby bottles in the US and Canada.  Then came an even bigger scare – a study that showed that a mother eating food packaged in BPA could cause lower thyroid levels in boys!

The something funny happened.  Concerned parent Sarah points out that the government of Canada released a study that says that consuming products packaged in BPA does not cause a health risk.

Sarah asks whom we would trust – the researcher who is dependent on grant money or the government of Canada.

I’ll take this even a step further and point out the existence of what is known as “publication bias”.  The studies that appear in journals – and thus are far more likely to end up in the mainstream media or be noticed by legislators – are more likely to be the results that are shocking or unexpected.

For example, let’s say that we’re studying whether or not drinking a quart of orange juice every day can prevent the growth of tumors.  Ninety nine studies do not show any link, but one study appears to show that drinking the orange juice does indeed prevent the tumors from growing.  Guess which one is going to end up in a scientific journal?  Certainly not one of the boring ones – the one with the surprising result is going to be published.

A basic concept with science is that an experiment must be repeatable by other scientists before the results can be considered valid.  This is to prevent unrelated factors (such as human error) from creating the result. 

The same concept applies to scientific studies.  When one study appears to shop a certain result, it’s very difficult to know of this is causation or merely correlation.  It’s possible that the result could be attributed to other factors, or even to selection bias.  In our hypothetical orange juice – cancer study, perhaps the control group and experimental group were located in different geographical areas.  The OJ drinkers received treatment at one hospital whereas the non-drinkers received treatment at another.  It’s possible that the OJ doctors were simply more effective in treating cancer, and that the orange juice really had no effect.

In a nutshell, exercise caution when you read about scientific studies.  Check to see if there have been other studies on the topic, and check to see what the results of those studies were.  Look at the entire body of work on the topic, in other words, and not just one study.  We’re in an age where we research topics with just a click of the mouse.  Put that power to work for you and become more educated on the topics you care about!

Enhanced by Zemanta