Interpreting and presenting the data correctly is the only way to unravel the Buttergate scandal
A recent email I received encouraged me to step back for a moment and consider if the last column I wrote about butter had errors in it. I decided it didn’t: it just expressed an opposing point of view.
This brought to mind considerations I have offered in previous columns about the difficulty in deciding which information sources to trust. For example, for years I studied pesticides and their impacts on people and wildlife and their legacy in the environment. I constantly faced challenges from focus groups (either pro or con pesticides) that used carefully selected data to their advantage to ‘prove’ that pesticides were good or bad, depending on their agenda.
Data is what you make it, and cunningly analyzing it to fit your argument and crafting your words can definitely influence the ‘facts’ in significant ways!
One of the challenges in trying to assess the validity of a source is to determine if their protocol is scientifically supportable and if they had any bias in their techniques and reporting.
One way to confirm this is to do a peer review, which is a mechanism whereby experts in the field study the report and determine if it is both scientifically and statistically valid and if the conclusions derived were reasonable and, most importantly, reproducible using normal scientific techniques.
The authors of the Expert Working Group on Feed Supplementation in Dairy report, cited by the email respondent, state that “the information provided in this report is strictly descriptive and cannot be submitted to a proper statistical analysis due to the nature of the sampling procedure” (e.g. only 40 butter sample).
This, in itself, doesn’t negate their conclusions but does indicate that more study is needed. Literature reviews, the primary tool they used in reaching their conclusions, have immense value as each was (hopefully) peer-reviewed before this analysis.
How you present your conclusion is equally important. In this case, they concluded that “there is no data to confirm that there has been a change in the consistency of butter over time”. These are interesting words as they don’t stipulate that the butter hasn’t changed – they just say that the available data doesn’t prove that.
They go on to say, “As a result of this lack of data, it is not possible to test for a causal relationship and therefore draw conclusions, on the link between the use of palm-derived supplements … and the consistency of butter …”.
This actually ignores the original question as to whether butter has changed and replaces it with a new question – if it did change, what caused that change? It’s paramount that, when conducting and reporting on studies, the fundamental questions behind the study are not lost.
One thing that is rarely included in these studies is the genesis of the question. Is awareness a factor that needs to be considered? For example, the authors state that they consulted with processors who determined that they had not made changes to the butter manufacturing processes in recent times but acknowledge that multiple processing factors can directly influence the characteristics of the butter.
But what does this mean?
Well, a home baker might notice that the butter is harder but a bread manufacturer who is busy making millions of loaves of bread might not. So the perspective of the original reporter should at least be explored.
Setting science aside for a moment, one challenge I face as a columnist is editorial limits. My columns are supposed to be about 700 words. But how do I broach a subject as complicated as this and try to tell the reader what is important? Admittedly, sometimes not enough information can be presented to support the argument.
However, the reader is always encouraged to look for other sources to try to find out more. Often a newspaper article is simply a stimulus to further research on the part of the reader.
Thank you to my email writer for expressing her thoughts; this makes us all better and more confident. Meanwhile, my butter is still hard!
Geoff Carpentier is a published author, expedition guide and environmental consultant.
For interview requests, click here.
The opinions expressed by our columnists and contributors are theirs alone and do not inherently or expressly reflect the views of our publication.
© Troy Media
Troy Media is an editorial content provider to media outlets and its own hosted community news outlets across Canada.
