Statistical ecology: everything we know isn't wrong
Mark Brewer, Biomathematics and Statistics Scotland
Debates over which statistical methods to use and which paradigm to follow have become increasingly polarised in ecology over recent years. But the lack of agreement may be due, at least in part, to
fundamental misunderstandings or even forgetting what we already know. The controversy surrounding AIC and the recommendations found in Burnham and Anderson (2002) for model selection is a case in point; simply asking "What are we modelling for?" can help resolve the situation, taking inspiration from Shmueli (2010). The distinction between AIC and p-values is not as clear as some would have us believe, and so it is important to clarify the implications for model selection - whether prediction
or explanation is our goal.
As a second example, concern has been expressed over reproducibility of scientific findings – a technical examination is found in Johnson (2013) - but discussion has focussed on significance rather than power or on Bayesian calculations; some authors have even appeared to bend over backwards to avoid mentioning "Bayes" even when, in effect, making their point using Bayes' Theorem.
At the recent ISEC conference, Ben Bolker discussed when novel statistical methods should be used and when not; a related but inverse question is: why am I often criticised for not using "fancy
statistical methods"? New methodology should augment, but not necessarily supplant older methods; my being told (a) to use mixed models with only three (or no!) groups, or (b) use GAMs for a set of binary explanatory variables suggests that some have forgotten what we already know. Appropriate methods are not "old school" or "outdated" (both actual criticisms received) simply because they weren't developed in the last ten years.
A brief discussion on the above will also consider the fledgling "data science" and where statistics as a discipline fits in (or should fit in).