Tuesday, February 18, 2014

Did I miss something?

Jeff Leeks at Simply Statistics has a post, "On the scalability of statistical procedures: why the p-value bashers just don't get it.", following last week's Nature op-ed by Regina Nuzzo.

I find the initial tone (especially the title) of Jeff Leeks's post to be confusing. My reading of Regina Nuzzo's article wasn't that p-values were "bad", simply misused. And, in fact, this seems to be the case that Jeff makes as his post continues. In fact, he has a great section of the pros and cons of several alternatives to the p-value.

I have to admit that I'm new to this whole debate. But it seems to me that Regina and Jeff differ only in that they affiliate with different sides. Both argue that misapplied p-values can be deceiving. Both argue that there is no one-size-fits-all solution.

I am confused.

That said, I'll re-quote Steven Goodman. (The title of my first post on this subject quoted Goodman's quote in Regina Nuzzo's article, so meta.)

"The numbers are where the scientific discussion should start, not end."

It seems banal to say this (again and again), but... Don't be dogmatic; there is no final answer and there never will be. You have to think critically about your analysis and interpret within a wider context.

And while Regina Nuzzo and Jeff Leeks approached the topic from different angles and raised different points, I got the same message from both: Don't be dogmatic.

** Edit 2/19/2014 **
After a brief Twitter conversation, Jeff Leeks and Regina Nuzzo have confirmed that they do agree about a lot. This is a relief (for me and probably for all of us) because both pieces raise many good points about how determining statistical "significance" is not straightforward.

No comments:

Post a Comment