💾 Archived View for dioskouroi.xyz › thread › 24996927 captured on 2020-11-07 at 00:54:03. Gemini links have been rewritten to link to archived content
-=-=-=-=-=-=-
________________________________________________________________________________
Andrew Gelman posted a good response to this:
https://statmodeling.stat.columbia.edu/2020/09/07/day-scienc...
(I found this while trying to find a definition of "day science" and "night science", which the original article kept mentioning but didn't define. [2] is another interesting post about those concepts, which sticks to seeing them as separate activities.)
[2]:
https://www.faraday.cam.ac.uk/churches/church-resources/post...
“Not all who wander are lost.”
Great quote. We might get caught on the idea that exploring is something to do with well defined purpose and focus. But there’s a lot of direct and indirect value to be gained by wandering.
I agree, that is exactly the reason is why I am enjoying so much my PhD.
But it's not so easy when the next quarterly report is all that matters.
Well, this one might be...
It is very often the case that a hypothesis is suggested by our partial knowledge, and in those cases, it would seem paradoxical to put it aside and set off on a random search for whatever.
Many of the problems outlined here seem to arise from attempting to force a hypothesis where there are no clear candidates.
We've seen so many cases where research is actually invalidated because the researcher made a hypothesis after collecting their data (most often discarding the hypothesis they had before collecting their data). This process occurs with enough frequency that it's even earned the moniker "p-hacking".
So I'd propose the opposite - and this is in line with people who are way smarter than I am - publish your hypothesis before you start your study. I'd recommend the "Everything Hertz" podcast for a lot of cogent discussion around both study quality and open access.
As an aside, I think that we should be rewarding those who publish the results of their "broken" research. There is value in knowing a negative result - something we use frequently in engineering (one reason we prototype everything before sending it off to production). Right now, there's almost an institutional shame in reporting "no results". In theory, there should be a bunch of really short papers "no results" published for every break-through study.
You know what is also a liability? N=33, p=0.034!
I think in this context - a classroom experiment - it's fine. The experiment was cleanly designed, there's little room for fancy p-hacking. The differences satisfy the Interocular Trauma Test. The theory (to guide our prior) is persuasive.
When I'm being polite, I have been calling insistence on doing Day Science, when the Night Science hasn't been done, Cargo Cult Science.
At other times, it's just
pig-headed BS.