Last Friday a rare event occurred at our golf course. My foursome all sank long putts for birdies on the second hole. What a great feeling and I could go on and on in detail but I won’t. Anyway we get back to the clubhouse eager to relate our story and all we hear is a collective, tepid “That’s interesting.” What is up with that?
My theory is that golfers understand that rare events happen often. They can’t predict which ones but they know they happen – a line drive tee shot hits the flagpole and drops in, a drive headed out-of-bounds hits a tree and lands in the middle of the fairway, or a ball hits a sprinkler head and bounds over the green. They have seen it all. Rare events happen. Andrew Gelman points us to an article on Vox that discusses outcome switching in medical research. The idea is, you design a study to test if a drug works. It doesn’t according to the statistical measures you set up, so you look for other outcomes. Maybe it worked on a sub-population or patients lost weight or something and then you publish results for that outcome instead. You switched outcomes after the fact. Of course you are going to find something. Rare events happen. Golfers know that and don’t get too excited. Apparently some medical researchers don’t and do.