At GoDaddy, the art and science of experimentation is a vital thread in the fabric of our product development and marketing. Our internal experimentation platform, Hivemind, is a testament to our commitment to cultivating a company-wide experimentation culture. Conceived in 2020, Hivemind phased out numerous external experimentation platforms in favor of a single standardized, in-house solution for configuring and analyzing controlled experiments.

When we first introduced Hivemind, it functioned as a minimum viable product, lacking the safeguards to prevent poor experimental practices. The basic setup and stats engine led to bad practices. These included p-hacking, ignoring the need for adequately powered experiments, and mixing up Bayesian with frequentist methods (among other issues).

Extensive training on experiment best practices wasn't an ideal solution to get experimenters company-wide up to speed quickly. Instead, we needed a mix of basic training and platform enhancements to help guide and incentivize experimenters to design optimal experiments. In this blog post, we'll delve into several key improvements we've made to Hivemind to shape the experimentation culture at GoDaddy.