Start normalizing data driven biotech
*** A quick plug: I recently sat down for a Q&A session with the folks at Ganymede and they posted the interview on their blog. Go check it out (after you read this post.) ***
In last week’s post, I explored the question of whether biotech startups tend to be data driven and came to the conclusion that they kind of are and kind of aren't. Shortly afterwards, Nicholas Larus-Stone wrote a response on Twitter and LinkedIn in which he argued that no, they're actually not at all data driven. Like, not even close.
And so this wouldn’t be the first time I was a bit generous in my assessment. But what I found most interesting about Nicolas' response was that it explored a slightly different angle on how biotechs should be able to get value from data, different from my last post even though it still falls under the umbrella of "data driven". And since this is the theme I was planning to explore, I think it’s a great excuse to dive deeper.
Broadening the scope
So, here’s the (somewhat subtle) difference that I see: My post last week focused on improving the decisions that biotechs consciously think about today. I argued that shifting the thinking towards data over intuition/experience will lead to better outcomes for these decisions. What Nicholas argues (or at least my interpretation of it) is that being data driven means actually expanding the set of decisions that organizations make deliberately, as opposed to implicitly.
Many of these decisions are what I would call strategic, as opposed to operational. Nicholas mentions increasing hit rates and reducing turn around time. If you use pure intuition to make decisions about improving these numbers, the work will still get done. It just won’t get done as fast or as effectively.
As Nicholas notes, Tech companies actively track operational metrics (or KPIs) and make sure everyone knows what they are. Very few, if any, biotechs do this for hit rates, experiment turnaround times, etc. But for cash strapped biotech startups with with ever shortening runways, you would think improving these metrics would be a top priority.
So what gives?
I usually chalk this up to lack of time/bandwidth. When you’re building the tracks ahead of a speeding train, it’s hard to take a step back and start measuring how fast you’re building the tracks or how fast the train is going. But the more I come back to that, the more it feels like an excuse. I mean, if tech startups can do it, why can’t we?
Then there’s the fact that the work biotechs do - especially early stage startups - is highly heterogeneous. Tracking and comparing how long users spend on your website makes sense given that all users have roughly the same experience. Tracking hit rates and turnaround times on experiments when every experiment uses a different assay and a different protocol seems a bit more dubious. But come on - we’re a smart group of people. I’m sure we can come up with metrics that make sense. So again, this increasingly feels like an excuse.
What I think it ultimately comes down to is that the ROI calculations are difficult to do. Because the drug development cycle is so long, we don’t have examples of startups that invested in being data driven early on, then got a big return.
Without those examples (aka data), decision makers have to fall back on intuition. And while those of you who read this newsletter may see a huge potential return, many decisions makers will (reasonably) mark it down until they see evidence.
On the other hand, they often over-estimate the costs. Data infrastructure was an order of magnitude more expensive to build just a few years ago. And today, working with someone who doesn’t know what they’re doing (even with the best intentions) can quickly blow up a any development budget. So their experience tells them to expect a huge investment.
But ultimately, maybe it’s just culture - Since decision makers don’t think of this as something that biotechs do, the safest option is to not do it either.
So how do we change this?
I think we can make progress on both lowering the expected investment and raising the expected return. Biotech software is getting better every day and biotechs are finding more effective ways to use it. We need to be pragmatic about how we use the resources we’re given while building solutions that lower investment costs even more. Then we need to get better at communicating the potential return that can come to the startups that make those investments.
Communicating these things is hard - I’ve been practicing on this newsletter for quite a few years now, and I still have a long way to go. But I believe we all have a duty to our employers and to the industry, to do the best we can. So if you’ll work at it with me, I’ll keep writing writing about it here.