It's 10pm. Do you know where your experiments are?
***A quick plug: After you’re done reading this, check out the latest episode of the Planet Bio Podcast where I joined to talk about metadata, communicating across teams, writing newsletters and much much more.***
As we continue to work our way backwards through the use cases that biotech software needs to support, we’re about to leave the realm of the data team and move squarely into the wet lab. That’s right, this week we’re tracking experiments. In particular, we’ll cover why tracking experiments is important, why its hard, and how you can get started anyway. (Also, a quick reference for any millennial readers confused about this week’s title.)
First, let me clarify what I mean by tracking experiments. Every scientist in the lab is tracking the experiments that they’re doing, one way or another. They wouldn’t be able to do their work otherwise. But what I’m talking about today is centralized tracking - a single system, or a set of processes, that puts all the information about the status and design of past, ongoing and planned experiments in one place that can be reviewed by teams members outside the lab. This is a large component of what’s sometimes called a digital twin of the lab.
There are two main reasons that biotechs typically want to track experiments. The first is that leadership wants to know what’s happening in the lab so they can plan accordingly - estimate budgets, report to the board, decide where to invest, and identify where they need to meddle get involved. However, this often ends up implicitly in the “nice to have” category, at least at startups, because it doesn’t address an urgent problem. It’s for long-term planning, which doesn’t happen much when you’re constantly putting out fires. The exception I’ve seen is startups whose core strategy centers around building a data asset, so the status of experiments is their most important metric.
The second major reason to centrally track experiments is that it’s the best way to ensure data teams can do their job. The information that goes into the metadata that data teams need is generated throughout the experiment design process, as early as the informal brainstorming of an experiment. The better you’re able to capture this information in the moment, the more accurate it will be and the easier it will be for stakeholders (such as data scientists) to provide feedback on experiment design. Plus, if data teams know when data will become available, they can plan to be ready for it. This is much more of an urgent need than the first reason, but data teams often don’t feel empowered to address it, or don’t even think to ask.
Now, note the one group that neither of these reasons applies to: The bench scientists who are actually in a position to address the problem. For the most part, they only need to know where their own experiments are, which they can often do with post-it notes, or just in their heads. A consistent, central system isn’t going to help them. In fact, it’s going to make it harder because they’ll have to figure out how to translate the way they think about experiments into this centralized form, which probably won’t be flexible enough to capture everything they’re doing anyway.
Now, I’m not saying that bench scientists will flat out refuse if you ask them to start tracking their experiments. They’re reasonable people. They genuinely want to help their colleagues in the dry lab. But it’s no small thing that you’re asking. It will slow them down. If it’s just a favor to the data team, they’ll skip it when deadlines are tight. Those instances will become more common over time. As the data team sees the system becoming inaccurate and incomplete, they’ll stop trusting it. There goes that.
But all is not lost. I think any biotech can get to an experiment tracking system that works if they focus on two key components:
Clearly define the scope of experiment tracking.
In the story above, the process degenerated because bench scientists decided to stop tracking urgent experiments, then the definition of urgent slowly slipped until it became meaningless. If you instead set a clear and reasonable boundary around the kinds of experiments that should be tracked, you can stop it from slipping, and even push it in the other direction. Start with simple, high-throughput experiments where tracking is easy. As the team gets used to that, start tracking more exploratory and more complex experiments.
Build a system that makes life easier for the bench scientists.
A lot of the things that bench scientists need to do as they move an experiment towards the lab are tedious and/or complex. If you can build a system that makes these tasks easier or does the work for them, they’ll actually want to use it. And if, as a side effect, this system tracks the status and design of the experiments they’re working on, that’s a win win. This strategy often goes hand in hand with lab automation. And like lab automation, it’s easier to implement for those later, more consistent, high-throughput experiments. So again, start tracking just those experiments. Once you’ve figured out what you can automate, start automating those aspects of other experiments. Expand the scope of both the tools and the tracking.
Now, obviously there’s a lot of details to work out for this strategy to work. It’s an incremental process, and sometimes requires some creativity. But I believe that every biotech startup that cares about their data can make this happen.
Scaling Biotech is brought to you by Merelogic - an independent consulting firm that helps early stage biotech startups get their data under control at every inflection point so they can focus on the science.