Another look at generative AI
*** A quick note: I’m hosting a second webinar in a couple of weeks, this time with Sphinx Bio’s Nicholas Larus-Stone. We’re calling it “Unlocking Biotech Data: How Sphinx leverages AI to help biotechs move faster and make better use of existing data.” It’ll be on Thursday November 21st at 2pm EST/11am PST and you can sign up here. ****
So, back in May, I wrote a post with my thoughts about the future of AI, and particularly Large Language Models (LLMs). Some readers seemed to think my take was kind of pessimistic, and given that I focused on the fundamental limits to the technology, they were probably right. But as I’ve been spending more time on the Biotech Reference Stack, I’ve also been thinking about the places where generative AI and LLMs could be used to make existing workflows better, as well as ways in which this new technology could completely change the Stack. So over the next few weeks, I want to explore what some of these look like. But first, this week I wanted to write about the framework I plan to use to evaluate them.
First off, I think it’s worth distinguishing three kinds of models that folks often mean when they say Generative AI:
Natural language models like ChatGPT that are trained entirely on text and can be used to generate text-based outputs. You can use carefully engineered prompts to make them do biology-specific things, but under the hood it’s the basic language model.
Biology foundation models that use the same kind of model architecture as these natural language models but are trained from the start on biological data/information so that they can be used for biology-specific tasks.
More narrowly scoped generative models for specific tasks like designing small molecules, proteins, DNA, etc. These models pre-date GPT and had already been used for a number of applications by the time GPT came out.
Most of the recent excitement has been about the first two kinds of models, and that’s what I plan to focus on in the next few posts.
The framework I want to use is based on trying to understand what these models do better than either existing digital tools or a human. And I think that’s essentially two things:
LLMs organize orders of magnitude more information than a human could remember, in a form that’s much more flexible than any previous digital tool could handle.
LLMs can use that information to generate artifacts, either in free text or in a structured form. They use the information both as the content of these artifacts and to interpret user instructions.
This is in contrast to some important limitations of LLMs:
LLMs often don’t “understand” the prompts and the information as well as you might expect based on the confidence, detail and eloquence of their answers.
Sometimes their answers are factually incorrect (They make stuff up.) despite that same level of confidence, detail and eloquence.
Putting this all together, here’s how I would summarize what I’ve seen: LLMs are much better than the average person, but not as good as an expert, at putting information into a requested form, but should only be used in situations where that information can be verified.
So the next question we need to answer is: What are the places in the biotech stack where we can benefit from putting information into a requested form that can be verified before it’s used. In some sense, the answer is probably “everywhere,” but that isn’t useful. So in the next few posts, I want to get more specific. Stay tuned!
Thanks for reading this week’s Scaling Biotech! I really appreciate your continued support, and I read every comment and reply.
As a reminder, I offer several services to help connect biotech teams with tools, practices and expertise to make their organizations more data driven.
The Biotech Reference Stack is a website designed to help biotech data teams identify the tools they need and figure out how to put them together.
For help navigating the Reference Stack, sign up for a free consultation call to clarify a problem you're facing and identify the best options to evaluate.
Or if you’re building software that makes biotech more data driven, find out how to add your app to the Reference Stack.