On AI and Information Theory (aka the obligatory LLMs post)
So far, I’ve deliberately avoided writing about LLMs on this newsletter, in favor of the more mundane aspects of biotech data. But I’ve been toying with an idea about how to think about them and I couldn’t resist the urge to share it. So I’ll return to the series I was working on next week. This week it’s all about AI hype.
As we all know, there’s nothing new under the sun. So when new technology comes around, you can usually come up with a model for how it’s likely to evolve based on analogy with past technology. In fact, you can often find many different analogies, but this week I want to explore one that I’ve been thinking about, which is to compare LLMs to programming languages.
The reason I’ve keep coming back to this one is that you can treat both LLMs and programming languages as ways of communicating what you want to a computer. With a programming language, you’re communicating an algorithm that you want it to carry out. With an LLM you’re communicating text that you want it to write.
The reason I like this analogy is that by framing things in terms of communication, you can use ideas from the field of information theory. In particular, there’s this idea that there are fundamental limits to the amount of information one can communicate across a given channel. If you know something about the distribution of types of messages that are going to be communicated then you can make it easier to communicate the more common types of messages, but only by making less common types of messages harder to communicate.
This is how file compression works: It translates the contents of the files into a “language” that makes common types of file contents very small. But information theory tells us that at least some of the time, these algorithms must make some files at least slightly bigger. Usually there’ll be some kind of trap door - a way of telling the de-compressor that the contents aren’t compressed. The extra size comes from putting in that message. But the kinds of contents that it makes smaller are carefully selected so that in practice you almost never run into a file that it makes bigger.
With programming languages, you have low-level languages like assembly in which you’re communicating directly how the computer “thinks about” the algorithm. This is the equivalent of no compression - you don’t optimize for any type of algorithm over any other.
Then you have low-level languages like C that implement common patterns like organizing instructions into functions that make things closer to how humans think. If you’re writing an algorithm that is organized in this way, it takes many times fewer lines of code to write it in a language like C. The trade off is that to write an algorithm that isn’t organized into functions - for example, because it would be more efficient that way - it’s much harder. Usually, you would use a “trap door” that lets you write directly in assembly and insert that into the C code. The code around this trap door makes it (slightly) harder to do this.
As you get into even more human-readable languages like Python and R, functional languages, etc. it gets easier to write the most common types of algorithms because you’re writing them in a human-friendly form. But again the tradeoff is that it’s much harder to write highly efficient code. Again there’s a “trap door” where you link in code from another language when you need a really efficient algorithm.
But most of the time you don’t need to do that because computers are so fast that for most things that efficiency doesn’t matter as much as your ability to communicate what you want. In other words, the bottleneck is no longer the computer’s ability to carry out the algorithm - it’s our ability to describe what we want to the computer. These new programming languages are optimized for the kinds of algorithms we’ll want to communicate in this context.
And even with these new, human-centered programming languages, while it’s many times easier to describe these algorithm, it’s still really hard. Partially that’s because of the information theoretic limits on the channel between our brains and the computer. But it’s also because as humans, we often have a hard time understanding what we want well enough to even try communicating it.
OK, so, what does this have to do with LLMs?
Think of an LLM as a way of communicating to a computer what you want it to create. It’s basically a fancy keyboard: With keyboard, it you want the computer to create an essay about Shakespeare, you type it word for word. With an LLM, you type “Write me an essay about Shakespeare.”
In other words, LLMs create a language that is optimized for communicating certain ideas to the computer - ideas that the system has picked out as being the ones that we will want to communicate most often. And it’s picked these ideas by reading all the text on the internet and deciding that these ideas that have been communicated in the past are probably the ones that are going to be transmitted most often in the future.
That’s great if you want the computer to create something like that. It also has the side benefit that LLMs seem to know information that you don’t. But information theory tell us that if LLMs make it easier to communicate these kinds of ideas to the computer, then by the fundamental, mathematical laws of information theory, they must make it harder to communicate other kinds ideas. That may be via a “trap door” i.e. if you can get increasingly specific about what essay you want the LLM to write until you’re basically writing it word for word. But one way or another, there’s a tradeoff.
More importantly suggests that there this is a fundamental limit that will always be there, no matter how many more GPUs we throw at the problem. The bottleneck is our ability, as humans, to get information out of our heads. (And for the record, I don’t think implanting a chip will fix this, but I’m definitely not going there today.) If we can understand how that limit will play out, we should be able to understand something about the future limitations of LLMs.
So, that’s my hot take on LLMs. I don’t (yet) have more concrete conclusions about what this means for the future of humanity. But as we see this all play out, that’s the analogy I seem to keep coming back to. You can draw your own conclusions.