Summer 2024 Thoughts on AI

Recently, preparing for an event, I started pulling together some of my thoughts on the current state of AI. Considering I haven't posted in a while, I figured sharing them could make for a good post. I would love to hear any thoughts this post prompts.

Short-Term

Generative AI is meant to be more of a reasoning engine than a retrieval engine. People open Generative AI applications and see a text box, then do what Google has trained them to do over the past two decades: search. Generative AI is fine at search, mainly when using RAG (retrieval-augmented generation) on the backend^[1], but its breakthrough use is reasoning. You can provide a model with a framework for thinking and input. The model can then use that framework to reason through the document and come to conclusions. For example, you don’t want the model to show you every hotel option in a city. You want the model to know that you are a Marriott person, prefer to be within walking distance of at least a few restaurants and public transit, and don’t mind staying in a worse hotel if it means a better location^[2]. Ultimately, these models aren't "thinking" since they are still just predicting the next word but show some emergent "reasoning" capabilities that shouldn’t be ignored.

The AI user interface will continue to move toward the end user. At first, I was going to predict the end of chatbots, but I don't think they will completely go away. A good version of Siri could be game-changing, and people seem to love chatbot friends. So, I will ask whether we will interact much with chat interfaces to do our work. Right now, prompting is all the rage, and there will be an interim time when people who know how to work with AI will have an advantage, but I think that will be replaced with custom solutions for vertical software. You won't spend days copying and pasting from your chat interface into your work software and back. The software will have custom LLMs with built-in prompts to help you execute small tasks, and you will be the human in the loop confirming everything is correct. There is lots of room for innovative interfaces, but instead of creating something novel, everyone is just focused on auto-complete...

Auto-complete for everything will become pervasive and make everything... lame? I have beat the dead horse enough in other posts, but defaults are critical. An easy and low-stakes way to integrate LLMs into software is to create forms of auto-complete or recommended next steps^[3]. These models are trained to insert what is most likely to be next, which in some cases could be helpful but is also the complete opposite of creativity. If you want to write anything new or different from what has come before, AI won't auto-complete your way there. Even using AI to help you get unstuck could be harmful since humans are prone to anchoring; once you hear one example, you get stuck on it instead of thinking more broadly. AI’s slightest input will leave a mark. We need to avoid the urge to let AI think for us. Instead, we could turn the tables and have the AI help guide us. I will caveat there are some excellent examples of using these models for auto-complete. Similar models have “tokenized” semiconductor chip designs for next-step recommendations and seem to unlock incremental improvement.

Creating intelligence benchmarks is incredibly difficult. The goalposts keep moving because something we believe we would identify as intelligence reaches new heights on the benchmark but still lacks some basic reasoning skills. How can you optimize for intelligence if it is never what you think it is?

Small data is more valuable than ever. Previously, if you had a small amount of data, there were limited ways to apply machine learning. With generative AI acting as a reasoning engine, you can pass your philosophy or process instead and let the model reason over the data. This is the difference between what I am calling implicit and explicit models. In implicit models, you provide millions of training examples, and then the model determines what is important by identifying what the positive examples have in common. In explicit training, you give the model a philosophy and allow it to reason over lots of data, applying it to each instance. You no longer need millions of data points. You need a clear, well-thought-out philosophy^[4].

Long-Term

Large Language Models (LLMs) will not take us to AGI (and the other options don't seem that close). LLMs are taught by predicting what they expect to happen next after being trained on what humans have written. This is inherently limiting since they imitate humans and thus could never surpass human intelligence. To get beyond human-level intelligence, we will need some form of long-term reinforcement learning (RL). RL provides an algorithm and a reward function, which defines what it gets rewarded for vs. punished for, then drops it into an environment where it can experiment with millions of iterations to find the best strategy. To steal the definition from Range, this works well in "kind" environments with clear boundaries, consequences are apparent, and similar challenges repeatedly occur. A good example is a board game, which is why AlphaGo could defeat grandmasters and even create novel strategies. I would argue that we live in a "wicked" environment where rules and boundaries are fuzzy at best, patterns may repeat but may not be obvious, and feedback is delayed, inaccurate, or both^[5]. On top of the difficulty of finding an environment appropriate for training these RL algorithms, they may be impossible to parallelize (use GPUs instead of CPUs), making them slow. LLMs have exploded because they can be trained in huge parallel batches where language processing was limited to reinforcement learning previously and thus had unrealistic compute requirements^[6].

Once we create AIs that are sufficiently "smart," we will have to trust them. If we develop intelligence that can surpass us, it will contribute to science and make connections that are too complex for us to comprehend^[7]. The world is full of incredibly complex relationships, but we like to simplify them. A few metrics drive what people believe about the economy, or neuroscientists claim there is a specific subset of cells that define location in our brain. Both of these are huge oversimplifications. Ultimately, we can only carry so much information in our brains and take it in at a relatively slow rate compared to a computer. What if location is driven by millions of different neurons interacting in a specific way instead of a single one? This version of intelligence, or better, could make discoveries with these complex interactions. The findings could take years to explain to a human if they could even understand. Right now, science is very focused on explanatory power. We don't want to know something is happening. We want to break it down and understand why. With a reasonably intelligent AI, we may never fully understand why. Then, we must decide whether to trust its conclusions and use its results.

The only thing I know for sure is that the world will only get more weird.

Let me know what you think.

1. Try Perplexity for this. If you haven't, it is very cool.
2. Apple Intelligence in this video seems the closest since it can access your data, but it is still not there.
3. Everyone’s product is AI-driven these days!
4. Generative AI could even help you create this philosophy. Using your small data, the model can help walk you through the process of documenting your philosophy using the examples.
5. In Range, David Epstein argues in a "kind" environment, you should start training very specifically early on, i.e., becoming a chess grandmaster or the best golfer in the world. In a "wicked" environment, you should instead gather varied experiences and specialize later in life, tennis or basketball, since they are more free-flowing and feedback isn't as straightforward.
6. Trying to improve reinforcement learning is why they first created the "attention" mechanism, leading to the Attention Is All You Need paper, which introduced the first transformer (aka, LLM).
7. Even in a world where we don't have intelligence better than us at reasoning, we could still have to trust in AI. Imagine we have an AI that can only reason as well as a dog, but it can work at 1,000x the speed of a human. That could be enough to make scientific breakthroughs.