Pod Insights No Priors What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang About this episode When AI research is evolving at warp speed and takes significant capital and compute power, what is the role of academia? Dr. Percy Liang – Stanford computer science professor and director of the Stanford Center for Research on Foundation Models talks about training costs, distributed infrastructure, model evaluation, alignment, and societal impact.
Sarah Guo and Elad Gil join Percy at his office to discuss the evolution of research in NLP, why AI developers should aim for superhuman levels of performance, the goals of the Center for Research on Foundation Models, and Together, a decentralized cloud for artificial intelligence.
No Priors is now on YouTube! Subscribe to the channel on YouTube and like this episode.
Show Links:
See Percy’s Research on Google Scholar
See Percy’s bio on Stanford’s website
Percy on Stanford’s Blog: What to Expect in 2023 in AI
Together, a decentralized cloud for artificial intelligence
Foundation AI models GPT-3 and DALL-E need release standards - Protocol
The Time Is Now to Develop Community Norms for the Release of Foundation Models - Stanford
Sign up for new podcasts every week. Email feedback to [email protected]
Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @PercyLiang
Show Notes:
[1:44] - How Percy got into machine learning research and started the Center for Research and Foundation Models at Stanford
[7:23] - The role of academia and academia’s competitive advantages
[13:30] - Research on natural language processing and computational semantics
[27:20] - Smaller scale architectures that are competitive with transformers
[35:08] - Helm, holistic evaluation of language models, a project with the the goal is to evaluate language models
[42:13] - Together, a decentralized cloud for artificial intelligence
Transcript Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.
0:07 Speaker 00 For ages, human level has been the target for AI. And that has really been kind of a north star that has fueled many dreams and efforts and so on over the decades. But I think we're getting to a point where along many axes, it's superhuman or should be superhuman. And I think we should maybe define more of an objective measure of like what we actually want. This is more of a general statement about how we should think about technology, not just chasing after mimicking a human, because we have a lot of humans.
0:45 Sarah Guo? This is the No Priors Podcast. I'm Sarah Guo.
0:48 Speaker 02 I'm Olad Guo.
0:49 Sarah Guo? We invest, advise, and help start technology companies.
0:52 Speaker 02 In this podcast, we're talking with the leading founders and researchers in AI about the biggest questions.
1:04 Sarah Guo? We're very pleased today to have Dr. Percy Liang, professor of computer science here at Stanford and director of the Center for Research on Foundation Models, a recently founded center here. Dr. Liang is the author of over 100 heavily cited research papers around helping machines understand natural language, helping humans reason about those models, and has contributed a number of novel technical approaches and creative uses of data to the machine learning field. And as a special treat, we're recording here in his office at Stanford today. Thanks, Percy.
1:36 Speaker 00 Great, welcome.
1:37 Sarah Guo? So I think just to start, can you tell us a little bit about how you got into the machine learning research field and your personal background?
1:44 Speaker 00 Yeah, so I've been in the field of machine learning and natural language processing for over 20 years. I started getting into it in undergrad. I was undergrad at MIT. I liked theory. I had a fascination with languages. I was fascinated by how humans could just be exposed to just strings of text, I mean, speech, and somehow acquire very sophisticated understanding of the world and also syntax and learn that in a fairly unsupervised way. And I wanted to, my dream was to get computers to do the same. So then I went to grad school at Berkeley. And then after that started at Stanford. And ever since, I've been pursuit of developing
2:26 Speaker 00 systems that can really, truly understand natural language. And of course, in the last four years, this once upon a time kind of dream has really taken off in a sense. Maybe in a, not a way that I would necessarily expect, but with the coming out of large language models such as GPT-3, it's truly kind of astonishing how much of the structure of language and the world that these models can can capture. In some ways, it kind of hearkens back when I actually first started in NLP. I was training language models, but of a very different type. It was based on hidden Markov models. And there the goal was to discover hidden structure in in text.
3:10 Speaker 00 And we were, I was very excited by the fact that it could learn about, tease apart what words were like city names versus days of the week and so on. But now it's kind of on a completely different level.
3:22 Sarah Guo? You've worked on multiple generations of NLP at this point, pushing the forefront of semantic parsing. Was there a moment at which you decided that, you know, you were going to focus on foundation models and large language models?
3:34 Speaker 00 Yeah, there was a very decisive moment. And that moment was when GPT-3 came out. That was in the middle of the pandemic. And it was just, it wasn't so much the capabilities of the model that shocked me, but it was the way that the model was trained, which was basically taking a massive amount of text and asking a model to predict the next word over and over again, you know, billions of times. What rose from it was not only a model that could generate fluent text, but also a model that could do in context learning, which means that you can prompt a language model with instructions. For example, summarize this document, give it some examples and have the model on the fly in context, figure out what the task was. And this was a paradigm shift, in my opinion,
4:18 Speaker 00 because it changed the way that we conceptualize machine learning and NLP systems from these bespoke systems where you're, it's trained to do question answering, to train to do this, to just a general substance. Where you can ask the model to do various things. And the idea of a task, which is so central to AI, I think, begins to dissolve. And I find that extremely exciting. And that's the reason later in two thousand twenty one, we founded the Center for Research on Foundation Models. We coined the term foundation models because we thought there was something that was happening in the world that was that somehow large language models didn't really capture the significance. And it was not just about language, it's about images and multimodality.
5:03 Speaker 00 It was a more general phenomenon. And we coined the term foundation models. And then the center started and it's been sort of, you know, a kind of a roller coaster ride ever since.
5:14 Speaker 02 We're going to be talking, I think, about both your experiences in research in academia. And then we'll also separately be talking about Together, which is a company you're involved with now. Could you tell us a little bit more about what the center does and what you're focused on?
5:25 Speaker 00 Yeah, so the Center for Research on Foundation Models, sort of two years ago, is under the Human Centered AI Institute at Stanford. And the main mission of the center is, I would say, to increase transparency and accessibility to foundation models. So foundation models are becoming more and more ubiquitous. But at the same time, one thing we have noticed is the lack of transparency and accessibility of these models. So if you think about the last decade of deep learning, it has profited a lot from having a culture of openness with tools like PyTorch or TensorFlow, data sets that are open,
6:08 Speaker 00 people publishing openly about their research. And this has led to a lot of community and progress, not just in academia, but also in industry with different startups and hobbyists and whoever just getting involved. And what we're seeing now is sort of a retreat of that open culture where models are now being only accessible via APIs. We don't really know all the secret sauce that's going behind them and there's sort of limited access.
6:40 Sarah Guo? What's your diagnosis of why that's happening?
6:42 Speaker 00 I think that this is very natural because these models take a lot of capital to train. There are enormous amount of, you can generate a lot of value and it's a competitive advantage. So incentives are to to keep these under control. There's also another factor, which is safety reasons. I think these models are extremely powerful. And maybe the models right now, I think, are, well, if they were out and open, it would be maybe okay. But in the future, these models could be extremely good and having them completely, anything goes. Running amok. We might have to think about that a little bit more carefully.
7:23 Speaker 02 How do you think all this evolves in terms of, if you look at the history of ML or NLP or AI, we've had these waves of innovation in academia and then we've had waves of innovation in implementation in industry. And it's in some cases, we've had both happening simultaneously, but it feels a little bit like it's ping ponged over time in different ways. Now that people are starting to be more closed in terms of some of these models on the industry side and publishing less and being less open, how do you view the role of academia and industry diverging, if at all? Like, do you think it'll be different types of research that each type of institution tackles? Do you think there'll be overlap? I'm sort of curious how you how you view all that evolving?
7:58 Speaker 00 I mean, I think industry and academia have very distinctive and important functions. And always when I tell my students, well, we should be working on things that are lean on academia's competitive advantage. And historically, I think this has meant different things. So before ML was that big, I think a lot of academic research was really about developing the tools to make these models work at all. I remember working on systems and being ML models back in grad school. And basically, it wasn't working. I mean, computer wasn't working, vision wasn't working, question answering wasn't working. And I think the goal of academia there was to make things work. And and a lot of the advances that were born out of academia
8:43 Speaker 00 then influenced other ideas and influenced other ideas before it started clicking. And now we're seeing this, a lot of the fruits of both academia and industry's research fueling this kind of industry drive that you see today. And now today, I think it's the dynamic is quite different because it's no longer academia's job isn't just to get things to work because you can do that in other ways. There's a lot of resources going into tech companies where there's, if you have data and compute, you can just sort of scale and blast through a lot of barriers. And I think a lot of the role of academia is understanding because these models, for all their impressive feats,
9:27 Speaker 00 we just don't understand what they work, how they work, what the principles are, how does this training data, how does moral architecture affect the different behaviors? What is the best way to weight data? How do you, what's the training objective? Many of these questions, I think, could benefit from a more rigorous analysis. The other piece, which is a different type of understanding, is understanding social impact. And this is going back to the question about what is CRFM's role is. CRFM is a center with over thirty different faculty across ten different departments at Stanford. So it's quite interdisciplinary. So we're looking at foundation models, not just from a technical
10:07 Speaker 00 perspective of how do you get these models to work, but also thinking about their economic impact, the challenges when it comes to copyright and legality. We're working on a paper that explores some of those questions. We're looking at, you know, different questions of, you know, social biases and thinking through carefully how the impact of these models have on issues of, you know, homogenization, where you have a central model that's making perhaps decisions for a single user across all the different aspects. So some of these are the types of questions. There are also people at the center looking at risks of disinformation,
10:48 Speaker 00 monitoring to what extent these, uh, these tools are so persuasive, which they are getting increasingly so. And what are the actual risks when it comes to, let's say, foreign state actors leveraging this technology? And there's also people at the center who are in medicine and we're exploring ways of leveraging foundation models and deployment in actual clinical practice.
11:13 Speaker 02 How near term do you think some of the deployments are? Because if you go back into the seventies, there was like the Mycin Project here at Stanford, which was an expert system that outperformed Stanford medical school staff at predicting what infectious disease somebody had, for example. And that was 50 years ago, or almost 50 years ago, and it never really got implemented in the real world. And so one of my concerns sometimes in terms of the impact of some of these things is, are there industries that are resistant to adoption or resistance to change? And is it exciting to hear that, you know, at Stanford, they're actually starting to look at, how do you actually integrate these things into real clinical care? Do you view those things as very far out on the healthcare side? Do you view them as sort of near? I know that isn't the main topic we're going to cover, but I'm still a bit curious, given how close you are to all this.
11:52 Speaker 00 Yeah, I think it's a, it's a good question. I think there are a bunch of different issues that need to be resolved. For example, foundation models are trained on a lot of data. How do you deal with privacy? How do you deal with robustness? Because once you're talking about, you know, in the healthcare spaces especially, there are cases where we know that these models can still hallucinate facts and sound very confident in doing so.
12:17 Speaker 02 I know some doctors like that, too.
12:20 Speaker 00 Yeah, there you go. So.
12:22 Sarah Guo? But you've also taken a point of view that we should, you know, expect superhuman, if we, if we see superhuman performance from these models, like, holding them to the standard of a human doctor is actually insufficient as well, right?
12:34 Speaker 00 Yeah, I think that's a, that's a great point, is that for ages, human level has been the target for, for AI. And that has really been kind of a north star that has fueled many dreams and efforts and so on over the decades. But I think we're getting to a point where along many axes, it's superhuman or should be superhuman. And I think we should maybe define more of an objective measure of like, what we actually want. We want something that's very reliable, is grounded. You know, I often want more statistical evidence when I speak to doctors and sometimes fail to get that. And have something that
13:16 Speaker 00 would be sort of a lot more principled and rational. This is more of a general statement about how we should think about technology, not just chasing after mimicking a human, because we already have a lot of humans.
13:29 Speaker 02 That's an interesting point. It's really fascinating to watch all this evolve right now. You've done extensive research on natural language processing and computational semantics. Can you explain what those terms mean and how they're relevant to the development of AI?
13:42 Speaker 00 So computational semantics is the process where you take.
13:47 Speaker 00 Language, text, and compute, quote unquote, meaning from it. And that is something I'm not gonna, you know, maybe try to attempt to define. There's a huge literature of linguistics and, you know, philosophy about what, what meaning is. I would say that a lot of my research in, in the past, maybe ten, five to 10 years ago, was adopting this view that language is a programming language. It computes, you can give orders, you can instruct, you can do things with language. And therefore, it was natural to model natural language as a formal language. So a lot of semantic parsing is about mapping natural language into a, you know, formal space
14:31 Speaker 00 so that machines could execute this. And so one concrete application of this that I worked on for a while is mapping natural language questions into essentially SQL queries, which obviously has many different applications as well. And what was nice about this framework is that to really do this, you had to understand how the words contribute to different parts of the SQL query. And then you could get something that was a program that you could execute and you deliver the results, as opposed to many question answering systems, which you ask a question, maybe retrieve some document, you're retrieving the answer, or either that or making something up, rather than computing rigorously.
15:14 Speaker 00 So, so that was a paradigm I, I was working in, maybe five to 10 years ago. But the main problem is that the world isn't a database. A small part of the world is a database, but most of the world is unstructured. And then I started thinking about question answering in general. And we developed the Squad Question Answering Benchmark to fuel progress in open domain question answering. And that, in turn, and many other data sets that were developed, both at Stanford and elsewhere, I think led to the development of these powerful language models that then, like BERT and Roberta and Elmo back in around 2018, to then
15:57 Sarah Guo? Many years ago.
15:57 Speaker 00 Many years ago, ancient history now, to more like 2020 generation of, you know, these large foundation models. So I think there's certainly a place for that type of thinking. There are cases where you want to just map natural language into, say, people call it tool use, like you ask some, some question that requires calculation, you should just use a calculator rather than try to sort of quote unquote do it in the transformer's head. But there's also a lot of aspects of reasoning which are not quite formal. We do this all the time. And a lot of that happens kind of natively in the, in the language
16:40 Speaker 00 model. And I think it's still an interesting question, how to kind of marry the two. I feel like the two are still in sort of, you know, jammed together in a way. And maybe it's natural because there's certain things you can do in your heads, certain things you can invoke a tool to use. But this has been also one of the, the classic debates in AI. There's neural versus symbolic. And, you know, for a while symbolic AI was dominant. Now neural AI has come, really taken off and become dominant. But some of those central problems of how do you do planning, how do you do reasoning, which was the focus in
17:24 Speaker 00 study of symbolic AI, are now again really relevant because now we've moved past just simple classification and just entity extraction, but now more to, to more ambitious tasks. What do you think of some of the more interesting research programs right now in that area? I think that it's interesting to remark on what's happening because there are, to a first order approximation, larger models trained on the relevant data seem to do well on various benchmarks. I think that maybe there isn't enough emphasis on data efficiency and how quickly you can get and how robustly you can get to these points. Because we know
18:06 Speaker 00 it has been well documented that benchmarks can be gamable. So even though you do well on benchmark doesn't mean you've necessarily solved the problem. So I think one has to be a little bit cautious about that. So obviously, scale and more data is just one clear direction. But in terms of orthogonal directions, what are the the methods? Several things have to happen. One is we have to have ability to handle a greater context length. If you think about a long reasoning chain, you know, transformers are fixed and there's ways to extend it. But fundamentally, it's sort of a fixed model. Let's say advanced problem solving. For example, if you want to solve a math problem, you'll improve something.
18:48 Speaker 00 The language model generates sort of this chain of thought and generates token by token, and then it generates something. But we know that humans, when they solve a problem, it's much more. You try different things, you backtrack. It's much more flexible, iterative, and it can last a lot longer than just going for a few iterations. And what is the architecture that can handle that level of complexity? I think is still an outstanding question.
19:19 Speaker 02 Is there any aspects of foundation or large language models that are emergent that you didn't anticipate or that really surprised you?
19:25 Speaker 00 I think going back to G P T three, I think in context, learning is something that surprised many people, including me. So here you're prompting a language model with an instruction and input output pairs. You know, here's a sentence, it's classified positive. Here's a sentence classified negative. And the model is somehow able to latch on to these examples and sort of figure out what you're trying to do and solve the task. And this is really intriguing because it's it's emergent. It wasn't hand coded by the designers to, oh, I want to do in context learning this way.
20:04 Speaker 00 Now, of course, you could have done that, but I think the the real sort of magic is you didn't have to do that, and yet it still does something. It's not completely reliable, but it's it's sort of can get better with better models and, you know, better data. Then there's chain of thought.
20:20 Sarah Guo? Do you want to explain what that is?
20:22 Speaker 00 So the idea is if I have a question that's presented to a language model. The language model could just answer and it'll maybe get it right or wrong. But if you ask a language model to generate an explanation of how it would solve the problems, kind of thinking out loud, then it's much more likely to get the answer right. And this is very natural that it would be the case for hu- humans as well. But the fact that, again, the chain of thought capability is something that, you know, emerges. The other thing I think is really wild is this, and I think it's maybe a general principle, which is the ability to mix and match.
21:03 Speaker 00 So you can ask the model to explain the quicksort algorithm in the style of Shakespeare, and it'll actually construct something that is semantically pretty on point, but also stylistically, you know, much, much better than what many people could come up with. Which means that it has learned different concepts of what Shakespeare and what quicksort are, and is able to fuse them. So if you think about creativity, I think this is sort of an example of creative use. People say that sometimes all language models are just memorized because they're so big and trained on clearly a lot of text. But these examples, I think, really indicate that there's no way that these language models
21:46 Speaker 00 are just memorizing, because this text just doesn't exist. And you have to have some creative juice and invent something new. And I, I think, just to kind of go on, riff on that a little bit. I think the creative aspects of these language models with the potential for scientific discovery or doing research or pushing the boundaries beyond what humans can do, I think is really, really fascinating. Because up until now, again, remember the, the AI dream tops out at humans, but, but now we can actually go beyond in many, many ways. And I think that unlocks a lot of possibilities.
22:25 Sarah Guo? Yeah, there are a lot of really interesting examples. I mean, you could actually argue that connecting new concepts in any novel way is creativity, but I love the one that is just discovering like new tactics and go that humans haven't discovered after thousands of years of play. Maybe, well, ask if you'll risk making a prediction that is impossible. Emergent behaviors of models at the next level of scale, anything you might predict. Emergent capabilities, if we wouldn't have thought chain of thought or in-context learning would work.
22:54 Speaker 00 So I I can give you an example of something I think is emerging. And I can give you an example of a hope, but I don't know if I would call it prediction. So what we're seeing today is the ability to instruct a model using natural language to do certain things. You see a lot of this online with ChatGPT and Bing Chat, where you can just, and some of Anthropic's work as well. You can instruct a model to be succinct, generate three paragraphs in the style of, and so on. You can lay out these guidelines and have the model actually follow. So this instruction-following ability is getting extremely good.
23:37 Speaker 00 Now, I I will say that how much is emergent and how much is not? It's hard to tell because a lot of these models, it's not just a language model that's trained to predict the next word. There's a lot of, you know, secret sauce that goes under the hood. So if you define emergence of, you know, it was not intended by the designers. I don't know how much of that is emergent, but at least it's a capability that I think is very striking.
24:03 Speaker 00 The hope is that language models currently mix stuff up. They hallucinate, and this is clearly a big problem. And almost in some ways, a very difficult problem to crack. The hope is that as models get better, that some of this will actually go away. I don't know if that will happen. I guess the way I think about these these models is that they're they're doing some sort of, if you think about predicting the next word, it seems very simple, but you have to really internalize a lot of what is going on in this context. What are the previous words? What's the syntax? What's who's saying them?
24:46 Speaker 00 And all of that information and context has to get compressed. And then that allows you to predict the next word. If you're able to do this extremely well, then you sort of.
25:00 Speaker 00 Have a model of what's happening in the world, at least the world that you've captured in in text. And so while the notion of truth might be, you know, ambiguous in in many cases, I think the model can get an idea of what certain, you know, parts of internet are maybe reliable and what parts of the internet are not and what kind of, you know, the idea of having, you know, entities and dates and locations and what activities there are. I I think that will maybe become more salient in the model. Like, if you think a model, the language model, that's just predicting the next word, and it's only trained to do that.
25:42 Speaker 00 And you say, "Elad, travel to blank." Of course, it's gonna make, you know, something up without further context. But if it has a better understanding of what's happening and of of course with more context, then maybe it can use that context to actually know that, well, okay. Well, I don't know, maybe I should ask where he went. Yeah, yeah.
26:04 Speaker 02 So scale is basically increasing the statistical accuracy of the prediction on the next word because you have more context and more data by which turn for what's coming. And therefore, it will reduce hallucinations because you're increasing accuracy.
26:15 Speaker 00 Yeah, so I I think there's pre-training, which is predicting the next word and developing a world model, so to speak. And with those capabilities, then you still have to say, "Don't hallucinate." But it will be much easier to control that model if it has a notion of what hallucination even is.
26:38 Speaker 02 I was talking to somebody who was close to the development of the transformer model. And his claim was that one of the reasons it's done so well is to your point around scale, right? Eventually, you hit enough scale that you see that it's it's clearly has these really interesting emergent properties. You keep scaling it up and you keep sort of growing it. And so therefore, it's like a self-reinforcing loop to keep using these types of models. And his claim was that it's expensive to do that sort of scale. And so therefore, there may be other architectures or approaches that we've just never scaled up sufficiently in order to actually see if they have the same emergent properties or certain characteristics that may be superior. How do you think about that from the perspective of, you know, just going down the path of the transformer side versus? Other architectures that may be really interesting and may be neglected because we just haven't thrown enough compute at them because it's expensive.
27:21 Speaker 00 Yeah, I really hope that in 10 years we won't be using the transformer. Because I think the transformer is, I mean, it's a very good architecture. People have tried to improve it, but it's sort of like kind of good enough for for people to press ahead. But scientifically, there's no reason to believe that this is the one. And there have been some efforts. So one of my colleagues, Chris Ray, and his students have developed other architectures, which are actually at smaller scales competitive with transformers and actually don't require the central operation of attention. And I would love to see much more research exploring other alternatives to transformers.
28:02 Speaker 00 This is something, again, that academia, I think, is very well suited to do because it involves kind of challenging the status quo. You're not really trying to just get it to work and get it out there, but you're trying to reflect on what are the principles? What can we learn from transformers? What is it trying to do? And how can we incorporate them in a much more principled way? At some level, it's still going to be about compute, right? So people have shown that LSTMs, scaling loss for LSTMs, show that if you're able to scale up LSTMs, maybe they would work, you know, pretty well as well. But the amount of compute is, you know, many times more. And given a fixed compute budget, we're always in a compute constrained environment.
28:47 Sarah Guo? It's an efficient enough architecture to keep trying.
28:49 Speaker 00 Yeah, you would not use an LSTM. The transformer strictly dominates an LSTM from the perspective of given a fixed compute budget. So this question of like, what if I could scale up LSTM, it becomes a little bit sort of irrelevant.
29:03 Speaker 02 So for the things where you see transformer-like performance, what sort of compute budget would you need in order to be able to test them out? Is it the scale of $1 million, $10 million, $100 million of compute? I know it changes based on compute pricing. And Ben, just trying to get a rough sense of. You know, how expensive is it to try today? And then if we extrapolate down to compute curve three years from now, maybe it's tractable again or something, so.
29:22 Speaker 00 Yeah, it really depends on the gaps that you're seeing. Right now, in academia, you can train one billion parameter models. I mean, it's not, it's not cheap by academia standards, but you can, you can do it. And, you know, here at CRFM, we're training like, you know, six or seven billion parameter models. And I think it's enough to be able to try out some ideas. But ultimately, because of emergent properties and importance of scale, you can only make a hypothesis. You can find something like, oh, this seems promising at smaller scales. You still have to go out and test whether it's really pans out or the gap just closes.
30:07 Speaker 00 And maybe this is a good segue to talk about compute together. So we founded together on the, the premise that compute was, is a central bottleneck in foundation models. On the other hand, there's a lot of compute that's decentralized, that's maybe underutilized or idle. And if we could harness that compute and bring it bear for both, you know, research and also commercial purposes, then we could actually do a lot more. There are some pretty hefty technical challenges around doing that because foundation models are typically trained in very high-end data center environments
30:50 Speaker 00 where the interconnect between devices is extremely good. Whereas if you just grab your average desktop or home interconnect, it's a hundred times or more slower. But Chris Ray and Sezhan and others, really, they deserve most of the credit for this. We've developed some techniques that allow you to leverage this. Weekly connected compute and actually get pretty interesting training going. So hopefully with that type of infrastructure, we can begin to unlock a bit more of compute, both for academic research, but also for, you know, other startups and so on.
31:31 Speaker 02 That's really cool. So it sounds a little bit like earlier predecessors of this, maybe things like folding at home, where people did protein folding collectively on their computers, or SETI at home, where there was search through different astronomical data. And now you can actually do this for training an AI system on your desktop or, you know, excess compute that exists at data centers or in other places.
31:52 Speaker 00 So folding at home is, I think, a great inspiration for a lot of this work. At some point during the middle of the pandemic, they actually had the world's largest supercomputer in terms of flop count, because it was used to do molecular dynamics simulations for COVID. The main challenge with foundation models is that there's a lot of big models and big data that needs to be shuffled around. So the tasky composition is much, much harder. So that's why many of the technical things that we've doing about scheduling and compression enable us to overcome these hurdles. And then there's the question of incentives. So I think there's two aspects of what Together is building. One is a sort of what I will
32:37 Speaker 00 call a research computer, which is for academic research purposes where people can contribute compute. And in the, in the process of contributing compute, they are able to use the decentralized cloud for doing training when they're not using it. And when they are using it, they can use much more of it. So the hope is that it provides a much more efficient use of the compute, because you're spreading it across a larger set of people. And then, you know, on the commercial side, the hope is that the open models that are developed in the open source ecosystem can, Together platform can allow people to find
33:23 Speaker 00 And tune and adapt these models to various different use cases. One thing I think is noteworthy is that, you know, we think of foundation models today as, you know, maybe there's a few foundation models that are, you know, very good and exist. But I think in the future, there's going to be many different ones for different kind of use cases as this space takes off. Many of them will be derived from maybe existing foundation models, but many of them will also be perhaps trained on from, from scratch as well.
33:58 Sarah Guo? I think this is actually a pretty uncommon viewpoint right now. Can you talk a little bit about like where you, um, or, you know, research efforts you're associated with, choose to train models, like maybe Biomed or, or whatever else you think is relevant here?
34:12 Speaker 00 So there's foundation models is a pretty broad category of, and many of the, the sort of the core center is, you know, large language models that are trained on lots of, you know, internet data. We've trained a model here at CRFM in collaboration with Mosaic on the bot called Biomed LM. And it's not a huge model, but it's trained on PubMed articles and it exhibits pretty good, you know, performance on various benchmarks. For a while, we were able to be state of the art on the US medical licensing exam. You know, Google did come up with a model that was, I think, 200 times larger and they, they, they beat that model.
34:56 Speaker 00 So, you know, scaled does matter, but, but I think there are many cases where you, for efficiency reasons, maybe you do want a, a smaller model since cost, I think, is, you know, a big concern.
35:08 Sarah Guo? I want to talk about some of the, I think, like most important or hopefully most important work that the center's done so far. Can you explain what HLM is and what the goal has been?
35:17 Speaker 00 Yeah. So HLM stands for holistic evaluation of language models, which is this project that happened over the last year. And the goal is to evaluate language models. So the trouble is that Language models is a very generic thing. It's like saying evaluate the internet. What does that mean? The language model takes text in and text out. And one of the features of a language model is that it can be used for a myriad different applications. And so what we did in that paper is to be as systematically and as rigorous as we could
36:00 Speaker 00 in laying out the different scenarios in which language models could be used. And also measure aspects of these uses, which include not just accuracy, which a lot of benchmarks focus on, but also issues of how robust it is, how well it's calibrated, meaning that whether does the model know what it doesn't know? Whether the models are fair according to some definition of fairness? Whether they're biased, whether they spew out toxic content, how efficient they are? And then we go and we basically grab every language model that's prominent that we could access, which includes open source models like OPT and Bloom,
36:44 Speaker 00 but also getting access to APIs from Cohere AI 21, OpenAI, and also Anthropic and Microsoft. So overall, there were thirty different models, forty-two scenarios and seven metrics. And we ran the same evaluations on all of that. We've put all the results on the Helm website so that you could see the top level statistics and accuracies. But also you can drill down into, hmm, on this particular benchmark, what are the instances? What are the predictions that these models are making? All the way down to what prompts are you using for the language models? So the idea here is that we're trying to provide transparency to this space.
37:30 Speaker 00 Right? We know that these models are powerful. They have some deficiencies. And we're trying to lay that all out in a kind of a scientific manner. So I'm pretty excited. About this project, the challenge in the about this project is since we put out the paper maybe three months ago. A bunch of different models have come out, including ChatGPT, Llama, you know, coherent AI twenty one have updated their models. GPT four might come out at some point. So what had this project has evolved into is this basically this dynamically updating.
38:09 Speaker 00 Where every two weeks we refresh it with new models that are coming out as well as new scenarios. Because one thing we also realized, which was made clear by ChatGPT, is that the type of things that we ask of a language model is changing. We don't ask it just to do question answering or just to do.
38:27 Sarah Guo? As they increase in capability.
38:28 Speaker 00 Increasing capabilities. Now they can do a lot more. They can, you know, write an email or give you, you know, life advice on XYZ if you've put in a scenario. And in or write a, you know, an essay about XYZ. And I think what we need to do with the benchmark is also add the scenarios that accordingly improve or capture these capabilities. As well as kind of new risks. So we are definitely interested in benchmarking how persuasive are these these language models, which governs. You know, what are the risks and also how secure they are. One thing I'm actually also worried about is.
39:10 Speaker 00 Given all that the jailbreaking that is extremely common with these models where you can basically bypass safety controls. If these models start interacting with the world and accepting external inputs, now you can not only just. Sort of jailbreak your own model, but you can jailbreak other people's model and get them to do various things. And then. So that could lead to sort of a cascade of errors. So some of these are the concerns that we hope to also capture with the model. I should also mention, we're also trying to look at multimodal models, which I think is going to be pretty pertinent. So lots to do.
39:47 Sarah Guo? A bunch of the things that you've described as sort of the role you see for the center or even like academia in the age of like. Foundation models broadly, like they have more of an intersection with policy than traditionally like machine learning research. Like, how do you think about that?
40:04 Speaker 00 Yeah, actually, we've, I'm glad you asked that because we've been thinking a lot about the social implications of these models and sort of the. Not the models themselves, which we focus a lot on talking about, but the environment in which these models are built. So there are a few players in this space with different opinions about how models should be built. Some are more closed, some are more open. And there's also, again, this sort of lack of transparency, where we have a model that's produced and it's aligned, apparently, to human values. But then, once you start kind of questioning, you can ask the question, okay, well,
40:48 Speaker 00 which values, which humans are we talking about? Who determines these values? What legitimacy does that have? And what's the sort of accountability? Then you start noticing that, well, a lot of this is just kind of completely a black box. So one thing that we've been working at the center on is developing norms, starting with transparency. I think transparency is necessary but not sufficient. You need some level of transparency to even have a conversation about any of the policy issues. So making sure that the public can understand how these models are built, at least some notion of like what the data is,
41:32 Speaker 00 what are the instructions that are given to align the models. We're trying to advocate for greater transparency there. And I think this will be really important as these models really get deployed at scale and start impacting our lives. You know, a kind of analogy I like to think about is nutrition labels or any sort of specification sheets on electronic devices. There's some sort of obligation, I think, that producers of some products should have to make sure that their product is used properly and has some bounds on it.
42:13 Speaker 02 I guess I'll ask two questions. One is, if people wanted to participate in Together, is there a client that can download and install or use? Or how can people help support the Together efforts?
42:22 Speaker 00 Yeah, so we are developing a client that will be made available. We both from the perspective of joining the Together Clouds so that you can contribute your compute, but also where we have an API that we're developing so that people can use Together infrastructure to do inference and fine tuning models. We are also training some open models, so we have this something called Open Chat Kit that we're releasing soon. And this is built on top of Eluther AI's NeoX model, but you know, improved to include various different types of capabilities. It's still of, you should think about it as really a work in progress.
43:06 Speaker 00 What we've tried to do is open it up so that people can play with it, give feedback and have the community improve this together, rather than us trying to produce some finished product and putting it out there. This goes back to the point about involving, you know, the spirit of open source and involving the community to build these foundation models together as opposed to someone unilaterally building them.
43:34 Sarah Guo? While we're talking timelines and predictions that you don't quite feel comfortable making, how do you think as a rigorous scientist about AGI?
43:41 Speaker 00 I must say that my opinions about AGI have changed over time. I think that for a while, it was, you know, perceived by most of the community as laughable. I will say that in the last 10 years, I have been aware of, you know, there's a kind of a certain community who think about AGI and also existential risk and things like that. You know, so I've been in touch with people who think about these. I think I see the world maybe differently. I think of perhaps, certainly these are powerful technologies and could have extreme social consequences. And but there's a lot of more near term issues.
44:23 Speaker 00 I focus a lot on kind of robustness of ML systems in the last, you know, five years. But, you know, one thing I've learned about Foundation models, because of their emergent qualities, I've learned to be very kind of no open-minded, I would say. I was asking earlier about what No Priors, where that comes from. And I think it's a fitting way to think about, you know, the world. Because I think everyone, including scientists, often get sort of drawn into a particular worldview and paradigm. And I think that, you know, the world is changing, both on the technical side, but also how we conceive of AI and, you know, maybe even humans at some level.
45:08 Speaker 00 And I think we have to be open-minded to, you know, how that's going to evolve over the next few years.
45:14 Sarah Guo? Awesome. Thanks for doing this conversation with us, Percy. It's great.
45:17 Speaker 00 Yeah, thanks for joining us. Yeah, thank you very much.
45:22 Sarah Guo? Thank you for listening to this week's episode of No Priors.
45:25 Speaker 02 Follow No Priors for new guests each week and let us know online what you think and who in AI you want to hear from.
45:30 Sarah Guo? You can keep in touch with me and Conviction by following at Sarah Normus.
45:34 Speaker 02 You can follow me on Twitter at @AladGill. Thanks for listening.
45:40 Sarah Guo? No Priors is produced in partnership with Podpeople. Special thanks to our team, Cynthia Galdea and Pranav Ruddy, and the production team at Podpeople. Alex McManus, Matt Saab, Amy Machado, Ashton Carter, Danielle Roth, Carter Wogan, and Billy Libby. Also, our parents, our children, the academy, and tyranny.ml, just your average friendly AGI world government.
Previous How AI can make drug discovery fail less, with Daphne Koller from Insitro Next What is the future of search? With Neeva’s Sridhar Ramaswamy Page updated October 26, 2026 · transcript and summary are machine-generated
