AI in Scientific Discovery & Ethics: Dr. Anima Anandkumar on the Future of AI

Transcript Disclaimer: This transcript has been generated using automated tools and reviewed by a human. However, some errors may still be present. For complete accuracy, please refer to the original audio.

00:00:11 Sangeeta Gupta: Welcome to the Nasscom Conversations. My name is Sangeeta Gupta and I'll be the host here today. Nasscom Conversations is the platform where we are bringing together global thought leaders to explore the transformative impact of technology. In each episode, we'll dive into the latest advancements on what's happening in the world of AI, how's the innovation and digital landscape changing and and you'll only hear from experts on how are they shaping the future worldwide. I'm very excited today because we have Dr. Anima AnandKumar joining us today. She's currently the brand professor of computing and mathematical sciences at the California Institute of Technology. Doctor Anima, welcome to the podcast.

00:00:51 Dr. Anima Anandkumar: Thank you Sangeeta for having me.

00:00:54 Sangeeta Gupta: So Dr. Anima, there is so much happening in the world of AI. While I know you would have been working in this field for a very, very long time, but but I think post ChatGPT moment, I think the whole world is enamoured with AI, both the positives, the negatives, the opportunities, the risks. How, how are you seeing the the fact that the timeline they are in today and how do you think about, you know, how and where is the shaping up as as you see it as a researcher, academician, as an innovator, what would be your perspectives around where is AI moving?

00:01:28 Dr. Anima Anandkumar: Yeah, no, it's been such an exciting journey as somebody who's been working for more than two decades in AI and seeing it grow from science fiction to reality, right? It's just been so interesting to kind of like start from theoretical foundations because, you know, back when I started my grad school, you know, you could do maybe small scale simulations, you could work with small data sets and neural networks were nowhere in the picture because, you know, they needed big data sets and big compute. So but starting there and having that right foundations, I think it's still very valuable even in this day, just as I'm teaching, I'm going to be teaching even today the course on foundations of machine learning. We kind of like chart through that journey, right? And and of course, once deep learning became practical and it turned out that, you know, the level of compute we had even a decade ago, along with data sets from the Internet scale, you know, collection was possible to trade practical deep learning. That to me, that's when the revolution just like jump started and ChatGPT is really to me a culmination of what you could do in my view, purely based on data from the Internet, right? So and even that over the last two years there's been a lot of private data that is assembled that is much more curated. So we're kind of like starting to see the limits of data availability and that's where there is the debate of what comes next. And to me, I think even as we've seen this revolution in terms of Internet scale data with deep learning, at the same time, especially over the last few years, I've been working on the intersection of AI and scientific domains. And, you know, once I joined Caltech, to me it was always this place I associated with Richard Feynman, with all the Nobel Prize winners in various areas of sciences. So what can AI do for these various areas? And not just that, to really tackle the foundations of scientific method itself. You know, can AI be deeply integrated into sciences? And that's where the last few years have really focused on now, you know, showing not only that AI can do as well as current methods, but surpass them in a really grand way, right? Being able to have simulations that are much faster, being able to have forecasts and predictions that are much better in a number of areas, being able to design better devices, drugs, all of these are to me. We are still at the infancy, you know, it may appear with ChatGPT, we've solved all the problems in AI. We already have practical working products. But when it comes to scientific domains, we are still early in the journey. And that's what excites me of things to come.

00:04:54 Sangeeta Gupta: No, I think that's fascinating Anima and you know, maybe just to build on the scientific discoveries and AI and scientific discoveries. Can you help us just paint your picture? Paint a picture of how you see the world changing with AI's integration with scientific discoveries. Maybe a couple of news cases that you think are visible on the horizon in the next few years.

00:05:17 Dr. Anima Anandkumar: Yeah. I mean we already have many cases that you know, are working in the real world. So more than three years ago we built the first AI based high resolution weather model that is not only accurate, in fact even more accurate than the existing weather models in many situations. And not only that, it's 10s of thousands of times faster. So what would take a big supercomputer that to run current weather models, the traditional weather models takes a single desktop gaming GPU that you may have at home. And we've open sourced the model. A number of weather agencies are using this as well as researchers are building on top of this. So it democratizes weather modeling and and not only that, you know, when we think about extreme weather events, you know, the unfortunate wildfires that happened close to Caltech here, as well as, you know, we think and think of hurricane storms. So worldwide we are seeing increase in these extreme weather events with the climate change. And so this ability to not only have early forecasts of extreme weather events, but also the risk assessment, right? The challenge that comes to events like these are it's chaotic. So you, you know, cannot possibly even with all the compute, all the data, it's still impossible to accurately predict what's going to happen. You know, what we saw with the wildfires here in Pasadena was unprecedented winds, right? So these are rare events, but of course, when they happen, the damage from them is also very outsized. So we can come up with a much better probabilistic forecasts or risk assessments using AI based weather models. And that's because they're so much faster. As I mentioned, they're 10s of thousands of times faster. So you can afford to Dow do much larger statistical ensembles. Meaning you look at all different scenarios, you perturb the current conditions like say wind conditions and check you know where. How does the path of the hurricane change if I add certain amounts of noise to the current conditions? And so the more scenarios you can generate like that, the better you have in terms of risk assessment because you can now count how many of those paths, say, make a landfall and can cause damage. And so those kinds of risk assessments are critical, you know, in terms of assessing when to evacuate a certain zone, for instance, you know, having valuable time to save human lives, maybe save property. So those aspects are I think will become even more important. So we've done these kinds of ensembles with weather models that are now much better what you can do with traditional models. So, so this is not just AI being faster. This is enabling now as to deal with extreme weather events that wasn't possible before. So that's one example. But more broadly, you know, the kind of techniques that have made these AI based weather models possible are the aspect that you can have AI not only learn at one fixed resolution or, you know, if you think of like the, you know, information across the globe, you can think of them as pixels. And if you fix the number of pixels and learn an AI model, it only works at that resolution. And that's how if you look at image generation tools, whether it's ChatGPT or any of the popular ones, that's how they work. They work using a fixed number of pixels, but that's not how the natural world is, right? So even if we represent the weather model with a fixed number of pixels, the reality is it's a continuous process. And the reality is, you know, the, if you zoom in, there's all kinds of local features and topography. There is clouds that have fine scale. And that's what you know is very important to model hurricanes, for instance. And so we are able to do that using an AI technique called neural operator. And what it's able to do is to be able to learn the mapping from continuous function to another continuous function in the output, which means it's not limited to one fixed resolution. So it can now capture information at multiple different scales. So if you zoom into a hurricane, it can still capture those fine scale details, whereas other standard computer vision models, other AI models are not able to do so. And this kind of aspect is really critical in many other settings. For instance, we designed, you know, the ability to stabilize drones under extremely turbulent conditions. We've designed a medical catheter that reduces bacterial contamination by a hundredfold. We have come up with AI that can predict how plasma evolves in a nuclear fusion reactor. And even here in a plasma can become extremely unstable, right? And you need fine scale that modeling because you need to really zoom in and look at a fine grid and how plasma evolves. And that's what traditional methods very expensive and we are able to do this kind of prediction more than a million times faster than what's possible with traditional methods. And because of that, we can now run these models faster than real time. So we can use this to control fusion and hopefully one day we would be able to avoid disruptions while having sustainable fusion. So AI is game changing in its ability to model these kind of complex processes.

00:11:59 Sangeeta Gupta: Very interesting. And you know, especially the, the fires example is just so live as we as we are living it. I just want to, you know, while we're talking about obviously the, the potential that AI is offering, there's also that equal concern on accuracy and reliability. And especially when you go into these very sensitive situations, how do you, how do you ensure that those, you know, accuracy and reliability? And I know there's a lot of testing validation that happens, but, but, but at the same time, and particularly, I think in some of the more consumer use cases, we're hearing a lot of concern on safety, reliability, and hence the risk of regulation. How do you, how do you perceive that that environment right?

00:12:46 Dr. Anima Anandkumar: I mean, as someone who's been working in AI for science, this is inherent to the problems at hand, right? So if like the weather example I mentioned, you know, you need to stress test it and ensure it works under all kinds of conditions. And when we started working on this, a lot of people were skeptical. You know, weather modelling has been done for decades. AI can't just one day come and we as outsiders can't just come and be able to up in the field. And you know, The thing is you expect because there are hurricanes are rare events, it wouldn't be able to model that well, right? But what we've seen over the last few years is these models are able to capture those kind of physical signatures because they are very clearly differentiated from other kinds of events. And so you may not need as many samples, which is great. But then in order to add robustness, can we add different kinds of laws of physics into them? So that way you, you know, conservation laws, for instance, so you have more confidence that, you know, it's sad. It's just doing something physical. Its outputs are physically valid. And we've done that in many other scenarios as well where you need like the guardrails of physics ensures that the model, even when it makes mistakes, is not going to do something that is not going to happen in nature. So I think that comes more naturally in these applications for sciences as well as in a robustness is inherently needed. If you think about for instance, you know, we designed a medical catheter and we simulated fluid dynamics and we wanted to prevent bacteria from swimming upstream along the walls of the pipe into the human body and cause infection. And to avoid that, we wanted to design these triangular proofs that create turbulences or vortices that can, that would make it hard for bacteria to swim across and and hence you prevent the bacteria from swimming into the body. So this is all nice in theory, right? Of course, the question is, once you design this in simulation, what happens when you go and test it in the lab? If you always need physical testing, you need it to work in the real world. And that's another kind of robustness check. And to our surprise, our model just worked for the first time and it led to a hundredfold reduction in bacteria contamination. So in other cases, if it didn't, you could go back and forth. And and so in many of these scientific applications, these requirements of robustness, you know, safety, we talked about how we looked at stabilizing the wing of drones and airplanes under extremely turbulent conditions and what is the right reinforcement learning methods to do for that. And again, right, so this we did it in a wind tunnel and a very turbulent wind. So, you know, so you need these like robustness, stability and safety, kind of like modeling that needs to go into this to work. So I I feel much better here because the guardrails of the physical world naturally require these kind of robustness. So you can very quickly check and ensure that you're designing methods that conform to those requirements.

00:16:26 Sangeeta Gupta: No, fully agree with that Anima, but you know, from a broader AI world, do you do you worry about the risks of AI, I mean we've seen those camps of, you know, the doomsday scenarios and the ones who are believing in the opportunity at the end of the tunnel. But but are there real risks that you worry about, not so much in the work you are doing, but the broader AI ecosystem?

00:16:50 Dr. Anima Anandkumar: So to me, AI has always been an open-source revolution. And you know, when I started working in the field and as deep learning started taking off, it was really right open data sets like Imagenet and competitions. And you know, people published, they open source the code and that's what led to this revolution, right? So ChatGPT didn't just happen in a vacuum. It was built on top of all the open source work, including some of the open source language models that when you know, I was at NVIDIA, right, NVIDIA released this code and open AI and others kind of further developed on that. So, you know, the important thing is to not lose track of that, right? And the worry is especially some of the recent attempts at regulation hit at the heart of open sourcing and really make that untenable in some cases. And I think that's the difficulty. It's very hard to get regulation right. That does not stop innovation. That does not come in the way of open sourcing because you know, there is all kinds of really, I feel like bad analogies made with nuclear weapons and what so I mean, AI is so different, right? It doesn't have, it's not just one specific technology. It's not one specific tool. It just, you know, like the kinds of scientific problems that I told you about are so broad. It's all of science. And so we cannot have this like one rule fits all or we cannot just say, oh, if the model is large enough, it it's probably dangerous. So we should regulate it. You know, every day we talk about innovations where the, you know, there is more efficient techniques to train these models. The sizes of these models is coming down. And at the same time, maybe you do more interesting things with larger models or more beneficial applications. So it's, you know, and, and that's kind of like where my worry has been over the last few years. You know, it's more to do with regulators getting it very wrong and, you know, and harming innovation and especially also innovation across the world. How do we ensure democratization? That AI reaches everyone and solves local problems? That local population can work on AI in meaningful ways?

00:19:36 Sangeeta Gupta: I think AI has become this geopolitical tool now where I think every country wants to build its own sovereignty around it. And I think it's fascinating to see how technology has become such a tool for geopolitics. But but I think that's the world we are living in. So you know, Anima, you mentioned about NVIDIA and NVIDIA I think recently talked about physical AI. And I know while in your world you've probably been doing it earlier, but I think suddenly that terminology of physical AI seem to become much more real. How how do you make that real for the, you know, the common person to understand what what do we mean by when we are saying physical AI and what goes behind it?

00:20:20 Dr. Anima Anandkumar: Indeed, you know when Jensen Huang, NVIDIA CEO talked about physical AI as the next revolution at CES, you know this was like something that I work closely at NVIDIA and also right, So something I know Jensen is personally very excited about. We know because when my team as a collaborative effect effort to develop these first AI based weather model, Jensen got very excited and that's how he created the NVIDIA Earth to affect effort. And you know, there is a lot of innovation that spurs from like kind of like, you know, just going and trying out, right. And with the physical AI, it's a very broad term. So it can mean weather models, it can mean robotics, it can mean the ability to do engineering design. And now the question is, what is the chat GPD for physical AI? I mean, I don't think it'll be a, first of all, a chat interface. So that's a misnomer. But to me, what?

00:21:26 Sangeeta Gupta: Is the chatGPT moment right? 

00:21:29 Dr. Anima Anandkumar: Yes, and also what does it mean to have universal physical understanding for AI? So can AI understand different forms of physical models? Be able to simulate design control, so all of these aspects that you expect, you know, currently there, it's either done through other kinds of numerical simulation or it's done in actual physical experiments. Can AI create physically valid digital twins? So not only digital twins that look good, you know, there's a lot of visualizations, right that may just like is about focused on things looking good, but instead is able to inherently understand the underlying physics of it. So it's able to do these simulations much, much faster. Like I mentioned the example of nuclear fusion where we are more than a million times faster. So it's able to do that while being also robust and accurate. And so that ability will be game changing in just such a broad range of domains. So.

00:22:45 Sangeeta Gupta: So when we think of a country like India and HIMA very more right now at the application layer in AI, yes, we are building a few local language models, etc. But I think most of the innovation that's happening in India is in the space of applications. And then when you think of, you know, the revolutions like physical AI coming to being, what do you think countries like India And how can we approach much more strategically some of these big changes that are happening? So that, you know, you're not just a use case capital for the world, but you're doing much more, especially in sectors like you said, right? Whether it's healthcare, whether it's manufacturing in India, there is just so many different use cases of scientific discovery. Healthcare, I think you can just think about just every use case, every possible solution that you can do at scale here. Any thoughts around you know what, what can be India's play in these areas?

00:23:42 Dr. Anima Anandkumar: Yeah, absolutely. I mean, to me as somebody, you know, who grew up in India and having, you know, my parents both be engineers and running a factory that brought some of the first computerized manufacturing to my hometown in Mysore, I really think, you know, manufacturing is the lifeblood of the country, right. And how do you leapfrog to the latest technologies in manufacturing and use AI embedded in that to further have gains and further have differentiation that really increases efficiency. And to me, like you mentioned, OAI is now a lot in the application aspect that India is kind of like building on. I think you need to think about going all the way down, right? So semiconductor space where Taiwan, Korea, you know there's a lot of leadership there, right, That is just, you know, like something that everybody is dependent in the NVIDIA is dependent in having chips manufactured with TSMC for instance. So that takes right a lot of focused effort to build that kind of expertise. And, and, and I know in India, the efforts are getting started to have that leadership in semiconductors. You know, I've talked to many leaders, business leaders in this space. And I think you should always start there. You should start at the source because, you know, we'll see a revolution in semiconductors. You know, there's a lot of physical limits that will become challenging as we go to, right, like more miniaturized processes, but also the thermal will become a big challenge. You know, you're packing more of these things. It gets harder, right? It's simple, but how do you overcome that and how do you do better designs and how do you manufacture with very high requirements of reliability? This is non trivial, but I think India can really bring in all of the engineering expertise, the ability to have like workforces that are very focused and discipline to tackle these kind of challenging problems. So I I would keep a lot of focus there. While of course the other aspect is in terms of like the right software, the ability to train models, build data centers. I know there have been efforts to also scale up many of the data centers and modernize them, right. So again, the aspect of being able to have models for local languages, right? So that's a that's a big one that you know, a lot of I know focus has been and the question is from there, how do you have meaningful applications that reaches all the population, right? So because India is so diverse and has so many languages, so many dialects, how do you go from just, you know, training on a few languages? Can you like capture the diversity of the country and can you really capture the cultural richness right to to then create meaningful applications? I know many entrepreneurs are are looking into this and this is an exciting time for sure.

00:27:21 Sangeeta Gupta: Definitely. And in India, given our diversity, I think speech to speech is becoming quite interesting, right? Because not everybody will be able to type. And you know, those live, those limitations may also exist. I know we're going to run out of time. So just couple of last questions and e-mail 1 is this whole debate around AGI and artificial super intelligence and what will it do to humanity? Where, where, where are you on that one, right? Are you seeing this in the near-term horizon or it'll come and go right or?

00:27:55 Dr. Anima Anandkumar: So I think to me the aspect that or AGI can just be language based is flawed. You know there is no AGI without understanding the physical world, right? Because ultimately you want AI to serve the needs of humanity and that means it has to understand the physical world, be able to reason, design, model. And if you see many of the logical flaws that ChatGPT or any of these chat bots have is because they, you know, they miss what happens in the spatial world right out. They miss many of the kind of like that experience of the physical world. And so I think that's a big limitation currently in these models. The other one is, you know, the definition of AGI itself is kind of all over the place. And, and I think a lot of people are amazed at the abilities of these chat bots, but we need to be careful to treat these the challenges as scientifically valid tasks, right? Because you know, if you show these models training data that look very similar to test data, then it's not AGI, it's more like supervised learning. And in many of the recent problems that, you know, like solving some of the math problems, there have been like lots of like heavy training that has happened on very similar problems. And so yes, it may be able to solve it. But you know, AI is very different from how human intelligence, right? When mathematicians, they build all of those foundations and then try to make connections and new mathematical discoveries. But here, if you're asking AI to solve lots of synthetically generated problems that look somewhat similar, for instance, in case of Olympiad, there's like a very structured way to design these problems. And so that becomes a big challenge. How do you then evaluate this model is capable or not right? And if you give it, you know, researchers have done some very interesting things where in these math problems they add something completely irrelevant, right? Like some rhyme or, you know, some poetry that has nothing to do with it. And then it gets completely confused. So, you know, so there are like these simple common sense tests that, you know, these models still fail. I mean, they're getting better as there's more training, more alignment. But if there are these fundamental flaws, to me that's not AGI.

00:30:44 Sangeeta Gupta: No, I totally agree with that. So just my last question, Anima is really about you being an advocate for diversity in AI research, right. So, So what do you think should be key steps that organisations or academic institutions or tech companies should be following to ensure the data itself is not biased? But, you know, many other things that are coming in are truly inclusive by nature.

00:31:09 Dr. Anima Anandkumar: I mean, to me, like the important thing to keep in mind is, you know, asking for better diversity and inclusion is not against merit, right? It's very pro merit. And that's I think there is a lot of misunderstanding around that. So I want to clear that first because, you know, to me the aspect is, you know, when there are very clear standards of what is excellent, you know, what deserves a promotion, then there is no room for bias or there's no room for, you know, which is actually a flaw in the current system that affects everybody, right? So the more transparent you make the, you know, requirements for, you know, different positions or, or the ability to advance in an organization, you also build better trust in the workforce. You, you know, remove that like politics and you remove the uncertainty around what people, you know, think their careers will be heading towards. And that's something and that's not easy. I think that's the challenge because a lot of organizations are more kind of like things expanding right in an unplanned manner, especially many of the tech organizations that experience sudden growth and, and I think it's but being thoughtful and having leadership that in really cares about its employees. I think that's important and this really is again an opportunity to fix maybe gaps in the organization that may just be rewarding wrong kind of behaviour that doesn't even benefit the growth of the organization. So that's how I see how we should think of diversity and inclusion.

00:33:05 Sangeeta Gupta: I think that's, it's a very debatable point at this stage, but but I think the core fundamentals of it are still very, very important, right? It's about how it's getting implemented and it's not taken as a quota for doing certain things. So I totally agree with you. So thank you so much, Anima. This has been a fascinating conversation. Truly appreciate the time you've taken to join us.