Let's Get Physical with AI with Ivan Poupyrev, CEO at Archetype AI

Enhance that computation and, and bringing this sort of incredible computation capabilities into the physical world. It's a very old idea and it came through many iterations and, you know, sort of in the history of bringing intelligence into the physical world, and IO OT was one of the early stages.

What they did, IOT did is able to connect. All the physi, a lot of physical devices, get them connected to the internet, building the pipeline. So until now, like my first sky I was driving had zero sensors. Right. And it was not connected internet. There's nothing to, you couldn't update it over over wifi or over wireless connection because there's nothing to update.

It was mechanical, mechanical beast. I was used to drive around. Right. But with iot. All the infrastructure start getting connected to the internet, right? And this whole directions of the big data and iot combined together, be able to take the data from the physical world and store it in the cloud.

Then use this data to understand how the physical infrastructure behaves and prove behavior and so, so forth. That was iot side of things. Sometimes people talk about OIT didn't believe to its promise. I actually think it. Actually exceeded the promise because if you think about like everything around us right now, particularly industrial world everything's connected to the cloud.

Everything's connected to internet. The data is completely available for, for all the new machinery has all connect connectivity there. So it was extremely successful. Connecting our physical world, digitizing and connected to the, to the, to the internet. Where it's got stuck is like, if you think about, talk about, you know, some of the partners we talk in the industry, you know, if you take a piece of Corus machinery and oil rig, for example, it will have, you know, thousands and thousands of sensors and thousands of thousands streams of data, and it all goes into this.

Some storage and some data lake on, on, on, on the Amazon or some other provider. Then there is this no capability to understand what this data means, because data by itself does not have any meaning. It's capturing a sort of slice of state of environment and time, but people need the meaning. And where it sets struggling is converting the data into the meaning, and that's where the model techniques for artificial diligence going and foundational model is going to change that equation and convert the data into the.

Actual meaning and insight, which actionable, you know, and could be used for something too. To improve the performance and so on so forth. 

Richie Cotton: Okay. The idea of building in, like analytics and getting through to that, well, okay, we've got something actionable just within the software rather than it having to be like, okay, well we've got this connected device and now we've gotta hand it over to a data analyst, and then they've gotta figure something out, and then you go back to like whoever's running the device.

And I guess there's a much slower process to happen. Things work automatically. And I guess in a lot of cases, I mean, certainly you would not have to do that with a self-driving car. It's gonna be way too too long to send it off to an analyst to say, should we turn left or not? So yeah.

Talk me through what are the big use cases then of physical ai? Like what are the coolest things you've seen? 

Ivan Poupyrev: If you think about the ultimate goal, of course is if you think about the data and data connected to the, you know, coming from the physical world we often talk to about three sort of levels.

Of physical ai, physical ai, and physical intelligence. The first one is insights, right? So getting the data and you know, in most common use cases, for example, for the personal use, if you think about it, all the data is being collected by the smartwatch. And you know, it measures how many steps you took or measures how many, you know, times you walked around, how to connect that to actual insight.

So the insight is first stack, it tells you what's what is it it means, right? And there's a mouth, a lot of use cases just to understand, you know utilization of, utilization of your machinery or equipment. Understand if there's any failures, known unknown failures. Is there something wrong with that?

And so, so forth, right? This inside generation, turning data into the knowledge is one of the use cases. Second use cases is rooting this knowledge. Data into the recommendations. It's not just enough to know that something's happening, but can we use a foundation model and this incredible capability to compress human knowledge to actually explain you what you need to be doing.

And you know when your, your car is broken, when your complex machinery, a piece of machine is not performing really well, how can you improve? The performance of, of, of that, that equipment. And we, we have, you know, multiple customers coming for different problems. You know, a lot of the use cases is to discover things you don't know about.

Like the quote element of a discovery is really important, right? Because machine learning, classic machine learning can detect things, you know, but you cannot know what it do, it doesn't know. With this foundation models and physical ai, you can start indicating things. Which I not necessarily being trained for this can be something unusual is happening and provide some recommendation how to do this.

And the ultimate goal, L three of applications is automation. We often talk about robots as, you know, independent as as this looks at causation as service robots walking around. But what does it mean to have a robotic factory? What does it mean to have robotic cities or robotic office buildings?

What it means if your home is automated what this means when the end. When you start talking about on this level, this whole idea of sort of superint intelligence changes direction, right? Because having just like a very intelligent, you know, washing machine and very intelligent car and very intelligent HVAC system in the home and electrical G grid, connecting all together is one story.

But then when they start talking to each other and actually optimize the overall system. To achieve better performance, safety, money saving reliability detective end, which need to be, prevent, prevented to really kind of taking care of you as a whole system. That's sort of like we often think about as like that will be physical super intelligence, which actually connect all these things together in achieve this sort of level automation. So there's three layers in size, recommendation, automations, we see this how, or physically I will progress. 

Richie Cotton: Well, I mean that's pretty incredible, the scale thing because I, I think in my mind I was thinking about this sort kind of slightly smarter smart appliances, you know, like you talked about the washing machine example or the self-driving car example.

Actually once you start cleaning these together, like. The idea of having a completely smart factory or completely smart city that, I mean, that's really big scale software. So I guess, how close are we to this sort of thing happening? 

Ivan Poupyrev: I think the, it's actually, I think we're pretty close. I mean, of course, obviously somebody who's working on physical ai, we tend to be over optimistic.

But, I do believe that the core challenges, as we mentioned with iot, we solve the core challenges with core. One of the core challenges was getting the data out of the physical world where we can apply these techniques. So building this infrastructure took years for this IO ot, the big data movement, right?

So this, the basic, basic infra is lie down. So right now we can start building on top of that and, while I think is maybe creating a all encompassing, you know, foundational model, which run your cd, maybe it's kind of further, further down the road, but in local use cases where you create a, you know, physical intelligence for I don't know, for the industrial process physical intelligence for a particular industrial piece of equipment, which would consist of multiple, multiple pieces together.

Or distribution center or logistics like supply chain in home use cases in, you know, more kind consumer use cases like smart home to the degree. Right? So it's, it's all this piece of stuff where you car can talk to your garage, why it doesn't happen right now. Right? You know, everything's on siloed, right?

So like, when comes to bring them together and create this kind of fully automated workflows around your house. I think it's pretty close. I would it's, it's, it's just a question of doing it. 

Richie Cotton: Okay. I like the idea of like you know, the, the fu the future's almost here. And I guess, yeah, certainly having agents talk to each other, that's been a big story on the sort of generative AI side in this last year.

And so yeah, I guess having machines talk to each other, that's maybe the next thing coming soon. Yeah. Hundred percent. Okay. So you mentioned the idea of a foundation model for physical ai. Now the big story of AI in the last few years, it's been large language models. So how how is a physical AI foundation model is, is it related to the idea of a foundation model for text?

Ivan Poupyrev: So I think the foundational piece is transformers, right? Is a piece of neural network design, which is. Which allows you basically train models on much larger body of data and yet be efficient during the inference time so you can actually build useful applications, right? So this kind of architecture is broadly used to build this new class as foundation models.

And the biggest difference, one of the key clear difference between LMS and physical world. Physical AI is a structure of the data. Large language model is mostly text. Maybe % images, % or so. And then everything else is, is much, has much smaller percentage of the data being used. When you go to physical ai.

The structure is very different. At the inference side of the use cases side, majority of the data about % give and take is measurements. So if you have a the oil rig there is, you know, gigabytes and gigabytes of, of data being generated every minute, right? So there is some data. We shown, I think, petabytes of measurements being generated in the physical world every every day or something like that.

It's this incredible in credible amount of data is being generated, mostly measurement, %. Then the next most popular data stream in the physical world is video. So about % is video and then Texas less than %, right? So physical world talks in, in the language of sensors, you know and the measurement will happen in physical world.

So training the model on sensor data requires different architecture to build those models. So you, because he would like to have a sense a physical models which can capture. The behaviors in the physical world based on measurement. So you're using with time series data, using as the piece you don't use necessarily text.

So that's biggest difference in terms of training between LMS and physical world data, if you want to get performance and the performance in physical AI models is a multi multiple. Benchmarks and benchmark is from foundational model is remains, is a big challenge. As as, as we know, there's lot of benchmarks with alls, but in physical ai.

The performance of the model have to be physically accurate, right? Like if you're generating a video of, of the, the, the, the bear on a, on, on, on a cycle on, on a, on a bicycle, you know, jabbing jabbing ice creams in the circus, you know, physical accuracy, probably not important, but if you joining control signals for nuclear power station.

To prevent, you know, emergency shutdown, you know, or, or in change ality you don't want to be physically accurate, right? So, so these are very different types of accuracy in this different type population you, you are looking for. So the models cannot just rely on text data for training. It has to be grounded in the physical world rather than the physical measurement so that it can do accurate.

Accurate controls and accurate insights and recommendations. 

Richie Cotton: Absolutely. I mean, certainly to go back to your nuclear power station example you really don't want hallucinations happy there. Like it's invented a problem of the station or, or worse, it's missed it. So I, I, I can certainly see how accuracy is important, but you mentioned that a lot of the data is time series data.

I, this is like sensor taking measurements. Over time you wanna see how things change. We've had time series analysis in it for, I mean, decades now. It is a fairly mature field. So how, how are these models different than to traditional time series analysis? 

Ivan Poupyrev: I think the model time series is, model itself is, requires a little bit of, of clarifications.

Time series is a format, right? This is how you store data. So you're basically array of the data in, in, in from just like, you know, images are stored in jpeg or gif, GIF images. And when we talk about. Analyzing visual data, we usually mean analyzing the content of the gif or or concept of conduct of a JPEG file, not necessarily jpeg file itself, right, or format.

So time, sly time series data. A lot of time series data analysis is, is basically grounded in financial world data. That's where a lot of techniques went there. And they work, you know, sufficiently well for financial raw data, right? But again, so time C is just a format if it's filled with the actual measurement for physical world, we found that the classic techniques statistically based techniques and other techniques they don't, they don't necessarily work as well for physical, physical sensor data.

And in particular, what it, where it fails it fail on generalizing, measurements, for example, across different physical conditions or across different types of sensors or different types of events, right? So in physical world literally the same kinds of sensors, I say measure in temperature produced by different types of suppliers, would have a different data coming out.

How can you make sure that the model you don't have to build individual model for every type of sensor, temperature sensor, which made by every, every, every, every supplier. Or if you take the same sensor in slightly mood in your device, in locations, the whole day measurement, main measurement changes.

So how you can make sure that the models can generalize and, and remain, remain reliable across kinds of sensors and across specific deployment of the sensors. Without having to retrain those models over and over and over again. Right. So the biggest people will, building people will building models.

I, we ourself, our team with IPI, we've been building models for sensor data for a long time. You know, I, I, I, I was established my career back at Sony and we will building, you know, like the sensor systems for, you know, PlayStation portables and, you know, all the devices. We we're very familiar. The problem there is.

They very fragile, brittle, and they very have to be, they have to be trained specifically for this sensor of a specific device, which is extremely expensive. Everything is like handmade and everything's kind of particularly trained with the devices. The beauty about this models is that can they generalize?

The one model can read any sensor in in a final conditions any sense, in any locations. And what we discovered with our model with Newton, which we're building. It can even generalize to different physical phenomena it hasn't been trained on. So we have that very, we have a demo like an office where you have a mechanical pendulum measured by the accelerometer, and the model is capable to predict behavior of this accelerometer.

Even. It's never seen this accelerometer before. Not only accelerometer, but also pendulum. So it's completely capability to generalize in physical behaviors, which never seen before, because these models they're not just simply predicting or, or. Simple, like trying to find the peaks and valleys in the temperature signals.

What they're trying to do is to build the world model of the measurement of the physical world. It said that's a difference. It's a representation of physical world and not just a analysis of particular measurements and going up and going down and what's gonna happen next. And with particularly important, I think what happens is that what I, when, when you have this kind of the model, a single model, which can takes all this, multiple sensors and a single presentation.

Then treats them together, is that now you can approach such problems like sensor fusion. So classic techniques have really always had trouble and, and struggled with fusion, multiple types of sensors. And what we working on now with, with, in our use case and applications, we working with our customers.

A typical, typical challenge is that you have a piece of machinery or equipment or process which measured by hundreds of sensors. And they would like to detect conditions where neither one of the sensors can detect it. It's only by fusion, either because the the effects is so small. So the sensors, individual sensor cannot capture the, those, those conditions.

A failure, for example, or surgeon's power or some sort of a very light solutions, which are very important 'cause it could be predictable to some failure in the future, or un un, you know, or unusual behavior. A single measurement cannot capture them. It's combination of all of them. We can capture those conditions.

We have a, we have a, just a simple example. We have a customer who was, who was, who was building control software for the windmills in electrical windowsills. And they collect maybe like measurements in real time for every windmill. And, and it's really, really important to keep this windmill running because if it shuts down, windmill shuts down.

Then they have to buy the electricity in the secondary market, which is at spot market and sell at much higher prices back to back to their, back, to their customers. And the whole idea of the windmill, you have a green energy, right? If you go and buy energy now from fossil fuels, kind of defeat the whole, the whole purpose of, of, of the green energy.

So for them measuring and completely predicting any potential failure is very important. And one of the conditions they have, like for example, which model our model was able to detect is that with the speed of, of the windmill. Of the the generator is increasing because the wheel is increasing, but the power dips individually.

Nothing wrong with this. Like if you have a power, if you, if you know there's this electrical machine that it's faster, you get ob obviously increasing speed. But the combination of the two is very unusual, right? Like, why, why, why would the speed increase or the power would be dipping? Clearly there's something, something wrong, right?

So. And model will be able to detect those kind of inconsistencies and behaviors because it has a world model of, of behavioral window would be able to detect it directly and, and without training, or without, without prompting, without structure. So that's very exciting. 

Richie Cotton: That's very cool. The ability to sort of generalize and be able to make predictions about, I guess about sensors it's not seen before and all that kind of stuff. That seems to be I guess the secret source here. So I know with things like computer vision, a, a lot of the models, they kind of work okay after the box, but then if you want like an industrial use case, you then gotta fine tune it and like show it video of like your own product or whatever.

Is that the same thing with these physical AI use cases? Or, or does it just kind of work straight out the box? 

Ivan Poupyrev: Yeah, I, I wanna talk about, we, we talk about our models, right? There is a physical, AI is expanding and. Multiple types of different variations. In our case, we really focused on the we also work with cameras, obviously in infusion between measurements and cameras is very important.

So when it comes to Vision v vision fine tuning is important indeed because particularly in industrial equip, in industrial use cases, you just don't have a data set to represent some of the, you know, piece of equipment people dealing with there, right? So like you have to teach the model. The model may understand.

Basics of human behavior. For example, you take piece A and piece two, connect them together and, and screw them together, but you won't be able to tell the, what kind of parts is that right? So some, some is important when it's kind of the physical, actually measurement models, the model, which will discuss temporal world model for the for, for the measurements of, of equipment.

We see that our model zero show generalized really, really well. And again, so exact variation of what it does mean. We, we work with this right now, it's all early work. Means not that early, but it's a you know, rapidly developing field. But what we see there out of the box, the model is capable to analyze data across the sensors, multiple sensors it hasn't been trained on.

And in recent examples when we've talked to customers customers will give us a raise of data. We don't even know what the data is. You know, it's like the, the, we don't even ask what sensors they are or what the measure we just we're looking at that data is, you know, the structure of the data is correct, this is formatted correctly.

And then the more, and then we provide a couple of examples of the data of, of use case of, of where the data was, was broken. This, okay, this measurement represents like failure one. This represent represent failure two. The model can go and without any of the training, zero should be able to find.

Those, those fillers across sample data they will provide. Right? So no, no training, no understanding what sensors are, and dude, in many cases doing better than specialized models which being built by hand. So, so this capability of generalization is really, really exciting. So, you know, philosophically you argue, you can ask yourself, does that mean that the model understand physics?

And what does it mean to understand physics? You know, then we, we often get this sort of. Philosophical debate. But practically it, it's, it's, it works out of the box now, you know, it's, it's there, there is a customers when, when things work out of the box, customers come with us for like really hard use cases, like some something which is really, really, really difficult.

They lot of noise or. The subtle, really subtle changes or data is missing. You know, like, can you solve those use cases? Because that's where like, and, and in this case, usually a small amount of fine tuning actually can solve a lot of those use cases. So this is, this is sort of outlines how it can work.

Richie Cotton: Okay. Alright. So it seems like you can get some kind of value straight away and if it's a difficult problem, then maybe you need to put in extra work. So actually on that note, I remember when a large language model sort of first hit the scene, it was like, it was not often clear what was gonna work well and what wasn't.

So you'd have some use case, might think, oh, maybe this is kind of tricky. And then. Simple prompted. Then other cases you think, oh, this can be easy and it gets stupid answers. So I'm curious what the equivalent is with physical ai. Are there are some classes of problems where it's works really well now and others where like, like what doesn't work yet?

Ivan Poupyrev: Yeah, so we the class of physical ai models we are building is both, is built and is not like for robots, right? So we could look in the broader field of, of, of use cases and so the, I think the biggest challenge in terms of research, right? What we're trying, trying to understand is the explainability, right?

So the model, I mean, as I, me exactly you mentioned is like one of the challenges, right? So the, as, as you, as you mentioned is, is that the model works quite well out of the books, but how can we be able to. Anticipate and which use case is gonna work better, in which case is going, going, gonna worse, right?

So like be able to, to kind of like be able to predict how well the model use particular kinds of data, particular kind of processes. Is, is interesting interactions. We, we, we explore. Right now it's, it's just like with lambs, a lot of that's a little bit of a there's in cool sort amount of trial and error, right?

So explainability and, and observability of the model. Understanding why works this way is something we are really deeply investing right now and essentially quite, quite, yeah, it's very interesting, very interaction because, you know, we, we, has to do a lot of IDing multidimensional mathematics and, and basically you convert it.

If you look at this high level, you're actually, you convert a time series measurement signal of measurements. You combine them together and map it into the multidimensional generic space. And now you, you treat measurement as a neuro genetic problem. So a lot of interesting work there for one of the example is, is explainability of, of results of the model.

Richie Cotton: Absolutely. I mean, I think as models has got. More complicated to explain. IES has been like an ongoing challenge. Like you get things like well, logistic regression decision trees, super easy to explain and then suddenly you, you start getting into neural networks and things like yeah, absolutely.

Ivan Poupyrev: Well, I think, I think we are finding, finding everything, there's all the work, which, which done for example, on visualization of embedding space, right? So you have basically, you, you map all live in embedding space and then you can. Visualize how data flows through embedding space and you know, but in many spaces, much bigger.

It's like a multi economic, you know, in our cases I think it's a hundred dimensions. So like how we can take the a hundred dimensional data points moving aha dimensional space and map it into something you can see. So this is this is all very interesting, interesting efforts. 'cause you have discovering the, this, how this math works actually.

Richie Cotton: Okay. So you mentioned like dimensions for the embedding space. I think a lot of the sort of text model to use a similar sort of size of embedding space, so I guess, 

Ivan Poupyrev: yeah. Text models must be immediate, like, you know, of thousands of dimensions like physical, like in our case physics model, which would build, which kind have like quote unquote understand physics.

It's actually surprisingly impact. We, what we discovered is that if you capture measure, if you measure the physical world through the measurement. You try to encode it into the into this embedding space neural network. The presentation is quite compact. It actually can run in our cases, it can, our models can like, you know, run on things like raspberry pi, for example.

Not very efficiently, not very fast, but can run on raspberry pie. So this is actually very, really important re very important requirement for physical ai. It does have to run on the embedded hardware or like on the prem, on the physical hardware like OMS run on the cloud. You know, you can send it, you can build a cluster and data center somewhere in, in, in any way you like.

Right? So, and then you can run this massive load on these data centers. But when you go to physical ai, if you have a goal, you have to work with equipment. It has to run on that asset, right? So we fundamentally, we and. When we started the company, we kind of look like thinking like, wow, you know, you run everything in the cloud, everything becomes so easy for everybody who's gonna do this stuff.

And fundamentally, like with most of our customers right now, it must run on the assets. So your model, you, you train your model and you kind model performance on the cloud. But deployment has to happen on the, on the physical asset. And the physical assets can be anything from a private data center to.

A a small one, single like GPU standing next to machine, to laptop, to a small ability of devices. This is a range of devices you have to be able to run your model on, right? So it's more than just build the model and, you know, or does results. Well, how deploy it is in a physical world is one of the key challenges.

Richie Cotton: That's very interesting. I think at least in the generative AI space. The model has been getting kind bigger and bigger and bigger and using more and more compute. So the fact that you've got these constraints in the A space is, is very interesting. Okay, so I, I'd love to talk about how you get started.

So maybe let's go with like what happens with the individuals first, and we can talk about companies afterwards, but if you are an individual who's interested in physical ai, like how do you, like, what do you need to learn? What skills do you need to make use of it? 

Ivan Poupyrev: This, there's two directions you can go from, right?

One direction is like a lot of people who broken physically are coming from the hardware world, right? So people who build hardware. So like we ourself you know, my, my, my personal path. I started by working in the consumer electronic industry. My, my first job with Sony in Japan, actually being consumer training devices.

So here you have a, a physical piece of hardware and a sensor, and you need to use a sensor to figure out what's happening around this device. So that device has to react appropriately, like, you know, whether it's interaction or temperature changes or whatever, like environmental data you need to do to control the device.

So like, you know, s of devices like that. You know, particularly Sony tv, you know, it's, it's like a lot of, a lot of works happening there. So a lot of, a lot of starting point for a lot of people. But like hardware could be interesting, a good starting point from that. Like, you know, getting a simple hardware kits, collecting the data from sensor data and then using a online physical physics physics model to analyze it and try it out.

Right. So. This is, this is this is probably one of the, one of the easy steps for like, for the actual where it's supposed to leave, right? So like, because you build your hardware and you're using app for the hardware. But overall, I think the physical ai, I don't, I don't think right now there is a clear path, like, except, so a lot of hackathons, for example, in San Francisco, they're based on building hardware with the sensors and trying to apply the models to those sensors.

The other approach is, is classic data analytics, right? So you, you, you run the data on the, on the cloud and using a existing models to analyze them. A lot of people trying to use chat GPT and, and chat and large language models to analysis or the starting point. It's really, really interesting result they can get.

But again, the physical accuracy of result is not necessarily granted. It can do maybe basic descriptions of the signals, but, you know, making sure that they're physically accurate is sometimes it's, it's, it's difficult to get this from us. 

Richie Cotton: I do like the idea of just like building something small and fun yourself with some sensors in it and then going away and encouraging to numbers.

And I guess, yeah, you can start with like traditional time series analysis. Then you, you can move to the the physical AI models or maybe even the other way around. I guess just dive straight in with the, with the physical foundation model. Okay. And then for organizations who are interested in this are there any sort of I guess it's mostly gonna be either industrial organizations or, or, or perhaps if you, if you're running a city or something like that then yeah.

What, what's a good first project? Like how do you get started with this? 

Ivan Poupyrev: I think the, the first, the best point of start for any organization is to actually talk to physical AI companies, right? Because if you look at the physical companies on physical ai. Most of the models right now are not necessarily either, you know, open source or you cannot really access them right now in, in easy way.

So talking to conducting a physical AI company like ours, for example, is the most straightforward way to, to do that. And, you know, we have, we, we work with multiple companies going, you know, ranging from manufacturing to energy to. Communication companies who all come in very similar problems that have a large amount of data where there are a combination between measurement and video usually, and some sort of language, and they would like to put them together to explain recommend and automate some sort of environment.

Right? So starting we're working with us is the, is the, is the fastest way to get going with physically ai. 

Richie Cotton: Okay. And is there a sort of hello world equivalent project for physical AI with, with companies? Like what's a good sort of easy place to start? 

Ivan Poupyrev: So, four Hazard, hello World is probably just running your data through the model and figuring out if, if the model can zero shot, find things you are looking for.

Like if you have a failure data and you want to see if the model can find this meal affiliated without you training the model, that's probably the first step that you know. Take the data, take a sample of things you want to discover, see and see if the model can, can find those things without you having to update a fine train model.

Because again, so the biggest point of this physical AI models is capability to generalize. So in theory, just like charge GPT can, you can ask pretty much any question in any language and it can be provide an answer. You may not look the answer, your wife a better answer, but it will answer. So I think same physical animal should be able to answer almost any question about your physical world situation without any just out of the box.

So that's probably the first, first step. 

Richie Cotton: Okay. And I suppose. Every business has sensors somewhere, even if it's like in your building heating system or something like that. I'm sure there's gonna be some data sets around that anyone can find. 

Ivan Poupyrev: Well, yeah, I mean, of course. I mean, but your phone produces so much sensor data.

You know, the easiest one is like basically stream accelerometer on the out, out of the phone, right. Every physical object they has, like every physical, physical device has a bunch of sensors. 

Richie Cotton: Okay. Actually that sounds like quite a fun thing, is just seeing what's my accelerometer doing on my phone? I have no idea how to access that data though.

Is there an easy way to get the streamed? Yeah, 

Ivan Poupyrev: there's open api. I access very, very easy doing politically today with the wide coding. You know, writing applications to get the data outta the form is relatively simple. They have acceleron, they have have, they have all the APIs to access acceleron data.

But more than that. Cameras and temperature sensors and phone is full of sensor data like barometers and, you know, and everything's there. So you can, you can get a lot of data of the phone and which can tell, you know, for you, for example, you know, like like how, how much you walking it can be, or how active you are or, or is it doing something unusual today?

So there's a lot of work been done early in, in, in, in my early career, we've done a lot of work like. Health and wellness data from the phone. To detect it straight up over the phone. 

Richie Cotton: Okay. Yeah, certainly. I'm not sure I necessarily want to know the answers to that question some days, but it still sounds like a fun project.

So I'm, I'm curious as to what the sort of the next thing is. It seems like physical AI is sort of on the verge of going mainstream. So I guess how close are we to that happening and, and what are the consequences of that? 

Ivan Poupyrev: It's a very good question. So the, it's, it's, we all can kind of like predict what's going, try to predict what's gonna happen.

I think these physical AI will, will get much more accessible ones various companies that making their models accessible to. Everybody in, in easy way to accept the basis, right? So that's where I think where everybody can go, either download the model on their laptop or accesses online. So I'm sure every, every single physical AI company, including ours, like working really hard to make it as accessible as possible, as complete as possible that at that stage it's gonna be people will be able to start analyzing data, which they were never able to be able to analyze it, don't even know where they exist.

To be able then to connect all the devices and truly bring intelligence in, in the, into their either industrial solutions or, you know personal solutions. There's multiple vectors where this can go, right? But one of the vector is that today our know charge PTs and, and, and the lambs of the world, you can ask them almost about anything which happens in the past, right?

It will not be able to ask what's happening now here. I cannot I cannot ask for GPT, you know what I'm doing right now? 'cause the, the mobile is not connected. It doesn't know what I'm doing. It doesn't know what I'm doing yesterday. It doesn't, it can recommend me what to do next. Right? So one of the visions of the physical AI is to build a real physical assistance, right?

Assistance in, in a simple use case, real physical assistance and agents, which can be. Deployed in a physical world and real physical system helping you because it's, it'll be connected to your, to, to your, to your, to your senses, to, to the world. They have some sort of knowledge about what's happening in your environment, and then be able to again, to either explain to you and tell you what happens or recommend what happens and then sell out automating.

So basically, I think this robotic robotic. Life and robotic robotic homes, robotic devices is much closer than, than we think. And potentially it could be done by individual customers. 

Richie Cotton: Absolutely. I mean, there's a, a fascinating sort of vision of the future where you've got AI that has context about like your surroundings and your, and just knows what's going on in your life because of the sensor data, and then can make additional recommendations on you.

Ivan Poupyrev: Yeah. So combination of the power of the power of knowledge can provide by all lamps. Combination with your, your current current currency patients. Combine the two, the language semantic understanding in the physical world and actual physical accurate measurements combined it together will give you kind of AI situated with you in physical world and be able to truly be like a real assistant which can help you and predict and anticipate your needs before you actually even know they have them.

So. And that's sort of like a lot of, a lot of vision if you think about, about AI is, is, is sort of based around that being, being helpful in the physical world. The real next thing. 

Richie Cotton: Okay. Yeah, certainly it's it's a very compelling vision of, of what could come next. Although actually I wonder are there any sort of privacy concerns around that?

Like if the air's got sort of awareness of I guess what's going on in your life in that sort of, I more intimate way with with the sensors? Are there any problems from that? 

Ivan Poupyrev: I think the privacy concerns there's a, I mean, we. A very short answer. The physically I'll be running probably on local devices.

So the data don't necessarily need to leave your environment and you should be able to completely control, control your data. Right. So the that's sort of, I think the most privacy secure way. So like, that's why I was saying before is that physically AI models have to be local. They have to run at the end on the on the edge devices because.

The data have to stay with you. This is what we discovered talking to. That's exactly the reason I mentioned before, is that when we start talking to our customers did not want the data leave the premises of their, their industry. So I would wanna talk maybe like less about privacy. I will talk about the data sovereignty.

The data sovereignty is probably the way to look at that because the data, you, physical data, which you do is. Kind of a core describing who you are and what, what happening to you, right? So you have to have a complete sovereign control of that data and you decide whether it's go somewhere and, and which way it goes and how, how it goes.

But the basic starting point is sovereign control, the data. And that's sort of the way we build our model to build our in case of our physical ai foundation model. And the whole platform to run the model and provide the data is very strongly based on. Concept of sovereign control over data by the people who run, run the user L model.

Richie Cotton: Absolutely. I mean, and it seems like sovereign AI's been sort of taken off as this thing about like which countries you are data center in, but actually there's a lot deeper sort of thinking needed around. 

Ivan Poupyrev: Yeah, exactly. So people thinking about the level of the countries, you know, but I would say like it should be on the level of individual.

So every. You are sovereign control. Have a sovereign control around your data, and AI is, is yours, right? It runs on your devices and it runs on your hardware. And if you don't want to share with anybody, you don't have to share with anybody. And then I think the way you build that goes back to Sation before, is that the difference between LMS and and s in, in physical ai LMS based on basically kind of vacuum all the data from the internet, right?

And everybody contributed to that data. In kind of believe that by giving data to the lms everybody's benefits, right? So this sort of approach but in case of physically ai this model kind of doesn't work. Doesn't fat falling apart, because if the data, you don't necessarily want to share your personal data with everybody else.

And, the way you build the platform to deploy the models has to take it as an account. So you cannot simply build everything in the cloud and then say, Hey, download all of the physical data there and we will tell you what's happening. Probably it's not gonna happen, it's not working right now for us, for sure.

Even in industrial use cases. Everybody wants to keep data on their own side and not to share at all. So I think the way you build, and that's why I think like the way we build the platform and the way we like build the, build the whole, the product which provides access to physical AI has to allow to build on the cloud, but deploy.

The end result on the edge, and so that customer can keep the data on the edge. 

Richie Cotton: Okay. I do think that's incredibly important part, particularly when it comes to things like you mentioned like a healthcare sort of tracker data. So if you've got like medical data, you, you don't share that with everyone.

Ivan Poupyrev: Yeah. And I personally will completely understand that. But if you come, if you talk to. Any industrial company would, would, would be hard pressed, would not ever share the industrial data outside of their prints. So you have to run the infrastructure. 

Richie Cotton: Okay. Alright. I like the privacy is is being thought of like front and center there.

Wonderful. Alright, so just to wrap up, I always want more people to learn from, so can you tell me whose work are you most interested in right now? 

Ivan Poupyrev: Actually, we, we very interested in, going back to basics. I'm rereading the call Shannon Information Theory right now because you're kind of looking through the all these new models you cannot like start looking at them kind like we are, we are looking them again from the all, all perspective, like how information theory applies to that.

Like how can I apply all these techniques, foundations to this need to new tools. So it's more like revisiting from now for me for now and looking Okay, well everything, how apply to this new, new world of ai, 

Richie Cotton: you know like Claude is one of the most popular sort of Ai like LM models at the moment.

So, yeah. Alright, super. Thank you so much for your time, Ivan. 

Ivan Poupyrev: Thank you. Appreciate it.