Edge AI with Derek Collison & Justyna Bak, CEO & VP of Marketing at Synadia

Richie Cotton: Hi, Derek and Justyna, welcome to the show.

Derek Collison: Thanks, Richie, for having us.

Justyna Bak: Thank you.

Richie Cotton: So, just to begin with, can you talk me through what's the difference between edge computing and more traditional computing?

Derek Collison: Well, I think it's a great question. And I think the reason that Synadia exists today is, for those that were around, we went through a pretty massive transformation of how we did things on premise, or in our own data centers, to cloud computing.

And what Synadia believes in is that the transition to edge computing will be as big, if not bigger, and will happen a lot faster than the change to cloud computing. most folks realize that whether we knew it at the very beginning, Now there's a very different set of rules on how you build, let's say, cloud native applications.

We believe the same transition is going to happen for edge native, applications and systems.

Richie Cotton: Okay, so it sounds like the next big thing is moving from cloud computing to edge computing. But can you talk me through what are the benefits of it? Why would you want edge computing?

Derek Collison: Well, I mean, I think the folks at Sanadia and Justine especially know this about me, I try to simplify things to make me understand them better. But one of our north stars, believe it or not, is the assumption that people are going to try to decrease latency to access distributed technology, whether it be services or data.

And that's just always been the case. And so we went from data centers to cloud and then cloud in multi, geos, CDN providers, you know, this notion of what we call far edge, you know, nearest the Cloud providers trying to hold on for dear life. I call it their blockbuster moment.

But the far are kind of like the Akamai's and the Fastly's and the Cloud players and Netlify's and, of course, Purcell and Dino deploys of the world. But what you're starting to see, and I think we started to see this right about when we started Cinedia, and that was one of our big best was, is that it won't stop there.

They'll keep pushing into, their vehicles, their manufacturing systems, factories, distribution centers, medical devices, whatever that is. And again, it's just this major driver to decrease latency to access either data or a service. And for us, it's kind of a combination of both.

Richie Cotton: Okay, so it seems like the big goal is reducing latency, so you've got your compute happening near where it needs to be used, but also companies don't want the hassle of sort of managing their own infrastructure as much, which was the original benefit of cloud, I guess. So is it a sort of best of both worlds situation there?

Derek Collison: Well, it's definitely both compute but also data. So, for example, if you have compute running locally, let's say inside of a vehicle, but it still needs to trombone back and forth, right, to get access to data, you could imagine a world where moving the data closer as well would make a lot of sense.

Richie Cotton: Okay, so compute and data together, that makes sense. Maybe we'll try and make this a bit more concrete, talk about like who's actually making use of edge computing. Justyna, can you talk me through some of the use cases? Sure.

Justyna Bak: Well, Edge Computing enables real time decision making, so it really unlocks the value hidden in the data. every industry and every use case that relies on real time insights into how the business is doing will benefit from edge computing. And so some of the examples that we've seen today could be industrial manufacturing, where you'll have sometimes hundreds of sensors monitoring different machines on the assembly line, monitoring the levels of vibration, the temperature.

If you can process the data, As soon as it's generated and you can identify anomaly. Maybe one of the machines is getting overheated. The moment you can act on this information, and that can save you money because you will keep your assembly line going. it can save you revenue because you keep producing, things that whatever your factory is producing, whether it's vehicles, whether it's other things. Um, also keeping things efficient. And by the time you would have to send this data into cloud, it would be inefficient because it would introduce a delay in the time when you can act on this data. It will be very expensive because this data will consume a lot of bandwidth. And in some cases, the data should not even leave the premises because traversing the regions would put you against the compliance regulations.

Derek Collison: To add to what Justine is saying a little bit more concretely in terms of the verticals, right, manufacturing, we really believe manufacturing is going through a renaissance right now. Not only just decreasing latency, but autonomous behavior, East west, you know, anomaly detection type stuff, meaning, hey, this part looks weird in our factory and we have 80 of these factories worldwide.

Does it look weird in your factory as well? The whole notion of, a vehicle becoming a technology platform that just happens to have wheels, so the connected car is a big one. We're also seeing it in kind of a revitalization of what we call the physical store experience. So a lot of people went to pure digital online, obviously with with Amazon, but Walmart, went to keep pace.

And I think what we've seen as we both entered into COVID as kind of a forcing function, which we can talk a little bit more about why we think that was, but exiting out of COVID of, how do we redefine the experience when someone's in a store? And you can imagine, we need lots of access to information at a very, very quick rate, right?

And to Justina's point, if for some reason the cell connection from our store to the cloud is down or our star link or whatever that might be, we want the store to have a certain level of autonomous behavior. And then the last example, believe it or not, is We're seeing a massive explosion in how people want to enhance the fan experience.

So think of live sporting events events like racing and things like that, where they want to immerse the fan more with what the drivers are experiencing, the data that's coming off of their car, the data that's coming off of other cars, the track. And you can imagine a world where, again, to Justina's point, if we were tromboning either to get the data to the cloud and then go to the cloud to get data back, right, to ask a question or to you know, enhance some experience either on their phone or whatever that is people are going to want to cut that out.

And what they found was is that you can't just forklift an app from the cloud to this edge, That that is a very different world. Like we talked about earlier on in the podcast when we went from our server room to a data center the cloud.

Richie Cotton: I would say I really like Justine's example of the manufacturing thing. I can certainly see how you don't want to wait until, like, shove things into a cloud and wait for some batch process to run. And then you get the answer the next day. You want to know when there's a problem immediately. And Derek, to add to your point, it seems like most of the examples where you want edge computing, it's like basically anywhere you've got Sensors.

Is that about accurate?

Derek Collison: think more broadly, and I'll let Justina go in as well, is there's A couple of things happening at the edge, right? The first thing is, is most of the data interacting with your, your customers or your partners or whatever is happening at the edge. So, you could easily reason that, taking it, moving it to the cloud just to bring it right back wouldn't make a lot of sense.

 Now, that being said, a lot of these AI inference models are still trained in the cloud, And I don't think that's going to change for quite some time. And so, Do you want the applications have to worry about the condition of the cell phone, you know, network to get the data to there? Can they just say, Hey, some text that here's the data.

Make sure it gets over there for training. And by the way, whenever you update a model, make sure I get that and then anything that the model needs. So, for example, prompt augmentation agentic workflows using multiple models. Make sure I know where those things are. you know, in a location independent way, which we can talk about and decrease the latency as fast as possible.

And again, you're even seeing some of newer models. I know we're, we're waiting a little bit into AI at the edge, but there's always going to be this massive scrutiny on how long did that take? What was the latency for me to get the first byte back or the first response back? Or in AI, the first meaningful context that I said, aha, it's answering my question.

Richie Cotton: So it seems like latency is the big driver then. So how long can you wait for an answer? Maybe can you give me some examples of like what a, an appropriate latency the different use cases you talked about?

Derek Collison: Well, it's, it's, you know, this is really interesting. And so, you know, way back when, you know, in the early 2000s I spent time at Google. I worked at Google. And Google's consumer was the person. And for the most part, the human brain switch context between 160, 200 milliseconds, depending on how fast your, your brain is oscillating, right?

And so Google had a very, very hard and fast rule that, Once a request came in, let's say a Google search query, it had to be back out the door in under 200 milliseconds, or else you didn't turn on the service. And so, anytime you're dealing with a person, most people generalize around those types of numbers, how much processing do we have to do to be able to get something so that the person, let's say me, doesn't context switch and like flip to something else.

Now, when you flip to other types of machinery and things like that, whether it's manufacturing or connected car, you could pick examples where that number needs to be a lot, lot lower. Should I be able to change a lane? You're not going to say, let me go ask an LLM in the cloud whether or not I should do that type stuff.

And so it's, it's semantically relevant what you're trying to do. Humans is about 160 to 200. But we're starting to see some of these processes within manufacturing and again, like connected car, things like that, that are getting very, very low. and you could imagine, again, anything that's actually physically controlling.

What the factory or what the car is actually doing, they have very, very strict tolerances around around latency and how fast they can respond all the way down, obviously, to the hardware and operating system, things like that,

Richie Cotton: Okay, so that's really interesting that people get bored after 0. 2 of a second, they'll continue to switch and go and do something else, but yeah, I can certainly see how that's even like way too slow in the context of a self driving car. Okay, let's try and figure out how we go about doing this. So you've got your cloud application and suppose you see it goes, right, let's migrate it to the edge.

What do you do? Like, where do you begin? So first of all, it all depends what's your destination. Because you're going from cloud to edge and your edge is a modern industrial manufacturing assembly line. You will have hundreds of sensors.

Justyna Bak: You will have rather stable connectivity. So it's almost like a small data center at the edge. Now things will be dramatically different when you go to a remote Oil plants. and you know, data is the new oil. So thinking about that and a remote rig where conditions are really wide, way harsher than in a modern factory because connectivity can be intermittent.

Latency is definitely going to be higher. Compute may even be limited. So an application that would thrive in the cloud, where all these resources are unlimited and always on, will definitely not do so well at the edge. So we need to fundamentally rethink. How we building applications who can thrive in the harshest of conditions at the edge.

And these applications, they have to be by design able to operate in offline scenarios because some activity can go down and you still have to be saving data. Analyzing data and using it for the critical decisions keeping that remote all operation up and running and making things safe for the folks who are there on site.

But at the same time the application needs to keep that data, even if the connectivity goes down, that the database in the cloud is not gonna help us in any way if we're not connected to the cloud. application has to be resilient. It has to be self-healing, handle these conditions.

And so the, the vision that we have at Syndia for enabling that future generation of applications is truly nomadic apps. The ones who can really thrive in the harshest of conditions. they don't have external dependencies. Everything they need in order to operate is captured in a small binary under 20 megabytes.

And they are fully resilient and fully ready to operate in offline scenarios.

Derek Collison: Justina's excellent response there. I think one of the reasons Sanity was very misunderstood the very beginning is because We, meaning Cinedi, decided to attack the problem in a very different way. Traditionally, you just said, okay, I've got an application, how do I get it to the edge?

Meaning that that's the first thing you, focus on. And by the way, in a, in a previous lifetime, I designed systems the exact same way. Focus on the workload first, how do you get it running somewhere else? Then you figure out, oh, it needs data, then it needs a network to connect it to the data. then you need to secure it, And so you see this workload, data, network security type of a pipeline. And so things that I've designed in the past, like platforms as a service, but Kubernetes, OpenStack, all did it that way. What we did was to say, to get to that end point where the application can truly run in any region, any cloud provider, all the way out into, let's say a connected car, type stuff.

 We started with the connectivity layer first. So we said, if this connectivity layer is intelligent, meaning everything is location independent, I don't need to know where you are. I can get access to you securely, because the network is secured at the very, very fundamental pillar. Then we move to data based on intelligent connectivity.

Things can move and ebb and flow in a secure way and be very location independent. And then at the very end, we care about the workload. And so at the very beginning of Snadia's lifetime, we were very misunderstood because we were taking a pretty much a totally reverse upside down path to the problem.

But to get to Justina's point around truly nomadic applications, you can't start with the application, right? You have to start with the connectivity, the networking, security, data layers before. And then in addition, there's a lot of things that make deploying things in the cloud on current architectures very easy because they're all right there.

They're like a little click away. And all of a sudden when you're inside of a, a retail store or a cafe or a manufacturing plant, it's like, oh, we don't have one click load balancers and GSLBs and service discoveries and next thing you know, you're, you're not just replicating your nomadic application or the one you want to be nomadic.

That might be a pretty simple application. It's like, Oh, wait a minute. When we were running that in the cloud, we had security, we had, you know, networking, we had VPCs, we had data services, we had load balancers, we had API gateways, we had all of this stuff to get around, at least in our opinion. the fundamental limitations of the current architectures, which mostly is around the connective pillar, which means everything in the current state of the art is location dependent.

I have to know where you are, and everything is a one to one request or apply. So anything based on HTTP, gRPC, all of these types of different protocols. And so we said, hey, how do we change that? such that fundamentally unlocks so many different opportunities as we go up the stack to data and to workloads.

Richie Cotton: That's absolutely fascinating. And it does sound like there are a lot of things you need to have in place before you can successfully create this, these applications. So you have all these worries about low connectivity, low resource requirement, compute capability, things like that. And you have all the, you need to put all this infrastructure in place.

So how much of this is going to be like hardware infrastructure you need to sort out, how do you need to change your own applications before you get going? And, you know, you can get access to this. Robust connectivity and things like that. What do you need to do before you can start building these, edge applications?

Derek Collison: Well, I think the important part for the audience to understand is, is that let's say we're three architects in a company, right? And we're in front of a whiteboard designing an application or designing a service. What we're going to be drawing and what we're going to be talking about in the modern world that Synedia sees is actually not changing.

We're going to be talking about microservices and access to relational stores and key value and object stores and streaming data and events and things like that, right? So the boxes and the circles and the lines, all are going to look the same. Where Sneddy comes in is the how. The how is radically different, almost like going from a petrol car to a Tesla.

Still has a gas pedal, Still has a steering wheel type stuff. And that's because that's what people are used to. So we're fundamentally not changing the what, but we are changing the how. And so we start at the connectivity layer and then the data layer. But for the audience to give them a little bit of, groundedness in what we're talking about.

Imagine a world where there's something inside of a connected vehicle that's asking the question. I don't know what the question is, but let's say, you know, it's asking something. But you can imagine that the tech company within the automotive industry saying, we want Richie to be able to design V2.

We want him to be able to test some sample data on his laptop, if that's allowed, right? Then he can scale it up into different clouds, different regions, put it on a telephone pole, and then eventually move it all the way into the vehicle without the thing that's running in the vehicle that's asking the questions, ever having to go down, be reconfigured, anything.

And hopefully the audience, if they kind of go, Hmm, how might I do that with today's technology of how I normally go about things with HTTP or PREST APIs or sockets and DNS and load balancers and GSLBs. And you realize that something that feels like it would be nice to have to be trivial to do this, like within, let's say, Richie, you finished the V2 of the really cool Richie microservice to be able to deploy that, the next day type stuff.

And what we see is, is that most companies are like, Once Richie's done, it's probably going to take us six plus months to actually get this all the way out to something like running inside of a vehicle inside of a factory or whatever. And so one of Synadia's big goals is to provide a modern tech stack for what we consider this modern technological landscape of where you need to deploy these things.

But you should be able to reduce complexity. To Justina's point, batteries should be included. So, all of those things we keep mentioning, GSLBs and, you know, DNS tricks and things like that, aren't needed with a Cinedia tech stack. And so, with that one binary, not your application, but that one binary of a Cinedia NAT server that can be Lego bricked anywhere and run anywhere.

Once that's actually running, all of those other pieces aren't needed. And so now you can see a very quick, rapid response to Ritchie Microservice v2, if that makes sense.

Richie Cotton: Okay. Yeah. That certainly sounds like, once you've got the prototype, then it takes six months to put things in place that's a lot of commitment. There's a lot of effort to create these things. So making that easier.

Derek Collison: you're finished. So you've done your job. You're like, hey, Ritchie v2 looks great and it's running. And then as you look at, oh, well, if we deploy it to the cloud in one region and one cloud provider that we're used to, we might be able to do that fairly quickly, right? We've got kubernetes set up.

We know how that works, where the friction and the time delays come into deploying. That is in that near, far and beyond edge?

Richie Cotton: Does it always have to be that long? Is there like a hello world equivalent for edge computing? Like what's the sort of simplest useful project you can do?

Derek Collison: that's where it becomes really interesting and why we took such a different approach. And again, I'm not pointing fingers at other people because I did the same thing, but you could say the hello world might be just a stateless application. How can I figure out how to get it to run inside, of a manufacturing plant or a distribution center or a connected car like we're talking about.

That's not hard, but that's also not reality. every technology is using multiple moving parts, lots of microservices, access the different data types and data stores and things like that that are all spread out, right? Some could be in the edge. Some could be, in a remote location, but close, for example, with connected cars that could be running in the base of the cell towers, or it could be in different regions within different cloud providers.

And so I think that's the big challenge. It's when you're trying to deploy into these unknown landscapes. That's where it becomes interesting. But it's not just unknown, it's you don't have all of the moving pieces that a cloud has to kind of what I do, what I call the unnatural acts to paper over the fact that all technologies, in my opinion, for the most part, that run what we're built on today are all location dependent.

I need to know where you are, your I. P. Believe it or not and people could argue, Oh, no, I have an IP for the load balancer. That, in my opinion, is an unnatural act to get around the basic limitations that everything we do today is location dependent and one to one. Request replied. And we're starting to see architectures that need quite a lot more, and they don't want to have to wait for six, twelve months sometimes, right, to have not only Richie, but let's say Derek and Justina as the supporting characters on platform engineering try to recreate everything in the cloud.

And so what we do for our customers and partners in the ecosystem is we drastically accelerate the time to value. We also drastically reduce the complexity. You can imagine a world where if the cloud provider is running all of these services. You don't necessarily have to worry about them. You know that they're running and all.

But now, all of a sudden, inside of a vehicle or inside of a factory, it's like, Oh, do we want to replicate every single thing that the cloud provider does? I mean, that's a massive, massive cost. And you could go down that path, or What Synadia did was we said, let's re frame the problem from the ground up and try to think differently about what that might look like.

Steps that we don't have to take six to 12 months to try to recreate everything that was in the cloud. And by the way, I love cloud, don't get me wrong. I do think it's the new mainframe, and hope that the audience, and I'm sure they are, because I'm sure they're very smart. You know, the cloud providers are incentivized to be a hotel California.

You can check in, but they don't want you to check out.

Richie Cotton: I'm sure. Okay. So yeah, it sounds like, there's still quite a lot of effort involved in creating these things. So I guess a few different teams within your organization are going to be involved. So if you're trying to create your first edge application, which teams or roles are going to be involved in this?

Yeah, go on, just in, you know, you've not had a go for a while talk us through it.

Justyna Bak: Well, building a new application always starts with the business outcome that the application needs to drive. during my time at Google, when I had the front row seat to the Generative AI application revolution, I saw that the customers who had the biggest success were the ones who focused on the very specific business outcome they wanted.

on one business process they wanted to optimize and to end rather than trying to ball the ocean and trying too many things the same time. So once you have this end goal, you'll assemble team that can help drive obviously designation. Uh, Alternative solutions to what we always used to build applications and considering approaches such as Cinedia Edge Native Tech Stack that allows you to build applications that will thrive under circumstances, even in the harshest of the conditions you will encounter at the edge, where you have intermittent connectivity, high latency, bandwidth may be scarce, and sometimes you may even have limited compute.

So solutions architects are definitely needed, but also your data team, because you, the applications need the data. And to Derek's point, you will not always have access to robust databases in the cloud. You may need to able to operate in offline scenarios, but still be able to write the data and eventually ensure data consistency.

think one of the really cool applications that we are seeing now and we'll be seeing more of is Edge AI. And the one functionality of this application that is so revolutionary is that you can start classifying the data that's generated into the high value versus low value data and that high value data that is really critical for you to act on in real time because it includes the business inside that will help you drive to the business outcome.

So for instance, if your business outcome develop a monitoring application for your manufacturing plans. and making sure that all the machines are working in concert. And if something, if an anomaly is detected, you act on it. And maybe you can even create a more advanced version of AI where you change several functions.

so first you detect the problem, then you classify it. Is it urgent or is it something I'm just monitoring? You triage it. And then if it's urgent, you start fixing it. So you also want to have the ML specialists who will be able do inference at the edge, then classify the data, use some of the data maybe to send back to cloud.

So train further the models with this interesting data that's just been captured. Or maybe you want to train this small AI model at the edge. but I think it's, we will see more collaboration across overall social architectures, looking at new approaches to building truly nomadic applications you will see the collaboration between the ML specialists and the data teams, because you cannot have AI without data.

Richie Cotton: Okay, I love that. So we've got obviously software developing and engineering, because you're building some kind of application, you've got the data people, you need data, and I love the use case of, you're doing anomaly detection, so you're going to need some data scientists or machine learning specialists in there as well.

I guess the teams that haven't been mentioned there's no mention of sort of business teams there. do any commercial teams need to be involved in these things?

Derek Collison: Well, I think Justyna led with the business case, the business teams and what we're trying to achieve. And so, And more succinctly to your point, it's less about the application itself that's running there, and it's more about what does it need access to that drives the business outcome. So, for example, if my application needs access to three other microservices, and they need access to data stores, and those all are remote, the business team might say, I can't wait for the display in the car to update when you touch your phone and it takes five seconds to update.

That's just a bad experience. It's a bad business outcome for our users. So then that trickles down and now it's less about the app itself. It's more about platform engineering, data engineering going. Well, what if we need to move those services and that data actually into the vehicle as well?

And that's where a lot of our customers, when they hit that crossover point, that's when they reach out to us.

Richie Cotton: Okay. Yeah, that certainly seems to make sense. You have the business people involved in the requirements that may be less concerned with some of the technicalities. Now you both mentioned the idea of AI at the edge. So I'd like to know what are some sort of examples of this? are we talking about sort of more traditional or predictive AI?

Or are there generative AI use cases? Like do you want chatbots on the edge? Talk me through it.

Derek Collison: Well, I think what you're going to see is, you know, inference, wherever it's running, right, is kind of a, it's its own ecosystem, in my opinion. And it has two distinct factors. Again, no matter where it's running. One is prompt augmentation, which I'm sure the audience is kind of familiar with. We can actually do it by hand.

You might hear things like RAG and RAG and things like that. And it's essentially saying, the raw prompt that you give me, I need a whole bunch of access to real time data sources to augment it, And so again, we're in that. Hey, do we want to pay the latency if all of that stuff is not where we really need it?

 The second one, which is just starting to come into fruition, but I think to your use case that you're talking about here is very applicable, is what most people term as a gentic system. But what I actually term it as is probably a DAG directed acyclic graph traversal through multiple models.

So it could be that, I'm going to be talking. You know, I think voice interactions, especially in certain situations, will dominate. And so LLMs and things like that might actually be the first thing that says, Okay, I get it. I get what you're trying do. And here's the plan. And oh, by the way, you need all of the access to these different data sources and models to kind of make this work.

And where I think Synadia and our partners can benefit from this, and where we have a lot of AI customers today, is you don't need to know where they are, but you need to know that it's secure and that you're really talking to the model that you want to. But it could be a mix of models between LLM and generative to predictive to on board vision models that are running, you know, in vehicle or on the manufacturing floor, the robots.

So you can see this ecosystem just keep playing out. And again, more access to more data, more latency sensitive access to these things, and then traversals through multiple models where you don't know where they're running. And you could even, in my opinion, have the ability to move these models again from the cloud to a cell tower to in the factory or in the vehicle, so to speak.

Richie Cotton: Okay, yeah, so two very cool examples. I like the idea of just using real time data sources to augment your prompts and that's, seems like a cool edge use case.

Derek Collison: think on that one just real quick, to kind of also highlight some of the things that we talk around, the connectivity layer. Imagine a world where there's inference, and it's a very large service, Let's say we go from 10 requests a second to a million. I'm making up a number, right? Obviously, you're going to have a very big bill if you're always talking to big LLMs.

But more importantly, if you're doing prompt augmentation, and everything is request reply, point to point, every data layer that you're accessing now also needs to scale with your inference layer. And so when you see a RAG, you know, RAG is retrieval. I ask and retrieve the value. And what we and our customers and partners kind of talk through is, Hey, let's say Richie's the main service, and Richie's now running at a million a second.

And Justyna and I are the data, you know, real time data things for prompt augmentation. Instead of you having to ask Justyna and me at a million times a second as well, I mean, we have to scale that, commensurate with your scaling. In our world, we have both push and pull. Push just means you can say, Hey, Derek, any time, let's say a sensor temperature changes on this, just let me know.

And I know it's only going to change maybe a couple times, you know, Over 30 seconds. But now, son, I don't have to scale to a million a second. I just know that I have a change, and I just, in our world, just send it out as an event that anyone who's interested in that event, meaning all 10, 000 of you, right, that are running this million requests per second, and maybe Justine is doing audit logs or something else with that, you know, In our world, that's just very natural and very simple.

And so even at the just prompt augmentation and access to real time data, when we talk about real time data, yes, you can ask for it. You know, you can do request response. But what's, I think, a highlight point of a system like Xenadius is you can architecturally move that to a push model and just say, Hey, you guys, whenever something changes, just let, let me know.

And then I'll hold on to that data for a certain period of time and just reuse it

Richie Cotton: Okay, so, not even having to ask the data, it's just being given to you and saying, here's something interesting completely automatically. Okay, I like that. It's the sort of zero work, work for humans you have to do. So yeah, everything just happens without you intervening. All right so, Do you have any success stories from organizations that have used agents?

So you mentioned that as your second use case. Everyone's talking about agents, but real world examples are a little hard to come by. So, have you seen this work in practice?

Justyna Bak: Oh, yes, we have lots of innovative AI startups as our customers, and they build a variety of interesting use cases. So for instance, we have a company who's building personal AI assistants, and that are trained on your own data, and they can become useful assistants helping you discover new things. some questions about your maybe even career choices based on the past data that you fed.

It's about your preferences. We have customers who are automating a DevOps workflows and helping DevOps teams be more productive and focus on what they do best by automating some of the mundane and repetitive tasks. And then we also have organizations who are building, search engines based on ai.

So there is a, a whole range of use cases that are very practical providing lots of value, and they are not far away in the future. They are happening right now. Mm-Hmm.

Derek Collison: to Justina's point, but also echo that what you were saying, Richie, is it is still very early on, And so we're we know, or at least I firmly believe that 2025 you will see these multimodal, multimodal, sorry, agentic workflows really kind of explode. I think And then the second half of 2025, I think we're going to see kind of an explosion of physical A.

I. So when these agents can actually start affecting physical environments, which I'll be honest, a couple years ago, I thought that was that physical A. I. component, like robotics and things like that was going to be very far out. And everything that A. I. is doing it's making a lot of us look kind of foolish because it's proving us wrong as we go.

So. We're definitely seeing customers starting to utilize our tech staff to make these easier, but it is very, very early days.

Richie Cotton: But second half of 2025, it's coming around fast. Um, What's changed to make the AI interactions with robotics easier?

Derek Collison: I think it's kind of like the iPhone. And what I mean by that, and the audience might go, what? but the iPhone was it was just a massive, perfect storm of all of these technologies reaching kind of an apex where the iPhone was possible. And so, you know, with generative AI, with the power of these models, with for the most part, you know, at least in our opinion, looking at the AI landscape, and I studied this in university, unfortunately, when we got out of university, or at least when I got out, We went into the second longest AI winter ever, right?

But now what you're seeing is, is that hardware is moving faster than I ever thought it would. You know, with NVIDIA's new, chips, you know, where they go with H100 and Broadwell, which is I think, I might have messed up that name coming out so fast. And then everyone was really concerned around energy.

And they're like, well, we'll just go to nuclear. And we really think fusion's probably within a decade, which will solve all those problems. So most of the forward thinkers in the AI space have said, let's assume that Data is plentiful. We're going to, we're going to figure out synthetic data.

Compute will become faster and cheaper, and energy will go to zero. So when you start thinking about that, you start seeing things like NVIDIA's Omniverse. Well, they've created a completely virtual reality environment to train virtual robots, but completely on the laws of physics. And they can really just transfer it to the physical thing, and they're getting amazing results.

But I think it's a, It's again, it's like this mini iPhone moment, which I just did not see happening for physical AI this fast. But for us, it's great because manufacturing is a big vertical for us, right? We have lots of customers in that space, so it's great to see.

Richie Cotton: That's a big gamble if you're assuming nuclear fusion is going to happen. Like, I remember learning about nuclear fusion maybe coming soon when I was at school and that's several decades ago. So, yeah but certainly it's a very interesting idea.

Derek Collison: but the interesting part is, you're right, and I do believe it will be within a decade, but you, I think every hyperscaler already has contracts to build nuclear power, data centers, AI data centers and things. And so, even in the short term, you can, at least on paper, I know this isn't reality, but you can mentally trend energy to zero, but at the same time, the energy requirements of the hardware is accelerating faster than, Any tech I've seen in my 36 plus year career.

So it's, definitely kind of bonkers where we're at. And it's exciting, at least for me. Very exciting. Absolutely. That's very cool stuff. All right. So, just to wrap up what are you most excited about in the world of edge computing?

Justyna Bak: So one of the most exciting use cases of AI at the Edge are definitely self driving cars. Lots of sensors capturing lots of data in real time. we need to act on this data in real time because it's a mission critical workload. Either it's safe to make a left turn in the intersection or it's not and you need to wait.

And if something is really going wrong with the vehicle, you need to pull over and wait for help. You need to have a tech stack that supports this mission critical workload. And connectivity is not a given. Sometimes these vehicles, they have to really operate fully uh, autonomously. all the decisions have to be made at the vehicle.

And so having a textile that supported key to making autonomous vehicles 

Derek Collison: can tell you, at least my perspective, Richie, I think that the biggest thing is, is that to Justina's point, we focus both on training and inference. But where we're really concentrating is inference. And what we've seen is it's always going to be at the edge. And for us, what at least I am most excited about is that all of the bets we made, for example, prompt augmentation, needing access to lots of different data in real time that can be spread out.

And then this multimodal, agentic workflow that's this exploding. All of it kind of needs the it. the tech stack that, that we bet on about seven years ago. So that's what I'm most excited about, that some of the bets that we, we crossed our fingers on, are kind of taking root.

Richie Cotton: All right. That's wonderful stuff. It's a very exciting future. So yeah. Thank you so much for your time.

Derek Collison: Absolutely. Thank you. I appreciate it.

Justyna Bak: thank you.