What is AIOps? With Assaf Resnick, Co-Founder & CEO of BigPanda

Richie Cotton: Hi Assaf, thank you for joining me on the show. So, just to begin, what is AIOps? 

Assaf Resnick: AIOps is the use of AI and machine learning to scale and automate IT operations. So, what does that mean? So, our mission as a company is to is to enable the teams that keep digital services running.

So what does that mean? So our customers are the world's largest enterprises. So think large banks, large stock exchanges, large, large airlines, so on and so forth. They have lots of digital services and, and. Under empowering those digital services is things like clouds and servers and applications and networks, lots of moving parts that generate lots and lots of data are customers, their job.

They are the essential workers that keep all that digital infrastructure running and when it breaks. They have to find and fix the problem before it affects customers, before it affects the business. That's getting harder and harder to do as enterprises move to the cloud, move to increasingly complex, uh, IT operational footprints.

So they're using cloud, microservices, containers, uh, they're deploying software a lot more. They're using all of these new technologies that let them move faster, be more innovative. On the other hand, now they've got a lot more moving parts, generating a lot more machine data. In fact, a tsunami of machine data that the human beings in IT operations, whose job it is to keep the business running, They can't keep up with all that data.

And so that's where AI comes in. AI is a generic technology is very good at taking large sets of fuzzy and fast moving data and turning that into insight and action AI ops. Is the application of AI specifically for IT operational use cases.

Richie Cotton: Okay, so this seems like another story of AI just invading absolutely every single industry.

And it seems like in this case it's helping out the IT operations and IT infrastructure people. And there are a lot of IT ops terms floating about at the moment. So you've got DevOps, you've got ML ops, you've got data ops, you've got all these related terms. How does AI ops fit in compared to these other ops things?

Assaf Resnick: Well, they all have to do with operations, but different parts of operations. So if you think of DevOps, that is a, uh, a philosophy that says developers, traditionally, when you look at, uh, IT services, uh, you had two groups. You had developers who built code, and you had IT operations who ran code. The concept of DevOps is one that says, hey, Developers should maintain the code base and the services they've built.

You build it, you own it. That's the concept of DevOps. Nothing to do with AI. MLOps is nothing to do with AI, there's a lot of tooling and operations that needs to happen in order to deliver better, um, ML and AI. So ML Ops is a whole branch around tooling and services to enable ML and AI developers. AI Ops is how do you use AI to scale and automate IT Ops.

It's a bit of a word salad. There's a lot of jargon out

Richie Cotton: there, hard to keep up. Absolutely, so three different related ideas, but for different groups of people then. Absolutely. Okay, so to make this a bit more concrete, can you give us some examples of what an AIOps project might involve? Yeah, of course.

Assaf Resnick: So if, um, if you think of, uh, human I.

T. teams whose job it is to keep digital services running, uh, I, those teams, there can be hundreds, there can be thousands of people in any given enterprise, uh, their job is to collect lots and lots of data, and that data can live in lots of different silos of the organization. It can be data about. The health of your machines and data about the health of your applications.

and data about the health of your customer experience. And so, health of your machines typically lives in a set of data called monitoring or observability data. And so that's lots and lots of software you're using to monitor the health of your, uh, servers and networks and storage and cloud and containers.

Lots and lots of data that lives in lots of different tools that all specialize in Monitoring the health of different tiers of your tech stack. Then there's, uh, information about the health of your applications. Right? So I've got lots of applications, lots of microservices. They're acting in different ways depending on what's happening.

Uh, I have to get that silo of information. Then I have to get siloed information around what's the health of my customer experience. And that just gives me enough information to know, Do I have a problem, right? Then I need to know, well, why do I have this problem? What's the root cause of this problem?

Typically, that can happen because someone changed something across my enterprise, and it's leading to un, uh, unpredictable, uh, situations. The problem there is. If I'm a large enterprise, I'm a Fortune, Global 2000 company, Fortune 500, there's thousands of people who can be changing lots and lots of things over the last two weeks that may have unintended consequences.

Developers who deploy new code, uh, security administrators who change security, uh, configurations, IT folks who change infrastructure configurations. 5, 000 changes in the last two weeks that could have led to, you know, an application not working like it should and delivering a suboptimal customer experience.

I have to filter through all of those 5, 000 changes to figure out which is the one that led to Uh, the problem that's a needle in the haystack issue, uh, then I have to filter through, uh, lots and lots of other data around who did what, who communicated what, have we seen this problem in the past, how did we fix it in the past, and so that's why for human beings today, to piece together all of that data in order to get context, It can take hours, uh, and in a digital world where seconds matter, if you're down for hours, that's very, very bad.

You know, there's a special place in hell, uh, for example, think of airlines, there's this very special place in hell called getting stranded in the airport and it turns out customers don't like it and they'll punish you for it. Uh, if you think of most large airlines, they've got like 50 to a hundred critical services that if any one of them is down for more than 10 minutes, they have to ground the entire fleet.

And so, you know, on the one hand, you've got 10 minutes to keep this service live and back, and you have to sift through a tsunami of data in order to figure out what's broken, why is it broken, what should I do about it? Uh, AIOps says, hey, let's piece together all that data, let's give you full context, let's give you, we have this concept called full context operations.

Let's take all these little silos of data that are, are buried in lots of different places. Let's piece them together and give people full context to do their job, make decisions better, make decisions faster.

Richie Cotton: Okay, I like that you point out that it does have real consequences when systems go down. It's going to cause genuine problems for your customers, like getting stranded in an airport or whatever.

And certainly the idea that once systems get really big, even the developers don't understand the changes that they make, they don't know what the impact is going to be. That seems very important, useful for motivating why you need AI to help out there. Okay, so do you have any examples of companies that have had success stories by going in on AIOps?

Assaf Resnick: Yeah, absolutely. Um, I would think of, uh, Intercontinental Hotel Groups, IHG. Uh, so the one of the largest hotel, uh, Uh, uh, groups in the world. So this got something like 6, 000 hotels representing, I think something like almost 300, 000 rooms, a lot of organizational complexity. They brought us in, uh, and, and really trying to help with help them.

Uh, A, automate how they detect issues to bring down that mean time to detection and then help them, uh, get to root cause much faster. So helping them automate how long it takes them to understand why did something break and how should I fixed it? And, and I think last year they achieved something like 99.

8 percent availability. after using BigPanda, after adopting BigPanda, which was the best they've ever done. Uh, and that obviously, level availability means that, uh, customers can book hotel rooms, uh, with less problems, uh, they can check in to their hotel with less problems, uh, and, uh, can, IHG can deliver much,

Richie Cotton: much better, uh, customer service.

Is the main focus just about avoiding downtime, avoiding problems or other, other use cases for AIOps?

Assaf Resnick: There's a number of use cases. So one of the biggest one is, uh, improving the availability of your digital services. That's a huge one. Uh, second one is enabling more efficient use of your resources. So instead of your people, Spending a lot of their time hunting and pecking for useful information.

Uh, the fact that they can get full context operations instantly lets them work a lot faster. And so, uh, you're making better use of your people. And you're also not doing, you're not bringing in as much disruption. So I, I remember, uh, sitting with an executive from, you know, one of the, uh, the world's largest insurance companies.

guys. And when they have what they call a P1 outage, which means, you know, the bells are ringing, things are on fire. They get these bridge calls from hell where, you know, 60 different people from 60 different parts of the company get on the line, including executives and trying to figure out what the heck is happening.

And they can take hours. And it's. hugely frustrating, hugely disruptive, uh, and we can help eliminate those and at the very least streamline those. So those bridge calls take minutes, not hours. And so bringing less, more efficiency and less disruption in the market or to our customers lets them continue to do their day job, which is to build services, build code, continue to innovate.

Richie Cotton: Okay, certainly not having horrible meetings with 60 people in them does seem like a pretty good motivator. And I would say having these outages, like I'm trying to debug what's going wrong. I've spoken to our engineers here at Datacamp, and yeah, this is one of the more stressful aspects of the job. So I think that people would prefer to focus on building stuff rather than dealing with problems.

And I'd like to talk a bit about how you go about implementing some of this stuff. So, which teams or roles tend to get involved when you're setting up an AIOps team?

Assaf Resnick: You know, I'd say there's kind of three primary personas involved. Uh, there is, uh, the users, so, uh, most of our other users who use something like a Big Panda or an AIOps tool are folks who are, uh, full time firefighters.

So most enterprises have what's called level one operators. They sit in an operations center. Looking at a bunch of screens that tell them about the health of their servers and networks and applications and so on and so forth. And, uh, the whole point of AIOps, or a big part of AIOps is, hey, instead of those people having to look at 30, 000 different symptoms of what's going on, You just want to come to work and say, Hey, I really only have one or two big problems.

It may have hundreds or thousands of symptoms, but I don't need to be a data scientist to connect the dots. And so those are the primary users. Uh, then there are typically the tool owners. So the, the, and the users, the operators are their customers. So the tool owner is typically someone who owns either, uh, AI initiatives.

For their company, so it'll be an executive or a VP who's tasked with bringing AI innovation into their company. Or it will be someone who is the owner of observability and monitoring tools. And AIOps is a natural extension of that. And so you have the tool owner. You have, uh, the tool users and typically the boss of both of those people is either the CIO or the CIO minus one or two, a VP or an SVP, uh, of IT operations.

And those are typically the, the folks involved. Um, when things really hit the fan and you have a big outage. Then the blast radius of that outage can also bring inside developers, subject matter experts, folks whose full time job is not operations and not incident management. But when something in their domain breaks, they get called in.

And they want easy access to information as well.

Richie Cotton: Okay, so it seems like it's pretty limited to IT teams and engineering teams. But it's going to be the CIO who is responsible for this in most cases. I do also like the idea of having a corporate blast radius. Uh, if something's going wrong, you've got an explosion rippling across your teams.

Very nice. I'm curious as to how much AI knowledge you need, because often this is something that not a skilled software developers are going to have. So what do you need to know about AI in order to make use of AIOps?

Assaf Resnick: Zero. Zero. And that's the whole point of AIOps, where, you know, I think that if you look at 10 years ago, you did see, you know, a lot of operations teams and IT teams try to kind of roll their own, uh, and create their own in home tools that took this tsunami of data and made sense of it.

And most of those projects ended up in failure, uh, because it's really, really hard. I mean, we've, we as, Big Panda as a company has raised something like 330 million. Uh, in order to build this offering, uh, because the back end of this offering is incredibly complex, but the front end of the offering is such that you don't need to be a data scientist.

You don't need to be an expert to use this thing. And like you said, most IT ops teams don't have data scientists on staff, don't have PhDs on staff. Uh, and so we've built AI ops That it's no experts required. Uh, and so the interface is one where you have to be aware of, you know, what's going on in IT ops, and you have to be able to speak the language of IT ops.

You don't need to be, have to be able to speak the language of.

Richie Cotton: Okay, so in some way, it's nice that all of, uh, mathematics and data is abstracted away since our audience, well, a lot of them care about data, but can you talk us through what sort of techniques, data and AI techniques are involved in Big Panda?

Assaf Resnick: You know, the first step, I'd say that there's kind of, uh, three steps here. So, the first step is, um, aggregating, normalizing, enriching all of the data. So, you know, what are human beings doing when an outage happens and it takes six hours to find the pro to fix the problem? Uh, they're hunting and pecking for their d for data and they're connecting the dots in their head.

That's the first thing we want to do as a programmatic piece of infrastructure. So the first step is collect all the data in the different silos where it lives. And so we have to shake hands with lots of different tools and data silos. And some of those provide, uh, very structured, very clean data. And a lot of those provide very messy data that there is no structure or every development team has a different structure.

And so we have to come in and we have to normalize all of that data into a, uh, extraction layer that we call the unified data fabric. And so there you're taking data from 250 different systems, you're normalizing it, you're structuring it, but more importantly, Uh, you're doing, uh, you're enriching it. So the data from tool A and tool B and tool C is now all connected to each other.

So you've got a ton more context. Uh, that's a big part of it. Then once you've got that kind of abstraction layer of data and unified data, then comes the AI. And there you're looking for patterns. You're looking for clusters. You're looking for time relationships. You're doing semantic processing, uh, you're doing a lot of different both algorithmic and AI processing to be able to glean, uh, insights out of that data.

Uh, and then the last part is to be able to actually visualize that data for teams, for IT ops folks to be able to quickly understand what's going on. That's where a lot of the magic happens as well. You know, we want to make sure that It's not just no experts required, but really, you know, minimal training required.

We want you, we want operators to come in to Big Panda and if you know how to use an email application, you'll naturally understand how to use Big Panda. It's just, you know, super, super easy.

Richie Cotton: Okay, that last pointing particularly important. 'cause once you. an incredibly complex system with lots of possible things going wrong, then you want to be able to display this information really clearly to people who don't necessarily have that much data expertise.

They're in a rush to find out what the problem is and it seems that that communication part is gonna be maybe the secret sauce. Can you talk me through any sorts of techniques you've used to try and make things clearer and quickly understandable for people?

Assaf Resnick: You know, it's one part data processing. and one part data visualization.

So the data processing is, um, how do you not just cluster data and, and correlate data, but how do you explain in very easy terms what is the logic behind that correlation? And so we've come up, you know, when we first brought AI to market, people were very skeptical. They said, hey, this is a black box. And now you're asking me to automate some of my business's most mission critical app, uh, uh, services and activities based on the logic that your AI creates.

But I don't know why the logic of the AI is making the decision. So we developed Openbox AI, which is kind of a very fancy way of saying we make the logic. Very explainable. The AI processes lots and lots of data. It finds patterns and it creates, uh, it creates pattern logic. So, uh, uh, it's a, when they seize the state in the future, it knows in a deterministic way, how to correlate all these things.

We make that logic, we expose that logic to our customers so they can see exactly why did the AI make the decisions that it does. And very importantly, Humans can not only see it, and therefore trust it, they can actually tweak it. They can say, hey, AI found some pretty good patterns, but I've got some tribal knowledge and some, you know, history in my environment that I know that this logic isn't quite right.

I want to tweak it a little bit. And I want to test that tweak based on historical data so I can see how it acts in the past before I release it in the wild. And so being able to expose that logic goes a long way. And then two is, is smart data visualizations to be able to visualize the kind of escalation of an incident and how it started here in servers and then moved to network and then affected your application and then affected your user experience.

Be able to see that kind of waterfall. Uh, is hugely important and, and it makes it much more intuitive for human beings to understand what's going on with my machines.

Richie Cotton: Okay. So just to make sure I've understood this correctly, it's going to learn what the standard time is to load a page on your website and it's going to learn some sort of a cutoff for what's the slowest it can load.

And if it goes past that, then that's when it's going to start telling you that there are problems. Is that the kind of idea of things? Thanks.

Assaf Resnick: That's kind of, so customers, companies have lots and lots of tools to monitor, like I said, things like the customer experience. So they have tools that say, hey, this website or this query should be answered in less than 50 milliseconds.

And if it's more than 50 milliseconds, That's a problem and a poor user experience. Raise a flag. We get that flag. It's called an event. And then we say, Hey, something over here is happening with user experience. Let's go see if something, you know, which servers support that user experience, which servers, which networks, uh, which clouds, which applications all touch that user experience.

So once we say, Hey, there's a problem that user experience, let's go look at the underlying infrastructure that we know to associate with that problem and see if there's a problem there.

Richie Cotton: Okay. So it's going beyond just alerting that there is a problem. It's going to help you triage where the problem came from.

Absolutely. And oftentimes,

Assaf Resnick: you know, the fact that you have a page that's slow to load, that's a, upstream symptom of something that might have started two, three, four hours ago, right? And you want to understand what's going on. And that problem that started two, three, four hours ago might be because of a change someone implemented two weeks ago.

And now I have to sift through hundreds of thousands of data points in order to find those breadcrumbs. That's what AI does.

Richie Cotton: Nice. Uh, that just seems like it's saying a lot of pain there. So, you've been through a few, uh, examples of problems, uh, I'd like to talk about how to get started. So, what do you need to do before you start adopting AIOps?

How do you prepare for this?

Assaf Resnick: You know, really not a lot, uh, to do. Uh, you know, part of the value proposition of AIOps is that you don't need perfect data. You know, you don't have to first, Uh, you know, I remember when I was a kid, my mom, you know, we'd have the cleaning lady come and my mom would say like, Hey, the cleaning lady's coming.

You have to clean the house first. So I was thinking like, but the cleaning lady's coming. Why do I have to clean my room? Uh, and, and you know, people often think something like that with AIOps. Yeah, I have to clean up all my data before I feed it to AIOps so it can make sense of my data. And the answer is no.

Part of the value prop of. And ops is, it can make sense of messy data, it can make sense of dirty data. Uh, and the fact that you have siloed data and every team in your organization has a different kind of framework and schema. That's okay. Uh, put the AI ops layer on top of that and it can help make sense of that.

So there's really not a lot of change you need to do to clean up your data before you bring AI. That's what AI, that's part of the value prop is, you know, we have this unified data fabric. that can come in and clean it, structure it, normalize it, enrich it, we do that work for you.

Richie Cotton: Okay, so what makes a good first project for AIOps?

What are you going to hook this up to first?

Assaf Resnick: There's not one rule. Typically what I tell large enterprises is, Start with a discreet air problem, you know, pick your most painful problem and start there. Don't try to boil the ocean and say, hey, I'm a large global bank and I want to roll this across my entire footprint.

You know, across, in the case of a bank, wealth management, consumer technology, trading, uh, and on and on and on. Typically. The right thing is, pick one area where you have a burning, burning pain, go fix it, go get a win, go socialize that win internally, get people excited, and get that snowball rolling. And so, you know, let's go get you a win within a few weeks or a few months.

Uh, and that typically leads to much broader adoption, because then people are coming to you and saying, Hey, heard you've got this big panda thing, and it's really cleaning up your data, and it's really streamlining your operations. How do I get in on the action?

Richie Cotton: Okay, so I suppose the good news is, if you're having a lot of outages, then you're going to get that first win a lot quicker, because you can see what's happening.

It's a target rich environment. It's a target rich environment, yeah. Nice. So, all the examples you've talked about so far have been with very complex IT systems in large enterprises. So, how big or complex do your projects need to be in order to get some benefit from AIOps?

Assaf Resnick: Typically, the larger you are, the messier you are, the more fragmented data you have, the more value you can get out of this.

If you're a small or mid sized business, if you're a young startup in Silicon Valley, you probably don't need AIOps. Uh, but if you are a medium to large enterprise, uh, you've been around for several decades, you've got different silos of technology, different generations of technology. The people that wrote and build your initial generation of technology might not be around anymore.

Uh, those are the kinds of environments where AIOps will really bring you outsized, uh, benefits.

Richie Cotton: Okay. Can you just talk me through, are there any particular stages to the adoption of AIOps? Is it a case of hook up one system at a time, or do you have to connect everything together at once to get the whole benefit from it?

And also, um, can you talk about what is a data fabric? I know this is of interest to our audience.

Assaf Resnick: Yeah, that's a great question. No, you can absolutely do it in steps and iterate your way to value. So, you know, again, I recommend as a first step, don't try to boil the ocean and connect your entire enterprise.

Pick one specific environment that's particularly painful and start there. Uh, step one is to connect all the data. So you, you want to connect Big Panda to the different systems, the different data silos. And so we're getting that data. And then we are, uh, storing that data, cleaning that data. Into this abstraction layer that we call the unified data fabric, and that's really a data store where we're taking all your disparate data were, uh, we're structuring your unstructured data.

We are normalizing all the data from the 15 different tools that speak 15 different languages that it's speaking the same data. And then very importantly, we're enriching that data. And so we know when, uh, you have a low, uh, page load time, we know that it's related to this application, we know this application sits on the server, we know this server speaks via this network, and we're enriching all of that data.

And if you just stop there, you're already going to get a ton of value. Because you've connected all that data, you've really cleaned it up, and you've made it way, way more actionable. That's step one. Typically takes several weeks. Step two is then to turn that clean data into insight. And that's where you want to unlock AI.

Uh, typically, you need, you know, enough statistically significant data for the AI to recognize patterns. The good news is in large enterprises, uh, that can really happen very quickly, because there's so much data going through large enterprises You can get to statistical significance in typically five to ten days.

Richie Cotton: It's interesting that First Step is just bringing all your data silos together and making sure the data is in a good state before you introduce the AI level. So, just to wrap up, what are you most excited about in the world of AIOps?

Assaf Resnick: I'd say the most exciting, uh, is Gen AI has really Um, just really change the landscape of what we can do.

I mean, if you think, we've been doing AI ops for 10 years. Um, previous to Gen AI, our AI and our ML development was really based on a small handful of data scientists who were doing research. And so these are PhDs. Uh, you're competing with these people with the likes of Facebook and Google. They're very expensive.

They're in high demand. Um, and they're very far away from understanding, you know, IT Ops. They're, they're data scientists. They're not IT Ops engineers. And so the research they're doing takes time. Uh, and there's not a ton of empathy with the customers and their pain. Now along comes Gen AI. And it means that our customers, Um, and our employees, folks like solution architects, sales engineers, our own DevOps engineers, are technical enough to do prompt engineering on Gen AI.

And so the beauty of that is you can now get folks that are deep domain expertise, have lots of familiarity and empathy with the problems that we're solving. They can now leverage Gen AI to innovate and prototype. Which means that, uh, for us, it's been a huge unlock in the velocity of our innovation and the quality of our innovation.

Uh, because we're taking the folks, you know, we're hiring our own customers and saying, Hey, now you can use Gen AI to solve the problems you've been suffering for from for the last 20 years. And that's been a real game changer in the velocity of the quality of our innovation.

Richie Cotton: Okay, yeah, so bringing people with domain expertise, your business people, uh, and getting them to be able to solve technical problems seems like an incredible leap forward.

Absolutely, absolutely.

Assaf Resnick: The other, the other aspect that Gen AI, uh, has helped, uh, unlock for us is it's dramatically expanding the surface area of the data that we can process. So up until before Gen AI, The data silos that we were really tapping into are things like observability data, change data, topology data, things that live within other systems.

But there's a lot of data that's buried in people's heads. There's a lot of data that's buried in lots of tiny little silos across the enterprise. Uh, that's really hard to get to. AI, uh, and a lot of that is in natural language, semantic, uh, environments. Um, AI, Gen AI can now help you get at those silos as well, so you can really tap into the long tail of enterprise data.

Uh, and that's very exciting as

Richie Cotton: well. Absolutely. It's amazing how much corporate knowledge is often just buried away in some throwaway comment in a Google Doc or something instead.

Assaf Resnick: In a Google Doc or PowerPoint or Gmail. Uh, and now you can get at it and turn that human knowledge into enterprise knowledge.

It's very

Richie Cotton: exciting. Exciting times indeed. Do you have any final advice for organizations wanting to adopt AIOps?

Assaf Resnick: My final advice is just do it. Just get started. Uh, there's not a lot of prep work you need, whatever state you're in. If you're suffering from the pain of data overload, get started with AIOps.

It's not a big lift.

Richie Cotton: Excellent. Just go and do it. I like that. All right. Thank you for your time, SF.