Scaling AI in the Enterprise with Abhas Ricky, Chief Strategy Officer at Cloudera

Richie Cotton: Hi Abhas, welcome to the show.

Abhas Ricky: Morning, great to be here. 

Richie Cotton: So to begin with, what are the most important trends you're seeing in the world of data?

Abhas Ricky: Yeah, so let's start with that we're starting to see. So as you know, for the last 10 odd years, majority of the focus has been about cloud applications. There wasn't technology in the last 40 years where You have three very large providers, each of them with 100 billion of compute or more, growing at 30, 40 percent CAGR.

And that fueled the data application business to next levels. But that also brought in new challenges around data security, data governance, data lineage, and all of those capabilities. So, over the last five years or so, when realistically the Transformer Library paper came out from OpenAI, there has been an increased focus on data as an asset.

And therefore, majority of the large enterprises we speak to, or I speak to on a weekly basis, they all want to say, hey, I want to be able to trust the data that I'm going to use to train my large language models on. And whether it's agency applications, whether it's RAG applications, whether it's fine tuning, the core premise is you want to be able to trust the data that you're going to use.

And I do think that the trust in the data is a big, big subject that's coming through. The second thing we're starting to get through is the fidelity of the outputs that we're starting to talk about. Oftentimes that depends upon the quality of the data sets that you have. And therefore, in AI parlance, we call it as enterprise context.

There's an increased amount of focus on getting the high value enterprise context. Thanks. whether that's for AI applications, whether that's for agency applications or their own. And I believe the third one really is to the original point that I said, which is there's a lot of data sprawl that is happening because remember architecturally, we went from data lakes to lake houses to now a slightly different architecture for AI technologies.

But along the way there have been application developers, there have been AI practitioners, there have been applications. SQL developers, front end and back end, and they've all created their own systems and tools. And there's been a lot of data sprawl that has happened. And that creates issues around data security, that creates issues around cost.

Total cost of ownership is a big element. So that is something that is coming to the fore more and more. And large enterprises are trying to tackle that with the best tools and services they have at their disposal.

Richie Cotton: Those all seem like, well, very important things. So, yeah, you want to be able to trust that your data is going to give you the right answers and you need to be able to find it rather than just. Working out which application it's sitting in or which database it's sitting in. So, you mentioned the idea of data as an asset there.

 Can you just tell me a bit more about what that means?

Abhas Ricky: Yeah, so technologically, a lot of the people have said they wanted to MSWordDoc Word. Document. 8 to data mesh architectures. And that's a source of capabilities, right? So that includes query federation, for example. That includes adding attributes on the data itself so that you can treat data as a product.

Because at the end of the day, that's where majority of the application developers want to get to so that their job becomes easier. But on top of that, I'll give you a real example. There's a very large factory I work with. They have close to one exabytes of data in the management. And that's a very large number of structured, unstructured images, all kinds of data.

But for two, three years back, that was all storage. And you could pull some of it out, For analytics, you can pull some of it out for your next best action or your sentiment analysis or whichever the use case might be. And then you worried about making sure you were compliant with the policies. If you're a regulated industry, in this case, they were, but also elements in Europe, as you know, GDPR in California, CCPA came through before.

So a lot of the value was around making sure you can use the data in parts. To get to a specific outcome, but with the onset of AI, what has happened is the data has been completely the primary focus because of democratization that's taking place. So, for example, if you were a SQL application developer and I was a data scientist.

Two years, three years back, you would have to come to me every time you wanted to change a parameter on a model. Now you don't need to do that because you can take the enterprise context, aka data asset, and apply it to the large language models without even having to come to me as the So you don't need to evolve or upskill necessarily, and you can get different outcomes for different use cases with the same data set.

So the data is becoming much more powerful because now you can use that as enterprise context. and apply italized language models or any of the new AI frameworks that might be on top of that. So I do think that there is an increased focus on the quality of the data, as I said, but also making it available, getting it to the hands of practitioners is equally important and data as a product definitely helps you get there.

Richie Cotton: Absolutely. So I like the idea that, well, you know, just storing your data somewhere, keeping track of it, that's fine, but it's not necessarily going to get to adding value to your business. You need to make sure that you're, you've got it in a place where you can use it easily. And then that way you can take advantage of it.

And you mentioned generative AI, obviously it's working its way into almost everything these days, but I get the sense that there are some organizations, you know, they've sort of got things together. They're quite advanced, other organizations are still just getting started. Do you have a general sense of where, enterprises are at typically in their AI journey?

Abhas Ricky: So I'll respond to the answer with two facets. So first one is depending upon the journey that the customer has been on or large enterprise has been on. So let's talk about leaders. So leaders are organizations who've had very large or relatively large engineering teams in house, skill sets, and they've been able to solve the data engineering problem rather succinctly, but also within the realms of the organization.

So those people are trying to take advantage of the generative AI applications. So, for example, there's a very large bank that we work with, and they started off with a low hanging fruit. They built a tech summarization application. They build copilots or something that is like a chat Q& A for the service operations.

So they spent the first year doing that largely with assistance. But this year, they're actually getting to scaling, but also they're going to agenting applications. So I say that move from assistants to agents. Now, an agent is largely a capability that allows you to do function calling. Back in the day, some people used to say API.

But an autonomous agent actually gets from not just data to insights, but also to action. because it takes actions on behalf of the human loop that would have been enforced by analyzing a set of parameters, understand the intent behind what the actions should have been, and being able to do that. So in this case, this bank, this last one here, They've productionized various use cases, such as loan underwriting, or such as tax invoice reconciliation through all agency applications they might have.

 But also, even the LLM calls example that I said, 70 percent or more of their LLM calls are now being done through agents. So they spend this year productionizing agents, but also incorporating generative AI as part of the normal processes Which I think is a super important part for scaling generative AI.

It can't be this standalone piece in the corner as an innovation arm. It has to be embedded as part of the normal R& D cycles. So that's for the leaders. Now coming to the laggards. And I don't want to have like a negative sentiment with laggards. I just mean to say people who didn't get softened up early enough.

I do think they're still at a slightly earlier stage, and they're still starting to get to a point where they're playing with the model of the choice, whichever flavor they like, proprietary or open source. They're still getting to build applications with whichever AI frameworks there is. And they're still starting to get to their first AI application.

And for those organizations, enterprises, or customers of ours, we oftentimes say that you have to make sure that your architecture is built for scale in the future, but also you're getting to a point whereby you have the core data platform. You have the core ability to serve up the data Before you can do any of the model hosting or fine tuning or write applications or so on.

So I think a lot of those customers are at that point in time and that's what we're seeing them to get to us.

Richie Cotton: Okay, so there's really quite a wide range. And of course, yeah, it's definitely okay to be just getting started now. Everyone's a bit like a beginner at everything to begin with. So, You mentioned the idea that you need to sort out your data before you really get into the AI stuff. So tell me what needs to happen there.

Like what do you need to do to your data infrastructure before you can be successful with AI?

Abhas Ricky: So I think the AI lifecycle is a very good representation of multiple steps that you would have had done when you were building a cloud application as well. But it's slightly different in that you need to have solved a data engineering problem before, because you still need to ingest the data. You still need to do data prep.

You still need to do data writing. You still need to do data engineering. And then serve it up to whichever platform you're using to be able to apply any applications, any model, any frameworks, any agents that might be on top of that. So what I meant by that is, as part of your AI application lifecycle, you have to make sure that the data ingest prep, rendering, etc.

has been done, and you're ready to serve it up to an environment or a platform. Whereby you can then start to apply AI or machine learning application building capabilities on top of that. That is slightly different from when we were doing lifecycle analytics. Because when we were just doing lifecycle analytics, the whole part of the lake house was that you had the data ingestion prepping, but you were also running applications on top of that.

And that is what was giving you outcome. And therefore, I use the word when you go from data to insights. In this case, you're going from data to insights to value to action. And that last mile is equally important. And it's super important for us to be able to get through that.

Richie Cotton: I really love the idea that there's a step by step framework there. So you start off with improving your data, then you try and get some insights out of that. And eventually you want to move towards getting some kind of action from that data. And then maybe you're sort of adding on the AI and you've got agents doing that all automatically.

So it sounds like they're going to be different challenges at each step of the way. So I guess to begin with, what are the big challenges with preparing your data, particularly in the enterprise?

Abhas Ricky: Absolutely. And I think the big context is whether you're doing life cycle analytics or whether you're doing AI, it's a team sport. I've always said that AI is a team sport. So for example, the example that the use case that I just talked about, which is AI lifecycle, you have to get to a point whereby your data management platform serves the data at the highest levels of fidelity, but with the security modules, with the metadata that you require to be able to go and apply forward, you need to have Models that you can apply to the data that you have that gives you the highest fidelity output, but also within the cost framework.

So, for example, you don't oftentimes need a large language models. You can potentially deal with small language models are. Even large action models as well. And then when you're starting to get to do vector embeddings for scientific search querying or you're trying to do inferencing of certain capabilities, you also have to have a governance layers around that.

So not only do you need fine tuning tools, but you also need monitoring capabilities, observability capabilities. You need to have MLOps capabilities around that. And then finally, when you start to deploy and build applications, you need to have agents. but also the orchestration around them because there are a bunch of them depending on which tool we use.

 And then you need to have an application hosting and UI capability that you can apply all of that. But across all of these things, you need to be able to get to a point whereby you have an infrastructure layer, so that you can run scale layer workloads on GPUs on an environment of your choice. Private cloud, public cloud, your desktop, your mobile.

At a price point of your choice. Why? Because compute is so prohibitive in today's world. And by the way, last I checked, electricity isn't becoming cheaper. So it's going to be there that way for a period of time. So that's the AI model. Now to the second part of your question, in terms of the specific challenges around data, to get that data prepped.

Well, there's a few. Number one is data scrub that's happening. As I gave you the example for application developers, we need to make sure that We are able to contest that, but also have a process and a set of processes within the systems, within the organizations, within LOB IT, lines of business IT, so that there is no concept of shadow IT driving that through.

The second one is getting to a point whereby you can access the right datasets. So I talked about agents. One of the customers told me about the hardest thing to do is to be able to expose the data To those agents, because that is the core fundamental for us getting to a higher fidelity output on the other side.

So getting that and making that available to whichever applications you use it in the format that you want, in a platform where you want, is not an easy thing. And obviously, lastly, is Everyone wants to be able to know who touched the data, who changed it, when did you change it, what did it look like before, what does it look like now?

That's all things around governance and operations, so data lineage, metadata governance, technical metadata, business metadata, having That capability of understanding the flow across systems when you move the data. So those are the three broad areas in which I'd say data operations as a concept is becoming more and more popular.

And that's where that's feeding the first part of the question you'd ask, which is how does data operations making inherently useful or increase the outputs for the AI lifecycle that we just

Richie Cotton: Oh, wow. That sounds, An awful lot more complicated than I was expecting. There's like so many different layers involved there. So you said you got the infrastructure stuff, like data lineage uh, MLOps. You've got monitoring, you've got all sorts of stuff. And that's before you even get into like, are we actually solving a business problem as well?

So, maybe before we get into like the problems later down the sort of life cycle, but it's worth talking about who needs to be involved, I guess. It sounds like there's gonna be so many different teams involved. Who needs to be in charge of all of this?

Abhas Ricky: we were doing in the whole cloud application era. naturally, we'll have the data stewards, who'll make sure everything around the data operations is running smoothly. But obviously, all of this starts with the use case. All of this starts with the business use case. Microsoft Mechanics www. microsoft.

com It's very rarely, and actually it's a wrong thing if you start to build an application and then force it onto the lines of businesses. So it should start with the line of business owner. So for example, if you're a bank or a telephone, if you want to run a churn model because you want to get better insights.

In your most loyal customer, that's where it starts to, that person will own the business definition and the conversion. Somebody on the team will own the conversion of business requirements into technical capabilities that the IT teams will have to do. Depending upon the old structure they might have, oftentimes you will have AI practitioners sitting within the lines of business as well.

And then obviously they will come in. And they will want to try and build an application to test it out. And sometimes, if you need to scale it, they'll lop it across the wall and say, Hey, IT, can you actually get into enterprise grade security, enterprise grade governance, and make it an enterprise wide application?

So in that case, obviously, the AI practitioners, but also the SQL application developers sitting within line of businesses, which we, you and I call it LLVIT, will need to be involved to get that test through. But when it gets to enterprise wide usage, it's not just the IT teams who are making sure that that's getting through.

You have the security teams involved. Because in a large bank, in a large telecom, in a large insurer, et cetera, there's elements around data security, which is super important, specifically in the world of generative AI. And we've all seen the multiple lawsuits that have been filed against OpenAI, for example.

The New York Times was one of the conversations. It's But do we have policies whereby which kind of data sources can we use within a model? Because that is something that needs to be driven by a compliance officer. Do we have policies around what can we actually put on the internet or can we use somewhere else?

So there are policy requirements whereby security teams and compliance offices will have a role to play as well. And then lastly, it's the core engineering capability that you have in house. There are the people who are building the picks and shovels and doing the plumbing to make sure that your platform looks cool.

And not to mention the practitioners who are actually using them. So there's a, as I said, AI is a team sport. Data is a team sport, and you have to be able to make sure you have the right organizational structures in place to be able to drive the most efficient processes forward.

Richie Cotton: Okay. Yeah, I absolutely agree that it's a team sport and it's interesting that you should always start with the business teams and like, what's the business problem? Because otherwise, I guess you're not going to get any value from it at the end. It's going to be a mismatch and then move it on to the data and AI people.

And then it's only when you start scaling that the IT people and engineering teams tend to get involved. So you also mentioned the legal challenges to making sure that things are secure. when do you need to start worrying about that? Should that be like at the start? Should be after you've come up with a business case?

Yeah, when do you need to worry about the the legal and security challenges?

Abhas Ricky: I'd say worry is probably not the right word. I think when, should you start to plant in corporate elements around uh, legal ramifications there might be. And I think right at the start when you're building plan and the plan is not just a business plan, but also a technical plan or a solutions plan.

 And the reason I say that is because, so right now you have DORA available in Europe, right? And you have the White House, precision initiatives from the guidelines around AI came through. More and more countries, more and more organizations will be subject to different levels of regulatory requirements and pressures that will come through.

And that is something that a large part of practitioners will need to deal with. So I do think that you have to be aware of what the ramifications are for specific policy directives. Whether it's governmental or intergovernmental I do think you have to incorporate the potential legal risk and exposure that you might get into if not applied through that right at the start and also throughout that.

But the big thing, which is, this is an ever changing paradigm. There's a model every Sunday, there's a framework every Monday, there's an agentic company and an application that's every Wednesday. People who are on the policy side are also evolving and they'll be coming up with newer guidelines over a period of time.

So not only do you have to do it right at the start, but also you have to make sure you're planning that through along the way. And the one thing that I will say, which is not long ago, we had multiple multi trillion dollar companies out there who have a lot of Buying power, but also the ability to dictate how the practitioners engage with them.

So in cases of partnerships for smaller organizations, for example, if you wanna be a strategic partner of choice with somebody, you have to make sure that you are taking care around elements such as indemnifications for libraries. And those are some of the core pieces that will come through to the forum more and more as people move from building RAG applications to financing.

And I think most people are starting to do that in six months, if they haven't already. So that is one of the core things that will evolve just like everything else.

Richie Cotton: Okay yeah, that certainly makes sense. Particularly if you want to partner with larger organizations, then you're gonna need to make sure that everything is working correctly from a legal perspective.

Abhas Ricky: And there's only so many companies who have chip providers. There are only so many companies who are providing with the compute capabilities. And at the end of the day, what's the monetization unit economics lever? It's compute. That's what everyone's after. Whether it's the large LLM providers, whether it's the hyperscalers, we're providing cloud applications, and now we're also getting to And some of them getting to chip manufacturing space, or whether it's the very large GPU providers who have just started a software business, they're all left to compute.

So I do think that, as I said, very large organizations have extremely outsized buying power. So doing generative AI at scale is a capital intensive game. And I've said this before, you have to take a leap of faith. You can't say that, oh, I'll get somebody to do this And then there will be an ROI after a year and a half, and they will bring in the consultants and we'll have the plan.

I think the train would have left the station if that's the case. And therefore, with so much capital and so much investments going into it, there will be a myriad of legal exposures that organizations can be subject to. We need to make sure that we're aligning to the new paradigm along the process, before the process, and after the process.

Richie Cotton: This is really interesting that you say compute is the thing that's most important because I think well, at least first sort of couple of decades this century there was this sort of sense that, well, computing is getting cheaper and then now I guess. Is Gentrify just this big game changer that says, well, actually, applications are expensive to run now.

need as much compute as you can, or you need to get it cost effectively. has the story changed, do you think?

Abhas Ricky: what I meant by saying is not that compute is the most important. I meant compute is the denominator for which people are monetizing against. That's the metric that people are using. When you monetize, whether it's a large language model, pick any one, whether it's OpenAI and throw a big moose trap, we're here for large language models.

Whether you look at large hyperspheres, whether it's Google, Microsoft, or Amazon, the services they provide, or whether even NVIDIA that we, An inferencing service with Nvidia through Nvidia names. So they have their models and they have a microservices package because the models are optimized to the GP performance that they have the data, but you also have industry specific APIs.

But I were to take that and give to a customer, the value proposition for them is. A, you can run scaled AI workloads on GPUs in a form factor of your choice of the best TCO. In certain cases, you can save millions of dollars in compute per month. But the second one is, you can do what I call as private AI.

And then the idea is, you can take any of the AI applications that you have, and you can do it in public cloud, private cloud, or desktop. And you can do that with the model of your choice. Open source or closed source. These are the value propositions for the customer. www. microsoft. com But the way the vendors are monetizing that is through compute.

So that's what I meant. Now, to your question on, is compute becoming cheaper? Well, there have been significant advancements in quantum computing. There have been a large set of organizations, a slew of billionaires, who've been, I would say, sponsoring and alternative sources of energy. I was at the Also Journal AI conference last week in California, and a Tier 1 CEO said that he believes right now, in the next three generations of the models that'll come through, it might take up the same amount of electricity as you would take for a small town for a week.

So it is pretty compute intensive. So in spite of the advancements, Of what quantum computing and the normal large companies have done through it'll be a while before compute becomes less prohibitive is what I would like to say, and therefore I meant that it's a capital intensive game. So don't mistake it for the fact that people aren't working on it.

They are whether it's nuclear fusion energy and Bill Gates talks about it as well or anything else. But I just think that it hasn't become mainstream yet for the average manufacturing company or the average retailer across the world. And therefore people will have to contend with the services that are being provided, the price fund that are being provided today for a foreseeable future.

Richie Cotton: Okay, yeah, I can certainly see how if you want to vastly improve on this cutting edge foundation models, it is going to get incredibly energy intensive. Is that something that organizations need to plan for now? A big increase in their electricity bill?

Abhas Ricky: I would say the first thing organizations need to plan for is which use cases do you actually want. to build an application for, because not every application requires an LLM. You can just use a small language model. Not everything requires x billion parameters. You don't need to ball the ocean. So the first thing is identifying which use cases you want to be able to deploy through a prioritization matrix, business values and execution.

And then after that, once you've done that, there will be certain cases in which you would want to use business values. Large language models, and obviously you need to make sure that you have the data to be able to train them because it's only compute intensive. It's running against a wide variety of data sets, so you might not need to.

And therefore, I said that there are action models that might be more effective, but in case if you do, obviously, you have to plan for, I'd say, a 10 bit increase in your IT budget insofar as, factoring in the complete cost is concerned. But the bigger thing is, just like cloud applications, when cloud applications became popular, SaaS as a service became popular.

Vertical SaaS offerings came to the fore. A similar thing will happen here, but the winners were the ones who had in house teams who were developing, playing with the products, fine tuning them, not as we say fine tuning here, but in the literal English way, trying to get through with that. So even in today's world, there will be armies of people within your own organizations who will want to get to the latest technology, but will also want to play with it.

And therefore you will have to factor in some amount of compute cost, some amount of GPU capabilities that you want to have internally. Even if you have outsourced your preliminary set of use cases to NSI who will build that for you, or a strategic partner of choice for a software vendor. So, net net, short answer, yes.

But how much? It depends upon your use case, the amount of data you use, and where it goes with it.

Richie Cotton: Okay that does seem very sensible is sort of using the minimum viable model for any given use case. So, earlier on you were saying that there is sort of very simple AI applications like text summarization and it goes through to very complicated things like agents. So is there an equivalent way of deciding, well, these particular use cases need a simple LLM, these other use cases need something a bit fancier, or you've just got to try it and see what performance you get.

Abhas Ricky: So I wouldn't say that there are easy ones or hard ones, like if you ask practitioners, they'll all say that all of them are complex ones, But let's say, for example, if you're a bank and if you're a trader and you come in the morning and you read the insights and it takes you two hours, for example, to compile the insights from different reports or different research works.

For that one, for example, you can actually build A text summarization tool, you can just have co pilot application. And all it does is does text summarization in two minutes. So your productivity is 99%. You don't necessarily need, to your original question, a lot of compute in that case, unless you have like volumes of data that you're actually scrolling through the internet, which you won't be in a normal case if you're a specific part of the trading desk, because you have a limited amount of information trying to get through with that.

But on the same front, if for example, your use case is slightly different, you're doing drug discovery, canonical drug discovery research for the next cancer drug, You want to look at as much canonical and empirical data you might have for the last decade or even more. And you will have multiple strata of different chemical composition that you are applying through.

And in that world, you obviously want an LLM that has the capability to handle majority of the input requirements that you're feeding in, which a lot of people call as parameters. Well, you must have a 7 billion, 8 billion a section and by doing so, you will get high fidelity output, but obviously it's compute intensive as well.

But the use case in that case is life and death. In fact, we have a very large pharmaceutical organization, over 60 billion dollars in revenue. And they're doing a lot of their drug discovery work with us and they're actually building generative AI applications in production and they have been able to move from not just a data platform to knowledge bases so that they can get the insights they need and they've been running different types of language models on top of us.

Now, I agree whilst it's compute intensive, the. Outcome is significantly important and it outweighs the cost that you might be going through because if you can get ahead for a specific drug by X number of years, that's an incredible achievement, not just monetarily, but also for humanity in terms of helping you provide cure and we have other organizations who are running them.

all of the immunization testing during the COVID timescales in the years 2020, 2021, 2022. And for them, for example, they'll want to use that as well. So I think it depends on the use case and then the outcome. And you have to have obviously the cost equation. Everything's a business case. But on top of that, it also is the quality of the data that is available to you.

And the skill sets that you might have that you can leverage to build applications going forward.

Richie Cotton: That's really interesting that you got two very distinct use cases that both make sense. So just summarizing a report or a meeting, it's like, well, you've just saved someone like 15, 20 minutes, but it's pretty cheap to do. And then on the other hand, you know, you're improving the drug discovery cycle.

That's going to make you millions of dollars, but obviously a lot more expensive to do. So it's about weighing up that sort of cost of implementation versus the value you're going to get. I'm curious as to how you go about scaling things. So you mentioned a lot of companies have tried playing around with prototypes.

Some of them have got things in production. What happens when you want to go really big, you found your use case, how do you go about scaling your AI application?

Abhas Ricky: There are three pieces that large organizations have to be aware of. Number one, scaling the generative AI infrastructure is an important piece. So it's not just the fact that you will use any application there might be. So I'll give you an example. There's a very large financial services organizations.

They have roughly around two dozen generative AI applications in production, but they're processing 75 million words through their LLMs on a daily basis. They're transcribing 600 hours of calls for the call sensors that they're getting through. And they have like 1, 200 direct users and 5, 000 daily indirect users for the generate output you're getting from LLMs.

And in that case, you have to make sure that you're scaling the Generate AI infrastructure as the first piece, because otherwise, that is a challenge. Second thing, you have to embed the Gen AI within the processes I mentioned. Embedding into the process is not just having a separate team, but also there are different levels of complexity that you need to be able to get to so that you can provide LLM as a service because you need the system integration with the various tools you have the applications for different capabilities, but also the serving capabilities.

You have a set of orchestration capabilities that you have to focus on, and then there's model management as well. So to be able to get all of that in a manner in this financial services organizations example, every 30 seconds, they receive the customer feedback across eight different channels. And to be able to feed that back into your social media campaign, to be able to get that through ratings that you might have, because people are posting about your CSAT, to be To be able to respond to the complaints you might have that impacts your customer journeys.

That impacts your brand perception that impacts your products and the value. So as you can see, you have to embed that as part of your normal processes. It cannot be a stand alone thing. And lastly, I would say. You have to get to a point whereby you are getting to advanced technologies like agent applications.

Now, a lot of organizations have used techniques such as reasoning, but also feature learning and summarization. And that obviously helps with getting information in real time. Integrating complex reasoning with agents is the next frontier. Doing it in an autonomous fashion is where the world is headed towards.

So I think I'll give you a real example, the same financial services organization that I talked to you about, they have vision language models. What is a vision language model? In plain English, it's a pair of eyes that you can use for doing processing. But they also have agents to streamline complex multistage tasks.

So the example that I gave you, tax invoice reconciliation. So they have an agent which does, they reads the tax documents. They have an agent that extracts additional sources and applies on top of that to apply the context. They have an agent that does benchmarking against the other relevant tax data.

They have an agent that writes a memo. They have an agent that fact checks the memo. They have an agent that formats the memo. So you need. A system that orchestrates and regenerates a lot of these agents so that when somebody asks the question, what is the source of this wealth document, you should be able to get through that, not just through the LLMs that you're using, but also in an autonomous fashion that you can do high volume processing to go through with that.

So to summarize those three things, scaling a generative AI infrastructure, embedding that in your processes, and getting to advanced technologies aka complex reasoning with agents is the core thing. But here's the key, the benefits is huge. Like not only do you get productive improvements like two days to 15 minutes.

You get consistency because agents enforce consistent format and quality across all the documents across all the six use case that I talked about validation. Agents provide document references in the example that I said to make it easy to validate the information. And lastly, and very importantly, You prevent hallucination because the agents can proofread them and you can get any out of concept detection and there'll be like a flair for automated hallucination rectification that you can act on.

Richie Cotton: That's absolutely fascinating that even just some of it sounds fairly simple, like writing a document. You've actually got a lot of different agents working together in, I guess, a swarm, is that the collective noun for agent? So yeah, you've got something writing the memo, something formatting it, something proofreading it.

And something doing citations or that kind of stuff. So it's not just one giant AI, it's lots of smaller AIs working together in tandem.

Abhas Ricky: And that's the whole neural network concept, right, whereby you want to get to a point wherein You have autonomous agents taking over a series of human actions. We never document the gazillion steps that we would take to do a simple task. We just do it because the brain functions and we're wired to do that.

But that's what we're training agents to become. So as you can see, when you're applying that to a business use case, there's a series of them and not to mention there will be regenerative agents, agents that will be regenerated as well, depending upon what you have before. Therefore, you need a platform to orchestrate that and make sure that you have these very complex reasoning capabilities.

coming together to deliver a use case. But as you can see, I'm super passionate about that, and I do believe that Agents will be the next version of applications, and that's where the world is headed towards.

Richie Cotton: All right. Wonderful. Okay. So just to wrap up then, do you have a recommendation for what should your first agent be?

Abhas Ricky: Again, it might sound cliche, but it depends on what the use case is. So it depends upon the use case, and that's how you should determine that. That's my simple answer. Having said that there is one topic that we haven't discussed, and that is hybrid. I do think whether it's cloud applications, or whether it's generative AI applications, or whether it's agentic applications, you want to be able to do All of that at the point of residency of the data and I've often said that I actually had a LinkedIn post a few courses back where I said you need to bring the models to the data and not the data to the models and the interpretation for that is you need to be able to invest in capabilities that allows you to do hardware acceleration because AI workloads on GPUs in a form factor of your choice to the best TCO.

The interpretation of that also is you need to be able to run these large language models in private cloud or public cloud, because if you're a large bank, if you're a large telco, there are certain use cases that it's never leaving your data center, because there's sensitive information, because you need to run 24 7 365, cybersecurity, and a host of those.

You need to be able to do that. And obviously, by the way, you need to do it with a wrapper of governance and operations. with the same level of fidelity, the same level of robustness around lineage, metadata, et cetera, on public cloud and private cloud. So I do think whether it's MLMs, whether it's agents, whether it's frameworks, a lot of these will get to a territory whereby very large organizations will say, I want to be able to do hybrid AI.

And that means I want to be able to do build applications, AI applications. in a form factor of my choice, but also ported depending on where the data resides, because otherwise it's super expensive. So that's the one core thing that's a large organizations and practitioners should be wary of if you aren't already.

Richie Cotton: That's wonderful. Yeah, so. That seems absolutely fascinating, the idea you bring your data and your models in the same place. And if you've got a big security concerns, that means you don't want it in a public cloud, you want it somewhere private. And so, but everything else, I guess, goes public cloud. So that's why you want the hybrid setup.

All right. Super. Any final advice then for organizations wanting to get better at AI?

Abhas Ricky: Three things. AI is a team sport. You need to make sure all of the pieces are in order. Second, you need to get to a point whereby you're able to trust the data you're going to use to train your models and applications on to solve all the data. And third, you have to embed generative AI within your normal processes to be able to scale that.

Otherwise, it's a science experiment which never scales.

Richie Cotton: All right. Great final wisdom there. Thank you so much for your time Abbas.

Abhas Ricky: Thank you for hosting me and thank you for having me here, Richie. It was a lovely speaking to you.