Generative AI at EY with John Thompson, Head of AI at EY

Richie Cotton: Welcome to DataFramed. This is Richie. At this point, hopefully you're convinced that generative AI chat, like ChatGPT, is useful at work. And yet, many executives are rightfully worried about the risks from having business and customer conversations recorded by the chatbot platforms. Some privacy and security conscious organizations are going so far as to block these AI platforms.

This eliminates the risk, but also eliminates any benefits that you could get from the tech. I've been trying to find leaders from companies with a strong privacy culture that have found a way to adopt generative AI. And as luck would have it, I bumped into John Thompson. He's the head of AI at EY, and he's just been involved in rolling out a ChatGPT implementation to 200,000 employees.

This definitely fits my criteria, since a lot of EY's value comes from its intellectual property, and so data privacy and security are paramount. I want to know how John pulled off this magic trick, and how it's impacted EY. In addition to his work at EY, John also runs the Marketing Sciences Consultancy, advising clients on monetization strategies for data.

And he's the author of four books on data, including Data for All and Causal Artificial Intelligence. The latter book is particularly interesting to me as Causal AI is the other exciting, up and coming branch of AI with big applications for decision making. Keep listening to hear me pick John's brains on both Generative and Causal AI.

Hi John. Welcome to the show.

John Thompson: How are you doing today?

Richie Cotton: Life is good. I'm excited to chat with you. just to begin with can you give me some context on what were the goals around you adopting generative AI at EY?

John Thompson: Yeah, you know, it all came out of the blue for everybody, I believe was the case, and no different at EY. I joined EY about 18 months ago, and about two months into my tenure, some of the executives came to me and said, Hey, you're the head of AI, what are we going to do about this stuff? So it was a trial by fire.

And, EY is a large corporation, 420, 000 employees all over the world, probably one of the most regulated companies in the world as well. Auditing and consulting and financial services, mergers and acquisitions, all the things you would expect. So they really wanted to make sure that the global employee base was able to use generative AI and large language models and related technologies, but in a safe way.

So the impetus was to build something that was actually going to work for the employee base and keep them safe. make sure that it was, foolproof, I guess is one way to say it. That, somebody couldn't take some information from EY that was very sensitive or information from one of our clients and upload it in a way that it became, public information when you don't want it to be.

So the driving force to build what we did build was to enable our clients or enable our employees to use leading edge technology for productivity and other benefits and yet be safe.

Richie Cotton: so it sounds like you got some pretty strict requirements there, so it's got to be able to scale to all 420, 000 employees. You've got all these data privacy challenges and regulatory challenges. Can you talk me through what was the setup you decided to build for this then?

John Thompson: you know, I think it's becoming more and more common now, you know, I've talked to a few different people that have done the same thing that we have done, and, you know, they're always very, everybody's always excited to say, hey, I didn't do something bad, you know, and we were out on the front end of it, so we're like, all right, this seems to be the right thing to do.

So we talked to people at Microsoft, we talked to people all over the world, and we said, all right, You know, our infosec, risk management, data privacy, and legal requirements put us into this kind of environment. And away we go. So, our entire environment is on Azure OpenAI. So, you know, we built an Azure environment that has instances in North America, Europe, and Asia.

And we looked at it and we did a lot of modeling, what's it going to look like when you have, all these people hitting it. Our effective population isn't 420, 000 because there are a number of people in countries that are restricted. So open AI restricts access or closes off access to countries like that.

China, North Korea, Iran, Iraq those kind of places. So, our effective population is 300, 000 people. So, we have 275, 000 people on the system pretty regularly. So, we built it tested it we did things like make sure, one of the things that was really important for EY, the governing functions that I listed earlier, InfoSec and all the rest of was that we weren't logging and storing the prompts or the completions.

So we actually went through the system multiple times to make sure that there were no unknown areas where things were being logged. And one of the things that we found was that Microsoft's abuse monitoring system was actually logging things that we didn't want it to log. So we had to go in and modify that as well.

So it's a, it's an Azure environment, runs on a global basis, geolocates users and routes them to the appropriate instance in North America, Europe and Asia.

Richie Cotton: that's pretty impressive. And I find the fact that you decided to turn off logging particularly interesting. So, first of all, I guess, how do you persuade Microsoft to turn off their abuse monitoring?

John Thompson: Well, it's our environment, so it's not as if we have anybody that's coming in from the general public. these are all employees of EY, they're in EY's tenant, they're behind EY's security measures. And if there's anybody abusing the system, it's really for us to find and track down. It's not, somebody from, doing something outside that Microsoft has to be concerned about.

So, you know, they're like, okay, well, if you're taking the responsibility for it, well, we can turn that off.

Richie Cotton: Okay so, you're taking responsibility for it. Are there any other steps you had to go through in order to get rid of all this logging and tracking of all the messages being sent?

John Thompson: It was actually fairly simple, you know, you have to know where it's being logged, and you have to be able to turn those things off, and those are mostly toggle, they're, configuration switches, it's not like you had to do a whole lot of coding. So, yeah, it was like three or four areas in the system, we had to find them first, and we had to turn them off.

But it was fairly straightforward. Now, of course, the world moves on and different opinions come to play. And EY is not a corporation, EY is a partnership. So you have different views inside the organization that say, you should really be logging all of it. And, you know, to get the system out the door, you know, we went with what the governing functions demanded of us, you know, we had to get it launched, the executive said, you got to get it done.

We're like, okay, well, these are hard stops. If we don't do this, we can't launch it. So now we come back and people are saying, well, what kind of intelligence can we get? And we said, well, you can't get any because we don't have any data. And they say, well, that's crazy. We should log all of it. We should be able to see, what are the people in the UK doing?

What are the people in Kazakhstan doing? What are the people in Japan doing? And, what kind of areas are they looking at? What prompting are they doing? How efficient is it? You know, those kind of things. And we're going back and saying, okay, well, let's have a conversation with InfoSec, Risk Management, Data Privacy, and the legal team and say, hey, here's the value.

Here's the upside of holding this information and keeping it, logging it, and analyzing it. Here's the downside. So there's a, a debate going on right now.

Richie Cotton: basically does sound fascinating because, yeah, on the one hand, if you're logging stuff, then there's the risk of some sensitive data being recorded somewhere and maybe leaking out and, you know, on the other hand, you don't get that sort of intelligence on like what people are actually doing with it.

So you're saying you're not tracking any of the, messages to GPT. Do you track any metadata, like how much people are using the services or anything like that?

John Thompson: Yeah, we do. We know, we where people are coming from and what kind of traffic they're putting in the system, what kind of token consumption they're driving. We know all that. And we're currently looking at that because we're having discussions about. how do you pay for the service?

last year, we paid for it through a centralized budget that the, the executive team said, we want this, we're going to pay for it. It's part of the ongoing operations. And now the question is, who pays for it? you know, that kind of debate is going on right now. we'll see how that comes out.

I have no idea which way it's going to go.

Richie Cotton: Okay, yeah arguing, like, which team has to pay for things is, I think, a common problem in any organization. Okay, so, are there any worries caused by not monitoring the conversations? So you said you're kind of assuming responsibility for it, so, is there anything you're worried about there?

John Thompson: Yeah, there's all sorts of conversations about, you know, how much personal use is going on. sure there's a certain percentage that people are, doing their vacation itineraries and writing sonnets to their puppies and things like that, but we don't know about it. And I think it's probably a low level of usage anyway.

So, you know, it all comes down to how strong is the argument that you just don't want to know, because if you do have it, it's legally And if it's subpoenaed, then you have to produce it versus, the value you get from understanding what people are doing. So, it's a pretty standard conversation you would expect in any organization.

If I had to bat, I would imagine we're going to go towards logging. I think

Richie Cotton: Okay, so you think logging is going to be the long term option. All right, so In the meantime, I mean, you said because you're not tracking anything, you're not quite sure what people are using it for. Do you have a sense of like what work tasks people might want to use the tools for?

John Thompson: Well, people are pretty open about what they're doing. they're very proud of what they're doing with generative AI. So, we have, as we talked about, a very large employee base. We have assurance and auditing. We have consulting, mergers and acquisitions, financial services.

We have, almost everything you can think of in the, in the business world, people using it to summarize documents. They're uploading all sorts of 10ks and 4ks and financial documents and summarizing those. They're uploading PowerPoints all sorts of public information about their clients.

They're uploading all sorts of private information inside EY and private information from their clients because it's all secure and you can upload that. and they're using it to generate proposals, understand pricing almost any use case you could think about, due diligence, summarization, they're doing those kind of things, it's happening.

the use cases are varied and in a myriad of them.

Richie Cotton: Are you able to measure productivity improvements from this? Can you see that there is some like real impact on the business?

John Thompson: It's really a great question, Richie, and something that we was debating it yesterday with people, some people have done some very simple surveys, self report kind of stuff, and you know, I'm very leery of those kind of numbers. I don't find those to be reliable or useful or, something that I would want to put my name on.

So there's all sorts of conversations, economists have talked about. breakthrough technologies taking, eight years to ten years to show up in the macroeconomic GDP figures and those kind of things. And I do think that it takes a while for productivity gains to settle in.

You know, somebody doing something once and saving two hours really isn't that impactful. a hundred thousand people doing something every day. and saving two hours. Well, that's worth talking about. We're not at that point. We're still at experimenting and one offing and, we haven't gotten to a point of doing process mining and understanding and revising and transforming to get to a point where productivity gains are repeatable, measurable, and locked in.

So, lots of productivity gains. had many conversations with people that say, it's saving me an hour a day, two hours a day, six hours a week. And you ask them, well, what are you doing? And some of them are not doing it and not taking those savings and putting it into work. They're leaving work early or they're, you know, improving their work life balance.

other people are saying, well, You know, instead of one proposal a week, I could do two. So there are some productivity gains. There's some work life balance improvements, but I don't think we're at a point where we truly understand where it's going to level out over time.

Richie Cotton: Okay, so it feels like at the moment it's sort of net positive, but there's still a lot of work to be done in terms of systematizing this and getting those large scale productivity boosts.

John Thompson: Yeah, my view, exactly.

Richie Cotton: And so, beyond productivity, I mean, you talked about work life balance being one thing. Are there any other benefits from using generative AI?

John Thompson: Oh, the benefits are, huge. people. are actually able to, we have to keep in mind that, before generative AI, we were using approximately 10 percent of the world's information. That's what we have in structured systems, ERP systems, CRM systems, you know, all that different kind of stuff revenue payroll, you know, all those kinds of numeric tabular kind of things.

Now, with generative AI, we've opened it up to the other 90 percent of information that is available. it's not like one day you go from 10, you get to 100, but you now have the capability of using that other 90 percent of the information. And I think that takes a while for non techno, it's taken a while for technologists and it takes even longer for non technologists to get their head wrapped around that.

let's say a law firm, is another type of partnership and you have lots of people that are pouring over books and over, repositories of information, you don't really need to do that anymore. you can actually ingest all that into a NLP or Gen AI environment and say, okay, I want to have a summarization of all the case law in the UK and the US that ever had to do with property disputes between two owners that ended in violence.

I mean, that's not a great example. It's not a very happy example, but it's one that you could do and you would have the answer in the, in this, probably 20 minutes. before that was impossible. I mean, it wasn't impossible, of course, and it's possible with enough time, manpower and money, but certainly wouldn't be 20 minutes.

Richie Cotton: Yeah, that's pretty impressive going from something that's like just not feasible in any kind of reasonable time span to 20 minutes. That's a pretty great sort of win. so. Does this change how reports are written? And if everyone's just summarizing these long reports, has it changed Corporate approach to like all the reports and presentations and things that you generate.

John Thompson: That is the 10 million question, isn't it, Richie? the knee of the curve, you know, right now to see where we go from a productivity and a transformation perspective. there's many people out there postulating their opinions and ideas. A guy named Azim Azar. Out of the UK wrote two months ago that, professional services, firms of all types, including law firms and people like Bain and McKinsey and all those things are going to have to transform dramatically.

some people have said, you can reduce the head count by 90%. I think that's pretty aggressive but you can certainly reduce headcount significantly. We've done a wide range of testing and showed that you can, generate reliable code much faster, computer code. You can write reports faster, you can do summarization much more quickly.

and we're seeing the first cut a 30 percent improvement, and then when you do refinement in improvements and, other information is added into it, you can get that up to 60, 70, 80, 90 percent in some cases. So the question of the future of work, again, a big, big debate that's really going on across economies,

Richie Cotton: Absolutely. And I suppose in some sense if particularly with the consultancy side of your business, where it's humans giving advice, if AI can also give advice, there's some sort of rivalry there.

John Thompson: for sure.

Richie Cotton: So have you thought about how that affects your business?

John Thompson: We're having those conversations at all levels of the UI right now, and I'm sure all the other professional services firms are doing it too. So it's, how can you work in a regulated industry like we do and provide reliable, accurate you know, 100 percent accurate results. I mean, that's one thing large language models don't do.

it's probabilistic. And that's another thing that people have a hard time with is that we have moved beyond a deterministic world. We're in a probabilistic world now. So the things that you get out of these large language models are probability based. And the probability could be that the accuracy is 50%, 60%, 80%, 90%.

You just don't know. In early days, it was really mind melting for some people. They would put into some of the early models, what is 1 plus 1, and the answer is Wednesday. And they're like, how could this be? And it's like, well, model thinks that Wednesday's a probable answer. So, yeah, it's, you know, where are we going to go with this?

No one knows.

Richie Cotton: Yeah. So, this is maybe get you back in a few years time and see what's happened. , Um, so, uh, one other area where I think is pretty interesting is for decision making. So have you seen any uses of generative AI for decision making?

John Thompson: Yeah, there's some really cool things going on in Jenner today, right now. A lot of discussion about agents and agentic behavior. we're working on the continuum of simple agents to intelligent agents to self replicating agents to polymorphic agents, which is really cool stuff. right now, early days when, when I'd see people go in and use large language model, OpenAI, Mistral, whatever they were using, they used it very much like a search engine.

What's the capital of Singapore? We know it's Singapore, but, you know, they would ask, you know, what's the capital of North Dakota or what's the capital of New York? And, that's a search engine. You don't really need a large language model for that. So we have people using them in very simplistic ways.

You know, they send in a prompt, they get a response. the models really offer a great deal more capability and functionality than that. I saw one of our groups write a prompt that was, stretched the model at the time. The context window would be easy to do now. But it was a set of entire financial statements that were made up.

And they had made them up in a way that there was mistakes, there were, outright errors, there were and gross fraud. And when they ran it into the model, it found all of it and came back and said, well, these numbers don't add up and this, this is not the right accounting treatment.

And this looks like it's suspicious. And, you know, the transfer pricing and different things like that. So, the future of large language models is more in the line of automated intelligence and decision making. So trying to bring it back and land the plane and answer your question is that, we are going to a point where simple things like deciding, should I fly on the 6 a.

m. flight or should I fly on the 820 flight? We won't make those at all. Our agents will do that for us. You know, we'll give them all the rules and the parameters and the top line of what we want to spend and when we want to get out of bed and all the other rules, and we'll code those into agents and they'll go into large language models and they'll go in and say, Hey, I need to fly from Chicago to New York on Tuesday of next week and return on Friday.

That's the last thing you'll say. The whole thing will be booked for you. So, those, Agent infrastructures are moving in a way that simple, repetitive decisions will be taken care of for us. Now, some people are wigged out about that and they say, well, geez, you know, it's going to be a lot of jobs and there's going to be people that lose their jobs.

Yes, that is true. People will lose their jobs. The upside is that the jobs that we will be creating with the augmentation of intelligence is that there will be more fulfilling jobs. Yes. They'll be more interesting jobs. They will be jobs that pay better. So, just like we went through transitions from horses to automotives to, you know, manual labor to industrial labor, we're going to go through a transition that is going to be focused mostly on white collar work, where the lower end repetitive stuff is going to go away.

and the higher end, more sophisticated work that we as humans are better at. Creativity, intuition, and all those things are going to be more enriched.

Marker

Richie Cotton: Okay. So that's really interesting. Two very different examples there. So your first example was around being able to critique a large document and just come up with intelligent answers to that. And the second one's more about automating away just boring manual tasks, like which flight should I pick and uh, going and booking things.

Okay. So, That's pretty interesting, those different examples. One thing I'm also interested in is around other types of AI. So I know you're a big fan of causal AI, and actually we had a guest on a previous show who was saying, the generator of AI isn't what you want for decision making, it's causal AI.

Uh, So I don't know whether you have an opinion on this, on one versus the other.

John Thompson: Well, I do have my book on causal AI just over my shoulder, so, you know, I do have an opinion, of course, but yeah, I think generative AI is good for simple decision making. Like I said, Do I want to go on the 820 flight or the 6 a. m. flight? That's a simple decision that generative AI can make for you. more complicated decision making, that's not really what generative AI is there for.

I'm writing my fifth book right now, which is focused on causal AI. foundation AI and generative AI. And the way I see that coming together is we're going to have composite applications of all three AIs working together. Now, causal AI is very good for understanding what's the true causative factors.

you always hear about this, that, there's firemen around fires, so firemen start fires. That's, no, that's not it. of course, you want to get to the true causal factors, and it's hard for people to understand causality. We understand it in the abstract as a conceptual framework.

We've been discussing it since Aristotle, so I think we've got a pretty good idea of what it is. But as far as modeling it and mathematically understanding it, we haven't been very good on it. Now, the interesting thing is, is that generative AI, having ingested the entirety or much of the internet and the corpus of human knowledge, has all the information about causality as well.

So one of the things generative AI can help you do is generate causal pairs. So you have a better, more reliable set of causal pairs that you can feed into causal AI. Now, what does causal AI do for you? Mathematically, thanks to Judea Pearl and his, all the people he's been working with over the last 30 years, gives you the new calculus where you can actually mathematically understand X did cause Y.

And Z was a confounding factor. So you now understand this at a very detailed level. So you can go in and say, okay, let's understand. What is the price and whereby? People in this demographic buy one of these, and which price causes them to buy 10? So you really can understand causality in a very detailed, mathematical, proven way.

So I believe that we're going to have causal AI to give us that. We're going to have traditional AI to predict how many of these people are actually going to show up and do what we expect them to do. And we're going to use generative AI to generate treatments and interactions and invitations and incentives to give to those people to make them do what we want them to do.

So I see causal, foundational, generative AI. all working together as one system.

Richie Cotton: Okay, that's pretty amazing. I like the idea of causal AI. It's like, why did something happen? And the foundational AI is like, what's going to happen? And then the generative AI has some many use cases, but it's like probably feeding into the other types of models and then

John Thompson: Yep, it just becomes a loop, you know, a generative loop or a closed loop on AI.

Richie Cotton: And are you actually making use of causal AI at EY?

John Thompson: Right now, we're not, launched service called Synthetic Data Generation as a Service, and that's kind of the process we're going to use for causal. There's about 10 or 15 causal companies out there. Last year, we looked at the 37 leading synthetic data generation vendors. We did bench research.

We, check them out from a financial and size and momentum and product and all that kind of stuff, perspectives, and we brought five in and then we tested two very heavily and we're going to do the same thing in the cost of space. So in about. Nine months from now, we will have picked the causal vendor that we want to use.

And a year from now, we'll probably launch a worldwide causal AI service.

Richie Cotton: Okay, so since you mentioned synthetic data, can you talk me through why you need to use synthetic data at EY?

John Thompson: Synthetic data is, really the saving grace of the AI industry. it's kind of surprising, you know, two years ago, we all thought, Oh, there's so much data in the world, you know, we'll never run out of it. And here we are now where, you know, many of the large language vendors have said, Well, we've ingested everything we can get our hands on, and we're licensing all sorts of proprietary information, and we expect to run out of new data sources in about a year.

So it's like, whoa, that's a crazy turnabout of events. So now we're looking at it and saying, Okay, if we've got our hands, if our large language models have ingested all the data that we possibly could have in the naturally occurring world, Which not all models and just all information. I get that. Don't, you know, don't send us hate mail.

But you get to a point where you say, okay, I can't really get my hands on this information because it's a national secret or it's protected by, you know, all sorts of policies and laws and regulations and the people don't want us to have this anyway, for all good reasons that we shouldn't have it.

But we understand how the data looks. We understand the metadata about it. We know what shape it is. We know the patterns that are in it for seasonality and consumption. And we know the interrelations of the different elements. We know all that. We don't really need any data, which is generated from scratch.

So, you know, going to the EU and looking at GDPR and the EU Data Act and all the other things that are out there, Synthetic data allows you to comply with all those acts because it's not tied to a natural person or a natural organization. And you can create all sorts of interesting data, then you can train your models on it.

So you really don't need anything from the natural world anymore, as long as you understand what it is that you're trying to generate. So that's why we have a synthetic data generation service.

Richie Cotton: I have to say, I do find it absolutely crazy that there isn't enough data, like, naturally occurring in the world. I mean, there's so much talk about, oh, big data and data sets are getting ever larger and larger and larger. That sounds like maybe we're peaking and demand for data to train your models is now greater than what's actually being produced.

John Thompson: It's amazing, Richie, it really is mind boggling to think about it, I'm glad you actually said that, you know, because now here's something to really, even twist the mind even further is that So you say, okay, I've got all these different data sets, millions and billions of data sets, maybe trillions of data sets.

And I can integrate these two together. All right, now I've integrated these two together, and this is a new data set because it didn't exist before. But on top of that data, I can derive new data. So then I've integrated others, and I've derived data, and I've integrated the derived data. And then I've derived further data.

So I've got first order effects, I've got second order effects, I've got third order effects, I've got fourth order effects. And it just keeps going. So as long as you have the interest, the intelligence, the patience, the time, the compute facility, and the cash, there's no limit to what you can make.

Richie Cotton: that seems pretty cool. Although it does sound like there's quite a lot of infrastructure you got to put in place in order to be able to do this effectively.

John Thompson: Yeah, you're not going to do it on your phone.

Richie Cotton: Okay. And I suppose maybe the big hesitation with synthetic data is making sure that it is representative of, like, the real data set that you want to simulate. Can you talk me through how you go about doing that? Like, what sort of quality control checks do you need for synthetic data? Okay.

John Thompson: You know, for people like me and my compatriots that have been doing this for decades, we used to generate synthetic data. We never called it that. We called it fake data. We called it test data. We called it all sorts of things. We used to do it through Excel. It's like, all right, copy, copy, copy, copy, copy, from one year of data out to 30 years and use it that way.

The problem there is that you don't really understand, you know, the patterns. you understand the patterns in a year. there are the holidays and Thanksgiving, U. S. Thanksgiving and Halloween and all those things that distort and change the way things work and summer holidays and travel and all those things.

Those are all well known. when you start generating data on, at scale, You need to start looking at it and saying, okay, the seasonality, that stuff's easy to understand. Do I have an understanding of the co correlation patterns between the data? does this go up?

Does this go down? what kind of effect does this have on it? Does it have a negative, a positive? So you have to write all, it's almost like a test harness. for a software product. So you're writing a test harness for your data product, and you're running it through, and you're looking at all the statistics.

You know, does this fit with Gaussian distribution? And is this, you know, is this looking so awful, so unusual that it can't ever happen? So a lot of it is just statistically testing the different patterns that are in the data.

Richie Cotton: Okay, yeah, so just looking for anomalies, looking for, just, does this feel right, I suppose, it's that kind of feeling.

John Thompson: Yeah, you ask all sorts of questions. It's like, okay, what about, all the people in the data set that are 16 year olds that are married and have seven kids? That's probably not happening.

Richie Cotton: Okay so it seems like you've got a very sophisticated set of round data and AI at EY. For other organizations that want to replicate this what's a good place to get started?

John Thompson: a great place to get started is right in your own backyard. I have people ask me all the time, what should I focus on? And, if you're technologist or an AI professional or someone, that's supporting the business, the best person to go talk to is one of your executives, one of your operating executives, because people say, Oh, I don't know what to work on.

We'll go talk to the head of sales. They'll tell you exactly what you should be working on. Or the head of manufacturing or supply chain or the CEO or, you know, and just sit down and they're generally not going to say, Oh, I need you to work on this. You can ask him a question along the lines of, what's one of the challenges that keeps you up at night?

Or what's one of the problems you're having in the operational aspect of the business? And they'll start telling you, it's like, well, we can't get product into this area or our margins are down here. Those are the things you should be working on. That's where you should be focused. Yeah, once you, once you get your use case, then you can start looking at the technology and saying, hey, you know, we can build a solution with one of the families of AI, traditional, causal, generative.

And mostly everything's generative right now. So, go in there and say, okay, let's try to see if we could do something to improve the supply chain using generative AI.

Richie Cotton: Okay, that does sound like great advice is like go and speak to the people in the commercial teams, you can find out like what the real business problems are, and then figure out how the technology fits into that rather than starting with the technology and then figuring out what the

John Thompson: It never works the other way around. You know, building a tech stack and then trying to figure something to jam something into it generally never works. I always like to go to the business people and have them tell me what their problems are. And then generally, I build an ROI case first. I build a business case before I do any technology.

So I go in and say, okay, you know, when a previous company I was working in CSL bearing was a biopharma company, and we knew that in the height of COVID, we needed to get more people out of their houses into the donation centers. So that was the problem. What do we do to get messaging to them to motivate them to leave their house and come and donate plasma into our centers?

And that was easy to work on, and we knew how much a plasma donation cost us to pay for. We knew what the end result was. We had all the factors, so we just built a case and went back and said, We should do this.

Richie Cotton: And since you work with a lot of other companies have you had any success in helping other companies get better at working with data and AI?

John Thompson: You know, that's an interesting question. I do work at EY, as we've talked about many times. My job is ostensibly to build the infrastructure for EY's employees to use. My job really isn't to work with other customers at EY, although I do. People ask me all the time, Hey, can you go talk to this company or that company?

And right now it's, I can't state any names because that's not my job to do it. But we do have, what we do internally to EY is that me and my team are software professionals. We're not professional services, consultants by any stretch of the imagination. But what we build can be taken and copied or cloned and deployed into our clients environments.

And we've got two handfuls of clients right now. that are taking the innovations that we've built and are implementing it in their environments right now.

Richie Cotton: Okay, so, yeah, sharing the wealth with your technology setup, I suppose. Are

John Thompson: And that was one of the selling points for EY, was that, we would build things that would give them acceleration into the marketplace. So rather than going somewhere and saying, hey, do you want, this system like we've built, this 275, 000 people environment that's using Gen AI, We said, well, we have this environment.

Is this something you think you could use? And we take and clone it and drop it into their environment. And they're up and running in the matter of a few weeks. And then they pay EY for the value. I

Richie Cotton: there any AI trends or data trends that you're paying close attention to at the moment?

John Thompson: do think, you know, we touched lightly on agents. Agents is going to be huge. You know, right now, as I said, people are stimulating models with prompts, and other models are stimulating them through APIs and those kind of things. Agents are going to be the next big thing, and it's going to come faster than most people would ever realize.

We've taken some of our early stage applications And we've agentized them, that's not a word, but we've turned them into agents, and they work exceedingly well. So there is going to be an entire infrastructure of agents built over the next couple years that take care of all the things that we talked about, like travel and all these repetitive kind of things.

those agents will just be running in the infrastructure. Problems that I foresee is that it's going to be costly. So, you don't want to have an agent going out there running wild, and if you don't give it the right level of, boundaries to work in, you know, an agent can be working with other agents and they're sitting there trying to optimize and keep the cost of that airfare, down to a penny, you know, and they could go back and forth a billion times trying to save that penny.

I don't think it's really worth it. you have to give them the framework and say, okay, optimize this airline cost, you know, within a couple hundred bucks. And when you get close to stop, so, people have to be careful or you could have, runaway agents, which we'll see. It'll get in the news and, you know, it'll be unfortunate for some people, but we're working really hard to make sure that that doesn't happen in our environment.

Richie Cotton: I like the idea of using agents for travel. It's like we've had travel agents for a long time and they're coming back again uh, in AI form. Okay so that's interesting, the idea of Putting these like boundaries on the agents just to make sure that they don't do things that You don't want them to like spending vast amounts of money to save a penny you have any advice for how you might go about making sure that these agents are constrained?

John Thompson: Yeah, it's, it's really not that hard. You know, some people are like, Oh, that sounds really difficult. It's not, it's really comes down to prompt engineering and putting. Either the system prompts in place or the prompts in the application in place that says just what I said earlier. Hey, once you get to the point of the airfare being optimized within 200, stop.

just don't work on that anymore. Go book the hotels and the transfers or whatever it is. So I do think agents are going to be a big thing. I also think synthetic data is going to be a big thing as well. We're one of the early movers, but I think Satya Nadella, in his most recent quarterly earnings call, called out a company called Gretel, which is the company we chose as our synthetic data vendor.

Called them out as one of the next big things out there. So agents, synthetic data and then I also believe, as we said earlier, composite applications that have three families of AI underpinning them also going to be a big one.

Richie Cotton: Okay, so yeah, three things to watch out for then. So agent, synthetic data, composite AI applications. Okay, so, just to wrap up, do you have any final advice for any businesses that want to make better use of AI?

John Thompson: I've now spoken to coming on to nearly 300 clients for EY. Like I said earlier, that's not my day job. My day job is to build the tech and make sure it works. But given our experience, lots of people want to hear what we've done. So my advice to all these organizations is get started now.

you have lots of data. You have lots of, lots of interesting information. that you could be leveraging better and could be leveraging today. So, yes, AI is moving fast, and new models are coming out daily, and everybody's, saying, Oh, my model's better, my model's Yeah, sure, everybody's got great models.

They all work really well. Get started. and, like we talked about earlier, you have challenges. You don't have to make these challenges up. You know, you have a pricing problem. All right. Give it a go. See how it works with it. And maybe Gen AI does work well with it. Maybe it doesn't. Then you can either try traditional AI or causal or lots of tools.

You know, we have an embarrassment of riches of tools right now. So my advice is just get started.

Richie Cotton: I love that. Yeah, just get started. Like, I don't think there are any organizations that are like, oh yeah, we don't have much data or we don't have any business challenges that we need to solve. So yeah the

John Thompson: We have no issues.

Richie Cotton: let's go. Yep. All right, super. Thank you so much for your time, John.

John Thompson: Richie, it was a pleasure. Look forward to the next time.