Skip to main content
HomePodcastsArtificial Intelligence (AI)

What Fortune 1000 Executives Believe about Data & AI in 2024 with Randy Bean, Innovation Fellow, Data Strategy, Wavestone

Randy and Richie explore the 2024 Data and AI Leadership Executive Survey, the impact of generative AI in 2023 and what to expect from it in 2024.
Jan 2024

Photo of Randy Bean
Randy Bean

Randy Bean is a start-up business founder, CEO, industry thought leader, author, and speaker in the field of data-driven business leadership. He serves as Innovation Fellow, Data Strategy for Paris-based consultancy Wavestone. Randy is the creator of the Data and AI Leadership Executive Survey discussed in today's episode. He is the author of the bestselling "Fail Fast, Learn Faster: Lessons in Data-Driven Leadership in an Age of Disruption, Big Data, and AI", and a current contributor to Forbes, Harvard Business Review, and MIT Sloan Management Review.

Photo of Richie Cotton
Richie Cotton

Richie helps individuals and organizations get better at using data and AI. He's been a data scientist since before it was called data science, and has written two books and created many DataCamp courses on the subject. He is a host of the DataFramed podcast, and runs DataCamp's webinar program.

Key Quotes

Artificial general intelligence, this notion where basically computers can perform any cognitive task better than a human being. People have asked leaders, is this like five years old or seven years out? And the answer was, well, three months ago we thought it was 5 to 7 years out. Last month we thought it was like three years out. We think it may even be in the lab today.

The important thing to understand is that data is the foundation to a good AI function. So if you don't have good data, you're not going to have good AI in terms of the outcomes, in terms of the predictive analysis that you're performing. So a successful AI is predicated upon data which means that if you want to be successful with AI, that increases the urgency, the importance, the momentum, everything around basically building out your data structure and making sure that you're working off a great foundation. So in that regard, it really, the use of AI only emphasizes the importance of data and as organizations go more and more with AI, they'll see that the quality of their data.

There's a direct correlation between the quality of their AI initiatives and results and the quality of their data.

Key Takeaways


Data is the foundation of a good AI function, there's a direct correlation between the quality of AI initiatives and results and the quality of data. If you want to be successful with AI, that increases the urgency and importance of building out your data structure and making sure that you're working off a great foundation.


Develop data literacy programs and initiatives to bridge the gap between data professionals and business leaders, ensuring effective communication and collaboration. This leads to greater buy-in from leadership into data, accelerating the effectiveness of data and AI functions as a whole.


Always maintain a 'human in the loop' approach to guide AI processes, balancing the benefits of automation with the critical input and oversight of human judgment.

Links From The Show


Richie Cotton: Welcome to DataFramed. This is Richie. It's very easy for managers to become solipsistic. That is, it's all too common to become focused on internal company issues and not think about what your competition might be doing. You know you're making progress, but you aren't quite sure if you're progressing fast enough.

That data transformation program seems to have been going on a long time, but you don't seem to have made money from it yet. Does everyone else really have their generative AI projects in production already? To clear your mind, today we're going to talk about the results of the Wavestone Data and AI Leadership Executive Survey.

This is one of the longest running surveys of chief data officers and other blue chip executives, so you'll get a chance to hear what other companies are thinking and doing about data and AI. Our guest is Randy Bean, the creator of the survey. Randy is the innovation fellow for data strategy at Wavestone, and he was previously the founder and CEO of New Vantage Partners before Wavestone acquired it.

He's the bestselling author of Fail Fast, Learn Faster, and he regularly writes in the Harvard Business Review, Forbes, and the MIT Sloan Management Review. He's also incredibly well connected amongst Fortune 1000 executives so hopefully we'll get some personal insights out of him as well as the latest survey results.

Let's find out what all these Chief Data Officers are up to.

Hi Randy, thank you for joining me on the show.

... See more

r="ltr">Randy Bean: Hi, Richie. How are you today?

Richie Cotton: Doing all right, thank you. I'd love to dive straight in. Can you just explain to me what are the goals of your data and AI leadership executive survey and who gets surveyed?

Randy Bean: Yeah, so this is the 12th year that the survey has been conducted. It first started in 2012. And at that time it was at the behest of a series of fortune 1000 data and CIO executives. And they basically want to understand. Data had reached that point. If you remember back the term big data had started to come into use And data had been elevated from something that was hidden off in the corners of the organization to now something that was a c suite concern So these executives want to And whether basically it was just their organization or whether it was pretty common across the industry.

So we start conducting the survey. The survey is aimed at fortune 1000 C executive. For example, to be precise, I'll just look up the numbers here this year. 8 percent of the respondents held the role of, chief data officer, chief data and analytics officer are basically the equivalent role in their organization, like head of data and analytics and of the respondents.

Roughly 94 percent were C suite executives. So that's the audience. That's the respondents and it's 4 to 1000 companies and global leaders.

Richie Cotton: Okay, so it's a pretty comprehensive survey of all these sort of leaders of big companies. I see, when you said like, 2012 was when sort of big data hit the mainstream and data grew popular. It seems, I suppose, it seems like a long time ago, but it's a decade. I guess the change this year has been generative AI has hit the mainstream.

So do you want to tell me how that's changed things?

Randy Bean: It's really changed everything and in the forward to the survey, which is co authored by Tom Davenport and myself, basically what we say it's generative AI has transformed everything. It's really elevated the whole discussion, the whole urgency around the importance of data and analytics and organizations.

We dedicated a major portion of the survey to generative AI and I could share some of the data. For example, we asked our. Investments in generative AI a top organizational priority and roughly two thirds said that it was, which is pretty good considering it's only been a year. As you affirm increasing investment in generative AI, 89.

6 percent said that they were. Do you believe generative AI has the potential to be the most transformative technology in a generation? Roughly two thirds answered that it was. So it's, it's definitely getting a C suite and board level attention. Here's a good one. Is there a need for safeguards and guardrails for governing generative AI?

99. 0 percent said yes. I'm wondering who the 1 percent are. Oh, yeah, no, just like, let it rip. Then we asked, you know, are the safeguards and guardrails for generative AI in place in your organization today? 62. 9 percent said yes. There's a pretty good gap. And some people initially I've shared this finding and some people said it's not even close to 62.

9 percent of people either lying or overly awful. And then we asked, is the talent in place to responsibly implement? Generative AI. And this is a nice answer because 50. 5 percent said yes and 49. 5 percent said no. So clearly uh, generative AI is top of mind. And the other thing that I'd add. is each year we ask a number of questions about the progress of data and analytics within your organization.

For the first time in the past five or six years, the numbers jumped pretty significantly, and we attribute it to generative AI and the elevated attention that all data and analytics initiatives are receiving now. Well,

Richie Cotton: That's pretty um, impressive. Just the shift just from 2022 to 2023. And one thing I'm wondering is basically everyone's said, yeah, generative AI is really important. we're thinking about it, it's top of mind. I'm not sure who's or how many people are actually implementing things yet. Like who's starting working on this?

Randy Bean: So we asked about the state of generative AI implementation efforts. 60. 4 percent said that they were in the experimentation and testing stage. 24. 5 percent said that they had implemented in limited production. So, you know, A case here, a case there.

6 percent said they were in the planning and design stage, so the earliest stage. Only 4. 7%, so less than 5%, that they'd implementate, implement, implemented in production at scale, so that's the highest. And then 3. 8 percent said they're doing absolutely nothing at all.

Richie Cotton: Okay, so two extremes, but it seems like the prototyping areas is maybe the most common thing. So people are toying around with what's possible at the moment, but not necessarily doing things at scale yet.

Randy Bean: Yeah, a lot of talk, a lot of experimentation, but I published an article in Forbes about Ally Financial, and they released some metrics, quantitative metrics about how they're using generative AI to increase the productivity of a series of their customer service tasks, and I wrote about the, 66 percent improvement, 88 percent improvement in various efficiencies that they're getting.

So they went public very quickly with some of the results. to show the industry what they're doing. And they're purely a online bank, so they're innovative by nature. But it was it was really some of the first data that I'd seen that organizations were quantifying or at least sharing their quantified results.

Richie Cotton: That's pretty impressive that they're already they've made that progress just in the last few months. So you gave an example of an online bank. Are there any areas or projects that you see a lot of organizations investing in around data and AI at the moment?

Randy Bean: I see most of them focused around productivity improvement. And we did ask the question primary business opportunity created by generative AI. And 49. 1 percent just about half said achieve exponential productivity gains. The other two areas were liberate knowledge workers from mundane tasks.

And that was another thing that Ally Financial was that they could free a lot of their workers to do a higher order type of activities, more creative or more synthesis type of activities. And then 22. 6 percent said improve customer service and experience. And then 4. 7 percent said other, I'm not sure what the other are.

But a lot of the activities around customer service, particularly in terms of activities that are pretty standard in terms of communication with the customer and how they can be improved and made more efficient and streamlined.

Richie Cotton: Okay. I mean, I like those three areas. They seem pretty sensible. So improve your productivity, automate boring stuff, and then give a better experience for your customers.

Randy Bean: Yeah, and by the way, I, I happened to go to this event, I live in Boston, at Harvard University in October, and it was with leaders from Harvard Medical School, and they said that they were AI. And they had two pieces of data to share, and I, as they were sharing them, I texted this data to my wife, who's in the healthcare industry.

So the first one was that it said that generative AI did a better job of diagnosing patient illnesses than doctors did. In 90 percent of cases, and they related an instance where there was a certain set of symptoms and the person went to 10 different doctors and it was slightly different diagnosis.

And then they asked them generative AI and it produced a result. And I went back to those 10 doctors and they said, ah, you know, actually that's the correct diagnosis. aNd so I texted that to my wife and she goes, yeah, makes sense. And then the second thing was, they said that in 80 percent of the time, generative AI was reported by the patients as being more empathetic than the doctors, which I thought was amazing.

And I sent that to my wife and she said, yeah, definitely. And a week later, I was hosting a chief data officer panel, and I had the chief data officer from Mayo Clinic. And so I said, Ah, you know, I basically surprised him on stage and said, Hey, you know, what about these results from this study at Harvard Medical School?

He said, Absolutely. He said, That's exactly what we see in terms of efficiency of diagnosis and empathy of the doctors.

Richie Cotton: I have to say the empathy one surprised me a lot, but now I think about it. Yeah, you can, can set the tone of voice for these generative AI tools. Yeah, it can be consistently nice. So one thing that we, talked about a lot on this podcast in the last year is the idea that In the near term, it's probably going to be humans and AI working together.

But in this case, you were saying that actually AI is just outperforming humans a lot of the time. So do you see that as being AI replacing humans rather than augmenting them?

Randy Bean: Probably the key thing is that I was at an event in Boston last week, a chief data officer summit and organized the kickoff keynote panel for chief data officers. And the second day was chief AI officers. It was billed as the first I've been bringing together chief AI officers and it was held at Northeastern University at a new center.

They've established the Institute for experiential AI. And was speaking with the director of that institute yesterday and the main and he's done, he has his PhD. He's been doing work in the space for 30 years. He led AI research for Microsoft for a number of years and Yahoo. He said the single most important thing is the human in the loop, because he said that basically, and this is interesting, so he, he's a leading international expert, and he said it was really that contextual human experience that could guide the AI process.

Now, in October, I went to the Wall Street Journal Tech Live event, which was interesting because it's an invitation only event. And they have people like Vinod Khosla and others, but the keynote speaker was Sam Altman. So he's there. There's about 100 people. And he shared some things.

A week later, he was fired. Two weeks later, he was rehired. So I had written about the event, and all of a sudden it became like viral. But, what he was talking about was artificial general intelligence, this notion where basically computers could perform any cognitive task better than a human being.

And, question of the discussion was, is this like five years out or seven years out? And the answer was last month we thought it was like five or three months ago, we thought it was five to seven years out. Last month we thought it was like three years out. We think it may be in the lap today.

So again, there's that importance of the human in the loop to guide the process, but there's also this notion of artificial general intelligence where a machine can perform any cognitive task better than a human being.

Richie Cotton: That would be an absolutely revolutionary occurrence if we do get to that point. And especially, like you say, it could be in the lab already. That's a fascinating and also mildly terrifying thought. Before we carry on with the artificial intelligence, let's get back to the survey. So we're talking a bit about what organizations generative AI at the moment.

Do you see a difference between industries? What's going on there?

Randy Bean: That's a good question. The survey respondents. I actually have a breakdown of that. So I can give you an idea this year. 50. 9 percent were from financial services and financial services have always been the largest respondent in these surveys because they. are very data and analytics driven and have invested in data and analytics initiatives for, over a generation.

But what's really exploding is healthcare and life sciences. So this year, 15 percent of the respondents were healthcare and life sciences company. And a lot of that was fueled during the COVID period, the race to develop vaccines and sharing data across clinical groups. We also had 11 percent from retail and consumer goods, and they've been very aggressive because whereas financial services firms are to a certain degree healthcare and life sciences, those are industries that are heavily regulated.

So a lot of the work is defensive. For consumer and retail, it's basically offensive. I, I moderated the keynote panel in Boston a year ago in October, and I had a woman, Diana Shilhouse, who's the chief data officer for Colgate Palmolive, or actually her title is the chief analytics and insight officer.

And I had three banking chief data officers and I asked, how much of your time do you spend on offensive activities for our new generation business growth? And how much time do you spend on defensive activities in terms of regulatory and risk management and compliance? And the bankers were all like, you know, we're aspiring to spend maybe 30, 40, if we could get to 50%, that would be great in terms of offensive activities.

And I came to her, she goes, 100 percent offensive. It sounded like cheering. So it's um, you know, it it depends what industry that you're in. I wrote a piece last year on Levi's, they're on the fashion business. So they're in the business of change trying to anticipate trends and needs as opposed to, when you're in financial services, it's about safety, security of the assets basically being able to And still the same level of confidence that you have for 150 years, if you're American Express or Wells Fargo or Bank of America or Citigroup, all of which have existed for basically 150 years.

Richie Cotton: I suppose it's going to be the more regulated industries that have to do a lot more defensive work because otherwise it's going to get sued into oblivion or fine. So can you tell me a bit more about what some of these business risks companies are worried about in terms of using AI?

Randy Bean: yEah, I can answer that in a couple of different ways. First, in terms of the defensive activities remarkably this year, particularly in financial services, there was much more time spent on defense more than there had been in the past five or six years, which was prompted by earlier this year, the failures of Silicon Valley Bank and First Republic Bank.

So now all of a sudden the regulators came back and particularly for the Kind of tier two and tier three banks. The really large banks had had to undergo that level of regulatory scrutiny. So a lot of the CDOs that I worked for, for the the large regional banks and so forth were like, yeah, no, we're not focused on business expansion activities.

We're spending all of our time with the regulators. So that's part of the answer in terms of defense versus offense in terms of could you just share the second part of that question about a,

Richie Cotton: Yeah it was just about what are the risks that executives are most worried about with respect to AI and generative AI in particular.

Randy Bean: yeah there's huge risks. we asked the question about what is the primary business risk posed by generative AI? So I think I have an answer for every question. By far the largest 44. 3 percent said spread of misinformation or disinformation.

Particularly when you look at social media or media of any kind, there's the opportunity to basically present anything that fits your audience. The second one was at 23. 6 percent was ethical bias at 5. 7 percent was job loss and job displacement. And then the other things were everything else.

The other thing I should note was. That we also asked about the state of data and AI responsibility and ethics. foR example, we said, Is data and AI ethics a top priority for your organization? And roughly three quarters, 73. 8 percent said it was. We asked, does the Board of Directors well versed in data and AI issues and responsibilities?

51. 4 percent said yes and 6 percent said no. I will add there that on November 1st, I was in New York and I did a panel at the National Association of Corporate Directors, which is basically 4 corporate directors. And it was really to talk to them about generative AI and the issues and concerns and what they should think about because they themselves didn't feel that they were sufficiently versed.

to bring those perspectives to the corporate boards they were on. We also asked does your organization have well established policies and practices in place around data and AI responsibility and ethics? Less than half, 42. 3 percent responded that they did, so a lot of work to be done there. And then we asked, has the industry done enough to address data and AI ethics, and only 15.

9 percent said yes. Roughly 85 percent said that the industry had a lot of work to be done.

Richie Cotton: So it sounds like there's, there really is a lot of work to do in terms of educating executives in terms of what does responsible AI actually mean, and how do you actually make sure it happens in your organization? Does that sound like a reasonable interpretation?

Randy Bean: Yeah. And earlier this week, I spoke to a group that I was just introduced to called the Responsible AI Institute. And they're working with it. They were and I shoot five or six years now out of University of Texas, Austin, and they're working with a number of fortune 1000 companies to help develop the responsible AI standards and practices for their or for those organizations.

So they have a template in place. And I was introduced by one of the fortune 1000 companies that work with.

Richie Cotton: Okay, it's good that these organizations exist just to try and help educate these leaders. We talked a bit about risks to businesses but you also mentioned, I think it was like 4 percent of companies just not touching generative AI at all. I'm wondering what the outright blockers are to adoption.

Randy Bean: would say it's the same issues that have been barriers to adoption for any data and analytics initiatives within organizations. And it's a question that we've asked over the years, and we say, basically, what is the principal challenge to becoming data driven or data and AI driven?

And the short answer is that the answer is consistent. We ask whether it's culture, people, process and organization of technology limitations and almost universally 77. 6 percent this year said it, had nothing to do with technology. It was all about culture, people, change management, communication.

really all of those issues that it takes to embed or launch a capability within an organization. There's a lot of natural resistance, a lot of lack of understanding, sometimes tell the story about going into organizations and I meet with the data leaders and they say, Oh, here's all the capabilities that we're creating.

And I said, that's very commendable. And I meet with the CIO and the technology organizations and they say, here's the architectures and the infrastructure that we're creating. And then I go and meet with the line of business leaders. And I say, so are you getting a value out of your data and analytics investments?

And they're like, Not as much as we'd like we actually don't trust the data, we're not getting the data that we need to make the decisions that we need to make. We're not getting into the timely fashion. And on top of that, we don't understand what the data and technology people are talking about.

They talk in a language that we don't understand. And master data management, data mesh, data fabrics. We don't know what that means. We're the ones running the business and they're speaking a language over our heads. And, we're the ones that, have to keep the customers coming in and make a profit and derive revenue.

So there's been this antipathy or antagonism where a lot of the businesses get frustrated because they're just saying, Hey, you know, we're, we're running a business. We need to sell stuff. And there was a story from a number of years ago where a chief data officer, I know, went into the president of the organization, said, I need 25 million to build the master data management capability.

And the president said, I have no idea what that means. So the answer is no, until you can speak in business terms, like, please leave.

Richie Cotton: It does sound like, although there's been a lot of progress this year and lots of success stories in terms of making progress with AI, it's actually the fundamentals, the data quality, the data governance is the hard part executives seem to be struggling. Does that sound right? Like where were the biggest pain points?

Randy Bean: So we've been asking this question for six years now. We ask about the progress of data and analytics initiatives. And I'm going to note that these numbers improved significantly this year. And, I had mentioned at the outset that we thought that this was due to generative AI, but I'll share last year's number, which had been basically flat for five years, and this year's number.

Yeah. Is your organization driving business innovation with data? Last year, 59. 5 percent said yes, which was the exact same number as 2019. This year, it jumped up to 77. 6%. So significant, whatever that is 20%, 30 percent increase. We said, is your organization competing on data and analytics?

Last year, 40. 8 percent said yes. So less than half. Which was down from 2019 when it was 47. 6%. This year it went up to 50%. So half said yes, half said no. It's a little bit like a glass half full or half empty. So a lot of progress, but a lot of work to be done and a lot of need and demand for employment in the future.

We asked, are you managing data as a business asset? Last year, it was only 39. 5%. This year, it jumped to 49. 1%. So pretty significant, again, all in this context of generative AI. We asked, have you created a data driven organization? Last year, it was pitiful. 23. 9 percent said that that's where they were this year.

It more than doubled to 48. 1%. So that's extraordinary. I hadn't seen that in any previous version of the survey. And lastly, we asked, have you established a data and analytics culture? Last year was 20. 6%. 1 in 5, but this year jumped to 42. 6%. So again, We attributed these massive increases to the attention that generative AI has brought to the organization, how it's really elevated data and analytics in all areas of the firms.

Richie Cotton: Okay, so that does sound incredibly positive then that there's been a lot of progress made in core analytics just as a result of the sort of hype from generative AI.

Randy Bean: And to your question one other data point we did ask Have efforts to improve data quality been successful? 37 percent said yes, so long way to go in data quality. I don't know if data quality will ever be solved.

Richie Cotton: Yeah, I mean, I guess you can always have higher quality data and it's always one of those things where no one really wants to do it, but it just needs to be done. Yeah, okay. Do you have any more of a sense of like what's causing some of these problems? Like, cause a lot of these cases where it's like, is your organization really good at data?

But it sounded like it was generally around half. I think that they're doing okay and the rest are struggling. Do you have a sense of like where the problems are here? Like what's preventing greater success with data?

Randy Bean: I would go back a little bit to the cultural issues. Maybe we could talk a little bit about the role of the chief data officer. But the point is that This role didn't really exist for the most part prior to 15 years ago, and for major banks who were the early adopters, it came about as a result of the 2007 2008 financial crisis.

For example, we asked has your firm appointed a chief data and analytics officer? And in 2012, Only 12 percent said yes. This year, 83. 2 percent said yes. A couple things to keep in mind. The rules still are relatively nascent. A generation ago, there was the old joke that CIO stood for career is over because it was so volatile.

And that's a little bit what chief data officers have been experiencing. Tom Davenport and I wrote in Harvard Business Review. A year and a half ago that the average 10 year was roughly 24 months. We've actually seen that go go down in a number of organizations in 2023. We saw more turnover in fortune 1000 chief data officers than in any previous year.

Just a couple other things.

Richie Cotton: Sorry, that's just quite a surprising statistic that like chief data officers last less than two years and it's it's getting a shorter tenure now. Do you have a sense of what might be driving that?

Randy Bean: yeah, absolutely. Two, two factors. One is this year there was a push to deliver business value. And a lot of that was driven by economic factors. So organizations had concerns how much money they'd have to spend and they had to make decisions about where they spent it and didn't spend it. And over, the past five, six, seven, eight, nine years, organizations every year.

You know, 98 percent of them said they were spending more on data and analytics initiatives. But there was still gaps in terms of achieving quantifiable, measurable business value in terms of profit. revenue generation, customer acquisition, things of that kind. So a lot of organizations have been spending time on building infrastructure, architecture, capabilities, and so forth.

So early on this year, a number of organizations, the leadership turned to the chief data officer and said, so where's the quantifiable results from this money that we've been investing over the past few years in data and analytics initiatives? And they're like well, you know, it's like we're building capabilities, but we don't necessarily have those numbers.

And in some cases, the executives said then, you know, you're, you're terminated. And in other cases, they said well, you know, you have like 60 days to like produce these numbers. And as a consequence, some people, basically they, they weren't in a position where they could do that.

And others just said, we're not going to deal with this. This is just like irrational in the sense that all of a sudden the clock has turned so suddenly and now it's these demands are being made that we're not set up organizationally to respond to. The other factor, in addition to the demand for demonstrating business value, Was generative AI and with its rapid emergence, many executive leaders turned to the chief data officer and said, Okay, so what is what's the impact of generative AI?

What should we be doing? And, those chief data officers like the rest of us were learning, you know, they didn't have, like Automatic answers. And so as a result of that, a lot of the organizations that, you know, we need to get somebody in place that can figure this out now, where we can't wait.

So it's the role this year has really evolved more than any other year, and it continues to shift. And as I mentioned, I was at the event last week, and there was a chief data officer, chief data analytics officer day, and And then the following day was Chief Artificial Intelligence Officers. This is the creation of this new role.

And one of the questions that we asked was the in terms of the Chief Data Officer mandate. So we asked a few things when the role first started, you know, as chief data officer, and it was a lot about data management defensive activities, and then as organizations start to move more onto the offense, they said well, we need the analytics combined in there.

So it evolved from CDO to CDAO role. So we asked, is our analytics today part of the CDO, CDAO function within your organization? 70. 8 percent said yes. So analytics is very much integrated for most organizations. But then we said, is generative AI part of the CDO, CDAO mandate? And 61. 7 percent said yes, so slightly more than half.

And then we said, should AI be part of the CDO, CDAO mandate? And 79. 4 percent said yes. So clearly, it's a work in progress. it's a turf thing, too, where organizations are starting to say, you know, this generative AI, it has transformational power for our organization. It's that.

Something that fits under the role of the chief data officer, because maybe without the chief data officer more as a specialized type of role. Is this too big for that? where does it fit within the organization? So it's very much a work in progress from what I'm saying.

Richie Cotton: It does seem like, someone who wants to be in charge of massive data management and making sure that metrics are harmonized across different departments. That's a very, very different role to how can I, improve productivity using generative AI.

Randy Bean: Yeah. And, I'm often asked about why is this gap between in the 10 years of chief data officers and why some chief data officers are successful and some are not successful. And initially, there was a lot of feedback or comments that the business leaders don't know what they're doing.

They don't really Have a firm plan in terms of how data and analytics fit into the organization and the overall picture of the other side of that is that a lot of people were promoted or appointed into the chief data officer and chief data officer, chief data and analytics officer roles were fundamentally subject matter experts.

So they weren't business people. They didn't necessarily Yeah, probably. Have the vision of how data and analytics fit into the overall profit picture and business goals of the organization. They were new to the C suite in terms of both the communication and the turf fighting and the sharp elbows and building relationships and sponsorships.

So they were kind of a little bit like, sheep to the wolves, if that's the right analogy. has been a learning experience. Some chief data officers now are saying, you know, I'd never do that again. And are opting out or opting for different goals. And now there's a new, a next generation of chief data officers stepping in, but the, expectations continue to change and evolve.

But, you know, this shouldn't really be shocking because, as noted, it's 10 to 15 years into it. And, early on in my career. I was working with a Fortune 1000 CIO who was vice chairman of their organization. He said to me, you got to understand that nothing happens in less than a decade.

and I was like, wow, that's a very cynical point of view, but now I have lived through many decades of experience that, particularly for legacy mainstream companies, they Undertake change and adopt change in a very gradual fashion, they're trying to be responsible, their jobs not to be bleeding edge, that's left with the Googles and Apples and Facebooks and Microsoft and others, they're competing with the others in this sector, if you're an insurance company, you know, you know, going to do like cartwheels, you're going to basically continue to serve your product.

That's why, organizations like Ally Financial that I mentioned or Capital One in Banking, have been revolutionary in the context of their particular industries because they've been very data analytics driven as opposed to traditional product and service approaches. Yeah, for mainstream companies, change takes time and it tends to happen more over period of decades.

I mean, we're 25 years into the digital revolution, digital transformation. And I know a lot of the organizations I work with said during COVID that that's where they made their exponential leaps in terms of the digital capabilities. There was no transformative urgency before that. But when people couldn't come into the branches, or their offices, they had to figure out how to deliver all their services in online digital fashion.

Richie Cotton: Like maybe I've spent too long in the startup because decade seems like a possibly long time. So I actually, I was going to ask you like how you think the chief data officer role is going to evolve over the next five years. Maybe it needs to be a longer time period. But yeah, what do you think is coming next chief data officers?

Randy Bean: You know, I think That's the crux of the issue. The most important question now in terms of that role that function know, right at this moment, it's a work in progress. It's the issues of does generative AI is the responsibility of the CDO or not?

I mentioned a panel that I did in October in Boston, the CDO event. I had three panelists. I asked that question to three panelists and two of them said it should absolutely be part of the CDO role. And the third said it should absolutely not be part of the CDO role. And some of the reasons they pointed out was that.

There's so much risk and regulatory and privacy and ethical issues that need to be dealt with first that it really needs that type of safeguards and people that are experienced with that and bringing new capabilities to bear as opposed to just saying, you know, let it rip.

It's a new function and the whole set of considerations that come into play relative to the role in general. You know, It's moved more from the defense to the offense integration of the analytics capabilities. Now we'll see where I fits into that. And also there's been an elevation into where the role reports in the organization.

For example this year, roughly 35 percent of chief data officers are now reporting to the CEO or chief operating officer. Five years ago, most of the chief data offices reported to the CIO. So it's really moved out from under the CIO function in terms of being a peer function now and greater elevation.

In the C suite hierarchy.

Richie Cotton: That's interesting how the reporting structure's changed. Now you mentioned data culture before, so I'm wondering if you've seen any changes in executive attitudes to data in the last year.

Randy Bean: Yeah, in the past year, there's been a significant transformation. And again, that's been driven by generative AI because it's produced an urgency to potentially see how this technology can be leveraged to you know, achieve the productivity gains. Elevation of job responsibilities, et cetera. So if before for many organizations, the idea of data and analytics was a little bit abstract.

In other words, yeah, of course, we should be data driven, a little bit like motherhood and apple pie. But, now it's experiencing that level urgency where organizations are saying, we really need to think about this in a serious fashion, we're at a transformative juncture because, for example, you know, when I went to the Wall Street Journal Tech Live event, people were asking me, oh, you know, what's happening with AI?

And also organizations were saying, can you come to speak to us about AI? I'm like, you know, I'm not, I don't feel qualified. I'm not an expert. It's moving too quickly, et cetera. But my biggest takeaway from the Wall Street Journal Tech Live event was that it's inevitable and it's happening quickly.

So whether you love it or hate it, it's going to happen. So you've got to get over it and you've got to get your arms around it and plan for it, in the same way that The railroad shortened, timeframes, the invention of the airplane and shortened timeframes. The engine increased productivity.

You know, I was reading something the other day where you talk about the railroad or the airplane, but it was about the development of north America and Native American peoples. And it said when the horse was introduced, from Spain and from Europe, all of a sudden it made, you know, there was.

Thousands of tribes, but they never interacted. There were a couple miles apart, but now with a horse, people could get there in, five minutes. Generative AI will undoubtedly have an impact of that. Magnitude and the other thing is that people say AI can destroy everything.

It can destroy mankind and a lot of arguments can be made for that. But at the same time, it's just a tool. And any tool Can be used for good or bad, you know, somebody was asking me the other day and I was just thinking about the most simple thing that came to mind. And I said a screwdriver, a screwdriver can be used to fasten things, but a screwdriver can also be used to like, bludgeon, somebody if you were slow and blind.

Technology, any type of technology or any tool can be put to very good and positive uses. All kinds of things in terms of curing diseases scientific knowledge medical healthcare advancements that can happen exponentially through generative AI. And organizations are working on that now, instead of going to a doctor once a year, you can be monitored like throughout the day, hour by hour.

But there's also, particularly around social media, elections, things of that kind uh, leadership. There's a lot of dangers of misuse, propaganda things of that kind, many positive opportunities, but also some really negative ones. I mean, you can create images now that, say, oh, here's the proof in the photograph, or here's the proof in the film pictures that we have but they can be created.

And I think we saw something the other day where they had police arresting Trump and cutting him away, but it was just generated

Richie Cotton: Absolutely. So I suppose Fake images have been around for a couple of decades now via Photoshop, but this really does scale up the ability to create these sorts of things.

Randy Bean: absolutely.

Richie Cotton: Okay. So I'd like to go back to data culture a little bit, cause you mentioned that at this point, executives are saying, yeah, we need to be more data driven, but I think the start was less than half think that companies actually are data driven.

What's going to change that culture to increase the stat?

Randy Bean: One answer is time. Because change, change management tends to be gradual, particularly with regular, mainstream legacy companies that I'm risk averse, and I'm going to move at the pace of their peers, and I'm not going to try to beat Amazon, Google, etc.

Richie Cotton: So one common refrain we've had from some of the other guests on the show is that data skills are still a common, or a lack of data skills is still a common problem. So do you think this is still true or has there some progress been made?

Randy Bean: I just came up with the second part of my answer to your previous question, and that is generational. younger generations are now more schooled and it's second nature to use various technologies as opposed to C suite executives that may be in their, late 50s, 60s and so forth.

So for them, it's a transformation effort. It's this is the way we've done things. Yes, you know, we're going to be sensitive. That's the. Data and A. I. Capabilities. But it's just not something that's in our D. N. A. And in that nature, particularly within mid management levels. But as next generation comes along, it seems like a long time because you know what's a generation 35 years.

But some of that is just That it's hard to teach an old dog new tricks, but a new dog where it's in the DNA, it just becomes something that's more natural. So some of that is just a function of time in that regard. In terms of data skills, I know that there's a lot of investment and initiatives like, data literacy and things of that kind.

All of that is good, but, too often I hear those terms that are just, that they use loosely like, oh, if only we were data literate, if only, you know, we had democratization of data. you know, What are those things really mean? I mean, the most successful organizations, from my experience of those organizations, where the, Data leaders, the chief data officers, are establishing strong partnerships with their business peers, the business leaders.

They're engendering their trust. They're using data to answer, one question at a time and being able to do that establishes the credibility and trust. And when you can do it two or three or four times, that establishes momentum and traction. But. Not enough data leaders are doing that.

So it's my experience that the data leaders that are really hand in hand partners with business leaders are the ones that are successful. They have somebody guarding their back. There's somebody that's an advocate for them. I know what JP Morgan Jamie Dimon at the most recent annual meeting.

He said, there's four factors that are most important in our results in our future, not success. And I think two or three was data and analytics. So he was out championing the importance of data and analytics as opposed to, a number of instances have seen some chief data offices say, Hey, you know, we're doing great things.

Why doesn't the business see this or championing us? So, When the business is taking the lead and saying you're doing a great job, you know, it's better to have them saying, Hey, you're doing a great job than you saying, I'm doing a great job, but nobody seems to appreciate it.

Richie Cotton: Okay what are you most excited about in the world of data and AI right now?

Randy Bean: The important thing to understand is that data is the foundation. Okay. So if you don't have good data. You're not going to have good AI in terms of the outcomes, in terms of the predictive analysis that you're performing. a successful AI is predicated upon data, which means that if you want to be successful with AI, that increases the Urgency, the importance the momentum, everything around basically building out your data structure and making sure that you're working off a great foundation.

So in that regard, it really, the use of AI only emphasizes the importance of data. And, as organizations go more and more with AI, you know, they'll see that the quality of their data there's a direct correlation between the quality of their AI story. Initiatives and results and the quality of the data.

Richie Cotton: I suppose there's two sides to this. So like the one side is that although everyone like really wants to be doing the fun work with AI, actually they need to be just doing the grunt work in terms of like making sure the data quality is high. And it's a sort of dirty secret. It's like, there's a lot of like, this sort of stuff goes on in order to get good performance, but on the second thing, it's like, we talked before that data quality has been a persistent problem, and maybe this is the motivation in order to increase data quality. All right, wonderful. Do you have any final advice for leaders who want to improve AI capabilities in their organization?

Randy Bean: I would just say the more of that that there's that partnership and sponsorship with business leaders. And I think that's the best advice I can give to any new Chief Data Officer is go out into the business, understand the business, understand what the key issues are. understand what the key mission and drivers are for the organization and basically wearing your data hat.

Think about where can make a difference and begin to articulate that and gain by and explain it and gain by and from the business leaders. So they say, you know, that's very interesting. We haven't thought of that. Let's try that. Let's try this. Let's answer some questions that we haven't been able to answer before.

And then when that's done again, that builds that trust and credibility and momentum. And that's when organizations start to really move the dial. And that's when the chief data officer really becomes a trusted partner and an integral member of the C suite of the organization.

Richie Cotton: Okay. So get your data leaders and your business leaders talking to each other and hopefully working together and that's how you get to success. Yeah, I think that's essential without strong business relationships without strong business sponsorship and without that business trust, you're really rowing upriver or against the tide.

All right. Super. Loads of great insights there and lots of great advice. Thank you very much for coming on the show, Randy.

Randy Bean: My pleasure. Nice to be here, Richie.



Data & AI Trends in 2024, with Tom Tunguz, General Partner at Theory Ventures

Richie and Tom explore trends in generative AI, the impact of AI on professional fields, cloud+local hybrid workflows, data security, the future of business intelligence and data analytics, the challenges and opportunities surrounding AI in the corporate sector and much more.

Richie Cotton

38 min


Data Trends & Predictions 2024 with DataCamp's CEO & COO, Jo Cornelissen & Martijn Theuwissen

Richie, Jo and Martijn discuss generative AI's mainstream impact in 2023, trends in AI and software development, how the programming languages for data are evolving, new roles in data & AI, and their predictions for 2024.
Richie Cotton's photo

Richie Cotton

32 min


Monetizing Data & AI with Vin Vashishta, Founder & AI Advisor at V Squared, & Tiffany Perkins-Munn, MD & Head of Data & Analytics at JPMC

Richie, Vin, and Tiffany explore the challenges of monetizing data and AI projects, the importance of aligning technical and business objectives to keep outputs focused on core business goals, how to assess your organization's data and AI maturity, why long-term vision and strategy matter, and much more.
Richie Cotton's photo

Richie Cotton

61 min


What to Expect from AI in 2024 with Craig S. Smith, Host of the Eye on A.I Podcast

Richie and Craig explore the 2023 advancements in generative AI, the promising future of world models and AI agents, the transformative potential of AI in various sectors and much more.
Richie Cotton's photo

Richie Cotton

49 min


The 2nd Wave of Generative AI with Sailesh Ramakrishnan & Madhu Iyer, Managing Partners at

Richie, Madhu and Sailesh explore the generative AI revolution, the impact of genAI across industries, investment philosophy and data-driven decision-making, the challenges and opportunities when investing in AI, future trends and predictions, and much more.
Richie Cotton's photo

Richie Cotton

51 min


How Generative AI is Changing Business and Society with Bernard Marr, AI Advisor, Best-Selling Author, and Futurist

Richie and Bernard explore how AI will impact society through the augmentation of jobs, the importance of developing skills that won’t be easily replaced by AI, why we should be optimistic about the future of AI, and much more. 
Richie Cotton's photo

Richie Cotton

48 min

See MoreSee More