Become an AI-Native Data Leader

Summary

Four AI leaders explain what it takes to run a data team that builds AI into everyday work without losing accountability.

Richie Cotton hosted Melissa Ramey, Director of AI Transformation for Data Insights and User Experience at Salesforce; Ieshika Chandra, who leads the ecommerce data science group at Walmart Global Tech; Femi Idowu, an enterprise AI architect with a background in regulated industries; and Hussnain Ahmed, Chief AI Officer and Co-founder at ekai. They agreed early on one test for an AI-native data team: it is defined by the work it has stopped doing and the evidence it can show, not by the number of tools it has bought.

The panel covered how to measure impact, which skills data professionals need now, why "human in the loop" falls short as a governance model, and how to bring a team along without a mandate. Ramey described an agent-led test that cut a product's hallucination rate to about 2.89% four days before a launch decision. Chandra explained why companies shifted from rewarding token use to rewarding efficient use. Idowu listed the teams a regulated company needs involved before a pilot reaches production. The session closed with audience questions on how long an AI transformation takes and how to launch AI in a regulated environment.

Key Takeaways

• An AI-native data team decides deliberately where AI improves the work, instead of adding it to every task.
• The best test of AI adoption is what your team has stopped doing, since the same work done faster means the AI is not working for you.
• Measure AI by the quality and speed of decisions it supports, not by the volume of code, models, or output it produces.
• Domain knowledge, critical thinking, curiosity, and ownership of outcomes matter more as AI takes over routine analysis.
• Build responsible AI judgment into data science work from the start, including guardrails on inputs, outputs, and MCP connectors.
• Match the level of human review to the consequence of the decision, and review exceptions instead of approving every low-risk output.
• Put human judgment at the start and throughout a governed process, because inspecting quality in at the end means paying for the same work three times.
• Start a transformation with one outcome you want to move, and find the moments of AI-native work that already exist in your legacy processes.

Deep Dives

What an AI-native data team looks like

Cotton opened by calling AI native "a little bit of a fuzzy term," then asked each guest to define it for a data team. The answers converged on judgment about where AI belongs.

Chandra said her team uses AI for day-to-day tasks and automates many processes, but she drew a line: "AI first or AI native does not need humans last." She wants guardrails at the input, output, and modeling stages so the team can tell when AI goes rogue and respond to it.

Idowu described an AI-native team as one that decides where AI improves the work. Not everything that can be automated needs AI, and the team should know its business outcomes, work from reliable data, and share definitions. It should also separate where AI assists from where AI acts, and keep people clear about who owns the results. His test for an AI-first team is how well it "redesigns and improves work," and tool purchases do not enter into it.

Ramey framed it as a division of labor between AI and humans, and the question she asks first is what a team has stopped doing. Her analogy: someone who buys a dishwasher can say they quit washing and drying dishes by hand. For an AI tool, she wants the same specificity. In an agentic onboarding program she described, human trainers did 67% less classroom work. "If you're doing the same work, but more efficiently, your AI is not working for you," she said.

Ahmed agreed that research, coding, and documentation move much faster in an AI-native team. He warned that speed shifts the burden to review. General-purpose models train on public data and "don't understand what your business definitions means," so the team has to define those meanings for the model and keep a human accountable for the results.

Taken together, the four answers give data leaders a working definition of an AI-native data team: it chooses where AI acts, supplies the business context the models lack, keeps a named human accountable, and can point to work it no longer does.

Measuring AI impact by decision quality

The second round of answers asked about impact. Each guest moved away from raw speed.

Idowu separated producing something faster from completing the whole workflow better. A draft query, design, or analysis arrives quickly but still needs validation. In regulated industries, teams also carry a burden of evidence: traceability, proof of compliance when a policy is set and when it is enforced, and a record that what the agent did matches what the team intended. He sees that as an opportunity as well as a limit, because a team can build a solution where "an independent regulator can also verify the outcome."

Ramey named two connected benefits, "speed to decision and quality of decision." Four days before a go or no-go meeting, a stakeholder told her team the launch would be blocked unless the product reached a 7% hallucination rate. Her team ran a one-day user acceptance test with agents it had built. The agents found 67% more problems than the human reviewers had in earlier testing, and the run took about an hour. The hallucination rate came in at roughly 2.89%, the steering committee saw every P0 through P3 issue, and the go decision happened that day. Without AI, she said, the same situation would have meant months of postmortems.

Ahmed picked a different benefit: "time from idea to prototype has reduced a lot." Prototypes that once took weeks of requirements gathering and iteration now arrive far sooner. Moving from prototype to production remains hard. Cotton noted that most data teams build analyses, dashboards, and simple web apps, so the prototype gain applies to them directly.

Chandra described several self-service agents at Walmart Global Tech, including a root cause analysis agent, that let business teams answer their own questions. Data scientists then spend less time on interruptions and more on new features and launches. She also flagged a side effect: AI generates far more code and output, and someone has to evaluate it. In her view "business impact definitely matters over output," and teams should measure which decisions improved and which outcomes they delivered.

Skills data professionals need now

Asked which technical or soft skill matters most for an AI-native data team, the guests put judgment ahead of tools.

Ahmed pointed to subject matter expertise. Fundamentals in statistics, data modeling, and data engineering still matter, but data people now need to work closer to the business and learn its processes. He described moving from paper manufacturing to biofuels and biochemicals, where AI helped him learn the basics quickly and domain experts filled in the rest. He also wants people who take ownership of delivering value instead of handing over a feature built to a specification, and he added curiosity to the list.

Ramey put critical thinking, judgment, and curiosity at the top and said AI has made them more important. She cited a study published that week in Nature Human Behavior. Researchers split 487 management consultants into two groups, gave one group ChatGPT, and asked both to complete data science tasks outside their expertise. The ChatGPT group performed better. When the researchers took AI away, those consultants did not know why their answers were right and could not have spotted a wrong one. Ramey called that "a little bit of a patina of expertise without the actual underlying knowledge." Her hiring filter follows: "I'm looking for the people who are using AI to learn everything."

Idowu added two points. People should question outputs without dismissing the technology, and they should be willing to ask for deeper understanding instead of hiding behind jargon. He also stressed listening, because "the best technical solution can solve the wrong problem" if the team does not understand how people actually work.

Chandra named responsible AI judgment, which she treats as part of good data science and not as a separate compliance step. Data scientists should ask whether the data is appropriate, how to evaluate data quality beyond one metric, and who is accountable for the outcome. Production adds risk that notebooks do not. She described MCP connectors where someone could change a context file, and nobody has time to read every line, so guardrails have to track those changes.

Human-led judgment instead of human in the loop

The most debated section of the session asked how to decide what to automate and where people stay involved. Ramey opened by challenging the phrase "human in the loop."

She argued that teams focus too much on a single moment of review after AI has produced something. She invoked W. Edwards Deming: "You cannot inspect quality into a product." Putting human judgment only at the end means paying a human to create the work, paying the AI to transform it, and paying a human expert again to repair it. She pointed to MIT research on AI decision rights, which treats risk and ambiguity as the two critical factors and describes decision making as framing, acting, and learning. Her preferred model is "human led" judgment throughout a governed process, starting with the choice of tool.

Ahmed added that humans must write the business semantics. A term like daily active user "means different thing in every business," and sometimes between two units of one business. He also named review fatigue. If a person must review 200 or 300 terms, they will start accepting suggestions after about ten. His fix is automated testing and validation built into the system so that most generated content earns a pass automatically, and only conflicting items reach human reviewers.

Chandra rejected a binary choice: "Level of human review should definitely match the consequence of the decision." For low-risk work such as summarizing or flagging anomalies, teams can review exceptions instead of approving every output. For decisions that affect customers, use sensitive data, or are hard to reverse, review should be explicit about which checks a human performs. Cotton illustrated the range: AI choosing a color scheme for a slide deck needs far less checking than AI drafting an operating procedure for a surgeon.

Idowu separated an AI recommendation from AI authority to act. He looks at reversibility, severity, data sensitivity, and whether errors can be caught before harm. A human approval step only counts if the reviewer has full information, enough time, and "the authority to say no." He starts with bounded tasks and widens authority as evidence supports it, with decision gates and automated safeguards in place, especially for multi-agent systems.

Leading the change, and launching in regulated environments

The last discussion topic was change management, followed by audience questions.

Chandra described an earlier phase in which leaders asked employees to use AI tools everywhere. Some companies tracked token usage as a proxy for efficiency, and employees found ways to inflate it. The goal later flipped to using as few tokens as possible for the same work. She argued that adoption needs enablement and governance together: training on tools, guidance on appropriate data, how to validate outputs, and when to escalate. She likes the NIST AI Risk Management Framework, which asks teams to govern, map context and risk, measure outcomes, and manage what they learn.

Idowu said teams succeed when they start from a problem they recognize, not a mandate. He recommends an approved, safe space to practice, "almost like a dojo," along with one useful example workflow and training on checking outputs and handling failures. Honest feedback about where a tool does not help beats adoption numbers that hide workarounds.

Ahmed told leaders to find the bottleneck in a business process and turn it into a data science problem before reaching for a language model. In his example, reading engineering diagrams was slow, and a language model alone failed. With digital image processing first, "the model can do wonders."

Ramey turned to the leadership side. Much of the resistance and apathy she sees comes from the top, and asking a team to transform while keeping every existing priority intact means "something's gotta give." In her view the issue is rarely employee adoption, since most staff already use more AI than leaders expect. The problem is prioritization.

In the audience Q&A, one attendee asked how long a team transformation takes. Ramey said it depends and offered a starting point: "Find one actual outcome that you are trying to move." On launching AI in a regulated environment, Idowu recommended an executive sponsor, a realistic use case, and outcomes defined in advance. He would also involve the chief information security office, the data protection office, and networking and infrastructure teams early, because the gap between a pilot and production is usually compliance controls and demonstrated risk mitigation.