Design an AI Learning Program That Sticks

Summary

Most AI training programs stall, and three practitioners explain what to build instead.

In a session hosted by Richie Cotton for DataCamp's AI for L&D week, Debbie Smith (head of learning experience at SpaceX AI), Iuliia Algasova (VP of skill development at BlackRock) and Egle Vinauskaite (director of AI and learning development at Nodes) compared how they run AI upskilling programs across thousands of employees. They agree that one course for everyone rarely changes behavior. Programs work when they are built in layers, matched to specific jobs, backed by managers and measured against business results.

The panel covers how to split training into foundation, function and leadership tiers, how to win over skeptical executives and anxious employees, how to protect learning time, and why peer sharing beats lectures. They also admit that most companies still rely on self-reported data to judge results. Anyone planning or rescuing an AI learning and development program will find concrete tactics, from hackathons to performance reviews to token-cost training, plus advice on what to measure and when.

Key Takeaways

• Training drives usage: at one customer, 3 to 4 percent of users were active on an AI coding tool, and after training, 95 percent of attendees started using it right away.
• An AI upskilling program works best in three tiers: a foundation for everyone, function-specific training tied to real use cases, and a leadership tier for managers.
• Start with durable skills, such as judging AI output and mapping work into workflows, then layer tool and role-specific content on top.
• Learning is only one part of capability building; time to practice, manager support and access to tools decide whether training gets used.
• Adding AI adoption to performance reviews and having leaders use AI in the open gives employees a reason to join.
• Protected learning time only counts if workload expectations shrink to match it.
• Peer learning ranked as the most effective intervention in research at BlackRock and at a company of more than 100,000 people.
• Tie each program to a team KPI before launch; most companies still rely on self-reported confidence and time saved to measure AI training ROI.

Deep Dives

How to Design an AI Upskilling Program in Three Tiers

Most AI upskilling programs start with a curriculum. Vinauskaite argues they should start with a system. She compares the structure to a beehive and says organizers "need to think about the system and not the curriculum." At the scale of thousands of employees, no central team can decide what every role should learn, so the job becomes building the conditions in which people find what they need.

Her system has three tiers. The foundational tier gives everyone a shared baseline and is the easiest to scale. The functional tier shows people how AI applies to their own role; it is the hardest to scale and the one that drives transfer to real work. The leadership tier prepares managers to lead the change. On the functional tier she is direct: "people don't start using AI because they've had foundational training." They need exact use cases for their own work. She recommends champions in each function who find use cases, run clinics and feed what they learn into a shared library, where votes push the best examples to the top.

Smith reaches the same place through the jobs-to-be-done framework. After a shared AI foundations course, she groups employees by job function and asks which tasks each group performs. In one program across hundreds of people in five countries, designers, project managers and developers received different training built on a standard template. Usage rose 98 percent, and the company pushed out hundreds of same-day interface updates that would once have taken weeks or months. Each session ended with an assignment to apply the examples to a real task, and the next session opened with participants showing what they built.

Algasova adds a safeguard against fast-moving tools: build an AI skills framework around skills that last. Her advice is to "start with the skills framework." She names critical evaluation of AI output and workflow thinking, meaning the ability to break work into tasks and spot which ones AI can handle, as skills that outlast any product. The skill spine stays the same across the company while the use cases change by function and proficiency level. Smith describes the practical rule as "using a particular tool to teach the underlying pieces," which keeps lessons away from click-by-click instructions that go stale.

Winning Manager and Employee Buy-In for AI Training

Algasova says securing money for AI training is often the easy part. The harder task is separating an event from a capability. "Learning is just one part of this capability building," she says, and the rest is environment, culture, manager support, access to tools and time to practice. If people have no time to experiment, even a strong course goes unused. She suggests sitting down with management and naming the workflows that should change before anyone builds content, then checking whether learning alone can change them.

Vinauskaite sees a different gap in many companies, where leaders hand out licenses, set productivity targets and delegate the rest. What moves those leaders, she says, is direct exposure to AI at the top, through sessions built for senior leaders. Once they see that AI does far more than save a few emails, support follows.

For employees, she starts with two questions: do people understand why AI matters, and do they have a personal reason to use it? Beneath those sit fears about job security, extra stress, or simple comfort with current methods. She also points to trust in leadership, which no communications campaign can create alone. Two tactics help. First, put AI adoption into performance reviews for individuals and managers, "because that creates stakes for people." Second, have leaders use AI where employees can see it, and let peers show what they do. That makes adoption look reachable and adds a little social pressure to keep up.

Smith adds that managers need a business outcome to chase, because usage alone helps no one. "Everything is outcome based," she says, and frontline managers must see AI improve their own team's results so the company is "not just burning tokens." SpaceX AI teaches a course on token economics, which covers when to use cheaper models for small tasks and when to save the expensive ones for work that justifies them. She says a vendor gains nothing from wasted tokens, since customers who burn them without results will cancel.

Keeping Learners Engaged by Protecting Time

When engagement drops, Algasova looks first at relevance. "It's the relevance problem," she says. A generic course that promises AI will help with everything fails because learners cannot connect it to their daily work, so they drop out. Her fix starts before design: talk to learners, find out what pressure they already feel from AI noise and fear of missing out, and ask what blocks them today. Ten-hour programs rarely fit a busy week, so she recommends smaller units of learning that remove specific blockers.

Smith relies on visible wins. At a previous employer, she says, staff were told to use AI without being told why, which felt like someone watching over their shoulder. At SpaceX AI the go-to-market team holds a monthly session to show results from its own products. Smith lives in Grokbot, runs 16 bots and does work that used to take four or five people. She had to show that work and share the wins before colleagues wanted to copy it.

Time is the second barrier. Vinauskaite says protected time has to be real. It comes from a leadership message that ranks learning above other work, and from manager expectations and incentives that match. If people must still deliver the same output at the same pace, the time exists only on paper. She calls that the gap between "real and phantom time."

She also wants training inside existing work. Foundations courses and a few workshops can pull people out of the flow, but most learning should be a challenge: a team picks a low-hanging process, spends a few weeks testing how AI could improve it, and returns to decide what to adopt.

Smith runs the same idea as one- or two-day hackathons, now for knowledge workers as well as developers. Cross-functional teams get a business problem, several groups solve the same one, and the winner earns a place in production. As she puts it, "the best solution actually goes into production for the company." Everyone learns from each other, the competition adds energy, and people want to join a team that fixes a real problem. She says in-person events work best, and she brings swag.

Spreading AI Skills Through Peer Learning

Algasova calls AI "the strongest argument we had in years for social learning." Her reasoning is practical. Learning designers in a large company cannot track new use cases in legal, marketing and IT, so peers become the source. She recommends relaxed sessions with few expectations, where people get used to sharing what they do. Research she ran at BlackRock and her previous employer put social learning at the top of what employees said helped them learn about AI. Vinauskaite heard the same from a conference speaker the day before, whose company of more than 100,000 people found that learning from peers beat every other intervention it tried.

Vinauskaite sorts social formats into three groups. Synchronous formats include hackathons, lunch and learns, show and tells and slots in all-hands meetings. Asynchronous formats include challenges that teams revisit after a week or two, chat spaces for sharing uses, and internal podcasts that collect use cases from across regions. An employee in Belgium can hear what a team in the US or Australia does with AI. The third group is a human space, where employees raise concerns and talk through the change around them. She says that space builds the culture and safety that make adoption possible.

Scale takes coordination. A group of 5,000 people sharing prompts cannot rely on a SharePoint or Notion page, so organizers must decide how big each space should be and how good examples get found and kept current.

Smith points to a shift that changes what people share. Chatbots are giving way to bots that carry skills and can update them. Colleagues can copy a bot and make it their own instead of pasting a prompt. "No longer do we reuse prompts," she says, and prompting itself is fading, with many people now talking to their bots by voice. SpaceX AI launched Grokbot in August, and she noted that OpenAI and Facebook have launched similar products. Cotton observed that each shift starts another round of upskilling.

Measuring AI Training ROI and Outcomes

Algasova says AI training should be measured like any other upskilling program, whether you prefer Kirkpatrick or another model. Her key step is to "measure the application and behavior change." That is the hardest layer, because learning designers often lack access to business data and have no baseline from before the program. The work starts early: agree with managers and leaders what should change, gather data on the current problem, and confirm you can see the same data afterward. Building experiments into the program, where people redesign real workflows and report back, makes later measurement easier.

Smith separates outcomes from ROI and says both matter, depending on the audience. Outcomes are the behavior change and the KPIs it moves. ROI is the calculation of the effect on the company's bottom line. She says "all training needs to be tied to a specific KPI": look at the team's goals, measure the KPI before and after training, then convert that outcome to ROI. From her customer education work, she cites customers who took training churning less, buying more and using more of the product, and one company where trained customers sent emails with higher open rates and better sales. She will cover her calculations at DevLearn.

Vinauskaite offers a reality check. Few organizations have solved this at scale, and most "rely on self reported confidence and self reported time saved." Isolating the effect of one strategic intervention among many is hard. Companies that go deeper do it at team level: pick a team and a workflow, then study how the new process saves time or changes how work gets done. She has not seen a good way to repeat that across the largest rollouts, so it stays spot checking.

At the business level, Smith asks whether the company solves problems faster, enters new markets or ships things it could not before. She also warns that most people overestimate their own AI skill and usage. At the end of the month she will release an AI skills model and matrix built around durable skills, plus a maturity model for individuals. Cotton's summary: decide the outcome you want before training starts, so you can check against it afterward.