Develop an AI Upskilling Strategy

Summary

Three workforce-learning experts argue that an AI upskilling program fails when it teaches tools instead of business results.

Richie Cotton moderated a panel with Lori Niles-Hofmann, an edtech strategy consultant, Kim Arellano, CEO of Workforce Evolution, and Makai Ruffin, an assistant professor of industrial-organizational psychology at San Jose State University. They covered how to link training to business goals, who should learn what, how managers build a learning culture, and how to tell whether a program worked. Examples came from Siemens and Amazon, and the panel laid out a measurement funnel that runs from participation to business impact. Audience questions turned to soft skills and the psychology of change. Watch it if you are planning a rollout and need arguments for executives, managers, and skeptical employees.

Key Takeaways

• An AI upskilling strategy should connect to making money, saving money, or reducing risk, the filter Lori Niles-Hofmann uses with clients.
• Leaders need to explain why the company is upskilling and use AI visibly in their own work.
• A training needs assessment at the organization, team, and individual level should come before any course assignments.
• Judging whether an AI output is relevant, along with systems thinking, will outlast any single tool skill.
• Managers turn training into results when they help employees apply what they learned and remove the roadblocks in their way.
• Executives have to carve out time for learning, because employee time is finite.
• Measure a funnel from participation to skill, on-the-job use, and business impact, with a fix ready at each stage.
• Weekly login counts say little about progress, so set targets tied to business output, such as a quota of new ideas per quarter.

Deep Dives

Build an AI Upskilling Strategy Around Business Results

Richie Cotton opened by asking for examples of AI training that paid off, and the panel kept returning to one idea: training works when it sits inside real work. Lori Niles-Hofmann described a Siemens factory program that used AI to predict machine downtime. Staff learned to act on those predictions, so the training solved a production problem while it built skills. She recalled that employees were able to "release about six thousand hours of employees' time that they could dedicate to higher tasks." Her conclusion was that teaching prompt engineering in isolation misses the point, and that skills have to feed into a company's workflow.

Makai Ruffin raised a second use case: moving people between roles. He pointed to Amazon's announced investment in generative AI, machine learning, and cloud training, which lets employees shift into AI-oriented jobs while the company builds the talent it needs. Richie added that this matters more as AI changes some roles faster than others.

Kim Arellano judged success on a smaller scale. She worked with the learning team at a large consultancy and read the room. "The hallmarks for me of a great class is when people are laughing, when they're going over, when they're not leaving the virtual classroom, when they're asking questions," she said. Skeptics who arrived doubtful started building skills through play and experimentation within a single session.

For strategy, Lori offered a filter she uses with clients. An upskilling plan "has to directly relate to one of three things. How does it help the business make money, save money, or mitigate risk?" A prompting workshop rarely passes that test. She also asks leaders to reset expectations, because many assume that switching on AI produces results. "Learning is not something that I can compress for you," she said, and a one-year target may land five or 10 percent short because people make mistakes while they learn. Richie agreed that the transformation runs over many years.

Leaders Set the Reason and Go First

Kim Arellano called leadership the starting point of any AI skills program. Employees fill silence with fear: "People are gonna default to, oh, they're gonna replace my job." She wants leaders to state the North Star at the outset, whether that is productivity or something else, admit what they do not know yet, and ask staff for a feedback loop on what works and what fails. Richie noted that bosses often say "go do this thing" and leave out the reason.

Her second point was modeling. Leaders can skip building agents in Copilot Studio. A better start is showing how they prepare for a meeting with AI or analyze one afterward, so the team sees small, repeatable uses. Kim also named the gap many organizations face: "a lot of the leader doesn't know any more than anybody else does." Her advice was to start from that admission and set a vision and strategy now.

Makai Ruffin tied the same idea to culture. A manager who refuses to use AI platforms cannot expect followers to try them. His research on technology rollouts shows what constant tool-chasing costs. When a company introduces technology without a positive learning culture and ongoing resources, he said, job satisfaction and confidence drop, and employees feel more threatened about pay and promotion. In his studies those effects lasted more than a year. Poorly explained decisions "leave a sticky residue for people psychologically." His recommendation: explain why the change is happening, then stay with the tool long enough to see how it works.

Lori Niles-Hofmann added that leaders should be open about how they use AI and what they expect from their teams. She wants to see leaders change their working structure, which an AI-written email does not show. Richie summed up the point: if the C-suite does not use AI, the rest of the organization will not follow.

Decide Who Learns What With a Needs Assessment

Makai Ruffin wanted planning to come first. Before a company launches an AI upskilling initiative, he recommends an "initial training needs assessment" that asks whether training is necessary at the organization level and then at the person level. A job analysis with people already in the role shows which skills the work requires. Conversations with employees show who is interested and ready. Without that groundwork, he said, results "fall flat" in engagement and in transfer back to the job. Richie added that informal learning is a signal too: what people pick up on their own can feed the formal program.

Lori Niles-Hofmann described a different model. Her team uses tools that unpack SCORM files, the packages that hold most e-learning content, so an AI tutor can draw on what a company already owns. The tutor works in two directions. It knows what the business needs people to know, and it watches how each person works, so content you already handle well never reaches you. She wants that tutor plugged into a CRM, so a salesperson can practice a conversation with a specific persona, and into project management software, so learning follows the projects that are slipping. Her buying advice: "don't be bedazzled" by a vendor's AI features, and look for technology that grows with you.

On skills, the panel favored durable ones over tool tricks. Makai said prompting still matters, but he is more interested in whether people can judge if an AI output is relevant to the task and then use it on the job. Kim Arellano named systems thinking and critical thinking, which start with asking whether AI should do a task at all or whether the task should change. She also teaches people to control how AI answers them, since a model that agrees with everything delivers "confirmation bias." She warned about the alternative: "doing the things we do today faster is gonna get us to obsolescence faster." She uses AI as a thinking partner for new ideas instead.

Build an AI Learning Culture Managers Can Support

Makai Ruffin recommended "error management training," which gives people room to make mistakes with new AI tools and learn from them. He also warned about late support. In a technology rollout he studied, employees did not get resources until seven months into the implementation, and the people who had learned first grew tired of training everyone else.

Kim Arellano put the manager at the center of whether learning sticks. "You can learn all day, but if you go back to work environment and it doesn't apply, you've wasted your time and the company's wasted money," she said. She wants managers to help each person apply what they learned, hold that learning accountable, and remove roadblocks. For employees who feel anxious about AI, she suggests a one-on-one that asks what they do best and how AI could amplify it, then invites them to learn something and teach it back to the team.

Lori Niles-Hofmann disagreed that this falls on managers alone. The C-suite has to carve out the time, she argued, because time for learning is finite. She quoted a Sesame Street line about Ernie and his saxophone: "put down the ducky" if you want to play. She also called for a reckoning inside learning and development. In her view, teams use AI to build the same bloated courses faster. "We don't need an escape room game to teach somebody about data privacy," she said, and subject matter experts should cut the history of a process when learners only need the steps.

Kim added that learning and development often arrives too late. "L and d comes after they've made all these decisions," she said, once executives have already planned reskilling. Inviting the team early would cut "scrap learning" and keep training in the flow of work. Richie agreed that learning teams should help shape the plan and not only deliver it.

Measure AI Training ROI Without Vanity Metrics

Makai Ruffin proposed a chain with four links: participation, skill acquisition, on-the-job application, and business impact. Before the program starts, a company should define the target skills, current and target proficiency levels, who needs to reach each level, and by when. Each link carries a trigger for early action. High completion with low assessment scores means the content needs revision. Good scores without application point to weak manager support or too little practice. Strong application with no business gain means the program targets the wrong skills. He added that on-the-ground evidence is thin so far, because the field is new.

Lori Niles-Hofmann starts earlier, with a skills taxonomy and a precise question: what would three or 17 trained people do to the bottom line? She asks two things when a learning request arrives. The first is whether there is evidence of need, and she said, "I want to see some numbers." The second is whether the skill could be automated, for example by recording an expert and turning the recording into a markdown document others can use.

Kim Arellano said traditional ROI models struggle because the technology moves faster than training can. Her answer is to define expectations first, then tie output to business indicators. A team might receive a target such as "We want 10 ideas by the end of the quarter," and the company can then trace which people produced them and which skills they used.

In the audience Q&A, Lori urged organizations to "stop reporting on how much people are using AI." She cited a large consulting study that reported 89 percent weekly use, which she said meant people who logged in once and ran one prompt. She would rather find the outliers, study what they do, and have leaders share their own results, including failures. One company she admired held "a public autopsy" for every failure, with a rule that nobody got in trouble.