How to Use Physical AI in Manufacturing & Construction

Summary

Large language models reshaped knowledge work. The factory floor and the construction site run on a different kind of AI.

In this session, Richie Cotton talks with three leaders building AI for the physical world: Greg Ombach, SVP of Disruptive Research at Airbus; Praveen Murugesan, VP of Engineering at Samsara; and Ori Silberberg, VP of Product Engineering at Buildots. They explain why physical AI looks so different from the chatbots most people now use. The data comes from cameras, lidar, and vehicle sensors rather than the internet. Many models have to run on the edge, in milliseconds, sometimes with no connection at all. The main bottleneck is rarely the algorithm; it is collecting enough real-world data to cover the cases that matter. The panel walks through working examples: adaptive factories that tolerate part variation, dashcams that talk a drowsy driver awake, and software that turns a chaotic building site into a structured record of what was built and when. They close on adoption and the skills that follow, including how a machine learning engineer can move into robotics today.

Key Takeaways

• Computer vision is the common foundation across aerospace, fleet operations, and construction, because perceiving the physical world is the prerequisite for acting in it.
• Physical AI runs on live sensor data and often has to work offline, so teams train large models in the cloud and distill smaller ones to run on the edge in milliseconds.
• Data scarcity, not algorithms, is the main constraint: Airbus estimates a robot foundation model needs about 10 million datasets for 99.9 percent accuracy and sits at roughly one to three million today.
• Safety AI only pays off when it closes the loop; Samsara's drowsy-driving feature changed behavior once an on-device voice agent intervened instead of just alerting.
• Adoption is a change-management problem, so Buildots pairs bottom-up early adopters with top-down mandates, the way a VP of Sales drives Salesforce use more than any individual rep.
• Starting where the pain already exists works: Samsara's voice note-taking for mechanics spread on its own once it fit how mechanics already worked.
• Software can beat hardware spend: Airbus sharpened satellite imagery from 30cm to 15cm with computer vision and generative AI, and launched no new satellites.
• Data literacy, robotics, and cybersecurity top the technical skill list; curiosity and adaptability top the human one.

Deep Dives

The adaptive factory replaces robots that break on variation

Greg Ombach's most exciting use case in manufacturing is the connected factory rather than any single machine. He described the value of physical AI as the adaptive factory, "not one robot just doing one task, but the connected robots which are really working as a chain of of execution," spanning engineering, production, quality, supply chain, and operations.

He contrasted this with his earlier automotive work on battery assembly. Those lines reached 80 to 85 percent automation, but only by programming KUKA robots to exacting tolerances. The team "had to specify all components very precisely," and "sometimes the tolerances were off from parts, and then suddenly the robot stopped," forcing a person to step in. Physical AI loosens that brittleness. Robots adapt to how a part actually presents itself, so engineers can specify less and still get a reliable result. The benefit reaches past the single station. Ombach said adaptability "helps with the entire change from engineering to the final product," which opens new ways to work with suppliers and design partners while cutting time to market and cost.

Airbus has already shipped a version of this. On one logistics task, workers used to pick bolts and screws from a shelf and arrange them in a tray in the right order. The company replaced that with computer vision and two robotic arms whose cameras work at different angles. Ombach was direct about why it worked: the computer vision is mature enough that, with a use case that creates value, "you are going to get a good return of investment on that one." Freeing people from low-value sorting lets them work where human judgment counts. The scale of the product makes the point, since one aircraft, he noted, "is composed of about 3,000,000 parts."

Why physical AI is a different discipline from LLMs

All three panelists agreed that industrial AI shares little with a chatbot beyond the word "AI." Praveen Murugesan framed the difference through data. Samsara works with "real world data sets and pretty much live real world datasets," including live camera feeds, IMU sensors such as accelerometers and gyroscopes, and readings pulled straight from a vehicle's CAN bus like fuel consumption rate and engine state. A model trained on internet text never sees any of that.

The harder constraint is where the model runs. You cannot wait for a cloud round trip while a vehicle is moving, and connectivity is not guaranteed. Murugesan pointed to customers "near the Arctic Circle in Canada where, like, there's no cloud connectivity or even disconnected for, like, weeks." So Samsara trains in the cloud, then distills compact models that run on the device under tight latency and hardware limits.

Ombach sees the same pattern in aerospace, with a higher safety bar and more sensors. An aircraft carries cameras, lidar, and radar across the airframe. Small models run on the edge today, and computer vision will push more compute onto the edge as it grows. He stressed response time. Asking a robot to fetch a bottle of water can tolerate a second or two, but for safety-critical work that lag "is going to be too long," so the architecture has to return millisecond responses by fusing kinematics, vision, language, and reasoning into one model.

Data scarcity sets the ceiling. To train a robot foundation model to the 99.9 percent accuracy industrial work demands, Ombach estimated "you did about 10,000,000 datasets" covering corner cases, while "today, we are on the one three million." Closing that gap, not inventing new algorithms, is the work ahead.

Turning a chaotic building site into structured data

Ori Silberberg described what AI does for construction in one line: "it is always about converting chaos into structure." Building sites generate unstructured everything. Reality on site, plans, 3D models, and schedules all vary by project and by country. As he put it, "Models are not built the same way all over the world, and even in the same country. They're built differently. Schedules are built differently."

No single tool handles that. Buildots runs a network of algorithms where each step checks the one before it. The company uses language models to parse incomplete documents that range from schedules to 500-page specification sheets, and computer vision on video from drones and from cameras mounted on cranes and fences. Working indoors removes GPS, so the team has to reach "centimeter grade, indoor positioning without GPS" from images and accelerometer data alone, then classify objects against the 3D model. Silberberg treats that model as a rough guide rather than ground truth, since crews often install things elsewhere.

The product grows more useful as the questions get harder. Buildots started by showing where a project stands, added history to explain how it got there, then began predicting outcomes such as finishing "two weeks, late, later than your your original duty." Knowing you are late means little without the cause, so the company keeps pulling in more data. It acquired a firm "called Agenda about half a year ago that collects manpower data," which helps separate a labor shortage from other root causes. The goal Silberberg named is a "construction expert that sits by your side that you can ask anything," so managers can focus on execution. Richie Cotton mapped the arc to the analytics pyramid: describe, diagnose, predict, then recommend.

From detection to action

Samsara's clearest lesson is that spotting a problem changes nothing until the system acts on it. The company makes "two way facing cameras that you can install on your vehicle," with AI on the edge that gives drivers instant feedback on unsafe habits such as tailgating or phone use. Its newer capability detects drowsy driving live.

Detection alone fell flat. Murugesan explained that an early version could flag a drowsy driver, but "there's no real feedback loop to close out on, like, a clear actionable outcome," so drivers generally ignored the warning. The fix was an on-device voice agent that intervenes, asks the driver a set of questions, checks whether the person is fit to drive, and records the exchange locally for later review. Since launch he has read transcripts of those interactions and watched drivers take action, which he called a game changer for cutting incidents on the road.

The same instinct shapes how Samsara picks what to build. Murugesan said the team treats itself as product people rather than operators, so "you have to get out on the field and really learn from the people who are operators." One result is an incident management center for investigating events after the fact, such as cargo theft. A customer supplies a location, time, or vehicle, the system packages the relevant camera and sensor data, runs models to flag anomalies, and returns a report people can audit and question.

He offered a simple test for whether a use case carries real value. When talking to customers, the team asks whether they would pay for a service if it existed. As he put it, "that's also a very good litmus test," because willingness to pay tracks closely with return on investment.

Getting AI adopted, and the skills that follow

Buying AI is easy. Getting people to use it is the hard part, and all three treated adoption as change management. Silberberg described a two-sided approach, "both the bottom up adoption and a top down standard setting approach." Early adopters embed the tool in daily work and teach the team what actually helps, but they are a small share of any workforce. Reaching the rest takes operational leaders willing to set a new standard, the same way, he noted, "it's the VP of sales that needs Salesforce for their organization, more than the, you know, the specific salespeople needed for themselves."

Murugesan's advice is to start where the pain already lives. Samsara added voice note-taking for mechanics who work with grease on their hands and cannot stop to type. The team built transcription into the app, a few people tried it, and usage grew on its own. He framed adoption as removing friction at the exact point of need rather than pushing a mandate from the top.

Ombach added governance. In aerospace, finding the right pain point is collaborative work with the business, and any model has to clear security and data sovereignty review before it scales from one use case to many.

On skills, the three converged. Ombach put data literacy first, then robotics and cybersecurity, since more connected devices widen the attack surface. On the human side he and the others named curiosity and adaptability. Silberberg said plainly that "AI is only as good as the data you feed it," and Murugesan argued the old 10x engineer gap is becoming a 100x gap, rewarding people with breadth who learn straight from customers. For a machine learning engineer eyeing the field, both pointed to simulators and cheap desktop robots, and to building hobby projects as the fastest way in.