I read two really insightful articles on robotics recently, and they left me with a question I would ask – what exactly is the model really responsible for?
A general-purpose model like the ones behind ChatGPT could interpret a request, choose the robot’s next movement or help improve the model controlling it. Those are very different jobs. I would make a different architecture and investment decision for each.
The two pieces were Martin’s (@0x_meden) guide to how π0, GR00T and Optimus think and Robotics’ New Era After Astra by FD at Robonomics. I recommend reading them together. Martin makes the relationship between understanding and movement easier to follow. FD pushes the argument further: how much of that work could general-purpose AI absorb, and what happens to the businesses built around it?
My instinct has been to favour smaller, specialised models running close to the robot. I still want that fast, dependable control. What I am reconsidering is how much general AI could contribute around it, especially to the engineering that makes the robot better.
The body still has deadlines
Ask a robot to clear a coffee cup from a table. Working out which cup you mean is one problem. Reaching around a laptop, gripping the handle and keeping the cup upright is another. Recovering when the cup starts slipping is more urgent still.
The System 2 / System 1 approach is useful because it recognises those different demands. System 2 interprets the scene and the instruction, and decides how to approach the task. System 1 uses what the robot sees and senses to generate movement. Some architectures add System 0 for the fastest control of balance, contact and the body itself. The labels describe responsibilities; each architecture draws its own boundaries. (Reference: developer.nvidia.com)
This is also where vision-language-action models, or VLAs, fit: they connect visual and language understanding to physical actions. Martin’s guide is a useful starting point for understanding that connection. NVIDIA’s GR00T architecture makes the relationship visible, while Figure’s Helix 02 shows how demanding the timing can become: its body controller operates 1,000 times a second.
I want that fast control close to the machine, obviously. A cloud connection dropping or a reasoning call taking longer than expected should not leave a robot waiting to recover its balance.
I would be equally cautious about declaring that these responsibilities must always belong to different kinds of models. FD’s suggestion that frontier models could absorb System 2 and parts of System 1 deserves serious attention. (Reference: robonomics.substack.com)
In Robocurve’s evaluation, GPT-6 Astra supplied gripper targets through existing robot-control software and placed a block in a bowl in 19 of 20 trials. It managed a puzzle-piece insertion in only two of 20. I find the gap between those results as interesting as the stronger result itself. General intelligence is reaching into action generation, while physical precision remains a substantial problem. openai.robocurve.org
Timing matters too. In NVIDIA’s Astranomy study, direct-policy inference calls averaged 18.56 seconds, and simulation paused while the model thought. A robot working beside people would have to handle those delays while its surroundings kept changing.
That is why I would choose the division of work around reaction time, reliability and cost. A general model might be worth calling when a robot encounters something unfamiliar. A fast local policy might handle the next thousand familiar movements. I expect the boundary to move as models improve, and I would design the product so I could move it.
The robot’s next engineer could be another model
There is another way general AI could power a robot: improve the policy that runs it, then leave that policy to execute locally. This is the possibility I find most interesting.
Suppose a robot repeatedly catches a mug against the edge of a microwave. Someone has to understand the failure, work out what to change and establish whether the fix helps. A general-purpose model could assist with that investigation, including changes to the robot policy itself. The policy is the model that turns observations into actions; its weights are the numerical parameters shaping that behaviour.
Playground’s experiments with weight-space programming make this possibility concrete. Frontier agents inspected demonstrations and recorded failures, then directly edited the weights of a π0.5 robot policy without gradient-based retraining or a simulator available during the editing process. On one simulated mug task, edits to 2,048 weights increased success from 34 to 44 of 50 held-out scenes. www.withplayground.ai
The researchers also published what the improvement cost: the edits fixed 16 failures and broke six previously successful cases. I appreciate that detail. It makes evaluation part of the result, and gives me a much better basis for judging what the technique might be useful for.
There is early physical evidence too. Ziyang Gong and colleagues had Opus 5.5 investigate and edit a robot policy, then tested the changes on real arms. Bottle placement became faster in their small test. Both versions already completed the task successfully, so this was an improvement in execution speed, with broader reliability still to establish.
An expensive model could spend time investigating a recurring failure, with the resulting improvement delivered through a fast model running on the robot. The reasoning cost could then be spread across many future tasks. I would happily pay for that investigation if the result reduced the need for someone to keep rescuing the same job.
Direct weight editing is one early method. I would also use general AI to inspect recordings, suggest better training examples, improve control code and design evaluations. I increasingly see general-purpose AI as part of the engineering capacity behind a robot. That gives it a much larger role than I had allowed for when I was thinking mainly about inference speed.
I would still insist that every proposed fix earn its way into deployment. The six regressions are a good reminder of why.
The data I would pay for
The people supplying physical experience. Figure’s reported Index contributor growth, August 2026. Credit: Figure
That engineering role also makes me more interested in the physical data market. A model investigating a failed grasp needs evidence of what happened. FD’s discussion of this market is one of the strongest parts of his article, particularly his question about which kinds of experience remain valuable as general AI improves.
If general AI becomes better at interpreting physical failures, those failures become more useful. A recording can support training and diagnosis, while the failure it captures can inspire a separate, reproducible test for future releases. I see the potential for a large market in collecting and preparing that experience.
There is already real money behind it. In its August Index announcement, Figure reported paying contributors $15 million and receiving more than 16 million videos. It also committed to spending more than $1 billion over the following year on data and compute combined. Scale’s Physical AI business offers collection through teleoperated robots, wearable systems and operating businesses. Collecting physical experience is becoming a commercial activity in its own right.
I would be careful about valuing all those hours equally. A general model may understand what a cup is and where it belongs, while still needing information about how a particular hand should hold it. Human video shows the variety of ways a task can be performed. Robot demonstrations connect observations to actual actions. Measurements of force and contact can explain a failure that looks perfectly reasonable on camera.
The transfer also takes work. NVIDIA’s EgoScale combines human-video pretraining with aligned human and robot data before adapting to robot tasks. Watching someone fold a shirt does not automatically give a particular gripper the ability to do it.
The work required to turn human experience into robot capability. Credit: NVIDIA Research, EgoScale.
Figure’s explanation of why it built Index is revealing too: it said outside vendors could not meet its requirements for throughput, diversity and quality. I read that as a warning to anyone assuming a large collection of videos automatically becomes a valuable data business.
My bet is that broad, repetitive footage becomes easier to source, while well-recorded difficult tasks, failures and recoveries retain a premium. I would want to know which weakness a dataset addresses, whether the relevant physical measurements are present, and whether a target robot improves after using it.
For an industrial or logistics operator in the UAE, I would take that possibility seriously when buying robotics. I would want a clear agreement on access to operating recordings, their permitted uses and how improvements return to the customer. A deployment could become a source of learning that improves the next deployment, provided someone has designed the collection and evaluation properly.
What I would build
FD’s argument that deployment could become the next bottleneck is the part I agree with most. Easier access to capable models still leaves someone responsible for making the complete service work in a customer’s operation.
If I were starting a robotics business, I would begin with a workflow whose value I could measure. I would use existing reasoning and action models where they met the requirement, and keep enough flexibility to replace them. I would invest heavily in collecting useful operating evidence, reproducing failures and testing improvements against everything the robot already did well.
That also changes the financial measure I would care about. Cheap inference is useful, but human interventions, slow recovery and interrupted work can overwhelm that saving. I would track cost per completed task at the required reliability, including supervision and recovery. A more expensive reasoning call can be a good purchase if it prevents an operator from having to rescue the job.
Specialised robotics models can still be valuable. I would expect them to prove their advantage in the work itself: faster execution, better manipulation, lower compute requirements or fewer interventions. General-purpose AI makes that a more demanding comparison, and could also give those same companies better tools to improve their models.
If I were evaluating a robotics company, I would ask it to walk me through its last recurring failure. How did the team identify it? What did they change? Which previously successful cases became worse? What happened to the need for human help after the update?
Those answers would tell me how effectively the company learns from deployment, and whether its economics have a reason to improve as it grows.
I would be comfortable buying much of the intelligence, while keeping control of the operating experience and the ability to improve from it. As general models become more capable, I would give them more responsibility wherever the results justified it.
A “ChatGPT-powered robot” would interest me. Before buying one, I would want to know which decisions the model owns, which physical deadlines it must meet, and what happens when it gets something wrong. I would then want to see how the team uses those failures to make the next deployment better. That is the business I would back. Or join 😉
Join the conversation