Skild AI's S1 learns robot tasks from video

Skild AI has unveiled its flagship S1 robot foundation model. The company, founded in 2023, has raised nearly $1.7 billion to develop what it calls a general-purpose robot brain.

According to CEO Deepak Pathak, S1 uses in-context learning, allowing robots to learn new, complex tasks by watching a single video. He told The Robot Report this method avoids the lengthy post-training typically required for AI models on robots.

S1 combines four data sources for training

Pathak outlined four primary types of robot training data, arguing that Skild's approach of using all of them is unique. He stated that the pros of one source compensate for the downsides of another.

Pathak said that while other companies primarily focus on one of these sources, Skild uses all of them. He explained that human video data, which is diverse but far from a robot's reality, can be combined with more direct sources like teleoperation. For more detailed data on performance, see our stats page.

Model targets long, complex tasks

The company is not focusing on a single industry. Instead, it is demonstrating S1 on long-horizon tasks that can take up to 10 minutes. Examples shown include repotting a plant, making coffee, and cooking pancakes.

Pathak highlighted the pancake-flipping task as an example of emergent behavior. The company checked its data and found no prior examples of flipping. He said this flipping emerged by looking at how the spatula is moving, and the robot could just do it.

The Skild model is also described as omni-bodied, capable of working on platforms from quadrupeds to humanoids or static arms. Pathak said future development will spend more time improving performance on humanoids, though scaling the current model to that form factor is pending. This adaptability across different platforms is a key focus in robotics squad development.

A step toward a robotics 'ChatGPT moment'

Pathak compared the evolution of robot foundation models to the development of large language models like ChatGPT. He noted that previous language models required fine-tuning for every new task, a barrier that was overcome with prompt-based systems.

While he believes S1 is a step in the right direction, Pathak does not think robotics has reached its equivalent breakthrough moment. He said the model is not quite ready to be rolled out to people's homes, but it is the first sign of what might come.

Looking ahead, Pathak said Skild has more releases planned to show how S1 is helping in production and acquiring customers. He cited the recent acquisition of Fetch Robotics as a move to gain the right talent for deployment and scaling. He stated that while the company will continue scaling frontier research, deployment is a high priority.