Boston Dynamics’ Spot robots perform autonomous patrols of property perimeters such as fence lines, exterior doors, and other critical assets located outside buildings.

Boston Dynamics presented the new robot. The company refused from the old hydraulic platform to introduce the new electric one under the same name, Atlas. The new robot is completely electronic, there are no hydraulic systems involved. The new robot will be stronger and more maneuverable as all developments of the previous generation of the robot will be improved. The company is ambitious to introduce humanoid robots and create infrastructure for them, including software.

Today, Boston Dynamics and the Toyota Research Institute (TRI) announced a new partnership “to accelerate the development of general-purpose humanoid robots utilizing TRI’s Large Behavior Models and Boston Dynamics’ Atlas robot.” Committing to working towards a general purpose robot may make this partnership sound like a every other commercial humanoid company right now, but that’s not at all that’s going on here: BD and TRI are talking about fundamental robotics research, focusing on hard problems, and (most importantly) sharing the results.

The broader context here is that Boston Dynamics has an exceptionally capable humanoid platform capable of advanced and occasionally painful-looking whole-body motion behaviors along with some relatively basic and brute force-y manipulation. Meanwhile, TRI has been working for quite a while on developing AI-based learning techniques to tackle a variety of complicated manipulation challenges. TRI is working toward what they’re calling large behavior models (LBMs), which you can think of as analogous to large language models (LLMs), except for robots doing useful stuff in the physical world. The appeal of this partnership is pretty clear: Boston Dynamics gets new useful capabilities for Atlas, while TRI gets Atlas to explore new useful capabilities on.

Here’s a bit more from the press release:

The project is designed to leverage the strengths and expertise of each partner equally. The physical capabilities of the new electric Atlas robot, coupled with the ability to programmatically command and teleoperate a broad range of whole-body bimanual manipulation behaviors, will allow research teams to deploy the robot across a range of tasks and collect data on its performance. This data will, in turn, be used to support the training of advanced LBMs, utilizing rigorous hardware and simulation evaluation to demonstrate that large, pre-trained models can enable the rapid acquisition of new robust, dexterous, whole-body skills.

The joint team will also conduct research to answer fundamental training questions for humanoid robots, the ability of research models to leverage whole-body sensing, and understanding human-robot interaction and safety/assurance cases to support these new capabilities.

For more details, we spoke with Scott Kuindersma (Senior Director of Robotics Research at Boston Dynamics) and Russ Tedrake (VP of Robotics Research at TRI).

How did this partnership happen?

Russ Tedrake: We have a ton of respect for the Boston Dynamics team and what they’ve done, not only in terms of the hardware, but also the controller on Atlas. They’ve been growing their machine learning effort as we’ve been working more and more on the machine learning side. On TRI’s side, we’re seeing the limits of what you can do in tabletop manipulation, and we want to explore beyond that.

Scott Kuindersma: The combination skills and tools that TRI brings the table with the existing platform capabilities we have at Boston Dynamics, in addition to the machine learning teams we’ve been building up for the last couple years, put us in a really great position to hit the ground running together and do some pretty amazing stuff with Atlas.

What will your approach be to communicating your work, especially in the context of all the craziness around humanoids right now?

Tedrake: There’s a ton of pressure right now to do something new and incredible every six months or so. In some ways, it’s healthy for the field to have that much energy and enthusiasm and ambition. But I also think that there are people in the field that are coming around to appreciate the slightly longer and deeper view of understanding what works and what doesn’t, so we do have to balance that.

The other thing that I’d say is that there’s so much hype out there. I am incredibly excited about the promise of all this new capability; I just want to make sure that as we’re pushing the science forward, we’re being also honest and transparent about how well it’s working.

Kuindersma: It’s not lost on either of our organizations that this is maybe one of the most exciting points in the history of robotics, but there’s still a tremendous amount of work to do.

What are some of the challenges that your partnership will be uniquely capable of solving?

Kuindersma: One of the things that we’re both really excited about is the scope of behaviors that are possible with humanoids—a humanoid robot is much more than a pair of grippers on a mobile base. I think the opportunity to explore the full behavioral capability space of humanoids is probably something that we’re uniquely positioned to do right now because of the historical work that we’ve done at Boston Dynamics. Atlas is a very physically capable robot—the most capable humanoid we’ve ever built. And the platform software that we have allows for things like data collection for whole body manipulation to be about as easy as it is anywhere in the world.

Tedrake: In my mind, we really have opened up a brand new science—there’s a new set of basic questions that need answering. Robotics has come into this era of big science where it takes a big team and a big budget and strong collaborators to basically build the massive data sets and train the models to be in a position to ask these fundamental questions.

Fundamental questions like what?

Tedrake: Nobody has the beginnings of an idea of what the right training mixture is for humanoids. Like, we want to do pre-training with language, that’s way better, but how early do we introduce vision? How early do we introduce actions? Nobody knows. What’s the right curriculum of tasks? Do we want some easy tasks where we get greater than zero performance right out of the box? Probably. Do we also want some really complicated tasks? Probably. We want to be just in the home? Just in the factory? What’s the right mixture? Do we want backflips? I don’t know. We have to figure it out.

There are more questions too, like whether we have enough data on the Internet to train robots, and how we could mix and transfer capabilities from Internet data sets into robotics. Is robot data fundamentally different than other data? Should we expect the same scaling laws? Should we expect the same long-term capabilities?

The other big one that you’ll hear the experts talk about is evaluation, which is a major bottleneck. If you look at some of these papers that show incredible results, the statistical strength of their results section is very weak and consequently we’re making a lot of claims about things that we don’t really have a lot of basis for. It will take a lot of engineering work to carefully build up empirical strength in our results. I think evaluation doesn’t get enough attention.

What has changed in robotics research in the last year or so that you think has enabled the kind of progress that you’re hoping to achieve?

Kuindersma: From my perspective, there are two high-level things that have changed how I’ve thought about work in this space. One is the convergence of the field around repeatable processes for training manipulation skills through demonstrations. The pioneering work of diffusion policy (which TRI was a big part of) is a really powerful thing—it takes the process of generating manipulation skills that previously were basically unfathomable, and turned it into something where you just collect a bunch of data, you train it on an architecture that’s more or less stable at this point, and you get a result.

The second thing is everything that’s happened in robotics-adjacent areas of AI showing that data scale and diversity are really the keys to generalizable behavior. We expect that to also be true for robotics. And so taking these two things together, it makes the path really clear, but I still think there are a ton of open research challenges and questions that we need to answer.

Do you think that simulation is an effective way of scaling data for robotics?

Tedrake: I think generally people underestimate simulation. The work we’ve been doing has made me very optimistic about the capabilities of simulation as long as you use it wisely. Focusing on a specific robot doing a specific task is asking the wrong question; you need to get the distribution of tasks and performance in simulation to be predictive of the distribution of tasks and performance in the real world. There are some things that are still hard to simulate well, but even when it comes to frictional contact and stuff like that, I think we’re getting pretty good at this point.

Is there a commercial future for this partnership that you’re able to talk about?

Kuindersma: For Boston Dynamics, clearly we think there’s long-term commercial value in this work, and that’s one of the main reasons why we want to invest in it. But the purpose of this collaboration is really about fundamental research—making sure that we do the work, advance the science, and do it in a rigorous enough way so that we actually understand and trust the results and we can communicate that out to the world. So yes, we see tremendous value in this commercially. Yes, we are commercializing Atlas, but this project is really about fundamental research.

What happens next?

Tedrake: There are questions at the intersection of things that BD has done and things that TRI has done that we need to do together to start, and that’ll get things going. And then we have big ambitions—getting a generalist capability that we’re calling LBM (large behavior models) running on Atlas is the goal. In the first year we’re trying to focus on these fundamental questions, push boundaries, and write and publish papers.

I want people to be excited about watching for our results, and I want people to trust our results when they see them. For me, that’s the most important message for the robotics community: Through this partnership we’re trying to take a longer view that balances our extreme optimism with being critical in our approach.

Boston Dynamics is the master of dropping amazing robot videos with no warning, and last week, we got a surprise look at the new electric Atlas going “hands on” with a practical factory task.

This video is notable because it’s the first real look we’ve had at the new Atlas doing something useful—or doing anything at all, really, as the introductory video from back in April (the first time we saw the robot) was less than a minute long. And the amount of progress that Boston Dynamics has made is immediately obvious, with the video showing a blend of autonomous perception, full body motion, and manipulation in a practical task.

We sent over some quick questions as soon as we saw the video, and we’ve got some extra detail from Scott Kuindersma, senior director of Robotics Research at Boston Dynamics.

If you haven’t seen this video yet, what kind of robotics person are you, and also here you go:

Atlas is autonomously moving engine covers between supplier containers and a mobile sequencing dolly. The robot receives as input a list of bin locations to move parts between.

Atlas uses a machine learning (ML) vision model to detect and localize the environment fixtures and individual bins [0:36]. The robot uses a specialized grasping policy and continuously estimates the state of manipulated objects to achieve the task.

There are no prescribed or teleoperated movements; all motions are generated autonomously online. The robot is able to detect and react to changes in the environment (e.g., moving fixtures) and action failures (e.g., failure to insert the cover, tripping, environment collisions [1:24]) using a combination of vision, force, and proprioceptive sensors.

Eagle-eyed viewers will have noticed that this task is very similar to what we saw hydraulic Atlas (Atlas classic?) working on just before it retired. We probably don’t need to read too much into the differences between how each robot performs that task, but it’s an interesting comparison to make.

For more details, here’s our Q&A with Kuindersma:

How many takes did this take?

Kuindersma: We ran this sequence a couple times that day, but typically we’re always filming as we continue developing and testing Atlas. Today we’re able to run that engine cover demo with high reliability, and we’re working to expand the scope and duration of tasks like these.

Is this a task that humans currently do?

Kuindersma: Yes.

What kind of world knowledge does Atlas have while doing this task?

Kuindersma: The robot has access to a CAD model of the engine cover that is used for object pose prediction from RGB images. Fixtures are represented more abstractly using a learned keypoint prediction model. The robot builds a map of the workcell at startup which is updated on the fly when changes are detected (e.g., moving fixture).

Does Atlas’s torso have a front or back in a meaningful way when it comes to how it operates?

Kuindersma: Its head/torso/pelvis/legs do have “forward” and “backward” directions, but the robot is able to rotate all of these relative to one another. The robot always knows which way is which, but sometimes the humans watching lose track.

Are the head and torso capable of unlimited rotation?

Kuindersma: Yes, many of Atlas’s joints are continuous.

How long did it take you folks to get used to the way Atlas moves?

Kuindersma: Atlas’s motions still surprise and delight the team.

OSHA recommends against squatting because it can lead to workplace injuries. How does Atlas feel about that?

Kuindersma: As might be evident by some of Atlas’s other motions, the kinds of behaviors that might be injurious for humans might be perfectly fine for robots.

Can you describe exactly what process Atlas goes through at 1:22?

Kuindersma: The engine cover gets caught on the fabric bins and triggers a learned failure detector on the robot. Right now this transitions into a general-purpose recovery controller, which results in a somewhat jarring motion (we will improve this). After recovery, the robot retries the insertion using visual feedback to estimate the state of both the part and fixture.

Were there other costume options you considered before going with the hot dog?

Kuindersma: Yes, but marketing wants to save them for next year.

How many important sensors does the hot dog costume occlude?

Kuindersma: None. The robot is using cameras in the head, proprioceptive sensors, IMU, and force sensors in the wrists and feet. We did have to cut the costume at the top so the head could still spin around.

Why are pickles always causing problems?

Kuindersma: Because pickles are pesky, polarizing pests.

Spot Release 2.0, launching today, includes improvements to navigation, autonomy, stair climbing, and more

Boston Dynamics robots imitate human and animal movements, making them impressive — and a little creepy.

Boston Dynamics began began leasing their robotic dogs to the public this year. One of their first customers: The Massachusetts State Police.

In a startling turn for a robot that has long haunted your dreams, Boston Dynamics' Spot has been tasked to encourage healthy behavior. 

According to the Straits Times, the robot "dog" has been assigned to patrol the Bishan-Ang Mo Kio park in Singapore with the express purpose of encouraging social distancing. 

"Let's keep Singapore healthy," sounds a recording from the robot as it trots by two terrified people relaxing on a park bench in the above video. "For your own safety, and those around you, please stand at least one meter apart. Thank you."

Notably, Spot's jaunt is part of a two-week trial that began Friday. The robot will supposedly not collect any personal information on the people it admonishes.  Read more...

Here's everything we know about Boston Dynamics' first commercial robot

Your weekly selection of awesome robot videos

An exclusive look at the world’s most dynamic robots

Boston Dynamics robot dog, known as Spot, is patroling Bishan-Ang Mo Kio Park in Singapore to help with social distancing practices during the coronavirus pandemic, the Singapore government announced.

Since announcing the commercial availability of Spot, Boston Dynamics has presented a range of different gigs for the robot, from construction to telepresence. Last month, the company announced it was partnering with local hospitals interested in using the platform to perform remote visits for COVID-19 victims. Turns out the global pandemic has spurred all manner […]

Spot will traverse a 4-mile swath of Bishan-Ang Mo Kio Park during off-peak hours while playing a recorded message that reminds park-goers 'observe safe distancing measures.'

Spot will traverse a 4-mile swath of Bishan-Ang Mo Kio Park during off-peak hours while playing a recorded message that reminds park-goers 'observe safe distancing measures.'

SpotMini, the Internet's favorite robot dog, appeared at WIRED25 with Boston Dynamics founder & CEO Marc Raibert. Raibert spoke with WIRED Editor in Chief Nicholas Thompson about robots and the future of robotic design as part of WIRED's 25th anniversary celebration in San Francisco.