Hurricane Helene hit a quartz mine in North Carolina that is key to global semiconductor production, which could impact the entire tech industry. Here is everything we know so far

Motion sensors in smartphones can be turned into makeshift microphones to eavesdrop on conversations, outsmarting security features designed to stop such attacks

It is thought that humans can only maintain relationships with around 150 people, a figure known as Dunbar's number, but it seems that AI models can outstrip this and reach consensus in far bigger groups

The UK government's decision to return the Chagos Islands to Mauritius surprisingly threatens the extinction of millions of website addresses ending in ".io", and no one is quite sure what will happen next

A simulator for the process of scientific discovery shows that AI models still fall short of human scientists and engineers in coming up with hypotheses and carrying out experiments on their own

Next-generation phone networks could dramatically outperform current ones thanks to a new technique for transmitting multiple streams of data over a wide range of frequencies

While quantum computers are still in their infancy, more and more people are training to become quantum software developers

AI models have safeguards in place to prevent them creating dangerous or illegal output, but a range of jailbreaks have been found to evade them. Now researchers show that writing backwards can trick AI models into revealing bomb-making instructions.

Google’s NotebookLM tool is billed as an AI-powered research assistant and can even turn your text history into a jovial fake podcast. But it could also tempt you into narcissism and nostalgia, says Jacob Aron

A type of mathematical problem that was previously impossible to solve can now be successfully analysed with artificial intelligence

What happens if AI is trained to write choral music by feeding it a specially created vocal dataset? Moving new exhibition The Call tackles some thorny questions about AI and creativity – and stirs the soul with music

A new supercapacitor design that uses only water, clay and graphene could source material on Mars and be more sustainable and accessible than traditional batteries

A lithium-ion battery made from three droplets of hydrogel is the smallest soft battery of its kind – and it could be used in biocompatible and biodegradable implants

A specialised algorithm could help autonomous vehicles track hidden objects, such as a pedestrian, a bicycle or another vehicle concealed behind a parked car

Scientists used AI to transform tourist photos into a 3D digital map of Antarctic penguin colonies – even as researchers debate whether to harness or discourage tourism in this remote region

Thanks to artificial intelligence, understanding animals may be closer than we think. But we may not like what they are going to tell us, says RSPCA chief executive Chris Sherwood

Feedback digs into a baffling ad for a mobile game and identifies a new and devilish way to advertise a product online: make it as confusing as possible to encourage people to click (it worked on Feedback)

An off-the-shelf millimetre wave sensor can pick out the tiny vibrations made by a smartphone's speaker, enabling an AI model to transcribe the conversation, even at a distance in a noisy room

One day, doctors might be able to 3D print copies of your organs in order to test a variety of drugs, thanks to a new technique that uses light and sound for rapid printing

Oil and water can be separated efficiently by pumping the mixture through thin channels between two semipermeable membranes

Hollywood has been imagining the impact AI might have on our lives for decades, but how accurate are these portrayals?

The controversies surrounding football’s video assistant referee (VAR) system highlight our troubled relationship with uncertainty – and point to potential solutions

Seven major datasets used to train audio-generating AI models are three times more likely to use the words "man" or "men" than "woman" or "women", raising fears of bias

An AI-assisted robot can listen to spoken commands and assemble 3D objects such as chairs and tables out of reusable building blocks

After seven years of voicing Elena Ferrante’s ambitious heroine, the Italian actress brings Lenù Greco home.

That’s not his deal, you guys.

Treat yourself to a Becky Lynch book signing, games with the Dropout stars, Kevin Smith’s Dogma, and so much more!

“It’s not funny,” he told her while hosting Saturday Night Live.

Here’s everything you need to know about the allegations, diss tracks, and that $12 million NDA.

It’s the second-shortest book to be awarded the prize.

Or, for the other half of you: Here Is Our Ivy Wolk Explainer.

Who built this ripped, anatomically graphic snowman? Is there a world of snowmen offscreen waiting to be turned into sex objects for widows?

Can we trust whatever is going on with Becky Minkoff?

Is Jessie available to babysit again?

The finale doesn’t look to provide a definitive answer to what drove Aaron’s actions, much to the show’s credit.

Krasinski scores the title of People’s Sexiest Man Alive 2024.

Ronnie O'Sullivan has agreed to compete in new snooker events.

Mike Tyson has not fought professionally since suffering a stoppage defeat to Kevin McBride in 2005.

Joe Marler discussed his recent comments about the Haka, which sparked backlash.

There has been another shock at the Grand Slam of Darts.

Emma Raducanu struck a deal to return to one of her favourite tournaments.

Former FIA race director Niels Wittich has rejected the motorsport governing body's version regarding his departure.

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

ICRA@40: 23–26 September 2024, ROTTERDAM, NETHERLANDS

IROS 2024: 14–18 October 2024, ABU DHABI, UAE

ICSR 2024: 23–26 October 2024, ODENSE, DENMARK

Cybathlon 2024: 25–27 October 2024, ZURICH

Enjoy today’s videos!

I think it’s time for us all to admit that some of the most interesting bipedal and humanoid research is being done by Disney.

[ Research Paper from ETH Zurich and Disney Research]

Over the past few months, Unitree G1 robot has been upgraded into a mass production version, with stronger performance, ultimate appearance, and being more in line with mass production requirements.

[ Unitree ]

This robot is from Kinisi Robotics, which was founded by Brennand Pierce, who also founded Bear Robotics. You can’t really tell from this video, but check out the website because the reach this robot has is bonkers.

Kinisi Robotics is on a mission to democratize access to advanced robotics with our latest innovation—a low-cost, dual-arm robot designed for warehouses, factories, and supermarkets. What sets our robot apart is its integration of LLM technology, enabling it to learn from demonstrations and perform complex tasks with minimal setup. Leveraging Brennand’s extensive experience in scaling robotic solutions, we’re able to produce this robot for under $20k, making it a game-changer in the industry.

[ Kinisi Robotics ]

Thanks Bren!

Finally, something that Atlas does that I am also physically capable of doing. In theory.

Okay, never mind. I don’t have those hips.

[ Boston Dynamics ]

Researchers in the Department of Mechanical Engineering at Carnegie Mellon University have created the first legged robot of its size to run, turn, push loads, and climb miniature stairs.

They say it can “run,” but I’m skeptical that there’s a flight phase unless someone sneezes nearby.

[ Carnegie Mellon University ]

The lights are cool and all, but it’s the pulsing soft skin that’s squigging me out.

[ Paper, Robotics Reports Vol.2 ]

Roofing is a difficult and dangerous enough job that it would be great if robots could take it over. It’ll be a challenge though.

[ Renovate Robotics ] via [ TechCrunch ]

Kento Kawaharazuka from JSK Robotics Laboratory at the University of Tokyo wrote in to share this paper, just accepted at RA-L, which (among other things) shows a robot using its flexible hands to identify objects through random finger motion.

[ Paper accepted by IEEE Robotics and Automation Letters ]

Thanks Kento!

It’s one thing to make robots that are reliable, and it’s another to make robots that are reliable and repairable by the end user. I don’t think iRobot gets enough credit for this.

[ iRobot ]

I like competitions where they say, “just relax and forget about the competition and show us what you can do.”

[ MBZIRC Maritime Grand Challenge ]

I kid you not, this used to be my job.

[ RoboHike ]

In the 1960s and 1970s, NASA spent a lot of time thinking about whether toroidal (donut-shaped) fuel tanks were the way to go with its spacecraft. Toroidal tanks have a bunch of potential advantages over conventional spherical fuel tanks. For example, you can fit nearly 40% more volume within a toroidal tank than if you were using multiple spherical tanks within the same space. And perhaps most interestingly, you can shove stuff (like the back of an engine) through the middle of a toroidal tank, which could lead to some substantial efficiency gains if the tanks could also handle structural loads.

Because of their relatively complex shape, toroidal tanks are much more difficult to make than spherical tanks. Even though these tanks can perform better, NASA simply doesn’t have the expertise to manufacture them anymore, since each one has to be hand-built by highly skilled humans. But a company called Machina Labs thinks that they can do this with robots instead. And their vision is to completely change how we make things out of metal.

The fundamental problem that Machina Labs is trying to solve is that if you want to build parts out of metal efficiently at scale, it’s a slow process. Large metal parts need their own custom dies, which are very expensive one-offs that are about as inflexible as it’s possible to get, and then entire factories are built around these parts. It’s a huge investment, which means that it doesn’t matter if you find some new geometry or technique or material or market, because you have to justify that enormous up-front cost by making as much of the original thing as you possibly can, stifling the potential for rapid and flexible innovation.

On the other end of the spectrum you have the also very slow and expensive process of making metal parts one at a time by hand. A few hundred years ago, this was the only way of making metal parts: skilled metalworkers using hand tools for months to make things like armor and weapons. The nice thing about an expert metalworker is that they can use their skills and experience to make anything at all, which is where Machina Labs’ vision comes from, explains CEO Edward Mehr who co-founded Machina Labs after spending time at SpaceX followed by leading the 3D printing team at Relativity Space.

“Craftsmen can pick up different tools and apply them creatively to metal to do all kinds of different things. One day they can pick up a hammer and form a shield out of a sheet of metal,” says Mehr. “Next, they pick up the same hammer, and create a sword out of a metal rod. They’re very flexible.”

The technique that a human metalworker uses to shape metal is called forging, which preserves the grain flow of the metal as it’s worked. Casting, stamping, or milling metal (which are all ways of automating metal part production) are simply not as strong or as durable as parts that are forged, which can be an important differentiator for (say) things that have to go into space. But more on that in a bit.

The problem with human metalworkers is that the throughput is bad—humans are slow, and highly skilled humans in particular don’t scale well. For Mehr and Machina Labs, this is where the robots come in.

“We want to automate and scale using a platform called the ‘robotic craftsman.’ Our core enablers are robots that give us the kinematics of a human craftsman, and artificial intelligence that gives us control over the process,” Mehr says. “The concept is that we can do any process that a human craftsman can do, and actually some that humans can’t do because we can apply more force with better accuracy.”

This flexibility that robot metalworkers offer also enables the crafting of bespoke parts that would be impractical to make in any other way. These include toroidal (donut-shaped) fuel tanks that NASA has had its eye on for the last half century or so.

Machina Labs’ CEO Edward Mehr (on right) stands behind a 15 foot toroidal fuel tank.Machina Labs

“The main challenge of these tanks is that the geometry is complex,” Mehr says. “Sixty years ago, NASA was bump-forming them with very skilled craftspeople, but a lot of them aren’t around anymore.” Mehr explains that the only other way to get that geometry is with dies, but for NASA, getting a die made for a fuel tank that’s necessarily been customized for one single spacecraft would be pretty much impossible to justify. “So one of the main reasons we’re not using toroidal tanks is because it’s just hard to make them.”

Machina Labs is now making toroidal tanks for NASA. For the moment, the robots are just doing the shaping, which is the tough part. Humans then weld the pieces together. But there’s no reason why the robots couldn’t do the entire process end-to-end and even more efficiently. Currently, they’re doing it the “human” way based on existing plans from NASA. “In the future,” Mehr tells us, “we can actually form these tanks in one or two pieces. That’s the next area that we’re exploring with NASA—how can we do things differently now that we don’t need to design around human ergonomics?”

Machina Labs’ ‘robotic craftsmen’ work in pairs to shape sheet metal, with one robot on each side of the sheet. The robots align their tools slightly offset from each other with the metal between them such that as the robots move across the sheet, it bends between the tools. Machina Labs

The video above shows Machina’s robots working on a tank that’s 4.572 m (15 feet) in diameter, likely destined for the Moon. “The main application is for lunar landers,” says Mehr. “The toroidal tanks bring the center of gravity of the vehicle lower than what you would have with spherical or pill-shaped tanks.”

Training these robots to work metal like this is done primarily through physics-based simulations that Machina developed in house (existing software being too slow), followed by human-guided iterations based on the resulting real-world data. The way that metal moves under pressure can be simulated pretty well, and although there’s certainly still a sim-to-real gap (simulating how the robot’s tool adheres to the surface of the material is particularly tricky), the robots are collecting so much empirical data that Machina is making substantial progress towards full autonomy, and even finding ways to improve the process.

An example of the kind of complex metal parts that Machina’s robots are able to make.Machina Labs

Ultimately, Machina wants to use robots to produce all kinds of metal parts. On the commercial side, they’re exploring things like car body panels, offering the option to change how your car looks in geometry rather than just color. The requirement for a couple of beefy robots to make this work means that roboforming is unlikely to become as pervasive as 3D printing, but the broader concept is the same: making physical objects a software problem rather than a hardware problem to enable customization at scale.

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

ICRA@40: 23–26 September 2024, ROTTERDAM, NETHERLANDS

IROS 2024: 14–18 October 2024, ABU DHABI, UAE

ICSR 2024: 23–26 October 2024, ODENSE, DENMARK

Cybathlon 2024: 25–27 October 2024, ZURICH

Enjoy today’s videos!

The National Science Foundation Human AugmentatioN via Dexterity Engineering Research Center (HAND ERC) was announced in August 2024. Funded for up to 10 years and $52 million, the HAND ERC is led by Northwestern University, with core members Texas A&M, Florida A&M, Carnegie Mellon, and MIT, and support from Wisconsin-Madison, Syracuse, and an innovation ecosystem consisting of companies, national labs, and civic and advocacy organizations. HAND will develop versatile, easy-to-use dexterous robot end effectors (hands).

[ HAND ]

The Environmental Robotics Lab at ETH Zurich, in partnership with Wilderness International (and some help from DJI and Audi), is using drones to sample DNA from the tops of trees in the Peruvian rainforest. Somehow, the treetops are where 60 to 90 percent of biodiversity is found, and these drones can help researchers determine what the heck is going on up there.

[ ERL ]

Thanks, Steffen!

1X introduces NEO Beta, “the pre-production build of our home humanoid.”

“Our priority is safety,” said Bernt Børnich, CEO at 1X. “Safety is the cornerstone that allows us to confidently introduce NEO Beta into homes, where it will gather essential feedback and demonstrate its capabilities in real-world settings. This year, we are deploying a limited number of NEO units in selected homes for research and development purposes. Doing so means we are taking another step toward achieving our mission.”

[ 1X ]

We love MangDang’s fun and affordable approach to robotics with Mini Pupper. The next generation of the little legged robot has just launched on Kickstarter, featuring new and updated robots that make it easy to explore embodied AI.

The Kickstarter is already fully funded after just a day or two, but there are still plenty of robots up for grabs.

[ Kickstarter ]

Quadrupeds in space can use their legs to reorient themselves. Or, if you throw one off a roof, it can learn to land on its feet.

To be presented at CoRL 2024.

[ ARL ]

HEBI Robotics, which apparently was once headquartered inside a Pittsburgh public bus, has imbued a table with actuators and a mind of its own.

[ HEBI Robotics ]

Carcinization is a concept in evolutionary biology where a crustacean that isn’t a crab eventually becomes a crab. So why not do the same thing with robots? Crab robots solve all problems!

[ KAIST ]

Waymo is smart, but also humans are really, really dumb sometimes.

[ Waymo ]

The Robotics Department of the University of Michigan created an interactive community art project. The group that led the creation believed that while roboticists typically take on critical and impactful problems in transportation, medicine, mobility, logistics, and manufacturing, there are many opportunities to find play and amusement. The final piece is a grid of art boxes, produced by different members of our robotics community, which offer an eight-inch-square view into their own work with robotics.

[ Michigan Robotics ]

I appreciate that UBTECH’s humanoid is doing an actual job, but why would you use a humanoid for this?

[ UBTECH ]

I’m sure most actuators go through some form of life-cycle testing. But if you really want to test an electric motor, put it into a BattleBot and see what happens.

[ Hardcore Robotics ]

Yes, but have you tried fighting a BattleBot?

[ AgileX ]

In this video, we present collaboration aerial grasping and transportation using multiple quadrotors with cable-suspended payloads. Grasping using a suspended gripper requires accurate tracking of the electromagnet to ensure a successful grasp while switching between different slack and taut modes. In this work, we grasp the payload using a hybrid control approach that switches between a quadrotor position control and a payload position control based on cable slackness. Finally, we use two quadrotors with suspended electromagnet systems to collaboratively grasp and pick up a larger payload for transportation.

[ Hybrid Robotics ]

I had not realized that the floretizing of broccoli was so violent.

[ Oxipital ]

While the RoboCup was held over a month ago, we still wanted to make a small summary of our results, the most memorable moments, and of course an homage to everyone who is involved with the B-Human team: the team members, the sponsors, and the fans at home. Thank you so much for making B-Human the team it is!

[ B-Human ]

This is a sponsored article brought to you by Khalifa University of Science and Technology.

Abu Dhabi-based Khalifa University of Science and Technology in the United Arab Emirates (UAE) will be hosting the 36th edition of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2024) to highlight the Middle East and North Africa (MENA) region’s rapidly advancing capabilities in the robotics and intelligent transport systems.

aspect_ratio

Themed “Robotics for Sustainable Development,” the IROS 2024 will be held from 14-18 October 2024 at the Abu Dhabi National Exhibition Center (ADNEC) in the UAE’s capital city. It will offer a platform for universities and research institutions to display their research and innovation activities and initiatives in robotics, gathering researchers, academics, leading corporate majors, and industry professionals from around the globe.

A total of 13 forums, nine global-level competitions and challenges covering various aspects of robotics and AI, an IROS Expo, as well as an exclusive Career Fair will also be part of IROS 2024. The challenges and competitions will focus on physical or athletic intelligence of robots, remote robot navigation, robot manipulation, underwater robotics, as well as perception and sensing.

Delegates for the event will represent sectors including manufacturing, healthcare, logistics, agriculture, defense, security, and mining sectors with 60 percent of the talent pool having over six years of experience in robotics. A major component of the conference will be the poster sessions, keynotes, panel discussions by researchers and scientists, and networking events.

Khalifa University will be hosting IROS 2024 to highlight the Middle East and North Africa (MENA) region’s rapidly advancing capabilities in the robotics and intelligent transport systems.Khalifa University

Abu Dhabi ranks first on the world’s safest cities list in 2024, according to online database Numbeo, out of 329 global cities in the 2024 standings, holding the title for eight consecutive years since 2017, reflecting the emirate’s ongoing efforts to ensure a good quality of life for citizens and residents.

With a multicultural community, Abu Dhabi is home to people from more than 200 nationalities and draws a large number of tourists to some of the top art galleries in the city such as Louvre Abu Dhabi and the Guggenheim Abu Dhabi, as well as other destinations such as Ferrari World Abu Dhabi and Warner Bros. World Abu Dhabi.

The UAE and Abu Dhabi have increasingly become a center for creative skillsets, human capital and advanced technologies, attracting several international and regional events such as the global COP28 UAE climate summit, in which more than 160 countries participated.

Abu Dhabi city itself has hosted a number of association conventions such as the 34th International Nursing Research Congress and is set to host the UNCTAD World Investment Forum, the 13th World Trade Organization (WTO) Ministerial Conference (MC13), the 12th World Environment Education Congress in 2024, and the IUCN World Conservation Congress in 2025.

Khalifa University’s Center for Robotics and Autonomous Systems (KU-CARS) includes a vibrant multidisciplinary environment for conducting robotics and autonomous vehicle-related research and innovation.Khalifa University

Dr. Jorge Dias, IROS 2024 General Chair, said: “Khalifa University is delighted to bring the Intelligent Robots and Systems 2024 to Abu Dhabi in the UAE and highlight the innovations in line with the theme Robotics for Sustainable Development. As the region’s rapidly advancing capabilities in robotics and intelligent transport systems gain momentum, this event serves as a platform to incubate ideas, exchange knowledge, foster collaboration, and showcase our research and innovation activities. By hosting IROS 2024, Khalifa University aims to reaffirm the UAE’s status as a global innovation hub and destination for all industry stakeholders to collaborate on cutting-edge research and explore opportunities for growth within the UAE’s innovation ecosystem.”

“This event serves as a platform to incubate ideas, exchange knowledge, foster collaboration, and showcase our research and innovation activities” —Dr. Jorge Dias, IROS 2024 General Chair

Dr. Dias added: “The organizing committee of IROS 2024 has received over 4000 submissions representing 60 countries, with China leading with 1,029 papers, followed by the U.S. (777), Germany (302), and Japan (253), as well as the U.K. and South Korea (173 each). The UAE with a total of 68 papers comes atop the Arab region.”

Driving innovation at Khalifa University is the Center for Robotics and Autonomous Systems (KU-CARS) with around 50 researchers and state-of-the-art laboratory facilities, including a vibrant multidisciplinary environment for conducting robotics and autonomous vehicle-related research and innovation.

IROS 2024 is sponsored by IEEE Robotics and Automation Society, Abu Dhabi Convention and Exhibition Bureau, the Robotics Society of Japan (RSJ), the Society of Instrument and Control Engineers (SICE), the New Technology Foundation, and the IEEE Industrial Electronics Society (IES).

More information at https://iros2024-abudhabi.org/

When we think of grasping robots, we think of manipulators of some sort on the ends of arms of some sort. Because of course we do—that’s how (most of us) are built, and that’s the mindset with which we have consequently optimized the world around us. But one of the great things about robots is that they don’t have to be constrained by our constraints, and at ICRA@40 in Rotterdam this week, we saw a novel new Thing: a robotic hand that can detach from its arm and then crawl around to grasp objects that would be otherwise out of reach, designed by roboticists from EPFL in Switzerland.

Fundamentally, robot hands and crawling robots share a lot of similarities, including a body along with some wiggly bits that stick out and do stuff. But most robotic hands are designed to grasp rather than crawl, and as far as I’m aware, no robotic hands have been designed to do both of those things at the same time. Since both capabilities are important, you don’t necessarily want to stick with a traditional grasping-focused hand design. The researchers employed a genetic algorithm and simulation to test a bunch of different configurations in order to optimize for the ability to hold things and to move.

You’ll notice that the fingers bend backwards as well as forwards, which effectively doubles the ways in which the hand (or, “Handcrawler”) can grasp objects. And it’s a little bit hard to tell from the video, but the Handcrawler attaches to the wrist using magnets for alignment along with a screw that extends to lock the hand into place.

“Although you see it in scary movies, I think we’re the first to introduce this idea to robotics.” —Xiao Gao, EPFL

The whole system is controlled manually in the video, but lead author Xiao Gao tells us that they already have an autonomous version (with external localization) working in the lab. In fact, they’ve managed to run an entire grasping sequence autonomously, with the Handcrawler detaching from the arm, crawling to a location the arm can’t reach, picking up an object, and then returning and reattaching itself to the arm again.

Today, Boston Dynamics and the Toyota Research Institute (TRI) announced a new partnership “to accelerate the development of general-purpose humanoid robots utilizing TRI’s Large Behavior Models and Boston Dynamics’ Atlas robot.” Committing to working towards a general purpose robot may make this partnership sound like a every other commercial humanoid company right now, but that’s not at all that’s going on here: BD and TRI are talking about fundamental robotics research, focusing on hard problems, and (most importantly) sharing the results.

The broader context here is that Boston Dynamics has an exceptionally capable humanoid platform capable of advanced and occasionally painful-looking whole-body motion behaviors along with some relatively basic and brute force-y manipulation. Meanwhile, TRI has been working for quite a while on developing AI-based learning techniques to tackle a variety of complicated manipulation challenges. TRI is working toward what they’re calling large behavior models (LBMs), which you can think of as analogous to large language models (LLMs), except for robots doing useful stuff in the physical world. The appeal of this partnership is pretty clear: Boston Dynamics gets new useful capabilities for Atlas, while TRI gets Atlas to explore new useful capabilities on.

Here’s a bit more from the press release:

The project is designed to leverage the strengths and expertise of each partner equally. The physical capabilities of the new electric Atlas robot, coupled with the ability to programmatically command and teleoperate a broad range of whole-body bimanual manipulation behaviors, will allow research teams to deploy the robot across a range of tasks and collect data on its performance. This data will, in turn, be used to support the training of advanced LBMs, utilizing rigorous hardware and simulation evaluation to demonstrate that large, pre-trained models can enable the rapid acquisition of new robust, dexterous, whole-body skills.

The joint team will also conduct research to answer fundamental training questions for humanoid robots, the ability of research models to leverage whole-body sensing, and understanding human-robot interaction and safety/assurance cases to support these new capabilities.

For more details, we spoke with Scott Kuindersma (Senior Director of Robotics Research at Boston Dynamics) and Russ Tedrake (VP of Robotics Research at TRI).

How did this partnership happen?

Russ Tedrake: We have a ton of respect for the Boston Dynamics team and what they’ve done, not only in terms of the hardware, but also the controller on Atlas. They’ve been growing their machine learning effort as we’ve been working more and more on the machine learning side. On TRI’s side, we’re seeing the limits of what you can do in tabletop manipulation, and we want to explore beyond that.

Scott Kuindersma: The combination skills and tools that TRI brings the table with the existing platform capabilities we have at Boston Dynamics, in addition to the machine learning teams we’ve been building up for the last couple years, put us in a really great position to hit the ground running together and do some pretty amazing stuff with Atlas.

What will your approach be to communicating your work, especially in the context of all the craziness around humanoids right now?

Tedrake: There’s a ton of pressure right now to do something new and incredible every six months or so. In some ways, it’s healthy for the field to have that much energy and enthusiasm and ambition. But I also think that there are people in the field that are coming around to appreciate the slightly longer and deeper view of understanding what works and what doesn’t, so we do have to balance that.

The other thing that I’d say is that there’s so much hype out there. I am incredibly excited about the promise of all this new capability; I just want to make sure that as we’re pushing the science forward, we’re being also honest and transparent about how well it’s working.

Kuindersma: It’s not lost on either of our organizations that this is maybe one of the most exciting points in the history of robotics, but there’s still a tremendous amount of work to do.

What are some of the challenges that your partnership will be uniquely capable of solving?

Kuindersma: One of the things that we’re both really excited about is the scope of behaviors that are possible with humanoids—a humanoid robot is much more than a pair of grippers on a mobile base. I think the opportunity to explore the full behavioral capability space of humanoids is probably something that we’re uniquely positioned to do right now because of the historical work that we’ve done at Boston Dynamics. Atlas is a very physically capable robot—the most capable humanoid we’ve ever built. And the platform software that we have allows for things like data collection for whole body manipulation to be about as easy as it is anywhere in the world.

Tedrake: In my mind, we really have opened up a brand new science—there’s a new set of basic questions that need answering. Robotics has come into this era of big science where it takes a big team and a big budget and strong collaborators to basically build the massive data sets and train the models to be in a position to ask these fundamental questions.

Fundamental questions like what?

Tedrake: Nobody has the beginnings of an idea of what the right training mixture is for humanoids. Like, we want to do pre-training with language, that’s way better, but how early do we introduce vision? How early do we introduce actions? Nobody knows. What’s the right curriculum of tasks? Do we want some easy tasks where we get greater than zero performance right out of the box? Probably. Do we also want some really complicated tasks? Probably. We want to be just in the home? Just in the factory? What’s the right mixture? Do we want backflips? I don’t know. We have to figure it out.

There are more questions too, like whether we have enough data on the Internet to train robots, and how we could mix and transfer capabilities from Internet data sets into robotics. Is robot data fundamentally different than other data? Should we expect the same scaling laws? Should we expect the same long-term capabilities?

The other big one that you’ll hear the experts talk about is evaluation, which is a major bottleneck. If you look at some of these papers that show incredible results, the statistical strength of their results section is very weak and consequently we’re making a lot of claims about things that we don’t really have a lot of basis for. It will take a lot of engineering work to carefully build up empirical strength in our results. I think evaluation doesn’t get enough attention.

What has changed in robotics research in the last year or so that you think has enabled the kind of progress that you’re hoping to achieve?

Kuindersma: From my perspective, there are two high-level things that have changed how I’ve thought about work in this space. One is the convergence of the field around repeatable processes for training manipulation skills through demonstrations. The pioneering work of diffusion policy (which TRI was a big part of) is a really powerful thing—it takes the process of generating manipulation skills that previously were basically unfathomable, and turned it into something where you just collect a bunch of data, you train it on an architecture that’s more or less stable at this point, and you get a result.

The second thing is everything that’s happened in robotics-adjacent areas of AI showing that data scale and diversity are really the keys to generalizable behavior. We expect that to also be true for robotics. And so taking these two things together, it makes the path really clear, but I still think there are a ton of open research challenges and questions that we need to answer.

Do you think that simulation is an effective way of scaling data for robotics?

Tedrake: I think generally people underestimate simulation. The work we’ve been doing has made me very optimistic about the capabilities of simulation as long as you use it wisely. Focusing on a specific robot doing a specific task is asking the wrong question; you need to get the distribution of tasks and performance in simulation to be predictive of the distribution of tasks and performance in the real world. There are some things that are still hard to simulate well, but even when it comes to frictional contact and stuff like that, I think we’re getting pretty good at this point.

Is there a commercial future for this partnership that you’re able to talk about?

Kuindersma: For Boston Dynamics, clearly we think there’s long-term commercial value in this work, and that’s one of the main reasons why we want to invest in it. But the purpose of this collaboration is really about fundamental research—making sure that we do the work, advance the science, and do it in a rigorous enough way so that we actually understand and trust the results and we can communicate that out to the world. So yes, we see tremendous value in this commercially. Yes, we are commercializing Atlas, but this project is really about fundamental research.

What happens next?

Tedrake: There are questions at the intersection of things that BD has done and things that TRI has done that we need to do together to start, and that’ll get things going. And then we have big ambitions—getting a generalist capability that we’re calling LBM (large behavior models) running on Atlas is the goal. In the first year we’re trying to focus on these fundamental questions, push boundaries, and write and publish papers.

I want people to be excited about watching for our results, and I want people to trust our results when they see them. For me, that’s the most important message for the robotics community: Through this partnership we’re trying to take a longer view that balances our extreme optimism with being critical in our approach.

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

Humanoids 2024: 22–24 November 2024, NANCY, FRANCE

Enjoy today’s videos!

Swiss-Mile’s robot (which is really any robot that meets the hardware requirement to run their software) is faster than “most humans.” So what does that mean, exactly?

The winner here is Riccardo Rancan, who doesn’t look like he was trying especially hard—he’s the world champion in high-speed urban orienteering, which is a sport that I did not know existed but sounds pretty awesome.

[ Swiss-Mile ]

Thanks, Marko!

Oh good, we’re building giant fruit fly robots now.

But seriously, this is useful and important research because understanding the relationship between a nervous system and a bunch of legs can only be helpful as we ask more and more of legged robotic platforms.

[ Paper ]

Thanks, Clarus!

Watching humanoids get up off the ground will never not be fascinating.

[ Fourier ]

The Kepler Forerunner K2 represents the Gen 5.0 robot model, showcasing a seamless integration of the humanoid robot’s cerebral, cerebellar, and high-load body functions.

[ Kepler ]

Diffusion Forcing combines the strength of full-sequence diffusion models (like SORA) and next-token models (like LLMs), acting as either or a mix at sampling time for different applications without retraining.

[ MIT ]

Testing robot arms for space is no joke.

[ GITAI ]

Welcome to the Modular Robotics Lab (ModLab), a subgroup of the GRASP Lab and the Mechanical Engineering and Applied Mechanics Department at the University of Pennsylvania under the supervision of Prof. Mark Yim.

[ ModLab ]

This is much more amusing than it has any right to be.

[ Westwood Robotics ]

Let’s go for a walk with Adam at IROS’24!

[ PNDbotics ]

From Reachy 1 in 2023 to our newly launched Reachy 2, our grippers have been designed to enhance precision and dexterity in object manipulation. Some of the models featured in the video are prototypes used for various tests, showing the innovation behind the scenes.

[ Pollen ]

I’m not sure how else you’d efficiently spray the tops of trees? Drones seem like a no-brainer here.

[ SUIND ]

Presented at ICRA40 in Rotterdam, we show the challenges faced by mobile manipulation platforms in the field. We at CSIRO Robotics are working steadily towards a collaborative approach to tackle such challenging technical problems.

[ CSIRO ]

ABB is best known for arms, but it looks like they’re exploring AMRs (autonomous mobile robots) for warehouse operations now.

[ ABB ]

Howie Choset, Lu Li, and Victoria Webster-Wood of the Manufacturing Futures Institute explain their work to create specialized sensors that allow robots to “feel” the world around them.

[ CMU ]

Columbia Engineering Lecture Series in AI: “How Could Machines Reach Human-Level Intelligence?” by Yann LeCun.

Animals and humans understand the physical world, have common sense, possess a persistent memory, can reason, and can plan complex sequences of subgoals and actions. These essential characteristics of intelligent behavior are still beyond the capabilities of today’s most powerful AI architectures, such as Auto-Regressive LLMs.
I will present a cognitive architecture that may constitute a path towards human-level AI. The centerpiece of the architecture is a predictive world model that allows the system to predict the consequences of its actions. and to plan sequences of actions that that fulfill a set of objectives. The objectives may include guardrails that guarantee the system’s controllability and safety. The world model employs a Joint Embedding Predictive Architecture (JEPA) trained with self-supervised learning, largely by observation.

[ Columbia ]