This is cool and interesting work but unfortunately extremely over-hyped. Their whole system is a low cost bimanual system which is interesting. However, beyond that there isn't any algorithmic improvements in here that would make robotics really work.

Basically, no real breakthrough has happened in robotics and everyone is hoping to be able to replicate LLMs/vision path for robotics (i.e. data). But no one really knows how to even collect so much high quality data and no one is even remotely close to executing it. The best we seem to have done is: https://robotics-transformer-x.github.io/

Overall, just too much hype.

I don't believe there's any way to handle the open-ended complexity of the real world other than data. Whether that comes from human demonstrations or self-collected with reinforcement learning, there's just no way around it.

But I disagree there has been no progress. Check out this lecture from Sergey Levine about sample-efficient RL in the real world (no simulator): https://www.youtube.com/watch?v=17NrtKHdPDw

Problem is behaviour cloning doesnt generalize, and even on tasks where you have infinite optimal data (solved board games) it yields lower scores than self play. (and still doesnt generalize to unseen states)

Sim to real, self play, and curriculum learning have yielded superhuman performances when done correctly in situations where they can be done. Behaviour cloning doesnt.

1-human tele-operating 10-robots seems like an obvious solution Is any one working on this?

Certainly for most commercial/industrial/agriculture/hotels/mining settings (in fact, everything except a private apartment butler thingy..?)

> Basically, no real breakthrough has happened in robotics and everyone is hoping to be able to replicate LLMs/vision path for robotics (i.e. data). But no one really knows how to even collect so much high quality data and no one is even remotely close to executing it. The best we seem to have done is: https://robotics-transformer-x.github.io/

ALOHA + RT-X is a great combo to collect a lot of somewhat uniform data to enable such a breakthrough.

> Overall, just too much hype.

True. :)

> ALOHA + RT-X is a great combo to collect a lot of somewhat uniform data to enable such a breakthrough.

Collecting data will not enable anything. Google, Apple, Facebook, Microsoft and others are doing it since 10 years and no "breakthrough" in sight.

> Collecting data will not enable anything.

Historically, this has not been the case. Collecting data enabled speech recognition, image recognition (and subsequent breakthroughs), machine translation.

What is so special about Robotics that collecting data will not enable anything?

The environment is changing and there are many more out-of-distribution events that can interfere.

This is true. But Waymo is dealing with the same kind of challenges and it seems that they are already at a point, where their technology (enabled, among other things, by collecting tons of data) is useful.

I expect Robotics (in particular, solving manipulation) to follow the same route: super expensive R&D, multiple false starts, but eventually (tons of data => breakthrough to get smarter robots).

Wondering whether some combination of rfid, volume mapping, to map (learn) an apartment/home could help bridge the gap, on home automation. Could the stove, washers, driers, furniture... 'help' ? I guess it's been tried to death but I lack a good view of SOTA on home automation and robotics.

I don't think it's necessary for an open-ended system.

This is why I really hope some highly popular consumer product (smartphone, smart glasses etc.) starts having high quality LiDAR sensors so we have a lot of 3D training data.