XRT
Unity
WebRTC
Link: XRT Whitepaper
XRT is a humanoid robot teleoperation interface for VR headsets that allows users to control robots using inside-out body tracking via wireless connection.
Most robot teleoperation interfaces (both hardware and software) have problems, and the field really needs a better de-facto standard/library for a teleop stack. For example:
- They are outdated and rely on bulky wired connections. However, having wire harnesses all over the user and the robot is suboptimal for not only safety, but also the quality of freedom.
- They often rely on physical controllers {hover: that looks like this:link} and do not make use of recent advances in real-time hand tracking technology, which has the potential of allowing users a much more intuitive control of dextrous robot hands
Gaming was the primary driver of research and adoption of VR/AR technology, and we can look towards them for some inspiration:
- Virtual Desktop, which is one of the most popular VR apps of all time, performs real-time encoding of video (from PC) and decoding (in VR) to essentially implement a virtual computer monitor with no perceived latency (which requires something like <80ms) purely over Wi-Fi!
VR user interfaces — like the one in Meta OS — makes great use of hand gestures, to allow a wider range of actions than would be possible with on-screen UI
It’s a no-brainer that it would be great to
We aimed to bring all of these great improvements to robotics, because it’d be great
More concretely, our vision was:
Screenshots [HIDE]
| Screenshot | Description | Why |
|---|---|---|
| someone using XRT (3rd person) | ||