Explainable OOHRI: Communicating Robot Capabilities and Limitations as AR Affordances

Demo video

Technical Demonstrations

1. Single-object pick-and-place: book repositioning

The user wants to place the book on a different shelf. With X-OOHRI, she moves the virtual twin to specify its placement while understanding the limitations.

2. Multi-object manipulation: room organization

The user tries to store foam blocks in an occluded basket. With X-OOHRI, she understands the limitation and selects the Auto resolution, which pulls the baskets out, stores the blocks, and pushes them back in.

3. Abstract action: blinds adjustment

The user finds the room too dark and wants to adjust the blinds. With X-OOHRI, she selects the Brighten action, sees the tilt wand is out of reach, and instead chooses Draw, opening the blinds through direct manipulation by raising the controller to specify the height.