We show a novel encountered-type haptic system, H-Wall, to enable haptic feedback using a 7-DoF manipulator suitable for simulating indoor VR environments, which are characterized and confined by a set of vertical walls and revolving doors.At runtime, our system tracks hand motion using an RGBD sensor and locates its configuration. Then, the robotic manipulator plans a trajectory for the end-effector, attached to a rectangular rigid board, to make a contact with the hand to deliver a sense of touch, as long as the perceived hand contact force is substantial. The force feedback is generated in a passive sense for static walls that the rigid board, corresponding to a vertical wall, holds its position as long as the perceived hand contact force is substantial. For a revolving door, the force feedback is generated in an active sense based on impedance control. In order to address the issue of limited workspace, we also propose a new reachability map, called per-plane reachability maps, that is optimized to answer whether passive haptic feedback can be generated by a manipulator when the user touches a vertical wall at a given orientation. These maps are generated offline using optimization and are looked up at runtime.

H-Wall  System

  • User Tracking
    1.  Head and hand positions of user are tracked by a RGBD Sensor.
    2.  Tracked data are transformed from the camera frame to the robot base frame.
  • Checking Reachability
    1.  Using pre-computed per-plane reachability maps, check the reachability of manipulator on hand position of the user.
  • Haptic Rendering
    1.  A 7-DoF manipulator, capable of human-robot-coexistence, operates as a haptic device to provide passive and active touch feedback to the user.
  • Visual Rendering
    1.  A user's virtual hand and an indoor virtual environment are rendered to the HMD.

  • System Setup

  • We use KUKA LBR IIWA 7 R800 manipulator as a haptic device.
  • A transparent, lightweight Acrylic Panel is attached to the IIWA end-effector and serves as a proxy for a virtual walls and doors.
  • We use Kinect for Windows for tracking usesr hand and head position.
  • Oculus Rift CV1 is used for rendering virtual environment and virtual hand.
  • Our  Approach

    Per-plane Reachability Maps

  • Reachability map is a set of points that can be reached by a reference point.
  • We impose the characteristics of vertical object as additional constraints.
    1.   1. Sample the workspace by n different orientation.
    2.   2. Sample again into m equidistance planes.
    3.   3. Compute the boundary points of each per-plane reachability map.

    Optimization-based Computation

    Per-plane reachability maps with orienrtation of the end-effector r=0.

    Per-plane reachability maps with orienrtation of the end-effector r=0.25pi

  • We formulate a set of equality constraints in terms of orientation and position.
  • We also formulate a objective function to calculate extremal y values.
  • Solve the non-linear optimization problem in order to coumpute the boundary points of each per-plane reachability map.

  • Cheking Reachability

  • Check the pre-computed reachability map in runtime.
  • The manipulator position and a virtual hand position are updated when an input value is inside the per-plane reachability map,
  • The manipulator and a virtual hand maintain their position and wait for the next update when an input value is outside the per-plane reachability map,.

  • results

    Passive Haptic Interaction Sequences

    Active Haptic Interaction Sequences


    Ewha Graphics Lab
    Department of Computer Science & Engineering, Ewha Womans University
      52, Ewhayeodae-gil, Seodaemun-gu, Seoul, Korea, 03760

      Yaesol Kim,
      Hyun Jung Kim,
      Young J. Kim,