Haptic Painting | 3D Surface Painting in VR using Force Feedback

Introduction

Since the concept of virtual reality was introduced, many researchers have made various efforts for virtual painting without being impeded by physical constraints, such as canvas size or painting material Recently, a few VR painting softwares have appeared such as Tilt Brush, Quill, and CanvoX to resurge such an interest.

Due to the nature of VR painting, which draws in the air users, only rely on visual feedback to check the progress of the painting. However, the visual depth cue that users can perceive does not always match the geometric depth in the VR, which is the main cause of unintended stroking results. Therefore, users have to repeatedly draw strokes to fix it This may reduce users’ overall productivity and also can result in significant physical fatigue as well.

We propose haptic interfaces using force feedback devices to address the problems of the existing VR painting systems Through this system, users can experience more effective and less stressful VR painting.

Systems

Overview

The Painting Module

For painting in VR using user’s haptic input can change the size and color of strokes

The Haptic Module

For communicating with a haptic device and handles stylus input and force output

The User Module

To engage a user with viewpoint navigation in a virtual environment

Systems

VR Painting System

• Painting strokes are represented by triangular mesh strips.
• Users employ a haptic stylus to create a drawing trajectory corresponding to painting strokes.
• The mesh is created by Mesh component on Unreal Engine 4. Users can draw strokes with four different textures.
• Users use an extra HMD controller to execute assisting functions for painting.

Systems

Haptic Interface

Force Drawing Anchor

• We display elastic forces in proportion to the distance between the users’ stylus position and the "anchored" position.
• This force display provides an intuitive sense of the relative depth and makes the system responsive to the user’s intention.

Virtual Drawing Plane

• We simulate a virtual planar canvas surface by generating penalty-based elastic forces.
• Our system uses the penetration depth of the haptics with the plane, and the resulting force size is proportional to the depth.
• This emulates a physical canvas and helps the users to make consistent contact with the virtual canvas.

Simulating Painting Material

• We simulate actual painting materials by applying different kinetic friction to the brush stroke.
• The friction is applied to the opposing direction to the haptic interface’s velocity, reflecting the viscosity of each texture