INTRODUCTION


   We present a robotic pen-drawing system that is capable of faithfully reproducing pen art on an unknown surface. Our robotic system relies on an industrial, seven degree-of-freedom(DoF) manipulator that can be both position- and impedance-controlled. In order to estimate a rough geometry of the target, continuous surface, we first generate a point cloud of the surface using an RGB-D camera, which is filtered to remove outliers and calibrated to the physical canvas surface. Then, our control algorithm physically reproduces digital drawing on the surface by impedance-controlling the manipulator. Our impedance-controlled drawing algorithm compensates for the uncertainty and incompleteness inherent to a point-cloud estimation of the drawing surface. Moreover, since drawing 2D vector pen art on a 3D surface requires surface parameterization that does not destroy the original 2D drawing, we rely on the least squares conformal mapping. Specifically, the conformal map reduces angle distortion during surface parameterization. As a result, our system can create distortion-free and complicated pen drawings on general surfaces with many unpredictable bumps robustly and faithfully.

SYSTEM SETUP


  • We use KUKA LBR IIWA 7 R800 manipulator, which has 7DoFs and can be position- and force-controlled.
  • A 3D-printed gripper is attached to the end-effector to hold various types of pens.
  • We use Intel RealSense ZR300 RGB-D camera to capture 3D point cloud of the target surface.

OUR APPROACH

Surface Estimation

  • In order to reproduce the drawing originally provided as a sequence of 2D vectors on a target surface, we fist acquire depth information of the target surface via RGBD camera.
  • Using the depth data, we generate a point cloud of the scene and calibrate it with the target surface by defining a local frame, attached to the target surface using three non-degenerated points on it.
  • Further, we also estimate the surface normals for the calibrated points to orient the robot's tool frame with respect to the robot base frame.
  • We use impedance-based control to compensate for the possible estimation error.

Conformal Mapping

  • Conformal mapping is one of the surface parameterization techniques that preserves both angles and shapes.
  • We adopted the least squares conformal mapping (LSCM) to parameterize the target surface.
  • Once we unfold the target surface into 2D parameter space, we search for proper parameter values of the 2D drawing data in the parameter space, and refold the surface into 3D space.
  • Lastly, we perform bi-linear interpolation in the parametric domain to estimate the height for every control points.

Impedance-controlled Pen Drawing

  • With impedance control, we compensate for possible estimation or calibration error and generate continuous contact motions.
  • The deviation δx between the target drawing position, determined by the bilinear interpolation aobve, and the physical position of the pen tip results in a spring force in Cartesian space, where k is the spring stiffness.
  • The controller is configured in such a way that the robot is compliant only in the normal direction of the surface.

DRAWING RESULTS

Pattern Drawing Results

Grid
Sierpiński curve
Snowflake

Artistic Drawing Results

Racoon
Kangaroo
Bear
Owl

Drawing Statistics

Grid Sierpiński curve Snowflake Racoon Kangaroo Bear Owl
# of Strokes
95 12 12 860 3,147 1,520 1,942
# of Control Points
6,930 2,360 4,128 69,350 80,580 66,910 159,895
Drawing Surface Size (mm)
384 x 216 432 x 216 384 x 192 252 x 491 252 x 491 252 x 491 252 x 491
Mapping Time (min.)
635 308 523 - - - -
Execution Time (min.)
54 14 19 186 293 221 317

PUBLICATIONS

CONTACT

Ewha Graphics Lab
Department of Computer Science & Engineering, Ewha Womans University
  52, Ewhayeodae-gil, Seodaemun-gu, Seoul, Korea, 03760
  +82-2-3277-6798

  Daeun Song, daeun7250@ewhain.net
  Young J. Kim, kimy@ewha.ac.kr