We present a semi-autonomous robotic pen-drawing system that is capable of creating pen art on an arbitrary surface with varying thickness of pen strokes but without reconstructing the surface explicitly. Our robotic system relies on an industrial, seven-degree-of-freedom (7DoF) manipulator that can be both position- and impedance-controlled. We use a vector-graphics engine to take an artist's pen drawing as input and generate Bézier spline curves with varying offsets. In order to estimate geometric details of the target, unknown surface, during drawing, we rely on incremental and adaptive sampling on the surface using a combination of position and impedance control. Then, our control algorithm physically replicates this drawing on any arbitrary, continuous surface by impedance-controlling the manipulator. We demonstrate that our system can create visually-pleasing and complicated artistic pen drawings on general surfaces without explicit surface-reconstruction nor visual feedback.


  • We use KUKA LBR IIWA 7 R800 manipulator, which has 7DoFs and can be position- and force-controlled.
  • A 3D-printed gripper is attached to the end-effector to hold various types of pens.
  • We use a Samsung Galaxy Tablet PC equipped with its S-pen for running vector graphics under Android OS.


Vector Graphics Engine

  • Vector graphics generates a sequence of continuous vectors that can be well mapped to manipulators' continuous motion.
  • Our algorithm first generates quadratic Bézier spline curves from a set of input points, each curve using three control points in R2 with associated pen pressures in the following steps:
    1. Calculate the mid-points of all successive input points.
    2. Choose an input point as well as its two adjacent mid-points, to constitute the three control points to define a single Bézier curve.
    3. To render a curve with varying thickness, classify each curve into six cases of bounding polygon of the offset curve and triangulate them.
    4. We use resolution independent curve rendering to render the offset curves from the triangulation.

Surface Estimation by Sampling

Before / After a new point is inserted
  • In order to estimate geometry of the target surface, our manipulator incrementally samples the surface before and during the drawing and builds an adaptive and implicit representation of the surface.
  • Every sampled points are stored in the quadtree, containing its x-y coordinate and the height value.
  • To project the control points for drawing strokes, the height value needs to be determined.
  • We first search the four nearest points of the drawing point from the quadtree, and bilinear interpolation is performed to yield the estimated height.
  • Note that our quadtree only grows but never shrinks, and thus does not require sophisticated tree re-fitting mechanism.

Impedance-controlled Pen Drawing

  • The deviation δx between the target drawing position, determined by the bilinear interpolation aobve, and the physical position of the pen tip results in a spring force in Cartesian space, where k is the spring stiffness.
  • The controller is configured in such a way that the robot is compliant only in the normal direction of the surface.
  • We still maintain the tangential motion of the target position using a position-based control.
  • In order to avoid self-collision and kinematic limits of the robot, We precompute the robot's free configuration space beforehand and use it as a starting configuration for evry stroke.


Comparisons with Original Digital Drawings


Reproduced on Arbitrary Surfaces


Drawing Statistics

Bear Owl Tiger Girl1 Girl2
# of Strokes
1,520 1,942 523 1,585 966
# of Control Points
66,910 159,895 72,845 59,205 35,600
Drawing Surface Size (mm)
252 x 491 252 x 491 126 x 262 126 x 262 94 x 184
Execution Time (min.)
221 317 216 125 104



Ewha Graphics Lab
Department of Computer Science & Engineering, Ewha Womans University
  52, Ewhayeodae-gil, Seodaemun-gu, Seoul, Korea, 03760

  Daeun Song,
  Taekhee Lee,
  Young J. Kim,