A Sim-to-Real Pipeline for Stroke-Based Robotic Painting

Representing art using a robotic system is part of artificial intelligence in our life, especially in the realm of emotional expression. Developing a painting robot involves addressing how to enable the robot to emulate human artistic processes, which often include imprecise techniques or errors akin to those made by human artists. This paper discusses our development of an innovative painting robot utilizing the sim-to-real approach within learning technology. Specifically, this pipeline operates under a deep reinforcement learning (DRL) framework designed to learn drawing strategies from training data derived from real-world settings, aiming for the robot's proficiency in emulating human artistic expressions. Accordingly, the framework comprises two modules when given a target drawing image: the first module trains in a simulated environment to break down the target image into individual strokes; the second module then learns how to execute these strokes in a real environment. Our experiments have shown that this system can meet our objectives effectively.