A robot that draws like a human

Graduate students of the Georgia Institute of Technology have created the first system of robots to draw graffiti that mimics the smoothness of human movement. The system, accurately named GTGraffiti, uses motion capture technology to record human painting movements, and then composes and processes gestures to program a robot with a cable that sprays graffiti images.

The project was developed by a candidate of robotics. student Jerry Chen in collaboration with Juan-Diego Flores, a fellow student; Frank Delart, professor of robotics at the School of Interactive Computing; Seth Hutchinson, Professor and Department of Robotics KUKA; and Sang-Won Lee, an associate professor at the School of Industrial Design. Reviewed by the research team robot the system will be published in the Proceedings of the International Conference on Robotics and Automation in June 2022.

How it works

In order for a robot to be able to draw in a human style, both the robot and the art must be designed with the other in mind – at least for now. The GTGraffiti system consists of three stages: capture of works of art, equipment of robots, and planning and control.

First, the team uses motion capture technology to record the paintings of people-artists – a strategy that allows you to understand the types of movements needed to create works of art written by a spray gun. For this study, Chen and the team invited two artists to draw a graffiti-style alphabet with letters. Like everyone artist painted, they recorded the movements of the artist’s hand on the canvas, as well as the movements of the spray can itself. Capturing the trajectory of hands and cans of paint is crucial for the robot to be able to draw using similar layers, composition and movement as a human artist.

The team then processed the data to analyze each movement for speed, acceleration and size, and used this information for the next step – designing the robot. Given this data, as well as the mobility and accuracy required for illustration, they decided to use a cable-controlled robot. Cable-controlled robots, such as Skycams, used in sports stadiums to shoot from aerial cameras, are characterized by the fact that they can scale to large sizes. The robot works on a system of cables, motors and pulleys. The robot team is currently mounted on a steel frame 9 by 10 feet high, but Chen says it should be possible to mount it directly on a flat structure of virtually any size, such as the side of a building.

For the third stage, the artist’s composition is converted into electrical signals. Taken together, the figures form a library of digital characters that can be programmed in any size, angle, and combination to create words to draw a robot. The artist selects figures from the library and uses them to create a work of art. For this study, the team chose to paint the letters “ATL”.

Once the team selects the sequence and position of the characters, they use mathematical equations to create the trajectories that the robot will follow. These algorithmically created paths ensure that the robot draws with the correct speed, location, orientation and perspective. Finally, the paths are converted into engine commands that must be executed.

With all the calculations and competing movements, the robot’s engines could potentially run against each other, threatening to break the robot. To solve this, the robot’s central controller is programmed to recalculate engine commands 1,000 times per second so that the robot can operate safely and reliably. Once assembled, the robot can draw a work of art in the style of a graffiti artist.

Why art? Why graffiti?

Some of the most typical industries for the application of robotics include manufacturing, biomedicine, automobiles, agriculture and the military. But art, it turns out, can demonstrate robotics in a particularly powerful way.

“Art, especially painting and dance, is an example of some of the most complex and nuanced movements that people can make,” Chen said. “So if we want to create robots that can do very technical things that people do, then creating robots that can dance or draw is a great goal. Apply to a variety of other applications.”

On a personal level, Chen is motivated by his hope that humans will perceive robots as beneficial to humanity, rather than viewing them as thefts of work or individuals that cause feelings of fear, sadness, or doom, as is often shown in the film.

“Graffiti is an art that is essentially meant to be seen by the masses,” Chen said. “In this regard, I hope we can use graffiti to convey this idea – that robots that work with humans can make a positive contribution to society.”

Future directions

Currently, Chen and the team’s plans regarding the robot are centered around two main areas: conservation and the expansion of art. To this end, they are currently experimenting with reproducing pre-recorded shapes at various scales and testing the robot’s ability to draw large surfaces. These abilities will allow the robot to draw enlarged versions of original works in different geographical locations and for artists who are physically unable to spray on site. Theoretically, an artist could paint a work of art in one part of the world, and the GTGraffiti bot can perform that work elsewhere.

In the future, Chen hopes to use GTGraffiti to capture artists who paint graffiti in the wild. With the data obtained, the GTGraffiti movement will be able to reproduce a work of art if it is ever painted over or destroyed.

“A robot does not create art by itself, but works with a human artist to allow them to achieve more than they could without a robot,” Chen said.

Chen suggests that the robot system will eventually have capabilities that allow the robot artist to interact in real time. He hopes to develop technology that will allow the artist standing at the foot of the building spray paint for example, graffiti in a small space, while a robot with a rope copies a picture with giant strokes on the side of the building, for example.

“We hope that our research can help artists create works of art that, made by superhuman robots, will convey messages more strongly than any work they could physically paint on their own,” Chen said.