Robotic Calligraphy Research Articles

Future Ink: The Collision of AI and Chinese Calligraphy

This survey delves deeply into the application, limitations, and opportunities of artificial intelligence (AI) technology within the realm of Chinese calligraphy. It examines AI's role in assessing calligraphic aesthetics, the automatic generation of artworks, style recognition, and robotic calligraphy, while addressing the capabilities and constraints of current AI algorithms and tools in the creation of Chinese calligraphy. Furthermore, it investigates the potential for AI to foster innovation and global heritage in the art of calligraphy, showcasing the potential and prospects for integrating AI with traditional Chinese calligraphy. The aim is to highlight the significance of AI in the preservation and innovation of traditional Chinese calligraphic art, to encourage interdisciplinary dialogue and collaboration between art and technology, and to offer new perspectives and approaches for the future development of calligraphy.

Internal Model Control Structure Inspired Robotic Calligraphy System

Internal Model Control Structure Inspired Robotic Calligraphy System

Solving Robotic Trajectory Sequential Writing Problem via Learning Character's Structural and Sequential Information.

The writing sequence of numerals or letters often affects aesthetic aspects of the writing outcomes. As such, it remains a challenge for robotic calligraphy systems to perform, mimicking human writers' implicit intention. This article presents a new robot calligraphy system that is able to learn writing sequences with limited sequential information, producing writing results compatible to human writers with good diversity. In particular, the system innovatively applies a gated recurrent unit (GRU) network to generate robotic writing actions with the support of a prelabeled trajectory sequence vector. Also, a new evaluation method is proposed that considers the shape, trajectory sequence, and structural information of the writing outcome, thereby helping ensure the writing quality. A swarm optimization algorithm is exploited to create an optimal set of parameters of the proposed system. The proposed approach is evaluated using Arabic numerals, and the experimental results demonstrate the competitive writing performance of the system against state-of-the-art approaches regarding multiple criteria (including FID, MAE, PSNR, SSIM, and PerLoss), as well as diversity performance concerning variance and entropy. Importantly, the proposed GRU-based robotic motion planning system, supported with swarm optimization can learn from a small dataset, while producing calligraphy writing with diverse and aesthetically pleasing outcomes.

Analysis of robotic calligraphy copying techniques based on linear regression models

Abstract A new robotic calligraphy system is constructed to address the problem of robotic calligraphy systems that only consider robotic writing in the straight state of the brush pen, enabling robots to write Chinese characters in the tilted state. A linear regression algorithm-based stroke model suitable for robotic brush writing is developed. Then a genetic algorithm is used to obtain trajectory information from the stroke images, and rules for starting, walking, and closing strokes are added to generate virtual stroke images. Finally, the B spline algorithm was used to realize the robot path planning and coordinate transformation to complete the pose calculation and perform the actual writing. The results show that the width of the pairs of strokes deepens as the stroke depth increases and the stroke speed decreases, corresponding to six sets of experimental speeds from 20mm/s to 120mm/s at stroke depths of 3mm and 7mm, respectively. From this, it can be seen that the relative error of most points is smaller than 10%, and the maximum relative error reaches 79.08%. The higher error was due to its lower benchmark value after comparison, which led to a huge error. In particular, as the independent variable increases in the experiment, making the benchmark value increase, its relative error decreases significantly, indicating the accuracy of the fitted model derived. It can be seen that the linear regression model-based robotic calligraphy copying technique helps to improve the robot’s purely repetitive trajectory in the writing process, adding its writing style to the original one and making it closer to humans.

Generative adversarial networks based motion learning towards robotic calligraphy synthesis

AbstractRobot calligraphy visually reflects the motion capability of robotic manipulators. While traditional researches mainly focus on image generation and the writing of simple calligraphic strokes or characters, this article presents a generative adversarial network (GAN)‐based motion learning method for robotic calligraphy synthesis (Gan2CS) that can enhance the efficiency in writing complex calligraphy words and reproducing classic calligraphy works. The key technologies in the proposed approach include: (1) adopting the GAN to learn the motion parameters from the robot writing operation; (2) converting the learnt motion data into the style font and realising the transition from static calligraphy images to dynamic writing demonstration; (3) reproducing high‐precision calligraphy works by synthesising the writing motion data hierarchically. In this study, the motion trajectories of sample calligraphy images are firstly extracted and converted into the robot module. The robot performs the writing with motion planning, and the writing motion parameters of calligraphy strokes are learnt with GANs. Then the motion data of basic strokes is synthesised based on the hierarchical process of ‘stroke‐radical‐part‐character’. And the robot re‐writes the synthesised characters whose similarity with the original calligraphy characters is evaluated. Regular calligraphy characters have been tested in the experiments for method validation and the results validated that the robot can actualise the robotic calligraphy synthesis of writing motion data with GAN.

Parallel Calligraphy Robot: Framework and System Implementation

Robot calligraphy is a combination of Chinese calligraphy art and modern machine intelligence. Aiming at the problem of insufficient flexibility and accuracy of robot calligraphy, this paper proposes a parallel calligraphy robot framework based on parallel robotics. To evaluate, a prototype parallel calligraphy robot system is built, including an execution subsystem, a software-defined subsystem and an acquisiton subsystem. A four-step paradigm including description, prediction, imitation and prescrition processes is proposed to perform closed-loop learning for robot calligraphy. Experimental results show that the system can realize accurate expert calligraphy collection, realistic calligraphy simulation and fine calligraphy writing results.

Automatic stroke generation for style-oriented robotic Chinese calligraphy

Intelligent robots, as an important type of Cyber–Physical systems, have promising potential to take the central stage in the development of the next-generation of efficient smart systems. Robotic calligraphy is such an attempt, and the current research focuses on the control algorithms of the robotic arms, which usually suffers from significant human inputs and limited writing styles. This paper presents an autonomous robotic writing system for Chinese calligraphy empowered by the proposed automatic stroke matching and generation mechanisms. Thanks to these mechanisms, the robot is able to effectively learn to write any Chinese characters in a style that is sampled by a small amount of handwritten Chinese characters with a certain target writing style. This is achieved by firstly disassembling each given Chinese character into individual strokes using the proposed character disassemble method; then, the writing style of the dissembled strokes is learned by a stroke generation module, which is built upon a generative adversarial learning model. From this, the robot can apply the learned writing style to any Chinese character from a given database, by dissembling the character and then generating the stroke trajectories based on the learned writing style. The experiments confirm the effectiveness of the proposed system in learning writing a certain style of characters based on a small style dataset, as evidenced by the high similarity between the robotic writing results and the handwritten ones according to the Fréchet Inception Distance.

An LSTM Based Generative Adversarial Architecture for Robotic Calligraphy Learning System

Robotic calligraphy is a very challenging task for the robotic manipulators, which can sustain industrial manufacturing. The active mechanism of writing robots require a large sized training set including sequence information of the writing trajectory. However, manual labelling work on those training data may cause the time wasting for researchers. This paper proposes a machine calligraphy learning system using a Long Short-Term Memory (LSTM) network and a generative adversarial network (GAN), which enables the robots to learn and generate the sequences of Chinese character stroke (i.e., writing trajectory). In order to reduce the size of the training set, a generative adversarial architecture combining an LSTM network and a discrimination network is established for a robotic manipulator to learn the Chinese calligraphy regarding its strokes. In particular, this learning system converts Chinese character stroke image into the trajectory sequences in the absence of the stroke trajectory writing sequence information. Due to its powerful learning ability in handling motion sequences, the LSTM network is used to explore the trajectory point writing sequences. Each generation process of the generative adversarial architecture contains a number of loops of LSTM. In each loop, the robot continues to write by following a new trajectory point, which is generated by LSTM according to the previously written strokes. The written stroke in an image format is taken as input to the next loop of the LSTM network until the complete stroke is finally written. Then, the final output of the LSTM network is evaluated by the discriminative network. In addition, a policy gradient algorithm based on reinforcement learning is employed to aid the robot to find the best policy. The experimental results show that the proposed learning system can effectively produce a variety of high-quality Chinese stroke writing.

Integration of an actor-critic model and generative adversarial networks for a Chinese calligraphy robot

As a combination of robotic motion planning and Chinese calligraphy culture, robotic calligraphy plays a significant role in the inheritance and education of Chinese calligraphy culture. Most existing calligraphy robots focus on enabling the robots to learn writing through human participation, such as human–robot interactions and manually designed evaluation functions. However, because of the subjectivity of art aesthetics, these existing methods require a large amount of implementation work from human engineers. In addition, the written results cannot be accurately evaluated. To overcome these limitations, in this paper, we propose a robotic calligraphy model that combines a generative adversarial network (GAN) and deep reinforcement learning to enable a calligraphy robot to learn to write Chinese character strokes directly from images captured from Chinese calligraphic textbooks. In our proposed model, to automatically establish an aesthetic evaluation system for Chinese calligraphy, a GAN is first trained to understand and reconstruct stroke images. Then, the discriminator network is independently extracted from the trained GAN and embedded into a variant of the reinforcement learning method, the “actor-critic model”, as a reward function. Thus, a calligraphy robot adopts the improved actor-critic model to learn to write multiple character strokes. The experimental results demonstrate that the proposed model successfully allows a calligraphy robot to write Chinese character strokes based on input stroke images. The performance of our model, compared with the state-of-the-art deep reinforcement learning method, shows the efficacy of the combination approach. In addition, the key technology in this work shows promise as a solution for robotic autonomous assembly.

GANCCRobot: Generative adversarial nets based chinese calligraphy robot

Robotic calligraphy, as a typical application of robot movement planning, is of great significance for the inheritance and education of calligraphy culture. The existing implementations of such robots often suffer from its limited ability for font generation and evaluation, leading to poor writing style diversity and writing quality. This paper proposes a calligraphic robotic framework based on the generative adversarial nets (GAN) to address such limitation. The robot implemented using such framework is able to learn to write fundamental Chinese character strokes with rich diversities and good quality that is close to the human level, without the requirement of specifically designed evaluation functions thanks to the employment of the revised GAN. In particular, the type information of the stroke is introduced as condition information, and the latent codes are applied to maximize the style quality of the generated strokes. Experimental results demonstrate that the proposed model enables a calligraphic robot to successfully write fundamental Chinese strokes based on a given type and style, with overall good quality. Although the proposed model was evaluated in this report using calligraphy writing, the underpinning research is readily applicable to many other applications, such as robotic graffiti and character style conversion.

A data-driven robotic Chinese calligraphy system using convolutional auto-encoder and differential evolution

The Chinese stroke evaluation and generation systems required in an autonomous calligraphy robot play a crucial role in producing high-quality writing results with good diversity. These systems often suffer from inefficiency and non-optima despite of intensive research effort investment by the robotic community. This paper proposes a new learning system to allow a robot to automatically learn to write Chinese calligraphy effectively. In the proposed system, the writing quality evaluation subsystem assesses written strokes using a convolutional auto-encoder network (CAE), which enables the generation of aesthetic strokes with various writing styles. The trained CAE network effectively excludes poorly written strokes through stroke reconstruction, but guarantees the inheritance of information from well-written ones. With the support of the evaluation subsystem, the writing trajectory model generation subsystem is realized by multivariate normal distributions optimized by differential evolution (DE), a type of heuristic optimization search algorithm. The proposed approach was validated and evaluated using a dataset of nine stroke categories; high-quality written strokes have been resulted with good diversity which shows the robustness and efficacy of the proposed approach and its potential in autonomous action-state space exploration for other real-world applications.

Learning Method of Pen Track Generation for Calligraphy Robot

Learning Method of Pen Track Generation for Calligraphy Robot

A Robotic Writing Framework–Learning Human Aesthetic Preferences via Human–Machine Interactions

Intelligent robots are required to fully understand human intentions and operations in order to support or collaborate with humans to complete complicated tasks, which is typically implemented by employing human-machine interaction techniques. This paper proposes a new robotic learning framework to perform numeral writing tasks by investigating human-machine interactions with human preferences. In particular, the framework implements a trajectory generative module using a generative adversarial network (GAN)-based method and develops a human preference feedback system to enable the robot to learn human preferences. In addition, a convolutional neural network, acting as a discriminative network, classifies numeral images to support the development of the basic numeral writing ability, and another convolutional neural network, acting as a human preference network, learns a human user’s aesthetic preference by taking the feedback on two written numerical images during the training process. The experimental results show that the written numerals based on the preferences of ten users were different from those of the training data set and that the writing models with the preferences from different users generate numerals in different styles, as evidenced by the Frechet inception distance (FID) scores. The FID scores of the proposed framework with a preference network were noticeably greater than those of the framework without a preference network. This phenomenon indicates that the human-machine interactions effectively guided the robotic system to learn different writing styles. These results prove that the proposed approach is able to enable the calligraphy robot to successfully write numerals in accordance with the preferences of a human user.

書道ロボット開発に関する研究(一般講演,ライフサイエンスとソフトンピューティング)

書道ロボット開発に関する研究(一般講演,ライフサイエンスとソフトンピューティング)

書道ロボットに関する研究 : 平仮名文字の書き順生成と軌道計画(ロボティクス・メカトロニクスI)

書道ロボットに関する研究 : 平仮名文字の書き順生成と軌道計画(ロボティクス・メカトロニクスI)

1202 書道ロボットに関する研究 : 書道運筆時の速度パターンの生成・解析とその適用

1202 書道ロボットに関する研究 : 書道運筆時の速度パターンの生成・解析とその適用

Kinematics and Moving Properties of a Four-Wheel-Drive, Omnidirectional Mobile Robot.

An omnidirectional vehicle (ODV), which can move in any direction without changing its body direction, was investigated. The principal driving motion involves four wheels equipped with rollers on their circumference. These rollers rotate along the axle direction of the wheels. The wheels are installed on all four sides of the vehicle. Moving properties of this vehicle were investigated through computer simulations and experiments. The simulation tests confirmed that the ODV can move to arbitrary places in arbitrary directions. Spinning motions and turning motions are also possible in addition to the omnidirectional motion. A variety of movements were examined in the ODV experiments. To analyze these results, an omnidirectional robot (ODR) called "calligraphy robot" was constructed, which drew a cursive style Chinese character with a writing brush on a sheet of Japanese writing paper. Movements and control of the ODR were evaluated, and satisfactory movement and control performances were confirmed.