RESEARCH AND IMPROVEMENT OF THE STABLE DIFFUSION MODEL USING LORA FOR AUTOMATIC GENERATION OF LITERATURE ILLUSTRATIONS

Abstract

In the context of today's technological advancements, the rapid development of artificial intelligence and machine learning has brought remarkable breakthroughs in various fields, including art and literature. Image generation models like Stable Diffusion are becoming powerful tools, enabling the transformation of ideas into images quickly and creatively. The application of advanced methods such as Low-Rank Adaptation (LoRA) not only enhances model performance but also opens up rich customization possibilities, catering to the diverse needs of users. The project titled "Research and improvement of the stable diffusion model using LoRA for automatic generation of literature illustrations" aims to leverage and optimize modern technologies to improve the capability of generating illustrations for children's literature. The Stable Diffusion model, known for its ability to create high-quality images from text descriptions, will be deeply studied to understand its mechanisms andthefactors determining image quality. The application of LoRA in this context allows for adjusting the model's weights without retraining the entire model, thereby minimizing costs and time while enhancingflexibility in customizing image styles and content. This is particularly significant indeveloping children's literary works, where vivid and engaging illustrations can create enjoyable reading experiences for children. The vision of this project goes beyond generating vibrant illustrations, aiming to build a robust and accessible creative platform for authors, artists, and publishers. Through this, the research not only enriches the content of children's literature but also fosters creativity and cognitive development in children. The illustrations produced are not merely visual accompaniments but serve as bridges that transport children into the magical world of imagination and discovery.