AI tool lets users relight portraits using descriptive text prompts

An innovative AI model has been developed to create dynamic lighting effects in portrait images and videos using only text input. This technology allows users to adjust colors easily with descriptive prompts, such as “warm, freshly cooked chicken” and “icy blue light,” eliminating the need for complicated editing tools.

Professor Seungryul Bak and his team at the UNIST Artificial Intelligence Graduate School have introduced Text2Relight, an AI-driven lighting-specific foundational model that can perform relighting of a single portrait image driven by a creative text prompt as shown in the image above.

This study, conducted in collaboration with Adobe, will be showcased at the 39th Annual AAAI Conference on Artificial Intelligence (AAAI 2025) in Philadelphia, taking place at the Pennsylvania Convention Center from February 25 to March 4, 2025. The research has also been accepted by the Association for the Advancement of Artificial Intelligence (AAAI), a conference in the field. It is available on the arXiv preprint server.

The new model excels in expressing diverse lighting characteristics, such as emotional ambiance, alongside color and brightness, all through natural language inputs. Notably, it adjusts the colors of both the subject and the background simultaneously, maintaining the integrity of the original image.

In contrast to existing text-based image editing models that lack specialization in lighting data and often result in image distortion or limited lighting control, Text2Relight provides a more refined solution.

To enable the AI to learn the correlation between creative texts and lighting, the research team developed a large-scale synthetic dataset. They utilized ChatGPT and text-based diffusion models to generate lighting data, while also implementing OLAT (One-Light-at-A-Time) techniques and Lighting Transfer methods to explore various lighting conditions.

In addition, the team further enhanced the model’s functionality by training auxiliary datasets focused on shadow removal and illumination positioning, thereby improving visual coherence and realism in lighting effects.

Professor Bak commented, “Text2Relight holds significant potential in content creation, including reducing editing time in photo and video production and enhancing immersion in virtual and augmented reality settings.”