Google’s latest concept, Lumiere, is a time-space diffusion model that transforms text and images into realistic AI-generated videos with on-demand editing capabilities.
Researchers at Google have released plans for a new time-and-space diffusion model called Lumiere that will turn text or an image into a realistic AI-generated video, with capabilities for on-demand editing.
Lumiere is designed to portray “realistic, diverse and coherent motion” through what it calls its “Space-Time U-Net architecture.” This instantly generates the whole duration of the video through a single pass of the model.
Lumiere was trained on a data set of 30 million videos and text captions and has the capability to generate 80 frames at 16 frames per second. However, there has been no mention of the source of the data Google used to train the model — a heated topic in the world of AI and copyright law.
