AI has been filling the gaps for illustrators and photographers for years now – literally, it cleverly fills the gaps with visual content. But the latest tools aim to allow an AI to give a helping hand to artists from the early stages of the blank canvas of a room. Nvidia’s new Canvas tool allows the creator to sketch a landscape like spots of paint by number, then fill it with compelling photorealistic content (if not quite gallery ready).
Each distinct color represents a different type of element: mountains, water, grass, ruins, etc. When the colors are projected onto the canvas, the rough sketch is transmitted to a generating antagonistic network. GANs essentially transmit content between a creative AI that tries to create (in this case) a realistic image and a detector AI that evaluates the realism of that image. These work together to make what they think is a fairly realistic representation of what has been suggested.
This is pretty much a more user-friendly version of the GauGAN prototype (understood?) Shown at CVPR in 2019. This one is much smoother around the edges, produces better images and can run on any computer. Windows with a decent Nvidia graphics card. .
This method has been used to create very realistic faces, animals, and landscapes, although there is usually some sort of “saying” that a human can spot. But the Canvas app isn’t trying to make something indistinguishable from reality – as concept artist Jama Jurabaev explains in the video below, it’s more about being able to freely experience more detailed images. than a doodle.
For example, if you want to have a moldy ruin in a field with a river on one side, a quick pencil sketch can only tell you a lot about what the final piece might look like. What if you had it one way and then two hours of painting and coloring later, you realize that because the sun is setting on the left side of the painting, it makes the shadows look awkward in the foreground?
If, instead, you just wrote these features in Canvas, you could see that was the case right away and move on to the next idea. There are even ways to quickly change the time of day, palette, and other high-level settings so they can be quickly evaluated as options.
“I am no longer afraid of the blank canvas,” Jurabaev said. “I’m not afraid to make really big changes, because I know AI always helps me with the details… I can put all my effort into the creative side of things, and I’ll let Canvas take care of the rest.”
It sounds a lot like Google’s Chimera Painter, if you remember that particular nightmare fuel, in which an almost identical process was used to create fantastic beasts. Instead of snow, rocks, and bushes, it had a hind leg, fur, teeth, etc., which made it a bit more complicated to use and easy to use.
Still, it might be better than the alternative, as certainly an amateur like me could never draw even the odd tube-shaped animals that resulted from the base dripping painting.
Unlike Chimera Creator, however, this app runs locally and requires a solid Nvidia video card to do so. GPUs have long been the hardware of choice for machine learning applications, and something like a real-time GAN definitely needs a big one. You can download the app for free here.