« Bacon Up® Bacon Grease | Home | Dragon Mantis »

September 15, 2021

Google's Spectacular New AI Photo Upscaling

111

From PetaPixel:

Google's New AI Photo Upscaling Tech is Jaw-Dropping

Photo enhancing in movies and TV shows is often ridiculed for being unbelievable, but research in real photo enhancing is actually creeping more and more into the realm of science fiction. Just take a look at Google's latest AI photo upscaling tech.

In a post titled "High Fidelity Image Generation Using Diffusion Models" published on the Google AI Blog (and spotted by DPR), Google researchers in the company's Brain Team share new breakthroughs they've made in image super-resolution.

222

In image super-resolution, a machine learning model is trained to turn a low-res photo into a detailed high-res photo, and potential applications of this range from restoring old family photos to improving medical imaging.

Google has been exploring a concept called "diffusion models," first proposed in 2015 but which has, up until recently, taken a backseat to a family of deep learning methods called "deep generative models." The company has found that its results with this new approach beat out existing technologies when humans are asked to judge.

333

The first approach is called SR3, or Super-Resolution via Repeated Refinement. Here's the technical explanation:

"SR3 is a super-resolution diffusion model that takes as input a low-resolution image, and builds a corresponding high resolution image from pure noise," Google writes. "The model is trained on an image corruption process in which noise is progressively added to a high-resolution image until only pure noise remains.

"It then learns to reverse this process, beginning from pure noise and progressively removing noise to reach a target distribution through the guidance of the input low-resolution image."

444

SR3 has been found to work well on upscaling portraits and natural images. When used to do 8x upscaling on faces, it has a "confusion rate" of nearly 50% while existing methods only go up to 34%, suggesting that the results are indeed photo-realistic.

Once Google saw how effective SR3 was in upscaling photos, the company went a step further with a second approach called CDM, a class-conditional diffusion model.

555

"CDM is a class-conditional diffusion model trained on ImageNet data to generate high-resolution natural images," Google writes. "Since ImageNet is a difficult, high-entropy dataset, we built CDM as a cascade of multiple diffusion models. This cascade approach involves chaining together multiple generative models over several spatial resolutions: one diffusion model that generates data at a low resolution, followed by a sequence of SR3 super-resolution diffusion models that gradually increase the resolution of the generated image to the highest resolution."

666

Once Google saw how effective SR3 was in upscaling photos, the company went a step further with a second approach called CDM, a class-conditional diffusion model.

777

As you can see, the results are impressive and the final photos, despite having some errors (such as gaps in the frames of glasses), would likely pass as actual original photographs for most viewers at first glance.

888

"With SR3 and CDM, we have pushed the performance of diffusion models to state-of-the-art on super-resolution and class-conditional ImageNet generation benchmarks," Google researchers write. "We are excited to further test the limits of diffusion models for a wide variety of generative modeling problems."

September 15, 2021 at 10:01 AM | Permalink


Comments

You guys familiar wih this individual?
https://pbs.twimg.com/media/D7VKYhJU0AEGJIN.jpg

Cheers from Kraków!

Posted by: Tomasso | Sep 16, 2021 11:25:58 AM

Eyes have it
https://techxplore.com/news/2021-09-computer-generated.html

Posted by: ag | Sep 15, 2021 9:01:18 PM

Post a comment