When we developed our new photo noise reduction algorithm in the upcoming ON1 NoNoise AI photo editing software, we took a holistic approach to understand noise and its causes. Thinking like a doctor, if we could prevent noise, that would be better than just treating it. A lot of the research bears this out. The earlier you can tackle noise, the better. A lot of the image noise we see in digital photographs is a side effect of the demosaicing process. If you are unfamiliar with that term, don’t worry, I’ll explain.

Digital camera sensors don’t see in color; they measure the intensity of light. In order to see color, there are tiny little colored filters over each pixel on a sensor. So in a way, each pixel can only measure the intensity of a given color. In most cameras, the colored filters are arranged in a pattern that samples one blue, one red, and two green pixels per block of four pixels. If you were to look at the photo taken at this stage, it would look like a microscopic mosaic made up of just these three colors. To us, it wouldn’t look anything like a normal photo. Yet, that’s essentially what a raw image looks like before it is processed.

The first step in converting raw data into a standard photo is the demosaicing process. This step is done by raw processor application if you shoot in raw, or by your camera if you shoot jpg. This step is where the magic happens. It breaks out the colors into four layers, one red, one blue, and two greens. Then it must compare what little information it has and guesses what the missing values are for each layer. That’s right, for each photo, two-thirds of the values are educated guesses made by your raw processor app. Most of the time, raw processors do an excellent job, and you get a photo that you would expect. However, sometimes things can go wrong.

The first and most obvious spot it can go wrong is noise. As your camera’s sensor struggles in low light, it must work really hard to measure the light intensity available. To do that, it must amplify the signal leading to noise being created in the raw photo. Then imagine how this noise confuses the demosaicing algorithm, which propagates even more noise.

This spot is also the first opportunity a raw processor has to reduce the noise. Training our AI neural network to detect and reduce noise before the photo goes through the demosaicing process ensures noise will not propagate further up in the image processing pipeline. It can better tell what noise is versus small details. This helps maintain sharper details and reduce false color. Other areas where demosaicing can struggle include fine lines, angled blocks, heavy patterns, and strong contrast edges. These can create artifacts such as jagged lines, zippering, false colors on edges, or maize and moire patterns. While these are generally rare, they can take a lot of work to reduce when they do appear.

Here are several side-by-side comparisons of demosaicing using a traditional method versus our new demosaicing and denoising working together in ON1 NoNoise AI.

NOTE: Click on any image to view the full side-by-side comparison.

Traditional Demosaic Algorithm (Adobe Camera Raw). 400% magnification. Note the softness of the lines and type, also the false color on diagonal edges of the black squares.
Preliminary Demosaic and Denoising Algorithm for NoNoise AI. Note the increased sharpness and reduced false color.
Traditional Demosaic Algorithm (Adobe Camera Raw). 400% magnification. Overall the text is softer and you can’t read the two lines.
Preliminary Demosaic and Denoising Algorithm for NoNoise AI. Overall the text is sharper and you can read one line lower.
Traditional Demosaic Algorithm (Adobe Camera Raw). 600% magnification. The diagonal lines begin to fall apart and exhibit false colors.
Preliminary Demosaic and Denoising Algorithm for NoNoise AI. The diagonal lines still resolve and show no false colors.

Now that I’ve explained how it works and shared some examples on resolution test targets, you might be asking yourself if this really matters in the real world? That’s a fair question. So often, photographers get too wrapped up in the technical side of photography, debate over minor things, and forget about capturing great photographs. So here are a few real-world examples where this does make an impact.

NOTE: Click on any image to view the full side-by-side comparison.

Traditional Demosaic Algorithm (Adobe Camera Raw). 400% magnification. The diagonal lines exhibit softness and a green color fringe.
Preliminary Demosaic and Denoising Algorithm for NoNoise AI. The diagonal lines are sharper and neutral in color.
Traditional Demosaic Algorithm (Adobe Camera Raw). 400% magnification. Note the magenta false colored spots on the specular highlights as well as the soft and green tree branch.
Preliminary Demosaic and Denoising Algorithm for NoNoise AI. There are no false colors on in the specular highlights and the tree branch is sharp and neutral in color.
Traditional Demosaic Algorithm (Adobe Camera Raw). 400% magnification. There is a general softness as well as a green fringe on the highlight at the top of the filter ring.
Preliminary Demosaic and Denoising Algorithm for NoNoise AI. Overall the image is sharper and the green color fringe is not present.

If you don’t shoot raw, that doesn’t mean you are doomed to have noisy photos. ON1 NoNoise AI works great on RAW images, but it also can remove noise just about as well from JPG, TIF or any other supported RGB photos. The robust AI network behind NoNoise AI has been trained to work with both RAW and RGB images.