RAW or JPEG on a phone: what the software throws away

Point your phone at the Milky Way, press the button, and for a second or two nothing happens. Then the photo appears. That photo is not what the sensor saw: it is a reconstruction, a burst of frames merged together, noise smoothed, shadows lifted, contrast put back in line, every call made by software raised on faces, dogs and sunsets. In daylight it does a superb job. At night it finds itself staring at a handful of dots two pixels wide on a black field, and it has to decide whether they are stars or sensor defects. Whatever it decides, it does not tell you.

Your phone does not take pictures, it builds them

The shutter button has not opened a shutter in years. It fires a sequence, which the processor aligns, averages, cleans, maps onto a tone curve and hands over as a JPEG or a HEIF: a developed file, ready for a screen. A JPEG keeps eight bits per channel, and eight bits are enough to make an image pleasant to look at, not enough to make it workable. The intermediate decisions have already been taken and baked into the pixels: how much noise to strip, where to lift the shadows, what colour temperature to assign to a sky that is nobody’s idea of neutral. Editing a JPEG afterwards means retouching somebody else’s conclusions.

The software in the middle is beautifully written. It just happens to be solving a different problem from yours.

The algorithm that hunts small bright dots

In 2019 Google published a detailed account of how the astrophotography mode in Night Sight works, and one passage repays attention even if you carry a completely different phone. The silicon in any sensor has flaws: some pixels build up current in the dark faster than their neighbours, and in a frame they show up as tiny bright specks. To get rid of them, the software compares each pixel’s value against the pixels around it and against the same pixel in the other frames of the sequence, looks for outliers, and when it finds one it replaces it with the average of its neighbours.

Now read that description again while thinking about a faint star. Through a phone’s wide angle a faint star is exactly that: one pixel far brighter than everything around it, alone in the black. There is a genuine difference between a star and a stuck pixel, because the star drifts across the frame as the Earth turns and as the tripod settles by a tenth of a millimetre, while the defect stays nailed to the same coordinates. That is precisely why Google compares along the sequence as well. Still, the difference is a fine one, and it depends on how much sky went past between frames and on where somebody set the threshold. Every manufacturer has its own pipeline, and nobody publishes theirs.

The same chain does more. Google identifies which part of the frame is sky so it can apply noise reduction tuned for sky and selectively raise contrast on clouds, colour gradients and the Milky Way. For the photo you are about to share tonight, that is a gift. For one brick in a six hundred frame stack it is trouble: each brick arrives carrying a different correction, computed on a slightly different framing, and the average of six hundred different corrections is not the average correction.

The same page marks out the boundaries of the game: a single Night Sight shot stops at fifteen frames of up to sixteen seconds each, roughly four minutes in total on a Pixel 4, and that sixteen second ceiling exists because beyond it stars stop being points and turn into short dashes. That is the entire reason stacks exist, and we told the long version of it in the piece on live stacking.

What the DNG gives back

RAW on a phone almost always means DNG, Adobe’s open format. On the iPhone it is called Apple ProRAW and you have to switch it on by hand in Settings, Camera, Formats. Apple describes it as a DNG that holds the raw data and part of the computational processing together: an openly declared halfway house compared with the naked Bayer file from a mirrorless body, but one that hands you back exposure, white balance and tone curve. Apple publishes the storage cost too: roughly 25 MB per shot at 12 megapixels, roughly 75 MB at 48.

One footnote is aimed straight at us. The 48 megapixel option only applies to the main camera at 1x, and photos taken in Night mode are saved at 12 regardless. At night the big file does not exist, which is fine: on a faint sky, smaller pixels bring more noise than detail.

On Android everything runs through the camera APIs. Camera2 provides DNG writing, and CameraX from version 1.5 supports DNG capture alongside the JPEG, so you go home with the finished file and the open one. In practice that means it depends on the handset and on the app. Plenty of mid-range phones never expose the raw stream at all, and no app can conjure it.

The arithmetic almost nobody does

Do two multiplications before deciding RAW is always the answer. Six hundred ProRAW frames at 25 MB come to fifteen gigabytes, for one evening, on a phone that holds 256 and is already half full of video. Add the write time between frames, which stretches the sequence out exactly while the sky is moving, and the fact that somebody eventually has to align and integrate those fifteen gigabytes. For a long session on a faint target the sensible route runs elsewhere: an app that works on the linear stream coming off the sensor, integrates as it goes and saves you the result instead of six hundred negatives. AstroStackerPro does that on iPhone, which is why it never asks you to fill up your storage.

The practical rule, in three lines

One frame, or a dozen, on a landscape with sky above it: RAW, always, no argument. A long night on a nebula: stack the stream, and keep RAW for the one framing worth developing by hand. Either way, lock white balance manually before you start, because an auto setting that slides three hundred kelvin between frame one and frame two hundred is one of the quietest ways to ruin a stack, alongside the classic beginner mistakes we all make the first time.

If you want to settle once and for all how much the processor is taking off you, there is a five minute test. Same field, same exposure, one JPEG and one DNG back to back, then open them side by side at home and zoom to a hundred percent on an empty patch of sky. The bright stars are in both. The ones down at the noise floor, the ones that separate a photograph from a sky, pick a side on their own.

What appears on your screen after the shutter is a well argued opinion. The DNG is the transcript.