Each new generation of mobile devices further erases the line between phone photography and so-called serious photography. Already the most popular cameras in the world by far, phones continue to encroach on the domain of standalone cameras.
There’s no better evidence of this than this week’s overhaul of DxOMark’s Mobile phone camera scores. Probably the world’s most-widely cited image benchmark for phones, DxOMark Mobile gauges how well mobile devices capture quality images. But until now, its tests have primarily focused on traditional phone use cases — standard focal length shots in typical lighting conditions, with limited emphasis on subject motion. The new version adds evaluating Zoom, Bokeh (which includes Portrait and Depth Effect), shots captured in low light, and scenes with subject motion.
Zoom and Portrait Mode
While not exclusively the province of dual-camera designs, those models have highlighted the growing demand for zooming in on scenes and mimicking the way standalone cameras can make a subject pop from the background. DxOMark Mobile now includes two new sub-scores, Zoom and Bokeh, to evaluate a phone’s performance in those circumstances. The Zoom score relies on capturing a test target from a range of distances, using whatever type of zoom the camera offers. Cameras with a second, telephoto, lens have a natural advantage here, but large sensor models like the Nokia PureView 808 also deliver very good results simply by cropping their high-resolution images.
Measuring Bokeh is more complex, as phones use a combination of estimated depth, facial recognition, and object recognition to analyze a scene and decide what to blur and by how much. In addition, in the case of dual-camera designs, the phone may combine the two images, which raises additional opportunities for artifacts and other image defects. DxOMark has created a series of new test scenes, like the one headlining this article, and several using a combination of Depth Rulers and subjects, to test how phones blur the foreground and background.
Apple’s iPhone 7 Plus Finally Gets to Show Off
Since the iPhone 7 and iPhone 7 Plus have the same main camera — used for traditional types of phone photography — they perform similarly in classic image quality tests. But with the new DxOMark Mobile’s inclusion of Zoom and Bokeh tests, the dual-camera system of the iPhone 7 Plus makes a clear difference, giving it an Overall score of 88 compared with the iPhone 7’s Overall score of 85 (it scored an 86 with the old protocol).
Here you can see that the iPhone 7 Plus uses its Portrait mode to create an image where the subject is isolated from the background.
Strengths and Weaknesses of Today’s Top Phones
There are a lot of “tricks” that phone makers use to create good-looking images in controlled conditions. For example, many models use long shutter times to reduce noise in low light. Shooting outdoors, or even indoors with a tripod mount, doesn’t show that. The new version of DxOMark Mobile adds low-light tests where the subject is moving, penalizing phones that greatly increase shutter times.
Chart comparing old versus new DxOMark Mobile scores for some of the best phones from the last few years
Computational Imaging Is Only Getting Started
While features like automatic multi-image assembly and dual camera designs have premiered in flagship phones, it’s only a matter of time until they are ubiquitous. Coming along behind them is likely to be another wave of even more advanced designs, like the one envisioned by startup Light.co at its launch two years ago. As a result, testing image quality on mobile devices will continue to require investment in new tests and test procedures given the rapid pace of innovation in mobile photography.
Disclaimer: I work extensively with DxOMark, and was involved in elements of the launch of the new DxOMark Mobile website.
[Images and chart data courtesy DxOMark]