The camera of Google's new Pixel 2 is known to take very stable videos thanks to its incredible Fused Video Stabilization technique. Although the company dedicated a substantial amount of research and development into the features, it was kind enough to share the secrets behind these groundbreaking techniques.
In a blog post, Chia-Kai Liang, Senior Staff Software Engineer and Fuhao Shi from the Android Camera Team discussed the methods being implemented by the Pixel 2 phones in its image stabilization processes. The technique, called Fused Video Stabilization, is a combination between two features, the optical image stabilization (OIS) and the electronic image stabilization (EIS).
The OIS is the most common type of image stabilization, and uses springs and electromagnets to suspend the lens in the middle its module. EIS, on the other hand, analyzes the phone's motion and removes the unwanted parts for the video. While other phones can only use one or the other, Google uses its machine learning algorithm to allow the Pixel phones to fuse the two features.
The phones first collect motion information from the handset's gyroscope. Then, Google employs a "lookahead" filtering algorithm to predict where the user is likely to move the phone. In effect, many common movements and quirks are counteracted by the software. Motion blur, camera shake, wobbling, rolling shutter, and focus hunting are also easily corrected by the combination of the features.
When needed, the Pixel phones also introduce a kind of virtual motion to cover up wild variations in sharpness when the user moves the phone too quickly.
The Pixel 2 flagship smartphones, which were released earlier this year, boast one of the best cameras in today's mobile phone market despite not having the dual camera sensors of other manufacturers. The phone was rated by DXOMark, a reference for image and camera quality, and received the highest mark ever for any smartphone. It topped the dual camera systems from competitors like the iPhone X, Huawei Mate 10, and Samsung Galaxy Note 8.
