FoggyStereo: Stereo Matching with Fog Volume Representation

(Code Coming Soon)

Chengtang Yao
Lidong Yu
[Code]
[Paper]
[Supp]
[Poster]

Abstract

Stereo matching in foggy scenes is challenging as the scattering effect of fog blurs the image and makes the matching ambiguous. Prior methods deem the fog as noise and discard it before matching. Different from them, we propose to explore depth hints from fog and improve stereo matching via these hints. The exploration of depth hints is designed from the perspective of rendering. The rendering is conducted by reversing the atmospheric scattering process and removing the fog within a selected depth range. The quality of the rendered image reflects the correctness of the selected depth, as the closer it is to the real depth, the clearer the rendered image is. We introduce a fog volume representation to collect these depth hints from the fog. We construct the fog volume by stacking images rendered with depths computed from disparity candidates that are also used to build the cost volume. We fuse the fog volume with cost volume to rectify the ambiguous matching caused by fog. Experiments show that our fog volume representation significantly promotes the SOTA result on foggy scenes by 10% ~ 30% while maintaining a comparable performance in clear scenes.
Paper thumbnail.


Pipeline



The overview of our method. We extract the features from left and right images to build the cost volume via warping ⓦ. We predict the atmospheric light $L_{\infty}$ and attenuation coefficient $\beta$ from the left image to render a series of images with different depth $Z_{i}$. The rendered images are concatenated along channel dimension and fused with cost volume for disparity estimation.

Comparison with Prior Work

The comparison of algorithms on the SceneFlow dataset. We compare the results testing on clear data and foggy data. * represents our re-implementation results.


The comparison of algorithms on KITTI 2015 and 2012 datasets. * represents our re-implementation results.


The comparison of algorithms on the clear data of PixelAccurateDepth dataset. * represents our re-implementation results.


The comparison of algorithms on the foggy data of PixelAccurateDepth dataset. * represents our re-implementation results.


The visualization of depth map on PixelAccurateDeth dataset with real foggy scenes.



Website template from here.