D-Flat: A Differentiable Framework for End-to-End Optimization of Flat Optic Vision Systems
Dean Hazineh, Soon Wei Daniel Lim, Zhujun Shi, Federico Capasso, Todd
Zickler, Qi Guo
Abstract
Optical metasurfaces are planar substrates with custom-designed, nanoscale
features that selectively modulate incident light with respect to direction,
wavelength, and polarization. When coupled with photodetectors and appropriate
post-capture processing, they provide a means to create computational
imagers and sensors that are exceptionally small and have distinctive capabilities.
We introduce D-Flat (D♭), a framework in Pytorch/TensorFlow that renders
physically-accurate images induced by metasurface optical systems. This
framework is fully differentiable with respect to metasurface shape and
post-capture computational parameters and allows simultaneous optimization
with respect to almost any measure of sensor performance. D♭ enables
simulation of millimeter to centimeter diameter metasurfaces on commodity
computers, and it is modular in the sense of accommodating a variety of
wave optics models for scattering at the metasurface and for propagation to
photosensors. We validate D♭ against symbolic calculations and previous experimental measurements, and we provide simulations that demonstrate its
ability to discover novel computational sensor designs for two applications:
single-shot depth sensing and single-shot spatial frequency filtering.
Talk Recording
Project Discussion
Note: Similar to other pages on this site, the following discussion is distinct but complementary to the main paper. The content is independent and not a duplicate of the paper.
Comments Section