Optical metasurfaces are planar substrates with custom-designed, nanoscale features that selectively modulate incident light with respect to direction, wavelength, and polarization. When coupled with photodetectors and appropriate post-capture processing, they provide a means to create computational imagers and sensors that are exceptionally small and have distinctive capabilities. We introduce D-Flat (D♭), a framework in Pytorch/TensorFlow that renders physically-accurate images induced by metasurface optical systems. This framework is fully differentiable with respect to metasurface shape and post-capture computational parameters and allows simultaneous optimization with respect to almost any measure of sensor performance. D♭ enables simulation of millimeter to centimeter diameter metasurfaces on commodity computers, and it is modular in the sense of accommodating a variety of wave optics models for scattering at the metasurface and for propagation to photosensors. We validate D♭ against symbolic calculations and previous experimental measurements, and we provide simulations that demonstrate its ability to discover novel computational sensor designs for two applications: single-shot depth sensing and single-shot spatial frequency filtering.

Talk Recording

Project Discussion

Note: Similar to other pages on this site, the following discussion is distinct but complementary to the main paper. The content is independent and not a duplicate of the paper.


Content in Development.

Visual depiction of end-to-end optimization for computational imaging systems
Visual depiction of what rendering operations DFlat does to enable computational imaging optimization

Optical Model

Content in Development.

Gif showing the use of DFlat to download and use pre-trained optical models
Qualitative Depction of a metasurface alongside a focusing metalens
Visual summary of the meta-atom approximation and the optical model for metasurfaces

Field Propagation

Content in Development.

Gif showing the use of DFlat to propagate a field to an output plane using Fourier Optics
Optimizaiton Diagram for Latent optimization of a lens with respect to a point-spread function target
Gif showing the use of DFlat to optimize the shapes on a metasurface in order to produce a multi-focci lens
Gif showing how to code DFlat for point-spread optimization using a pre-trained optical model and field propagation

Convolutional Rendering

Content in Development.

Future of DFlat

Content in Development.

Comments Section