Differentiable Direct Volume Rendering

Sebastian. Weiss¹, Rüdiger Westermann¹

¹Computer Graphics & Visualization Group, Technische Universität München, Germany

A fully differentiable direct volume renderer is used for a) viewpoint optimization, b) transfer function optimization, andoptimization of voxel properties using c) an absorption-only model and d) an emission-absorption model with rgbαtransfer functions.a), c) and d) show intermediate results of the optimization process until convergence.

Abstract

We present a differentiable volume rendering solution that provides differentiability of all continuous parameters of the volume rendering process. This differentiable renderer is used to steer the parameters towards a setting with an optimal solution of a problem-specific objective function. We have tailored the approach to volume rendering by enforcing a constant memory footprint via analytic inversion of the blending functions. This makes it independent of the number of sampling steps through the volume and facilitates the consideration of small-scale changes. The approach forms the basis for automatic optimizations regarding external parameters of the rendering process and the volumetric density field itself. We demonstrate its use for automatic viewpoint selection using differentiable entropy as objective, and for optimizing a transfer function from rendered images of a given volume. Optimization of per-voxel densities is addressed in two different ways: First, we mimic inverse tomography and optimize a 3D density field from images using an absorption model. This simplification enables comparisons with algebraic reconstruction techniques and state-of-the-art differentiable path tracers. Second, we introduce a novel approach for tomographic reconstruction from images using an emission-absorption model with post-shading via an arbitrary transfer function.

Associated publications

Differentiable Direct Volume Rendering

S. Weiss, R. Westermann

IEEE VIS 2021 [PDF] [Preprint] [BIB] [Code]

This is the accepted version of the following article: "Differentiable Direct Volume Rendering (Weiss & Westermann, 2021)", which will be published in final form at onlinelibrary.wiley.com. This article may be used for non-commercial purposes in accordance with the

Wiley Self-Archiving Policy.