Overview

TLDR: I developed a method for direct volume rendering of highly multivariate volumetric datasets, so datasets where each voxel does not store just a single value (scalar field) but many values (multi-attribute or multivariate field). The main challenge here is not changing the math or making it run in real time on the computer (though there are also some challenges there as these datasets can get very large, think hundreds of dimensions per voxel, and more than a terrabyte in size but this was not my main focus) but rather how to design transfer functions (how the user interacts with the data) in such a way that it can still explore the data and find intresting connections between multiple variables, and at the same time not user unfriendly.

ABSTRACT: Standard direct volume rendering (DVR) techniques are designed for scalar data and rely on transfer functions (TFs) that map scalar values to color and opacity. When a voxel contains many variables (multivariate volumetric data), defining TFs that expose relevant structure becomes difficult, especially when datasets have dozens or hundreds of dimensions, such as CycIF and other high-dimensional bio-medical volumes.

Common multi-dimensional workarounds — assigning hues to individual dimensions or using parallel-coordinate/star plots — either provide limited expressivity or do not scale cleanly. A promising alternative is to apply neighborhood-preserving dimensionality reduction (DR) such as t-SNE or UMAP to reduce the high-dimensional data to a low-dimensional embedding and design TFs in that embedded space. This lets users leverage familiar 2D TF interfaces while preserving local similarity relationships that are meaningful for exploration and analysis.

However, t-SNE and UMAP are nonlinear: interpolating in the embedded space does not trivially correspond to interpolated high-dimensional values, which is a problem for DVR sampling and reconstruction. In my thesis I designed and evaluated multiple interpolation strategies to address this: the most accurate approach uses approximate nearest neighbor (ANN) lookups to approximate where an interpolated high-dimensional sample would fall in the embedding. I also propose an adapted two-step transfer function that enables gradient-like opacity and surface shading effects for multivariate data — effects that scalar TFs provide but that are not directly available for high-dimensional data.

The methods were implemented and evaluated on CycIF datasets as part of the Bio+MedVis 2025 challenge. The work demonstrates a very high-fidelity rendering method (used for final-quality outputs) and a hybrid interaction workflow pairing that method with faster approximations to maintain interactivity.

Implementation & Skills

The project is a full research-to-prototype effort. I implemented a plugin for the ManiVault framework and a stand-alone application featuring an interactive Qt-based UI. The rendering pipeline is implemented in OpenGL and relies on extensive GLSL shader programming; though some rendermodes use compute shaders. This project combined rigorous research (literature review, algorithm design, evaluation) with engineering: C++ development, UI design (Qt), OpenGL/GLSL.

The end result is a usable tool for exploring multivariate volumes that balances visual fidelity with interactive performance, and which served as the basis for the publication and demo linked below.

Video

Demonstration video:

Paper & Resources

For details see the publication: Ravi Snellenberg and Thomas Höllt, t-SNE based Transfer Functions for Multi-attribute Volume Rendering, Challenge entry for the Bio+MedVis Challenge at IEEE VIS, 2025.

For code see the github: Link to the github repository: https://github.com/Rsnelllenberg/VolumeProjectorPlugin

Conference Photos

Photos from the IEEE VIS 2025 presentation.

Technologies Used