The ongoing or envisioned research projects I’m working on all concern the exploration of appearance in Nature at different scales (microscopic, naked eye) and through different prisms (optics, perception, art).

Some aspects of natural appearance have already been touched upon in past projects (click on below images for videos), but there is much more to be done.

Structural colors & iridescence

Structural colors are obtained when materials are structured at scales comparable to visible wavelengths, yielding interference effects. The simplest of structural color effects is obtained with Thin-film Interference, on which we worked in the past. I have more recently empirically studied the appearance of iridescent flavobacteria in collaboration with researchers in design.

In general, natural materials occurring on insects, birds, and some bacteria or gems show very complex structures, usually called photonic crystals.They have the potential to produce extremely bright, saturated and iridescent colors. However, due to irregularities, iridescence as well as scattering properties are often affected and hard to model. A classic example is the Morpho Butterfly, which adds an additional layer of complexity due to structures at multiple scales.

Thin-film interference
Siggraph paper |
Morpho butterfly (photograph)

Layered & porous media

Many materials are layered, either due to manufacturing or growing processes. In a recent work, we have studied Layered Material Appearance, and identified materials with different layered structures that have similar appearances (equivalence classes). However, many of those materials will not occur in Nature, due to limited media availability and to irregularities.

Porosities may be seen as a particular kind of irregularity, and they have the potential to affect appearance in interesting ways (e.g., in retro-reflection). Another source of appearance variation is water, as it fills the pores to different extents and affects appearance in ways that depend on surface tension. A water puddle is thus a relevant subject of study.

Layered Material Appearance
Siggraph paper
Water puddle (photograph)

Light-material interactions

Appearance is not solely dependent on material properties: lighting can play a fundamental role in affecting how things look. We have studied Light-material Interactions in previous work, and identified a few canonical lighting modes and their influence on appearance. Color may also change depending on the type of lighting (metamerism) or the optical depth of light paths going through a medium (vathochromism). We have introduced a new tool for One-to-many spectral upsampling that helps explore such effects.

As a follow-up, it would be interesting to study the specific ways in which lighting is modulated in natural habitats, such as forests or shallow watters. These habitats raise a host of questions: How do different types of environments differ in terms of incoming lighting at different heights? How do these lighting environments vary throughout a day and across seasons?

Light-material Interactions
Journal of Vision paper
Vathochromic effects
Computer Graphics Forum paper

Specular image structure

Shiny materials create sharp reflection patterns that compress and stretch in distinct ways on surfaces, a topic we have touched upon in digital drawing tools such as Surface Flows and Flow-guided Warping to convey surface shape. In previous work, we have also studied the relationships between Specular Motion & Shape Perception.

To date, there is no unified account of how specular or glossy reflections convey diagnostic cues about shape and materials. Such an analysis should take into account not only geometric and radiometric properties of objects, but also motion and lighting in the scene since they strongly affect appearance as is well illustrated by reflections on water waves.

Specular Motion & Shape Perception
Journal of Vision paper
Reflections on water waves (photograph)

Dynamic material perception

The Human Vision litterature usually considers two different aspects of material perception: optical and mechanical. In previous work, we studied each of them on specific cases: Hazy Reflections for optical cues, and Viscosity for mechanical cues. However, the way different types of cues are integrated in perceptual processes is unclear.

In recent work, we have shown that Specular Reflections & Material Categories interact in a complex manner that is inconsistent with a simple bottom-up process. It would thus be interesting to consider more complex feedforward+feedback dynamic mechanisms to explain material perception. A source of inspiration might by the color blocking and sculpting technique, where the general gist of objects is first roughly painted, before details are added to refine appearance.

Specular Reflections & Material Categories
Nature Human Behavior paper

Motion & layout depiction

Our work on 2D animation has initially been concerned with 2D shape morphing, as demonstrated in another of our techniques, N-way 2D Morphing. However, with such tools, animation is restricted to fronto-parallel motion: (dis)occlusions, topological changes, and overal control over the dynamics of animation are lacking. In a more recent work, we have introduced a system based on Transent Embeddings, which grants non-linear editing over dynamics, trajectories and topological changes in 2D animations.

The next step is to design tools that convey complex depth, 3D volumes, weight and layout relationships, which would not only be useful to assist 2D animation artists, but would also help study which motion and layout cues are required to convey a plausible 3D-like appearance.

N-way 2D Morphing
CASA paper |
Transient Embeddings
CGF paper |