Present…
The ongoing or envisioned research projects I’m working on all concern the exploration of appearance in Nature at different scales (microscopic, naked eye) and through different prisms (optics, perception, art).
Some aspects of natural appearance have already been touched upon in recent projects (click on ▶ below images for videos), but there is much more to be done.
Structural colors & iridescence
Structural colors are obtained when materials are structured at scales comparable to visible wavelengths, yielding interference effects. The simplest of structural color effects is obtained with Thin-film Interference, on which we worked in the past. I have also empirically studied the appearance of iridescent flavobacteria in collaboration with researchers in design.
In general, natural materials occurring on insects, birds, and some bacteria or gems show very complex structures, called photonic crystals.They have the potential to produce extremely bright, saturated and iridescent colors, as we have shown in a paper on Bragg Mirrors. Other types of photonic crystals and the impact of irregularities on iridescence and scattering remain to be studied.
Layered & porous media
Many materials are layered, either due to manufacturing or growing processes. In a recent work, we have studied Layered Material Appearance, and identified materials with different layered structures that have similar appearances (equivalence classes). However, many of those materials will not occur in Nature, due to limited media availability and to irregularities.
Porosities may be seen as a particular kind of irregularity, and they have the potential to affect appearance in interesting ways (e.g., grazing angles, in retro-reflection), as demonstrated in our micrograin framework. Another source of appearance variation is water, as it fills the pores to different extents and affects appearance in ways that depend on surface tension.
Light-material interactions
Appearance is not solely dependent on material properties: lighting can play a fundamental role in affecting how things look. We have studied Light-material Interactions in previous work, and identified a few canonical lighting modes and their influence on appearance. Color may also change depending on the type of lighting (metamerism) or the optical depth of light paths going through a medium (vathochromism). We have introduced a new tool for One-to-many spectral upsampling that helps explore such effects. Fluorescent materials also react in markedly different ways to different types of lighting, and we have shown that a spectral representation is not necessary to capture these effects.
As a follow-up, it would be interesting to study the specific ways in which lighting is modulated in natural habitats, such as forests or shallow watters. These habitats raise a host of questions: How do different types of environments differ in terms of incoming lighting at different heights? How do these lighting environments vary throughout a day and across seasons?
Specular image structure
Shiny materials create sharp reflection patterns that compress and stretch in distinct ways on surfaces, a topic we have touched upon in digital drawing tools such as Surface Flows and Flow-guided Warping to convey surface shape. In previous work, we have also studied the relationships between Specular Motion & Shape Perception.
To date, there is no unified account of how specular or glossy reflections convey diagnostic cues about shape and materials. Such an analysis should take into account not only geometric and radiometric properties of objects, but also motion and lighting in the scene since they strongly affect appearance as is well illustrated by reflections on water waves.
Dynamic material perception
The Human Vision litterature usually considers two different aspects of material perception: optical and mechanical. In previous work, we studied each of them on specific cases: Hazy Reflections for optical cues, and Viscosity for mechanical cues. However, the way different types of cues are integrated in perceptual processes is unclear.
In recent work, we have shown that Specular Reflections & Material Categories interact in a complex manner that is inconsistent with a simple bottom-up process. It would thus be interesting to consider more complex feedforward+feedback dynamic mechanisms to explain material perception. A source of inspiration might by the color blocking and sculpting technique, where the general gist of objects is first roughly painted, before details are added to refine appearance.
Motion & layout depiction
Our work on 2D animation has initially been concerned with 2D shape morphing, as demonstrated in a previous work on N-way 2D Morphing. However, with such tools, animation is restricted to fronto-parallel motion: (dis)occlusions, topological changes, and overal control over the dynamics of animation are lacking. In a more recent work, we have introduced a system based on Transent Embeddings, which grants non-linear editing over dynamics, trajectories and topological changes in 2D animations. Conversely, we have taken inspiration from 2D animation to create 3D smearing effects to convey motion.
The next step for 2D animation is to design tools that convey complex depth, 3D volumes, weight and layout relationships, which would not only be useful to assist 2D animation artists, but would also help study which motion and layout cues are required to convey a plausible 3D-like appearance.