Image and Sound
| Head : H. Nicolas |  |
Images and sounds, essential elements for the perception and representation of the physical world, are continuous phenomena which were initially manipulated by analog processes based on the signal processing approach. The development of computers quickly motivated the digitisation of visual and sound signals and their representation by discrete data suitable for computation. The activities of the Image and Sound team are central to this rapidly evolving approach. They cover a wide range of topics in digital image and sound with scientific studies of problems of synthesis, treatment, analysis, coding and interaction, organised in four themes.
The Video analysis and indexing theme concerns mutimedia computing with three major directions in multimedia flow analysis. The first direction concerns the segmentation of animated images based on movement and luminance/colour with the object of assisting the interpretation and efficient coding of video. The second direction concerns macro-segmentation of video documents into sequences and scenes with the double objective of indexing and navigation. The third concerns cross-media segmentation, that is the joint analysis of video and audio flow in digital audio-visual content.
The Sound and music modelling theme concerns modelling, analysis and synthesis of sound and music. Most current models of sound are hybrid, separating the stochastic noise component from the deterministic sinusoidal component. Concerning sound, the work being done aims at a unified spectral hierarchy model allowing a representation of sound at different levels, the sound itself, the control of the sound and variations in the control. Concerning music, the team's activities concern principally models of various forms of interaction with interactive music scores allowing the specification of musical material at the moment of its composition. The results of this research have applications in musical creation within SCRIME (Creation and Research in Computing and Electroacoustic Music Studio) and in musical teaching methods.
The 3D Modelling, Visualisation and Interaction theme lies within the more general virtual reality (VR) area and is devoted to the various stages from the creation of virtual environments (VE) to their visualisation and manipulation. Cocretely, we work on automatic methods of creation of VEs from digitised plans and photographs, on various rendering methods (point based, non photo-realist) and on the 3D interaction between user and application. These activities started with standard workstation type hardware, and were then extended to systems with large screens (several tens of square metres) thanks to the installation of the Hémicyclia VR room. In the last two years we have started to use a new type of equipment: communicating mobile terminals (telephone, PDA) as part of the IPARLA project (LaBRI - INRIA Futurs).
The Image structure and analysis theme concerns models describing the geometrical and topological structure of discrete images, in two or more dimensions, and the applications of these models in image analysis. More precisely, an image analysis is based on a stage of segmentation which divides the image in objects or components and a stage of extraction of information about these objects: topological relationships and geometrical measures and characteristics. Our work in this field principally concerns definition of new models in discrete topology and geometry, development of frontier based and region based segmentation techniques, and implementation of software and libraries applying these models, algorithms and tools.
Exchanges between the group's different themes are frequent and fruitful, allowing us to attack real problems applying a wide range of competencies and skills. This cooperation of mathematical models and algorithmic techniques drawn from historically disjoint research areas constitutes one of the original features of the "Image and Sound" team. Also noteworthy are our transdisciplinary collaborations, notably in the areas of man machine interaction and medical imagery.
Research themes:
- Image structure and analysis
- Head: A. Braquelaire.
- Keywords: image segmentation and analysis - geometrical and topological image models - discrete object representation, measure and analysis - medical imagery - surface reconstruction.
- Image Modelling and Synthesis
- Head: P. Guitton.
- Keywords: Virtual reality - 3D modelling and visualisation - 3D interaction - mobile terminals.
- Video analysis and indexing
- Head: J. Benois Pineau
- Keywords: Movement and video analysis - video sequence and compressed multimedia flow spatio-temporal segmentation - cross-media indexing - super-resolution and mosaics.
- Sound and music modelling
- Head: M. De Sainte Catherine.
- Keywords: Sound models - sound analysis and synthesis - musical composition assistance.
One of the original features of the "Image and Sound" team's work is the very strong cooperation of mathematical models and algorithmic techniques drawn from historically disjoint research areas, such as geometrical modelling and image synthesis, image analysis and processing, sound modelling and synthesis, and more recently video sequence analysis and treatment.
The success of SCRIME (Creation and Research in Computing and Electroacoustic Music Studio) and Hémicyclia (Virtual Reality Competency Centre) show the vitality of this work, notably in national and international collaboration, the ability to attract external funding, and the creation of many technology transfer operations with industry.