Intended Audience: This tutorial is specifically intended for researchers studying signal processing in interactive and cognitive environments. Data fusion plays an important role for developing this kind of systems. Typical audience of this tutorial comprehends experts studying methods that involve distributed data sources from small to large sensors. The audience should have a working knowledge of mathematics and probability theory. Some familiarity with data fusion and tracking is desirable but not required.

Description: The tutorial aims at providing an overview of new insights in extending Dynamic Bayesian Networks techniques for representing, modeling and automatically interpreting and managing complex interaction situations occurring in cognitive environments starting from observations provided by multidimensional signals collected through a distributed network of embedded systems. A uniform representation is discussed that can also be used to support decisions concerning interactions between operators and the status of the observed environment. Solutions, which are based on an extension of traditional Bayesian filters for object assessment, are the basis background of discussion from which techniques in this tutorial.

The common representation and processing framework is based on a statistical interpretation of data fusion principles, and is suitable to be applied within fusion architecture models like recent versions of JDL. As a whole, the described approach can be figured as a multi-level joint tracking of objects and situations, including adaptive modeling of dynamic interactions among observation and dynamic models of single object trackers. It is shown that in this way the gap between signals and sensors level and semantic level can be bridged according to a bottom-up, uniform approach Applications domain like of scene analysis and abnormal situation assessment are discussed in scenarios where multiple interacting objects are observed through from multi-camera video sequences.

Some applications are discussed as case studies (i.e. crowd management, port security monitoring, etc.) where activities are described by using Bayesian filters of more than one observed pattern, including patterns that correspond to active parts controlled by a the system using the proposed representation. It is shown that interactions among activities of single patterns can be described as larger and more hierarchically structured Dynamic Bayesian Networks exhibiting globally a better level of adaptation to changing context. Properties and appropriate taxonomies of collaborative trackers included in such networks, are shown to be related to adaptive and self-aware selection of observation and dynamic models inside each tracker obtained through a set of messages exchanging probabilistic information within the proposed DBNs. Mechanisms are discussed that allow such networks to maintain an updated probabilistic knowledge simultaneously at signal and semantic (up to interaction event) levels. Deviation from most probable predictions is described as an abnormal detection method naturally emerging and being available at different time and event resolution scales.

The tutorial will demonstrate on case studies how the described DBN based situation assessment methods can be successfully applied to objects described with higher and lower precision at the state level. To this end, attention will be used to the description and the identification of dimensionality reduction techniques most suitable to be used. The choice of appropriate machine learning methods to learn from experience structure and parameters of the DBNs will be discussed, too.

Multisensor Surveillance and Physical and Cyber Security are the application fields for which examples will be provided in the tutorial: in particular, it will be highlighted how the new described techniques can be useful within large smart systems aiming at situation assessment at different spatial scales, like buildings, open environments, complex and critical infrastrustures.

Prerequisites:

Presenter: Carlo Regazzoni and Lucio Marcenaro