Intended Audience: This tutorial is specifically intended for researchers studying signal processing in interactive and cognitive environments. Data fusion plays an important role for developing this kind of systems. Typical audience of this tutorial comprehends experts studying methods that involve distributed data sources from small to large sensors. The audience should have a working knowledge of mathematics and probability theory. Some familiarity with data fusion and tracking is desirable but not required.

Description: The tutorial aims at providing an overview of new insights in extending Dynamic Bayesian Networks techniques for representing, modeling and automatically interpreting and managing complex interaction situations occurring in cognitive environments starting from observations provided by multidimensional signals collected through a distributed network of embedded systems. A uniform representation is discussed that can also be used to support decisions concerning interactions between operators and the status of the observed environment. Solutions, which are based on an extension of traditional Bayesian filters for object assessment, are the basis background of discussion from which techniques in this tutorial.

The common representation and processing framework is based on a statistical interpretation of data fusion principles, and is suitable to be applied within fusion architecture models like recent versions of JDL. As a whole, the described approach can be figured as a multi-level joint tracking of objects and situations, including adaptive modeling of dynamic interactions among observation and dynamic models of single object trackers. It is shown that in this way the gap between signals and sensors level and semantic level can be bridged according to a bottom-up, uniform approach Applications domain like of scene analysis and abnormal situation assessment are discussed in scenarios where multiple interacting objects are observed through from multi-camera video sequences.

Some applications are discussed as case studies (i.e. crowd management, port security monitoring, etc.) where activities are described by using Bayesian filters of more than one observed pattern, including patterns that correspond to active parts controlled by a the system using the proposed representation. It is shown that interactions among activities of single patterns can be described as larger and more hierarchically structured Dynamic Bayesian Networks exhibiting globally a better level of adaptation to changing context. Properties and appropriate taxonomies of collaborative trackers included in such networks, are shown to be related to adaptive and self-aware selection of observation and dynamic models inside each tracker obtained through a set of messages exchanging probabilistic information within the proposed DBNs. Mechanisms are discussed that allow such networks to maintain an updated probabilistic knowledge simultaneously at signal and semantic (up to interaction event) levels. Deviation from most probable predictions is described as an abnormal detection method naturally emerging and being available at different time and event resolution scales.

The tutorial will demonstrate on case studies how the described DBN based situation assessment methods can be successfully applied to objects described with higher and lower precision at the state level. To this end, attention will be used to the description and the identification of dimensionality reduction techniques most suitable to be used. The choice of appropriate machine learning methods to learn from experience structure and parameters of the DBNs will be discussed, too.

Multisensor Surveillance and Physical and Cyber Security are the application fields for which examples will be provided in the tutorial: in particular, it will be highlighted how the new described techniques can be useful within large smart systems aiming at situation assessment at different spatial scales, like buildings, open environments, complex and critical infrastrustures.

Prerequisites:

Presenter: Carlo Regazzoni and Lucio Marcenaro

Carlo S. Regazzoni received the Laurea degree in Electronic Engineering and the Ph.D. in Telecommunications and Signal Processing from the University of Genoa (UniGE), in 1987 and 1992, respectively. Since 2005 he is Full Professor of Telecommunications Systems.
Dr. Regazzoni is involved in research on Signal and Video processing and Data Fusion in Cognitive Telecommunication Systems since 1988. His main current research interests are: Bio-inspired Signal and Video Processing and Recognition, Distributed Data Fusion, Signal Processing for Wireless Communications and Localization, Ambient Intelligence, Cognitive Radio, Multimodal Intelligent Interfaces, Pervasive adaptation in embodied cognitive systems. Since 1998 he is responsible of the Video and Signal Processing for Telecommunications (ISIP40, http://www.isip40.it) Research Group at the Department of Biophysical and Electronic Engineering (now Department of Electrical, Electronic, Telecommunications Engineering and Naval Architecture - DITEN), within the Engineering Faculty of UniGE. Intelligent Distributed Video Surveillance and Wireless Mobile Communications/ Interaction Systems are the main focus of applications studied in this group.
Dr. Regazzoni has been involved and coordinating several EU research and development projects and of several research contracts with Italian industries. He is responsible of joint research labs with Technoaware (A2lab /Ambient Awareness Lab), Telecom Italia, and Selex-ES and of the Cognitive Radio Lab (CorLab) at DITEN. Dr. Regazzoni since 2009 is coordinator of the Erasmus Mundus Joint Doctorate on Interactive and Cognitive Environments (ICE), one of the 13 courses international PhD Courses selected in 2009 by European agency EACEA providing joint PhD titles. ICE is composed by a consortium of five top European Universities.
Since 1997 he has served as certified quality system reviewer of Higher Education Courses in the context of the Campus and Campusone projects organized by Italian Conference of Deans (CRUI). He has evaluated more than 40 courses at the Master and Bachelor level. He has also served as quality system reviewer under the European EURACE project. Dr. Regazzoni is the academic responsible of the security lab section of the Region Liguria industrial- academic consortium on Intelligent Integrated Systems (SIIT) as well as member of the Scientifical Technical Committee of SIIT.
He was in the funding board of the International Conferences on Advanced Signal and Video Based Surveillance Systems, IEEE AVSS, now at its 6th edition. He serves as General Chair of Genova edition AVSS 2009. He served as Technical Program chairman for the IEEE SPS ICIP05 Conference on Image Processing. He is currently external reviewer of a EU-FP7 project (Sense) and of several research programmes in Italy. Dr. Regazzoni is Associate Editor of several international journals: IEEE Signal Processing Letters, IEEE Transactions on Mobile Computing, Eurasip Journal on Information Security, International Journal on Image and Graphics. He serves as associate editor for the IEEE Transactions on Mobile Computing, and the International Journal on Image and Graphics. He has been Guest editor of many special issues on Proceedings of the IEEE, IEEE Signal Processing Magazine, and other journals. He has been co-editor of 4 edited books (Kluwer) on intelligent video surveillance from 1999 to 2003. He has been awarded of best IEEE Vehicular Electronics VT paper award in 2002. Dr. Regazzoni is author or co-author of 70 papers on International Scientific Journals and of more than 250 papers presented at peer reviewed International Conferences. He is member of Image and Multidimensional Signal Processing (IMDSP) and Multimedia Signal Processing (MMSP) committees of the IEEE Signal Processing Society.

Lucio Marcenaro enjoys over 15 years experience in image and video sequence analysis, and authored about 100 technical papers related to signal and video processing for computer vision. An Electronic Engineering graduate from Genova University in 1999, he received his PhD in Computer Science and Electronic Engineering from University of Genova in 2003. From 2003 to 2010 he was CEO and development manager at TechnoAware srl. From March 2011, he became Assistant Professor in Telecommunications for the Faculty of Engineering at the Department of Electrical, Electronic, Telecommunications Engineering and Naval Architecture (DITEN) at the University of Genova where he teaches the courses of Pervasive Electronics and Computer Programming and Telematics Lab. He is the principal scientific and technical coordinator of the Ambient Awareness Lab (A2Lab), with TechnoAware srl.
His main current research interests are: video processing for event recognition, detection and localization of objects in complex scenes, distributed heterogeneous sensors ambient awareness systems, ambient intelligence and bio-inspired cognitive systems.
Lucio Marcenaro was involved in many video-surveillance projects including: “Sistemi intelligenti per l’elaborazione e la trasmissione di segnali multidimensionali per applicazioni di video-sorveglianza in tempo reale” (1999-2000) funded by the Italian Ministry of University and Scientific Research; UE-FP5 REOST (2002-2004): Railway Electro Optical System for Safe Transportation; UE-FP6 INMOVE (EU IST-2001-37422) (2002-2004): Intelligent Mobile Video Environments; VICOM-FIRB (2002-2006) VICom (Virtual Immersive Communications); Elsag PSA (2001-2004); Architetture distribuite ed eterogenee per sistemi di sorveglianza multi-sensoriali (Prot. N. 7280/297 MIUR) (2002-2005); Context Awarness and Autonomic Network (2007-2009), joint lab with Telecom Italia spa; Sintesis (Sistema integrato per la sicurezza ad intelligenza distribuita) (2008-2010): n.S.HI.E.L.D. (new embedded Systems arcHItecturE for multi-Layer Dependable solutions) (ARTEMIS-2009-1 G.A.269317) (2011-2014).
Lucio Marcenaro was Industrial Chair of the 6th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS2009), Genoa, Italy September 2-4 2009. He is in the technical committee of many surveillance related international conferences (IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), IEEE International Conference on Imaging for Crime Detection and Prevention (ICDP), IEEE International Conference on Image Processing (ICIP), IEEE International Workshop on Visual Surveillance (VS) 2009) and international journals (IET Image Processing, IET Computer Vision, IET Communications, Machine Vision and Applications, Circuit and Systems for Video Technology, Pattern Recognition Letters, Transactions on Sensor Networks, Signal Image and Video Processing).
Since 2015 Lucio Marcenaro is an Expert on video analytics for ERNCIP Video Analytics & Surveillance for Security of Critical Infrastructures Thematic Group of the European Union.