COGNIRON (Cognitive Robot Companion ; 2004-07)

Description of the project

The overall objectives of this project are to study the perceptual, representational, reasoning and learning capabilities of embodied robots in human centered environments. The project develops methods and technologies for the construction of such cognitive robots, able to evolve and grow their capacities in close interaction with humans in an open-ended fashion.

Expected results are basic methods, algorithms and architectures and their integration and long-term experimentation and scientific evaluation on embodied robotic systems in different settings and situations.

In the focus of this research endeavor, is the development of a robot whose ultimate task is to serve humans as a companion in their daily life. The robot is not only considered as a ready-made device but as an artificial creature, which improves its capabilities in a continuous process of acquiring new knowledge and skills.

Besides the necessary functions for sensing, moving and acting, such a robot will exhibit the cognitive capacities enabling it to focus its attention, to understand the spatial and dynamic structure of its environment and to interact with it, to exhibit a social behavior and communicate with other agents and with humans at the appropriate level of abstraction according to context.

The design of the cognitive functions of this artificial creature and the study and development of the continuous learning, training and education process in the course of which it will mature to a true companion, are the central research themes of the project.

Research Area within project (Spatial Cognition and Multimodal Situational Awareness)

To address the understanding of how an embodied system can come to a conceptualization of sensory-motor data, generate plans and actions to navigate and manipulate in typical home settings. The concept formation is considered in a broad sense, i.e. the ability to interpret situations, i.e. states of the environment and relationships between components of the environment that are static or evolving over time. The robot can observe these states or be part of the evolving action itself.

The robot will use different sensing modalities, and can also act in order to improve its understanding of the situation or to disambiguate its interpretation.

The following questions will be studied within the framework
the representation of space
the representation of situations
the representation and recognition of objects
dynamics and actions

For more information, please refer the project webpage.

Spatial Cognition for Mobile Robots: A Hierarchical Probabilistic Concept-Oriented Representation of Space - Shrihari Vasudevan

Movies

A 5 min 48 sec summary (AUDIO VIDEO PRESENTATION) of my research can be found here

Bayesian Space Conceptualization and Place Classification by a Mobile Robot

• Office ( imagesOffice.avi & objectmapOffice.avi )
• Kitchen ( imagesKitchen.avi & objectmapKitchen.avi )

The file imagesX.avi shows the robot exploring the scenario and recognizing various objects.

The file objectmapX.avi shows the internal representation of the robot. Specifically this shows the objects that are mapped, the clusters that are formed, the concepts that these clusters are inferred as being instances of (with appropriate beliefs) and the current place hypothesis (with appropriate belief).

The movies depict a robot exploring a room, recognizing objects, creating a probabilistic object graph based map of the room, using the objects and inter-object relationships perceived to probabilistically form higher level concepts and finally using these concepts to classify/infer on the place. The representation thus obtained is a semantically enriched one; the robot can thus demonstrate a greater degree of spatial awareness. Stereo vision together with a SIFT based object recognition system are used in conjunction with odometry to perform these experiments. Note that the deviation in the robots path is expected as only odometry is used. This can be corrected using scan-matching techniques, for instance. Objects are incrementally clustered (as perceived) and each cluster is inferred to be an instance of a particular concept. Objects of a particular cluster are denoted by a single color and a number in parenthesis. The legend of the image displays the result of the conceptualization process with the concept inferred and the belief in that concept being shown. The end product is a hierarchical probabilistic concept oriented representation of the place.

Please see the project poster for more details. The abstract of my thesis should provide a quick overview of the entire work. You can also find more details in a thesis summary presentation that I have made online here. Finally, the papers below have the most detailed presentation of the ideas.

Publications

Please email me for any of the articles that are not yet online

Thesis

• Shrihari Vasudevan, Spatial Cognition for Mobile Robots: A Hierarchical Probabilistic Concept-Oriented Representation of Space, Thesis Number 17612, March 2008, Swiss Federal Institute of Technology Zurich.
  • The abstract of my thesis is available here.
  • An outline of the chapters is available here.
  • A summary presentation of my work is available here.
    

Book Chapters

• S. Vasudevan, S. Gachter, A. Harati and R. Siegwart (2007) A hierarchical Concept-oriented Representation for Spatial Cognition in Mobile Robots, In M. Lungarella, F. IIda, J. Bongard and R. Pfeifer (eds.), 50 Years of Artificial Intelligence, Springer Lecture Notes in Artificial Intelligence, Vol. 4850.

Journals

• S. Vasudevan, S. Gaechter, C. Kobe and R. Siegwart (2008) Cognitive Maps for Mobile Robots: Perspectives from a User Study, Submitted.
  
• S.Vasudevan and R. Siegwart (2008) Bayesian Space Conceptualization and Place Classification for Semantic Maps in Mobile Robotics. Volume 56, Issue6, From Sensors to  Human Spatial Concepts, Pages 522-537, 30 June 2008.
• S. Vasudevan, S. Gachter, V.T. Nguyen and R. Siegwart (2007) Cognitive Maps for Mobile Robots - An object based approach. Robotics and Autonomous Systems, Volume 55, Issue 5, From Sensors to Human Spatial Concepts, 31 May 2007, Pages 359-371.

Conferences & Workshops (peer reviewed)

• A. Ramisa, S. Vasudevan, R.L. de Mantaras and R. Siegwart (2008), A tale of two object recognition methods for mobile robots, Accepted for publication in the proceedings of the International Conference on Computer Vision Systems (ICVS), Santorini, Greece.
  
• S. Vasudevan and R. Siegwart (2007) A Bayesian approach to Conceptualization and Place Classification: Incorporating Spatial Relationships (distances) to Infer Concepts, in the proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'07) workshop "From Sensors to Human Spatial Concepts (FS2HSC), San Diego, USA

• S. Vasudevan, A.Harati and R. Siegwart (2007) A Bayesian approach to Conceptualization and Place Classification: using the number of occurrences of objects to infer concepts, in the proceedings of the 3rd European Conference on Mobile Robotics (ECMR) 2007, Freiburg, Germany.

• S. Vasudevan and R. Siegwart (2007) A Bayesian Conceptualization of Space for mobile robots. in the proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2007), San Diego, USA. (more detailed technical report version below)
  

• S. Vasudevan, S. Gachter and R. Siegwart (2007) Cognitive Spatial Representations for Mobile Robots – Perspectives from a user study. In the proceedings of the IEEE International Conference on Robotics and Automattion - Workshop: Semantic Information in Robotics (ICRA - SIR 2007), Rome, Italy. (more detailed technical report version below)

• S. Vasudevan, S. Gachter, M. Berger and R. Siegwart (2006) Cognitive Maps for Mobile Robots – An Object based Approach. In the proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems - Workshop: From Sensors to Human Spatial Concepts (IROS - FS2HSC 2006), Beijing, China.
  

• S. Vasudevan, V.T. Nguyen and R. Siegwart (2006) Towards a Cognitive Probabilistic Representation of Space for Mobile Robots. In the Proceedings of the IEEE International Coference on Information Acquisition (ICIA) 2006, Shandong, China.
  

• A. Tapus, S. Vasudevan and R. Siegwart (2005) Toward a Multilevel Cognitive Probabilistic Representation of Space. In Proceedings of the International Conference on Human Vision and Electronic Imaging X, part of the IS&T/SPIE Symposium on Electronic Imaging 2005, 16-20 January 2005, CA, USA.

Talks (partial list ! to be updated...)

• Cogniron Winter School on Human Robot Interaction (January 2008) : Session on Cognitive Maps
  • My part alone
  • For other parts that I covered (including contributions from colleagues)
    
• ICRA 2005 Workshop on Cognitive Robots and Systems
  
  • My Talk
    

Software

• Object Recognition Toolkit based on SIFT (Davide Lowe)
  
  • SIFT object recognition software including GUI for ease of use
    
  • Released under General Public License (GPL)
    
  • Reference Publication : Ramisa, Vasudevan et al @ ICVS 2008 (please see above)

Datasets

ASL object recognition database

• 9 objects (3 textured, 3 uniformly textured and 3 with no texture)
• 36 test images containing random subsets of these objects.
• Ground truth
• Notes about dataset: Challenging dataset + Blurred images + changing illumination
• Reference Publication : Ramisa, Vasudevan et al @ ICVS 2008 (please see above)