Grants

The main measurable objective of the project is to improve significantly the performance of existing single-modality approaches in terms of scalability and precision. Towards that goal, BOEMIE will deliver a new methodology for extraction and evolution, using a rich multimedia semantic model, and realized as an open architecture. The architecture will be coupled with the appropriate set of tools, implementing the advanced methods that will be developed in BOEMIE. Furthermore, BOEMIE aims to initiate a new research activity on the automation of knowledge acquisition from multimedia content, through ontology evolution. BOEMIE will pave the way towards automation of the process of knowledge acquisition from multimedia content, by introducing the notion of evolving multimedia ontologies, which will be used for the extraction of information from multimedia content in networked sources, both public and proprietary. BOEMIE advocates a synergistic approach that combines multimedia extraction and ontology evolution in a bootstrapping process involving, on the one hand, the continuous extraction of semantic information from multimedia content in order to populate and enrich the ontologies and, on the other hand, the deployment of these ontologies to enhance the robustness of the extraction system. The ambitious scope of the BOEMIE project and the proven specialized competence of the carefully composed project consortium ensure that the project will achieve the significant advancement of the state of the art needed to successfully merge the component technologies.

Funded under: FP6-IST

K-Space is a network of leading research teams from academia and industry conducting integrative research and dissemination activities in semantic inference for automatic and semi-automatic annotation and retrieval of multimedia content. K-Space exploits the complementary expertise of project partners, enables resource optimization and fosters innovative research in the field. The aim of K-Space research is to narrow the gap between low-level content descriptions that can be computed automatically by a machine and the richness and subjectivity of semantics in high-level human interpretations of audiovisual media: The Semantic Gap. Specifically, K-Space integrative research focus on three areas: 
- Content-based multimedia analysis: Tools and methodologies for low-level signal processing, object segmentation, audio/speech processing and text analysis, and audiovisual content structuring and description. 
- Knowledge extraction: Building of a multimedia ontology infrastructure, knowledge acquisition from multimedia content, knowledge-assisted multimedia analysis, context based multimedia mining and intelligent exploitation of user relevance feedback. 
- Semantic multimedia: knowledge representation for multimedia, distributed semantic management of multimedia data, semantics-based interaction with multimedia and multimodal media analysis. An objective of the Network is to implement an open and expandable framework for collaborative research based on a common reference system. Specific dissemination objectives of K-Space include: 
- To disseminate the technical developments of the network across the broad research community 
- To boost technology transfer to industry and contribute to related standardisation activities.

Funded under: FP6-IST

This project aims at the theoretical examination and the experimental justification of a model which collects, semantically interprets and uses audiovisual information collected from humans. The goals are (a) supporting non-verbal human-machine interaction and (b) the reconstruction of the motion and “behaviour” of the human subjects with in virtual environments.

Funded by the greek General Secretariat of Research and Technology PENED 2003

ASK-IT integrated project aims to establishAmbient Intelligence (Ami) in semantic wed enabled services, to support and promote the mobility of the MobilityImpaired people, enabling the provision of personalised, self-configurable, intuitive and context-related applications and services and facilitating knowledge and content organisation and processing. Mobility Impaired (Ml) people have a wide variety of functional limitations, from different types of physical impairments to activity limitations. ICT systems following the "design for all" and adequate content are required, so as to take advantage of both internet and mobile-based services. ASK-IT integrated project aims to establishAmbient Intelligence (Ami) in semantic wed enabled services, to support and promote the mobility of the MobilityImpaired people, enabling the provision of personalised, self-configurable, intuitive and context-related applications and services and facilitating knowledge and content organisation and processing. Within it, Mlpeople related infomobility content is collected, interfaced and managed in SP1 (Content for All), encompassing transport, tourism and leisure, personal support services, work, business and education, social relations and community building related content. To offer the content, a number of advanced tools are developed within SP2(Tools for All), such as enhanced accuracy localisation, accessible intermodal route guidance modules and interfaces to eCommerce / ePayment, domotics, health and emergency management, driver support, computeraccessibility, eWorking, eLearning systems and assistive devices. Content and tools are integrated within an Ambient Intelligent Framework (SP3), by a Multi Agent System of Intelligent Agents and a self-configurable UserInterface, that offer service personalisation according to user profile, habits, preferences and context of use. This framework is interoperable in terms of mobile devices, local and wide area networks used, entrusted and based on intuitive web-semantics; thus offering seamless and device independent service everywhere. The integrated ASK-IT service and system will be tested in 7 interconnected sites Europewide in SP4 (Accessible Europe), to prove that full travel accessibility for Ml users can be achieved in a reliable and viable.

Funded under: FP6-IST

MUSCLE aims at creating and supporting a pan-European Network of Excellence to foster close collaboration between research groups in multimedia data mining on the one hand and machine learning on the other in order to make breakthrough progress towards the following objectives.(i) Harnessing the full potential of machine learning and cross-modal interaction for the (semi-)automatic generation of metadata with high semantic content for multimedia documents.(ii) Applying machine learning for the creation of expressive, context-aware, self-learning, and human centred interfaces that will be able to effectively assist users in the exploration of complex and rich multimedia content.(iii) Improving interoperability and exchangeability of heterogeneous and distributed (meta)data try enabling data descriptions of high semantic content (e.g. ontologies, MPEG7 and XML schemata) an conference schemes that can reason about these at the appropriate levels.(iv) Through dissemination, training and industrial liaison, contribute to the distribution and uptake the technology by relevant end-users such as industry, education, and the service sector. Due to the convergence of several strands of scientific and technological progress we are witnessing the emergence of unprecedented opportunities for the creation of a knowledge driven society. Indeed, databases are accruing large amounts of complex multimedia documents, networks allow fast and almost ubiquitous access to an abundance of resources and processors have the computational power to perform sophisticated and demanding algorithms. However, progress is hampered by the sheer amount and diversity of the available data. As a consequence, access can only be efficient if based directly on content and semantics, the extraction and indexing of which is only feasible if achieved automatically. Given the above, we feel that there is both a need and an opportunity to systematically incorporate machine learning into an integrated approach to multimedia data mining. Indeed, enriching multimedia databases with additional layers of automatically generated semantic metadata as well as with artificial intelligence to reason about these (meta)data, is the only conceivable way that we will be able to mine for complex content, and it is at this level that MUSCLE will focus its main effort. Realising this vision will require breakthrough progress to alleviate a number of key bottlenecks along the path from data to understanding.

Funded under: FP6-IST

HUMAINE aims to lay the foundations for European development of systems that can register, model and/or influence human emotional and emotion-related states and processes - emotion-oriented systems. Such systems may be central to future interfaces, but their conceptual underpinnings are not sufficiently advanced to be sure of their real potential or the best way to develop them. One of the reasons is that relevant knowledge is dispersed across many disciplines. It identifies six thematic areas that cut across traditional groupings and offer a framework for an appropriate division of labour - theory of emotion; signal/sign interfaces; the structure of emotionally coloured interactions; emotion in cognition and action; emotion in communication and persuasion; and usability of emotion-oriented systems.

Funded by FP6-IST 

aceMedia built a system to extract and exploit meaning inherent to the content in order to automate annotation and to add functionality that makes it easier for all users to create, communicate, find, consume and re-use content.Long term market viability of multimedia services requires significant improvements to the tools, functionality, and systems to support target users. aceMedia seeks to overcome the barriers to market success which include user difficulties in finding desired content, limitations in the tools available to manage personal and purchased content, and high costs to commercial content owners for multimedia content processing and distribution, by creation of means to generate semantic-based, context and user aware content, able to adapt itself to users` preferences and environments. aceMedia will build a system to extract and exploit meaning inherent to the content in order to automate annotation and to add functionality that makes it easier for all users to create, communicate, find, consume and re-use content. aceMedia targets knowledge discovery and embedded self-adaptability to enable content to be self organising, self annotating, self associating; more readily searched (faster, more relevant results); and adaptable to user requirements (self reformatting). aceMedia introduces the novel concept of the Autonomous Content Entity (ACE), which has three layers: content, its associated metadata, and an intelligence layer consisting of distributed functions that enable the content to instantiate itself according to its context (e.g. network, user terminal, user preferences). The ACE may be created by a commercial content provider, to enable personalised self-announcement and automatic content collections, or may be created in a personal content system in order to make summaries of personal content, or automatically create personal albums of linked content. The ACE concept will be verified by two user focused application prototypes, enabled for both home network and mobile communication environments. This enables the aceMedia partners to evaluate the technical feasibility and user acceptance of the ACE concept, with a view to market exploitation after the end of the project.

Funded under: FP6-IST

The current World Wide Web (WWW) is, by its function, the syntactic web where structure of the content has been presented while the content itself is inaccessible to computers. The next generation of the Web (the Semantic Web) aims to alleviate such problem and provide specific solutions targeted the concrete problems. The Web resources will be much easier and more readily accessible by both human and computers with the added semantic information in a machine-understandable and machine-processable fashion. It will have much higher impact on eWork and eCommerce as the current version of the web already had. Still, there is a long way to go transfer the semantic web from an academic adventure into a technology provided by software industry. Supporting this transition process of Ontology technology from Academia to Industry is the main and major goal of Knowledge Web. This main goal naturally translates into three main objectives given the nature of such a transformation. (1) Industry requires immediate support in tacking up this complex and new technology. Languages and interfaces need to be standardized to reduce the effort and provide scalability to solutions. Methods and use cases need to be provided to convince and to provide guidelines for how to work with this technology. (2) Important support to industry is provided by developing high-class education in the area of semantic web, web services, and Ontologies. (3) Research on Ontologies and the semantic web have not yet reached its goals. New areas such as the combination of semantic web with web services realizing intelligent web services require serious new research efforts. Spoken in a nutshell, it is the mission of Knowledge Web to strengthen the European software industry in one of the most important areas of current computer technology: Semantic web enabled eWork and eCommerce. Naturally, this includes education and research efforts to ensure the durability of impact text.

Funded under: FP6-IST

The main scope of the proposed framework is the development of innovative analysis techniques, export of semantic information and management of multimedia content in general and multimedia documents in particular. The proposed system, called Visual Asset, will be constituted by a number of distinguishable subsystems that will undertake the different stages of processing, analysis, storage and access to content provided by multimedia documents (texts, images, video, audio and 3D representations) and are summarized in the following:

• Image and video processing subsystem aiming at automatic export of low level characteristics (color, texture, form, movement, speed, etc) and export of characteristic parts of objects (object segmentation) (e.g. segmentation of persons in a video sequence). These results will be used in the system integrating visual and textual information via the use of ontologies, but also in the effective search and retrieval system, based on visual information.
• Integration of visual and textual information subsystem. Object of this subsystem will be representation of knowledge with use of ontologies, analysis of multimedia content based on visual and textual and production of metadata with a common way of representation, regarding both textual and visual information.
• Modeling and logical analysis of documents subsystem, aiming at automatic categorization and creation of effective search and retrieval applications, providing advanced functionalities in organization and management of big document volumes. The subsystem will integrate visual and textual information results and will use the notion of context in a document in order to fulfill the tasks of automatic categorization, logical analysis (table of contents, automatic recognition of chapters, titles, notes, reports in images, video, etc) and efficient search and retrieval.

Funded under: SP6 (National)

The main objective of the ERMIS Project is the development of a prototype system for human computer interaction that can interpret its users' attitude or emotional state, e.g. activation/interest, boredom, and anger, in terms of their speech and/or their facial gestures and expressions. The adopted technologies include linguistic and paralinguistic speech analysis and robust speech recognition, facial expression analysis, interpretation of the user's emotional state using hybrid, neurofussy, techniques, while being in accordance with the MPEG-4 standard. Specific attention is given to the evaluation of the system's ability to improve effectiveness, user friendliness and user satisfaction, while examining and resolving related ethical issues. Real life applications, where users interact with machines, and in particular with call/ information centres and next generation PC interfaces, have been selected to test the performance of the ERMIS system The ERMIS project has conducted a systematic analysis of speech and facial input signals, in separate, as well as in common; the aim was to extract parameters and features which can provide human computer interaction (HCI) systems with the ability to recognize the basic emotional state of their users and interact with them in a more natural and user friendly way. Testbed applications have been selected for testing and evaluating the ERMIS system performance, referring to everyday interaction of users with their PCs and with service or information providing call centers, with successful developments prospecting a large market size.The ERMIS system is able to rely on prior knowledge related to the emotional analysis of speech and/or facial expressions, and to accommodate for the different expressive styles of humans. The continuity of emotion space, the uncertainty involved in the feature estimation process and the required ability of the system to use prior knowledge, while being also capable of adapting its behaviour to its users' characteristics, is handled by using intelligent hybrid, neurofuzzy, approaches.

Funded under: IST-2000