To effectively deal with the management and retrieval of multimedia content, ITI-CERTH has developed web-based interactive multimedia search engines, which are capable of searching in video and image collections using different modalities. These multimodal search systems integrate technologies from semantic multimedia analysis and indexing, which are summarized below:
- Multimedia browsing with fast video preview.
- Extraction of low level visual features tailored for the special needs of a certain multimedia collection. For general case content the state of the art features (i.e. SIFT, MPEG-7) are employed. Depending on the collection, optimized features or new algorithms are developed (e.g. for patent images an algorithm suitable to deal with the complexity and the geometry of patent figures has been invented).
- Content-based retrieval of images or video frames based on low level visual characteristics (i.e. color, texture, etc) and on R-tree indexing.
- Text-based search exploiting existing metadata or annotations, while for video the text provided by Automatic Speech Recognition is considered. Depending on the application, either full text search or ontology-based retrieval is considered.
- Concept-based retrieval by extracting high level concept and event information from images and video shots using low level visual descriptors and supervised machine learning techniques.
- Generation of recommendations by combining the aforementioned modalities and considering past user interaction.
These functionalities integrated in a multimedia search system can drastically support the user in the task of fast and efficient retrieval of multimedia content as it was demonstrated in real user experiments and evaluation workshops.
· Identification of R&D activity
|Acronym:||Multimedia Search Engines|
|Author of the R&D result:||Dr. Ioannis Kompatsiaris, Stefanos Vrochidis|
|Company or University:||Centre for Research and Technology Hellas, Informatics & Telematics Institute Multimedia Knowledge Group|
|Phone:||Tel: +30 2311 257774 (I. Kompatsiaris), +30 2311 257754 (S. Vrochidis)|
|Type of the R&D result:||Software platform|