International Journal of Semantic Computing

Today's metadata models and metadata standards often focus on a specific media type only, lack combinability with other metadata models, or are limited with respect to the features they support. Thus they are not sufficient to describe the semantics of rich, structured multimedia documents. To overcome these limitations, we have developed a comprehensive model for representing multimedia metadata, the Multimedia Metadata Ontology (M3O). The M3O has been developed by an extensive analysis of related work and abstracts from the features of existing metadata models and metadata standards. It is based on the foundational ontology DOLCE+DnS Ultralight and makes use of ontology design patterns. The M3O serves as generic modeling framework for integrating the existing metadata models and metadata standards rather than replacing them. As such, the M3O can be used internally as semantic data model within complex multimedia applications such as authoring tools or multimedia management systems. To make use of the M3O in concrete multimedia applications, a generic application programming interface (API) has been implemented based on a sophisticated persistence layer that provides explicit support for ontology design patterns. To demonstrate applicability of the M3O API, we have integrated and applied it with our SemanticMM4U framework for the multi-channel generation of semantically annotated multimedia documents.

Current multimedia applications in Web 2.0 have generated large repositories for multimedia resources and annotations, so there is an urgent requirement to interlink annotations of these resources across different repositories to achieve better indexing and searching. To solve this problem, many researchers have been trying to apply semantic Web technologies to media fragments and annotations. Linked data has brought forward a promising way to expose, index and search media fragments and annotations which used to be isolated in different applications. This paper discusses in depth three key research problems when applying linked data principles in multimedia annotations: choosing URIs for media fragments, dereferencing media fragments and ontology alignment. An architecture is designed based on the possible solutions of the research problems. The key idea of the architecture is that it should act like an extra layer built on top of existing applications when publishing linked data. A demo is built as an implementation of the architecture to show that media fragments can be published and linked to various datasets in the linked data cloud. In the future, some algorithms should be designed to make full use of the interlinked media fragments and annotations for indexing and searching.

Today's search interfaces typically offer keyword searches and facets for the retrieval of cultural heritage multimedia objects. Facets, however, are usually based on a static set of metadata fields. This set is often called an indexing profile. Graph-based repositories based on predicates about resources allow for more precise semantics. They offer stronger support for retrieval, and they can be adopted to almost any metadata format. Technically, those predicates may be serialized as RDF triples, but handling a huge amount of objects with numerous predicates puts an unpredictable load on the query engine. In this paper, we present an approach on analysing transition paths in the RDF triples at ingest time and using the results to create facets in the search index.

The aim of this paper is to provide a comparison of various algorithms and parameters to build reduced semantic spaces. The effect of dimension reduction, the stability of the representation and the effect of word order are examined in the context of the five algorithms bearing on semantic vectors: Random projection (RP), singular value decomposition (SVD), non-negative matrix factorization (NMF), permutations and holographic reduced representations (HRR). The quality of semantic representation was tested by means of synonym finding task using the TOEFL test on the TASA corpus. Dimension reduction was found to improve the quality of semantic representation but it is hard to find the optimal parameter settings. Even though dimension reduction by RP was found to be more generally applicable than SVD, the semantic vectors produced by RP are somewhat unstable. The effect of encoding word order into the semantic vector representation via HRR did not lead to any increase in scores over vectors constructed from word co-occurrence in context information. In this regard, very small context windows resulted in better semantic vectors for the TOEFL test.

A better understanding of events many times requires the association and the efficient representation of multi-modal information. A good approach to this important issue is the development of a common platform for converting different modalities (such as images, text, etc.) into the same medium and associating them for efficient processing and understanding. In a previous paper we have presented a Local-Global graph model for the conversion of images into graphs with attributes and then into natural language (NL) text sentences [25]. Here, in this paper we propose the conversion of NL text sentences into graphs and then into Stochastic Petri-nets (SPN) descriptions in order to efficiently offer a model of associating "activities or changes" in multimodal information for events representation and understanding. The selection of the SPN graph model is due to its capability for efficiently representing structural and functional knowledge. Simple illustrative examples are provided for proving the concept proposed here.