The 4th workshop on Structuring and Understanding of Multimedia heritAge Contents (SUMAC)

co-located with
10-14 October 2022, Lisbon, Portugal

Submission Deadline Extended to 8 July 2022

Aims and scope of the workshop

The digitization of large quantities of analogue data and the massive production of born-digital documents for many years now provide us with large volumes of varied multimedia data (images, maps, text, video, multisensor data, etc.), an important feature of which is that they are cross-domain. "Cross-domain" reflects the fact that these data may have been acquired in very different conditions: different acquisition systems, times and points of view (e.g. a 1962 postcard from the Arc de Triomphe vs. a recent street-view acquisition by mobile mapping of the same monument). These data represent an extremely rich heritage that can be exploited in a wide variety of fields, from SSH to land use and territorial policies, including smart city, urban planning, tourism, creative media and entertainment. In terms of research in computer science, they address challenging problems related to the diversity and volume of the media across time, the variety of content descriptors (potentially including the time dimension), the veracity of the data, and the different user needs with respect to engaging with this rich material and the extraction of value out of the data. These challenges are reflected in research topics such as multimodal and mixed media search, automatic content analysis, multimedia linking and recommendation, and big data analysis and visualisation, where scientific bottlenecks may be exacerbated by the time dimension, which also provides topics of interest such as multimodal time series analysis.


Keynote 1

Prof. Dr.-Ing. habil. Andreas Maier

was born on 26th of November 1980 in Erlangen. He studied Computer Science, graduated in 2005, and received his PhD in 2009. From 2005 to 2009 he was working at the Pattern Recognition Lab at the Computer Science Department of the University of Erlangen-Nuremberg. His major research subject was medical signal processing in speech data. In this period, he developed the first online speech intelligibility assessment tool – PEAKS – that has been used to analyze over 4.000 patient and control subjects so far.

From 2009 to 2010, he started working on flat-panel C-arm CT as post-doctoral fellow at the Radiological Sciences Laboratory in the Department of Radiology at the Stanford University. From 2011 to 2012 he joined Siemens Healthcare as innovation project manager and was responsible for reconstruction topics in the Angiography and X-ray business unit. In 2012, he returned the University of Erlangen-Nuremberg as head of the Medical Reconstruction Group at the Pattern Recognition lab. In 2015 he became professor and head of the Pattern Recognition Lab. Since 2016, he is member of the steering committee of the European Time Machine Consortium. In 2018, he was awarded an ERC Synergy Grant “4D nanoscope”. Current research interests focuses on medical imaging, image and audio processing, digital humanities, and interpretable machine learning and the use of known operators.

Talk Title:

Building Blocks for a Virtual Time Machine


Time travel is an old dream of mankind that is fuelled by fascination and curiosity. Of course, such a journey through time goes far beyond our physical possibilities today. Historical science largely recapitulates the past using text, while the creative industry reproduces it as a more or less well-researched fiction. The Time Machine Initiative has now taken on the task of digitizing and processing the cultural heritage on a large scale in order to create new virtual accesses to the past, which - taking into account the fragmentary tradition - come close to a journey through time. In a large-scale interdisciplinary and trans-European research project, a kind of edition of European history is to be created which, as a data-saturated reconstruction, can create a new form of comprehensibility and experience. The time machine would therefore be a virtual research environment, the results of which would also be communicated directly and immersively to a broader public. In the presentation, we will briefly describe the project and present concrete research results that have been obtained in this direction ranging from book CT, over art image analysis up to writer and font identification.

Keynote 2

Prof. Georgios Artopoulos

works at Science and Technology in Archaeology and Culture Research Center, Cyprus Institute on immersive and virtual environments, urban modeling and digital simulation for the study of built heritage and the creative exploration of historical narratives. Georgios holds a Master of Philosophy and a PhD, University of Cambridge (UK) with a Doctoral Award from the Arts and Humanities Research Council.

Together with the team of Virtual Environments Lab, at the CyI, Georgios is developing ICT-enabled user-driven tools for social resilience and inclusion, with an application in historical context. The social aspects of historic space and the cross-disciplinary nature of the pressing challenges facing our cities are explored through the externally funded projects he is contributing to or coordinating (under H2020, ENI-CBC-MED, and Cyprus Research and Innovation Foundation frameworks), his role as a co-Head of Virtual Competency Centre e-Infrastructure of the DARIAH ERIC, and as a Member of the Scientific Advisory Board of JPI Urban Europe, where he works on matters of sustainable and liveable cities and urban areas. His work was presented in the International Exhibition Computational Turn in Architecture, MAV, Marseille; Seoul Biennale of Architecture and Urbanism 2017; Hong Kong and Shenzhen Bi-City Biennale of Architecture and Urbanism; 63rd Venice Film Festival, La Biennale di Venezia; Royal Institute of British Architects, London; London Design Festival; Festival of Architecture 2018, Israel.

Talk Title:

Creating a Time Machine of future pasts: data integration and interoperability for cross-disciplinary research on urban heritage clusters.


Historic clusters of heritage buildings comprise the core of a great number of European cities and represent the fabric based on which today’s municipalities have developed historically. The sustainable development of these environments is often threatened by urbanization, gentrification or depopulation phenomena. These urban environments should not be studied and analysed as static formations disconnected from the contemporary fabric of a city, but rather as an assemblage of tangible and intangible assets subjected to dynamic pressures of economic, environmental, and social activities. The cross-disciplinary nature of the pressing challenges posed by said phenomena requires the development of novel data-driven methods for the re-use, regeneration and safeguarding of neglected areas of our cities’ existing building stock. The presentation will discuss about the methodological and technical framework required for the creation of a platform that will function as a time machine of our cities in the future. A time machine that does not aim only at representing how our cities used to be in the past, but rather one that curates and stores current transformations of our built environment, with the objective to enable dynamic observation of the existing building stock at neighborhood scale in present and future times. In this context, the presentation will be occupied with the significance of bringing the building scale (architectural) data together with neighbourhood scale (environmental) data in the same digital environment to enable deeper and cross-disciplinary insights of built heritage assets’ conditions. This data-driven study is enabled by the use of Building Information Modelling (BIM) tools for the common management of multi-scale and multi-discipline datasets generated by the 3D documentation, non-destructive testing and metadata integration of conservation state analyses and historic architecture information of building assets. Finally the presentation will offer a description of the requirements for integrating these datasets in online repositories for the open access of the public and relevant stakeholders to spatial data analytics that can be used for territorial planning, energy monitoring, educational purposes and smart historic city applications.

Program Committee

  • Nathalie Abadie (IGN, France)
  • Peter Bell (Philipps-Universität Marburg, Germany)
  • Marin Ferecatu (Cnam, France)
  • Noa Garcia (IDS, Osaka University, Japan)
  • Nicolas Gonthier (LTCI lab-Telecom Paris, France)
  • Ronak Gupta (E-infochips Pvt. Ltd., India)
  • Leonardo Impett (Cambridge University, UK)
  • Margarita Khokhlova (Fujistu, France)
  • Prathmesh Madhu (FAU, Germany)
  • Stephane Marchand-Maillet (University of Geneva, Switzerland)
  • Prerana Mukherjee (School of Engineering Jawaharlal Nehru University, India)
  • Milind Padalkar (PAVIS-IIT, Italy)
  • Benjamin Renoust (Osaka University, Japan)
  • Edgar Román (ITAM, Mexico)
  • John Samuel (CPE Lyon, France)
  • Jing Zhang (The University of Sydney, Australia)

Call for papers

The objective of the 4th edition of this workshop is to present and discuss the latest and most significant trends in the analysis, structuring and understanding of multimedia contents dedicated to the valorization of heritage, with emphasis on the unlocking of and access to the big data of the past. We welcome research contributions for the following (but not limited to) topics:

  • Multimedia and cross-domain data interlinking and recommendation
  • Dating and spatialization of historical data
  • Mixed media data access and indexing
  • Deep learning in adverse conditions (transfer learning, learning with side information, etc.)
  • Multi-modal time series analysis, evolution modelling
  • Multi-modal and multi-temporal data rendering
  • Heritage - Building Information Modelling, Art Virtualisation
  • HCI / Interfaces for large-scale datasets
  • Smart digitisation of massive quantities of data
  • Bench-marking, Open Data Movement
  • Generative modelling of cultural heritage

Submission (Deadlines Extended)

Submission Due: 8 July 2022 | 4 July 2022 AoE

Acceptance Notification: 26 July 2022 | 22 July 2022

Camera Ready Submission: 7 August 2022

Workshop Date: 10/14 October 2022

Submission formats

All submissions must be original work not under review at any other workshop, conference, or journal. The workshop will accept papers describing completed work as well as work in progress. One submission format is accepted: full paper, which must follow the formatting guidelines of the main conference ACM MM 2022. Full papers should be from 6 to 8 pages (plus 2 additional pages for the references), encoded as PDF and using the ACM Article Template. For paper guidelines, please visit the conference website, and refer to the 'Paper Format' under 'Submission Instructions'.

Peer Review and publication in ACM Digital Library

Paper submissions must conform with the “double-blind” review policy. All papers will be peer-reviewed by experts in the field, they will receive at least two reviews. Acceptance will be based on relevance to the workshop, scientific novelty, and technical quality. Depending on the number, maturity and topics of the accepted submissions, the work will be presented via oral or poster sessions. The workshop papers will be published in the ACM Digital Library.

Submission Portal

Submit via: Easy Chair Portal

Accepted papers (TBA)

Program (TBA)

Contact Information

Any questions? Please contact us!

Previous Editions