SUMAC 2021
The 3rd workshop on Structuring and Understanding of Multimedia heritAge Contents

20 October 2021, 13:30-17:00 (GMT+8 Time)
Online Event

20 October 2021, Chengdu, China

About the workshop

Aims and scope of the workshop

The digitization of large quantities of analogue data and the massive production of born-digital documents for many years now provide us with large volumes of varied multimedia data (images, maps, text, video, multisensor data, etc.), an important feature of which is that they are cross-domain. "Cross-domain" reflects the fact that these data may have been acquired in very different conditions: different acquisition systems, times and points of view (e.g. a 1962 postcard from the Arc de Triomphe vs. a recent street-view acquisition by mobile mapping of the same monument). These data represent an extremely rich heritage that can be exploited in a wide variety of fields, from SSH to land use and territorial policies, including smart city, urban planning, tourism, creative media and entertainment. In terms of research in computer science, they address challenging problems related to the diversity and volume of the media across time, the variety of content descriptors (potentially including the time dimension), the veracity of the data, and the different user needs with respect to engaging with this rich material and the extraction of value out of the data. These challenges are reflected in research topics such as multimodal and mixed media search, automatic content analysis, multimedia linking and recommendation, and big data analysis and visualisation, where scientific bottlenecks may be exacerbated by the time dimension, which also provides topics of interest such as multimodal time series analysis.


Keynote 1 (talk details)

Jon Yngve Hardeberg

is Professor at the Department of Computer Science at NTNU in Gjøvik. He has a MSc in Signal Processing from NTNU, and a PhD in Signal and Image Processing from the Ecole Nationale Supérieure des Télécommunications in Paris, France. Professor Hardeberg is a member of the Norwegian Colour and Visual Computing Laboratory where he teaches, supervises graduate students, manages international study programs and research projects. He has co-authored more than 200 publications.

Keynote 2 (talk details)

Mathieu Aubry

is a tenured researcher in the Imagine team of Ecole des Ponts. His work is mainly focussed on Computer Vision and Deep Learning, and their intersection with Computer Graphics, Machine Learning, and Digital Humanities. His PhD on 3D shapes representations at ENS was co-advised by Josef Sivic (INRIA) and Daniel Cremers (TUM). In 2015, he spent a year working as a postdoc with Alexei Efros in UC Berkeley. Since 2018, he is leading the EnHerit ANR project on enhancing heritage image databases. He is also co-organizing the DHAI seminar.


Program Committee

  • Edgar Román (ITAM, Mexico)
  • Jing Zhang (The University of Sydney, Australia)
  • Leonardo Impett (Durham University, UK)
  • Marin Ferecatu (Cnam, France)
  • Milind Padalkar (PAVIS-IIT, Italy)
  • Noa Garcia (Osaka University, Japan)
  • Olga Sushkova (Institute of Radio-engineering and Electronics of RAS, Russia)
  • Stephane Marchand-Maillet (University of Geneva)
  • Prathmesh Madhu (FAU, Germany)

Call for papers

The objective of the third edition of this workshop is to present and discuss the latest and most significant trends in the analysis, structuring and understanding of multimedia contents dedicated to the valorization of heritage, with emphasis on the unlocking of and access to the big data of the past. We welcome research contributions for the following (but not limited to) topics:

  • Multimedia and cross-domain data interlinking and recommendation
  • Dating and spatialization of historical data
  • Mixed media data access and indexing
  • Deep learning in adverse conditions (transfer learning, learning with side information, etc.)
  • Multi-modal time series analysis, evolution modelling
  • Multi-modal and multi-temporal data rendering
  • Heritage - Building Information Modelling, Art Virtualisation
  • HCI / Interfaces for large-scale datasets
  • Smart digitisation of massive quantities of data
  • Bench-marking, Open Data Movement
  • Generative modelling of cultural heritage


Submission Due: 30 July 2021 AoE

Acceptance Notification: 11 August 2021

Camera Ready Submission: 31 August 2021

Workshop Date: 20 October 2021

Submission formats

All submissions must be original work not under review at any other workshop, conference, or journal. The workshop will accept papers describing completed work as well as work in progress. One submission format is accepted: full paper, which must follow the formatting guidelines of the main conference ACM MM 2021. Full papers should be from 6 to 8 pages (plus 2 additional pages for the references), encoded as PDF and using the ACM Article Template. For paper guidelines, please visit the conference website, and refer to the 'Paper Format' under 'Submission Instructions'.

Peer Review and publication in ACM Digital Library

Paper submissions must conform with the “double-blind” review policy. All papers will be peer-reviewed by experts in the field, they will receive at least two reviews. Acceptance will be based on relevance to the workshop, scientific novelty, and technical quality. Depending on the number, maturity and topics of the accepted submissions, the work will be presented via oral or poster sessions. The workshop papers will be published in the ACM Digital Library.

Submission Portal

Go to the link: Submission Portal & in the Author's Console, create a new submission for the track: "3rd workshop on Structuring and Understanding of Multimedia heritAge Contents"

Accepted papers

Built Year Prediction from Buddha Face with Heterogeneous Labels Yiming Qian (Osaka University); Cheikh Brahim EL VAIGH (Irisa/Inria); Yuta Nakashima (Osaka University); Benjamin Renoust (Osaka University); Hajime Nagahara (Osaka University); Yutaka Fujioka (Osaka University)

Software and Content Design of a Browser-based Mobile 4D VR Application to Explore Historical City Architecture Sander Muenster (FSU Jena); Jonas Bruschke (FSU Jena); Ferdinand Maiwald (FSU Jena); Constantin Kleiner (FSU Jena)

Evaluation of Deep Learning Techniques for Content Extraction in Spanish Colonial Notary Records Nouf Alrasheed (University of Missouri-Kansas City); Shivika Prasanna (University of Missouri-Columbia); Ryan Rowland (University of Missouri-Kansas City); Praveen Rao (University of Missouri-Columbia); Viviana Grieco (University of Missouri-Kansas City); Martín Wasserman (University of Buenos Aires & CONICET)

How to spatialize geographical iconographic heritage Emile Blettery (IGN); Nelson Fernandes (LASTIG/IGN-UGE); Valérie Gouet-Brunet (LASTIG/IGN-UGE)

Searching Silk Fabrics by Images Leveraging on Knowledge Graph and Domain Expert Rules Thomas Schleider (EURECOM); Raphael Troncy (EURECOM); Mareike Dorozynski (Leibniz Universitat Hannover"); Franz Rottensteiner ("Leibniz Universitat Hannover, Germany"); Jorge Sebastián Lozano (Universitat de València); Georgia Lo Cicero (University di Palermo); Thibault Ehrhart (EURECOM)


Date: 20 October 2021

Venue: Virtual Event, 13:30-17:00 (GMT+8 Time)

This year, like the previous year, we have the pleasure to award a prize of 500 euros for the best article. The best article will be chosen by a jury, and announced at the end of the workshop. The award is co-sponsored by Friedrich-Alexander-Universität Erlangen, the French Mapping Agency (IGN) and the French National Research Agency (ANR, Alegoria project). The workshop will also feature the following two keynote talks.

Keynote 1

Jon Hardeberg

Professor at the Computer Science Department of the Nor-wegian University of Science and Technology, Norway

Analyzing CHANGE in cultural heritage objects through images

Cultural heritage (CH) objects have been constantly undergoing changes/degradation over time. In order to pass the legacy of these objects to future generations, it is important to monitor, estimate and understand these changes as accurately as possible. These investigations will support the conservators to plan necessary treatments in advance or to slow down the specific deterioration processes. The dynamic characteristics of materials vary from one object to another and are influenced by several factors. To detect and predict their changes, accurate documentation and analysis are necessary. Over the years, CH digitization using scientific imaging techniques has become more widespread and has created a massive amount of datasets of different forms in 2D and 3D. Several past projects focused on different aspects of technological developments for better digitization methods. There has been less focus on the processing and analysis of these datasets to make the greatest use of them and to their further exploration for monitoring ‘changes’ in CH artifacts for conservation purposes. The CHANGE project takes cultural heritage digitization to a new level by exploring digital datasets for deeper analysis and interpretation by developing methodologies for the assessment of changes in CH objects by comparing and combining digital datasets captured at different time periods.

Keynote 2

Mathieu Aubry

Senior researcher, Imagine team, LIGM lab, ENPC, Écoledes Ponts ParisTech, France)

Deep Learning for Historical Data Analysis

This presentation will give an overview of projects on leveraging deep learning for historical data analysis my group did in the last 3 years, partly in the context of the ANR EnHerit project. I will discuss in particular how deep learning can be used to establish links between artworks and historical documents: repeated patterns discovery in artwork collections, fine artwork alignment, document images segmentation, historical watermarks recognition, generic clustering, and scientific illustration propagation analysis. In all cases, I will show that standard approaches can give useful baseline results when tuned adequately, but that developing dedicated approaches that take into account the specificity of the data and the problem significantly improves the results.

Contact Information

Any questions? Please contact us!

Previous Editions