The XR4DRAMA project is driven by two pilot use cases: disaster management and media production planning. The two user partners, Alto Adriatico Water Authority (AAWA, Italy) and Deutsche Welle (DW, Germany), have spent the first months of the project to specify the use cases and to define the user requirements:
The disaster management use case
The first use case describes a situation where clear and comprehensible information is vital, and a confusing impression could be fatal: the management of (natural) disasters. AAWA is an institution that is responsible for preventing flooding as far as possible and to manage emergency situations in case of one. In this project the focus lies on Vicenza, a city located in north-eastern Italy. The city and the surrounding area lay at the confluence of four rivers and therefore have experienced severe floodings in the past. Due to the high flood risk the city of Vicenza and AAWA have already developed alert systems, advanced emergency action plans, and can also count on well qualified civil protection volunteers. However, disasters never play out according to plan. It is highly challenging to manage unforeseen emergencies from a remote control room with only little awareness of the real circumstances in the field.
The main purpose of the XR4DRAMA system is therefore to support disaster managers who are working from remote positions and to connect them with the first responders in the field. The whole scenario is split into two operational phases: the preparation for a potential crisis and the time during the emergency.
The pre-emergency phase begins with an official warning that informs about an alarming rise of water levels. At this point, the staff inside the control room should be able to access various relevant information through the XR4DRAMA system. This information includes map data, satellite images and water level reports and delivers a comprehensive overview of the situation. This will allow the management team to initiate first measures to minimise the fallout of the upcoming disaster.
During the emergency itself, the team in the control room depends on information from the field, either reported by first responders or by citizens on-site. This information is supposed to be communicated through the XR4DRAMA mobile app. It undergoes automated textual, visual and audio analysis to provide a new information layer in the XR4DRAMA system. Together with this layer visualisations in virtual and augmented reality improve situation awareness for disaster managers and help them to make the right decisions consciously and efficiently even from their remote position. Additionally, the control room staff is able to assign tasks and to monitor their execution in real time. Furthermore, the project uses physiological and environmental sensors to allow the management to continuously assess the physical status of first responders and potential risks for their wellbeing.
The media production planning use case
The second use case is not as life-threatening as the first case, but not less worthwhile. Media production planning is a complex process utilising a variety of approaches, strategies, and tools to ensure a smooth operation. This planning process becomes more difficult when production managers have not visited and experienced the proposed location in person. Their lack of grasping the conditions on-site may bring down production quality.
The showcase that we have chosen to evaluate the XR4DRAMA system is a fictious documentary about the old town on the Greek island of Corfu. Throughout the centuries, the island was ruled by various powers and often subject to unrest and war which large fortifications and ancient ruins bear witness to. In 2007, the old town of Corfu was added to the UNESCO Cultural World Heritage list. The fictious documentary intends to cover the changeful history as well as the life of the islanders today on the occasion of the 15th anniversary of being a world heritage site.
The media production planning workflow is split into three steps, starting with the decision for the topic and location. Based on this initial idea, the production management can use the XR4DRAMA system to receive fundamental information about the location in question. The results of these automated queries are then analysed, organised and presented in a comprehensible way by the system.
In a second step, the production management assigns a location scout to further investigate the location. Their task is to verify, adjust and add information such as local data, images, videos and audio recordings. They feed this data into the system which enables the production staff to gain a better overview and feeling for the location even from their remote position.
The final phase is the actual planning of the production itself based on all the information the system and the location scout provided. The production management needs to be able to make creative and reasonable decisions without being on site which requires a certain understanding of and feel for the respective location. Therefore, the XR4DRAMA system displays virtual, immersive environments in order to gain a certain level of spatial sense regarding the location. Additionally, the remote production management team will be able to simulate camera movements and shot angles under various lighting conditions in VR. Thus, xR4DRAMA will contribute to save resources, to optimise the production and – ultimately – to boost the creative outcome overall.
For both use cases, it’s going to be very interesting to see how professional users deal with immersive environments and how these might improve situation awareness and the quality of their work. We are going to keep you posted.
4 out 5 team members: Sotiris Diplaris (2nd from the left) with his colleagues Stefanos Vrochidis, Ioannis Kompatsiaris, and Anastasios Karakostas (from left to right)
In this series of blog interviews, we further introduce the people and organizations behind XR4DRAMA by asking them about their work and their particular interest in the project. Our second interview partner is Dr Sotiris Diplaris of ITI/CERTH, the Information Technology Institute at one of Greece’s (and Europe’s) leading research centres.
1) When did you first get in touch with the concept of situation awareness and XR technology?
It’s hard to say when exactly, but we’ve worked on SA- and XR-related projects for quite a while now. For instance, there was beAWARE, which focused on decision making in emergency situations and also featured AAWA as a partner in disaster management. beAWARE led to the idea of employing XR tech for real-time situation monitoring and collaboration between field teams and control room staff. But we’ve also had several talks with DW, who told us that media professionals would really benefit from having detailed, immersive models of a filming location – before going there with a production team. As for XR tech, we’ve explored that in projects like MindSpaces, SO-CLOSE, and V4Design (the latter also together with DW).
2) Who is in your team – and what are your colleagues working on at the moment?
Dr Stefanos Vrochidis does overall coordination. He’s the head of M4D; that’s the Multimodal Data Fusion and Analytics Group of the Multimedia Knowledge and Social Media Analytics Lab (MKLab) of ITI-CERTH. Stefanos leads a project management team that consists of three more people: Dr. Anastasios Karakostas supports the coordination of XR4DRAMA by supervising research activities and use cases. Spyridon Symeonidisconducts the daily technical management and the implementation of the web data collection mechanisms. As for me, my duties are the scientific and technical management of XR4DRAMA. Finally, there’s Dr. Ioannis Kompatsiaris. As head of MKLab, he’s involved in the high level supervision of the project.
…and the team member missing in the group photo (hi there, Pandemic): Spyridon Symeonidis
3) From your point of view: What are the most interesting aspects about XR4DRAMA?
The most exciting part of this project is that we use several technologies to improve areas of work and life that are really important and depend on situation awareness and decision making. Hopefully, we’ll be able to show how realistic VR or semantically enriched AR representations of environments can improve the way professionals work together. The concepts and applications we’re developing will add a whole new dimension to workflows that has never been explored before.
4) What could be a challenge for the consortium?
The main challenge lies in combining all kinds of data sources and technologies that usually belong to different research fields. We have multimedia web content, drones and satellites, and sensors that enrich our repositories with environmental and physiological information. There’s analysis on a text, audio, video and sensor level, and interlinked results created via semantic technologies. 3D reconstruction techniques generate the elements of immersive environments while geographic information systems (GIS) are used to properly place the content in 3D space. It’s easy to understand that it won’t be trivial to make optimal use of all these components. At the same time, our designated users will only benefit if we put it all together really well.
5) When the project is over in late 2022, what kind of outcome do you expect?
We hope that the end result of XR4DRAMA will be a complete set of XR software that has been evaluated in two real world use cases, namely disaster management and media production planning. The infrastructure we have will be versatile and suitable for other scenarios and domains. Front-end tools will include a desktop and a mobile application as well as an AR and VR environment. We’ll equip professionals with a powerful solution that makes their work easier and safer. And of course, depending on the use case, the impact of XR4DRAMA may go well beyond professionals in the field: For example in the event of a flood, better decision making and disaster management will have positive effects for all people in an affected community.
Dear Sotiris, thank you for you time!
Our next interview partner will be Nicolaus Heise of DW.
While writing concepts, technical documentations and tweets, we’ve realized that the XR4DRAMA consortium uses a lot of abbreviations and acronyms, many of which aren’t necessarily known to a non-expert audience. And even within the team, nobody has heard of all of them, as it’s almost impossible to be an expert in disaster management and media planning and data processing and computer linguistics and smart clothes and immersive technologies.
So in order to everybody a good overview, we’ve compiled this handy glossary – which also defines/explains some of the more exotic or difficult terms:
AR: Augmented reality (a type of extended reality where a users see their “real” environment, but with digital overlays)
CEM: Certified emergency manager
COWM: Citizen Observatories for natural hazards and Water Management
DP: Disaster Preparedness (coordinated actions taken to prepare for disaster, prevent them, or mitigate their impact)
DRR: Disaster Risk Reduction (coordinated actions that aim to reduce the damage caused by natural hazards, via an ethic of prevention)
ECCA: European Climate Change Adaptation Conference
ECG: Electrocardiogram (the process of recording the electrical activity of the heart, usually to check for cardiac abnormalities)
EGU: European Geosciences Union
EO data: Earth observation data
GIS: geographic information system (a conceptualized framework that provides the ability to capture and analyze spatial and geographic data)
GNSS: Global navigation satellite system
H2020: Horizon 2020 (an EU program for research and technological development – and the funding source of XR4DRAMA)
HCI: Human-computer interface (basically any device that lets human interact with a machine, e.g. a keyboard, a touchscreen, a dataglove)
IA: Innovation action (a certain type of R&D project; XR4DRAMA is an IA)
ISCRAM community: an international community of people working in the field of Information Systems for Crisis Response and Management
IFAFRI: The International Forum to Advance First Responder Innovation
LIDAR: Light detection and ranging (also: laser imaging, detection, and ranging): a remote sensing method that uses light (a pulsed laser) to measure variable distances
MR: Mixed reality (like AR, but with interactive virtual objects anchored in the real world)
NLP/NLProc: Natural language processing (a subfield of computer science/artifical intelligence and linguistics that is focused on creating programs to handle and analyze large amounts of natural – i.e. human – languages.
NLU: Natural language understanding (a subfield of NLP/NLProc that is focused on creating program to comprehend what has been collected and processed)
PPE: Personal protective equipment
R&D: Research & development
SAR: Synthetic Aperture Radar (a type of radar used to create 2D or 3D reconstructions of landscapes and objects)
SVI mapping: Social vulnerability index mapping (efforts to visualize/map U.S. Census data that determines the social vulnerability of specific geographic regions)
VR: Virtual reality (a type of extended reality where users don’t see their “real environment”, but are rather fully immersed in a digital sphere)
WWS: Wearable wellness system (a body worn system designed to monitor all kinds of physiological parameters)
XR: Extended reality (AR + MR + VR + all other forms of immersive media)
Have we missed a term that’s important in the context of XR4DRAMA? Send us an email: consortium@xr4drama.eu
Martina Monego during a surveying mission in her district (Eastern Alps)
In this series of blog interviews, we further introduce the people and organizations behind XR4DRAMA by asking them about their work and their particular interest in the project. Our first interview partner is Martina Monego of AAWA, an Italian public body dedicated to the management and regulation of the Alpi Orientali (Eastern Alps) hydrographic district.
1) Martina, when did you first get in touch with the concept of situation awareness (SA) and or XR technology?
As a disaster manager, I’ve been familiar with SA for quite a while, but XR is a rather new thing to me. However, next to XR4DRAMA, I’ve also been involved in a (still unnamed) project that uses immersive technology for educational and training purposes. This one is about better engagement in learning processes and helping students improve their visualization skills. The basic idea is to simulate a flood in a very realistic way, so students can better understand the risks, the relevant aspects of preparing for a disaster like this, and the right behavior in case of emergency.
2) Who is in your team – and what are your colleagues working on at the moment?
The team consists of Michele Ferri, Daniele Nobiato, Franceso Zaffanella and myself. Michele Ferri is our development and innovation manager, in charge of coordinating hydrological research in the context of flood risk management, and the scientific lead. Daniele, Francesco and I are experts in hydrologic and hydraulic modelling, which includes data assimilation, flood forecasting, and flood risk assessment. As a team, we’re responsible for all kinds of projects. For example, we’re currently implementing a so-called Citizen Observatory (CO) on water in our district. In the scope of this CO, citizens provide information (e.g. on water and snow levels or flooded areas) via a mobile app. We then combine this information with other data and use it for early warning systems. The goal is to get a better picture of developments before and during a flood event and to facilitate communications between citizens, authorities, and agents in the field. In this way, we can increase the effectiveness of civil protection efforts. We also do presentations and training sessions for teachers, students, and civil protection volunteers.
Daniele Nobiato and Franceso Zaffanella in the Vicenza municipality control room
3) From your point of view: What are the most interesting aspects about XR4DRAMA?
Well, during disasters like floods, decision makers and first responders face a lot of stress – and need to understand the situation as clearly as possible, so they can act promptly, make the right call, and not waste valuable resources. X4DRAMA will hopefully help them do a better job – and stay safe. At AAWA, we’re very interested in achieving a level of situation awareness that is as detailed and reliable as possible. In our pilot – which we’ll explain in detail later on – we can hopefully do two very interesting things: In phase number one, we’ll collect web information, sensor data, and other sources to predict and simulate a specific scenario, so that control room staff can check and verify all necessary emergency procedures.
4) What could be a challenge for the consortium?
In my opinion, the main challenge is to have enough data and repositories available, so we can get to a good level or SA – or the representation thereof – and fully exploit the potential of XR. Furthermore, our simulation should not be about high-end scenography, but about meaningful and tailor-made content that serves disaster managers. We need to effectively use the technology to support the work and ensure the safety of our first responders.
5) When the project is over in late 2022, what kind of outcome do you expect?
Our vision is to have an innovative and effective tool that improves emergency management in the control room, increases the safety of rescue units, and optimizes our resources.
Dear Martina, thank you for your time!
Our next interview partner will be Sotiris Diplaris of CERTH.
We use cookies on our website to give you the most relevant experience by remembering your preferences and repeat visits. By clicking “Accept”, you consent to the use of ALL the cookies.
This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.
