ARCADIO REYES-LECUONA – Universidad de Malaga, Spain

3D Audio for VR Applications: Fundamentals and Practicalities

Immersive Virtual Reality has been experimenting a constant development and has become more and more popular in the last times. This development has been mainly focused in the visual modality. However, auditory stimuli can also be very powerful in creating immersive perceptions. In this context, three-dimensional localization of sound sources plays an important role in these systems.
3D audio techniques allow to produce the perception in the listener that a sound source is virtually located anywhere in the 3D space, including behind or above the listener. They are not new, but, with the advancements in computational power, it is now possible to perform the required processing in real time using affordable equipment. Therefore, the interest for 3D sound has been increasing.
Additionally, sound is modified by the environment and this is especially relevant when simulating spatial sound. The perception of sound includes a combination of the original sound emitted by the source, modifications due to the environment and modifications produced by the listener’s head. All those modifications can be characterized and applied in a simulation in order to virtually locate a source in a specific place within a given environment.
In this talk, the fundamentals of several techniques of 3D audio will be presented, with special attention to those more suitable for affordable Virtual Reality systems. More specifically, the potential of binaural audio and virtual Ambisonics will be presented. In addition to this, the role of the environment and how it can be considered will also be discussed. Finally, using 3D audio in a Virtual Reality system requires to know some basic concepts of real time audio, which will be addressed as well. Moreover, the main decisions and trade-off to be taken when implementing a 3D sound renderer will be discussed, presenting the practical implications of each of them.

Arcadio Reyes-Lecuona is an Associate Professor lecturing in the Telecommunication Engineering School and the Faculty of Fine Arts at the University of Málaga (SPAIN). He is the head of DIANA research group also at the University of Málaga, working in Virtual reality and 3D interaction. He is a Telecommunication Engineer (BSc & MSc) and obtained his PhD degree in Telecommunication Engineering in 2001. He is also Psychologist (BSc & MSc) since 2017. He has worked since 2001 in VR as project manager in several national and EU projects in the areas of Haptics, 3D interaction and 3D sound, being one of the coordinators of the 3D Tune-In audio toolkit, an open source library for 3D audio rendering (https://github.com/3DTune-In/3dti_AudioToolkit).

 

 

ROBERTO PIERDICCA – Università Politecnica della Marche, Italy

Sensing Cultural Heritage: User-Centered Approaches towards Senseable Spaces

Cultural Heritage domain (both tangible and intangible) has witnessed, in the last decade, to a tremendous improvement on the way in which the users can be in contact with cultural goods. The reasons are many, but one can summarize all of them: ICT are everywhere, pervasive like never in the past, and at the same time more and more cheaper and available in the market. Following this wave, 3D reconstructions, new advanced interfaces, wireless connections and interactions are becoming the backbone for AR/VR experiences, which can be definitively defined as the mainstream for communicating and valorizing the priceless values of cultural goods in a more efficient way. State of art solutions for the development of such experiences are mature enough to allow an effective storytelling and are designed to be exploited by heterogeneous users.
However, several limitations still prevent the adoption of a real and efficient digital agenda for the management of Cultural Heritage. One of them, and probably the most urgent, is that digital experiences are developed by experts or insiders, without taking into account the real needs of the users. There is thus the necessity of adopting strategies to understand the users’ behaviours by analysing and studying their habits, preferences and knowledge. Nowadays this process is made possible by the increasing miniaturization of new technologies, which allows to provide contextual information to the users and, at the same time, to infer information from the digital footprints they leave (the so-called Users’ Generated Data).
The talk, besides providing a complete overview of the latest achievements in the field of DCH, will provide prospective visions about a new paradigm of spaces (both indoor and outdoor) where a bidirectional exchange of information from the space to the user and vice versa is possible. These Spaces can be defined Senseable Spaces, which can be both indoor and outdoor scenario where the service to the users are designed following their behaviours and need, in a seamless way. To show the feasibility of such approach, some research projects (developed by a multi-disciplinary group) will be broadly discussed.

Roberto Pierdicca is a post doc research fellow at Università Politecnica delle Marche, Department of Costruction, Civil Engineering and Architecture. Graduated in Building Engineering and Architecture, he get the Ph.D. in Informatics Engineering in 2017, defending the thesis titled “Senseable Spaces, from a theoretical perspective to the application in augmented environments.” He currently belongs to the research group GAP (Geomatics Applications and processing), conducting his research with a multi-disciplinary approach, ranging among the following topics: Virtual Archaeology, Digital Cultural Heritage, Augmented and Virtual Reality, Ambient Intelligence, Mobile Development, Human Behaviour Analysis, Geospatial Applications and Remote Sensing. He is author of more than 40 international publications and actually engaged in more than 10 national and international research projects.

 

 

MARCELLO CARROZZINO – Scuola Superiore Sant’Anna, Italy

Opportunities of the use of Embodied Agents in Virtual Reality for Cultural Heritage

Virtual Reality is becoming an increasingly important tool for the research, the communication and the popularization of cultural heritage. However most of the available 3D interactive reconstructions of artefacts, monuments and sites often miss an important factor: human presence. Thanks to the advancements in the technology, in latest years Virtual Humans have started being used in a variety of cultural-related VR applications. From simple 2D characters to complex 3D avatars, technology continues to evolve and so is the adoption of virtual assistants in digital heritage. The acceptance of such tools deserves a greater attention from the scientific community.
This talk will explore the state-of-the-art on this subject, underlining the technological challenges and also analysing the effects of avatar interaction on user engagement, sense of immersion and learning effectiveness.

Marcello Carrozzino is Assistant Professor of Computer Engineering of Scuola Superiore Sant’Anna (SSSA) in Pisa. His activities involve mainly the design, implementation and integration of Virtual and Augmented Reality systems and their applications to industry and cultural heritage. He received a Master degree in Computer Engineering at University of Pisa and a Ph.D. at Scuola Superiore Sant’Anna. Since 2005 he has been teaching Virtual Reality at the Computer Science department of the University of Pisa. Since 2006 he has been coordinating the Art, Culture and Education group at the Laboratory of Perceptual Robotics of SSSA.  He has been coordinating several research projects at international and national level. He is the cofounder of VRMedia s.r.l, a spin-off company of Scuola Superiore Sant’Anna in Pisa, dealing with the development of Virtual and Augmented Reality solutions for industry. He is also the cofounder of the Mnemosyne Digital Culture spin-off association dealing with the dissemination of digital culture and technology-based learning. He has authored and co-authored many publications in international journals, conference proceedings and book chapters.