In the same way as a tour, a temporary exhibition or mediation activities, an immersive experience can be the subject of an evaluation. This assessment can be carried out at several levels. A first level may consist of carrying out a diagnosis of the proposed experience. This audit could help find concrete ways to improve the reception, support and immersion of visitors. A second level can allow us to better understand these visitors, their practices and the reception of this experience (qualitative study of the public). Finally, a third level can make it possible to analyze what this immersive experience brings in terms of notoriety, image but also the development of audiences at a place (analysis of attendance and the impact of the experience). This article proposes to return to these 3 levels of evaluation by putting into perspective both the challenges of such an approach and the tools that could be mobilized.
1st level of evaluation: carry out a diagnosis to adjust an immersive experience
To carry out this diagnosis, two methodologies can be used: first, an analysis of the proper integration of the immersive experience into the reception area (issue no. 1), then, a study of the uses and practices of visitors (issue No. 2).
Challenge n°1 – Analyze the integration of the immersive experience in a cultural place (audit of the spaces and arrangements implemented).
A topographical study of an immersive device, accompanied by some shots (photos/videos) can make it possible to evaluate the integration of a virtual reality device into a visit route or in a dedicated space.
This analysis can be carried out before installing an immersive experience or following its implementation to gauge its relevance (in connection with the proposed visit route or with the signage implemented). The formalization of such a mapping can help anticipate possible usability problems between devices (e.g.: a sound shower located near an immersive device which would interfere with the experience). Better situating the experience can also make it possible to note the lack or abundance of directional signs or written explanations offered in addition to the experience. Finally, it can help identify the adjustments to be made.
As such, the Chambord 360° VR experience is an interesting example. This 12-minute VR film is only offered in situ to visitors who have purchased a dedicated ticket. Located in a dedicated area of the tour route (chapel on the 1st floor), the decision was therefore to facilitate access to this experience as much as possible. However, a more in-depth analysis of this visit plan would make it possible to see to what extent the directional signage is sufficient to guide visitors towards this experience and make other audiences want to participate in this experience.
Installed in a dedicated 250m2 room (the largest room in the castle), the furniture has been adapted (backless benches to turn 360° more easily), 28 people can carry out the experience per session (60 VR headsets available du château) – more information on this project in this Culture Matin article. Different shots (photos / videos) could make it possible to better analyze the scenography and arrangements of the place (including the furniture) and to see to what extent the reception and human support could facilitate the handling of the system by the participants. visitors.
Challenge no. 2 – Experiment and observe visitors carrying out the experience to improve their welcome and support (behavioral analysis).
An analysis and observation grid can make it possible to analyze the practices and uses of the public of an immersive device. This analysis can be accompanied by shots (photos/videos) to factually illustrate the observations made.
This analysis can be carried out following the launch of an immersive device. It allows you to see how visitors behave during the experience: are they comfortable when they are equipped? Are they comfortably seated or do they move easily (in the case of a course)? Do they require additional support at the beginning or during the experience? How do they interact with each other during or at the end of the experience? How do they feel once it’s over?
For this study to be as detailed as possible, it needs to be reproduced over different analysis periods and with a diverse panel of visitors. To carry out these observations, a logbook can, for example, be given to the people responsible for welcoming the public in order to note daily feedback, comments heard by visitors or adjustments to be made.
This analysis can be enriched by an analysis of ticketing data and a more in-depth study of audiences (based on interviews and/or questionnaires).
2nd level of evaluation: better understand the audiences and practices of an immersive experience
In this ambition, two issues can be distinguished: on the one hand, better knowing visitors interested in the immersive experience (issue no. 1), then, a study of the uses and practices of visitors (issue no. 2).
Challenge no. 1 – Better understand visitors interested in the immersive experience (registration and ticketing data).
To guarantee a fluid experience, many VR projects are accessible by reservation. The data collected at registration is valuable for better understanding the types of visitors interested in this type of experience.
The collection of this data can be carried out regularly. It can make it possible to track the date of reservation, the mode of visit (alone, with family or in a group), the price paid, the age categories interested in this type of experience, and possibly the other offers selected. These data are those, for example, that the National Museum of Natural History collects with each registration for the virtual reality cabinet.
Challenge no. 2 – Better study the uses and practices of visitors interested in the immersive experience (interviews and surveys).
Depending on the effort envisaged by the place hosting a VR experience, an evaluation of practices and uses can be pragmatically proposed via a questionnaire made freely available to visitors (paper or online format) or administered by mediators. If the issues and the operational possibility of freeing up time or a budget allow it, a more in-depth and qualitative analysis can be considered through interviews.
In 2018/2019, as part of the VR experience L’obsession des Nymphéas hosted within its walls for the Clémenceau / Monet exhibition, the Musée de l’Orangerie with Lucid Realities set up a campaign of questionnaires distributed by the installation facilitators. Among the responses collected, indications made it possible to better understand the audiences for this experience but also to evaluate the experience as such. In terms of visitor data, the study made it possible to define typical user profiles: 41% were international audiences (only 20% Parisians and Ile-de-France residents), 51.3% had never tested an experience in VR, 71% were men, the majority aged 11 to 27. In terms of experience, the responses collected show that 86% really appreciated the experience offered. They particularly appreciated: the support provided by the facilitators, the ease of using the system or the fact of learning something. Equally fundamental learning: this virtual reality experience made them want (more than 60%) to rediscover physical works. Such a result is in line with an English study carried out in 2016 on the Geevor Tin Museum demonstrating that a virtual reality experience outside the walls of a museum makes you want to visit it (to find this study, bibliography on the Immersive devices carried out as part of a study between Mêtis / Universcience).
To refine the results of a questionnaire or survey, a panel of audiences can also be questioned in the form of interviews. This method of investigation makes it possible to collect verbatim comments on visitors’ feelings after an experience. As part of a public survey around the theme of space and the uses of immersive digital tools for Universcience, the Mêtis association carried out semi-directed interviews with 20 visitors aged 26 to 75 recruited for half at the Cité des Sciences et de l’Industries and half at the planetarium and the Natural History Museum of Marseille. The choice of this method was justified by the need to identify “unanticipated needs, expectations or positions, on subjects little explored in the scientific literature”. Regarding virtual reality, this study allowed Universicence to highlight the curiosity that visitors, the majority of whom have never tested a virtual reality experience, have about immersive digital devices, partly thanks to their novelty nature. . These interviews thus make it possible to collect data upstream of an immersive experience on visitors’ expectations and downstream on their feedback.
3rd level of evaluation: evaluate what an immersive experience brings to a place (awareness, brand, audience development).
This evaluation is essential to see to what extent a VR experience can increase the notoriety of a venue (issue no. 1) or contribute to the development of its audiences (issue no. 2).
Challenge n°1 – Increase the notoriety of a reception venue (analysis of communication media).
Notoriety is complex to measure precisely. On the other hand, a set of indicators can be analyzed to measure the contribution of a VR experience to the notoriety of an establishment: monitoring the number of press coverage (if a press campaign has been implemented), evolution of web pages , social media publications or opening and click rates for newsletters dedicated to this experience, etc.
The collection of this data can be carried out during the high points of a reception venue’s communication on its VR experience. It may consist of monitoring the evolution of traffic to its website (particularly the dedicated presentation or ticketing page), mentions of this project which are made on the Web or in its communities on social networks. In a more qualitative way, an analysis of the verbatims on the social networks of the host location, Trip Advisor or Google Reviews can be considered.
Challenge no. 2 – Contribute to the development of audiences at a venue (ticketing/reservation data/surveys/interviews).
The analysis of ticketing data can be considered to evaluate the development of audiences linked to an immersive experience. As mentioned above in this article, various data concerning visitors may be collected when they register or purchase their tickets. Surveys or interviews can supplement this collected data in situ.
Thus, the Cité de l’architecture, which hosted the ScanPyramids virtual reality experience in 2019 and 2020, recalls that of the approximately 1,000 sessions recorded, 57% of visitors had never visited the establishment before (for to find out more, consult the article Virtual reality on the attack on museums in Quotidien de l’art). The development of new audiences both to carry out this immersive experience and to discover the other cultural offerings of a reception venue are, therefore, all potential economic resources for a reception venue.
The evaluation of an immersive experience can therefore be considered by mobilizing a variety of tools. The experience as such can be analyzed in its reception area and through the behavioral observation of visitors. It can also be enriched with a set of qualitative and qualitative means: ticketing/registration data, analysis of communication media, surveys, interviews, etc. These different approaches can make it possible to improve a proposed experience in the short term but also (in the longer term) to better determine what VR can bring to the places that host it.
Antoine ROLAND and Baudouin DUCHANGE