Académique Documents
Professionnel Documents
Culture Documents
This deliverable outlines the design and plan for EXPERIMEDIAs embedded experiment focusing on shared, real-time, immersive and interactive cultural and educational and executed by the Foundation of the Hellenic World at its premises at Hellenic Cosmos in Athens. Starting from the more abstract scenario description provided in the earlier D2.1.2, exploiting the architectural blueprint described in the D2.1.3 that is being developed in parallel, and taking into consideration the methodological guidelines described in D2.1.1 as well as the ethical oversight principles described in D5.1.1, the document provides an overview of the current status as well as a detailed design and time plan for the future of the experiment.
www.experimedia.eu
EXPERIMEDIA
Dissemination Level: PU
Project acronym EXPERIMEDIA Full title Experiments in live social and networked media experiences Grant agreement number 287966 Funding scheme Large-scale Integrating Project (IP) Work programme topic Objective ICT-2011.1.6 Future Internet Research and Experimentation (FIRE) Project start date 2011-10-01 Project duration 36 months Activity 4 Experimentation Workpackage 4.3 EX3: shared, real-time, immersive and interactive cultural and educational experiences Deliverable lead organisation IME Authors Manolis Wallace, Anthousis Andreadis, Efstathia Chatzi (IME) Reviewers Wolfgang Halb (JRS), Stephen C. Phillips (ITInnov) Version 1.0 Status Final Dissemination level PU: Public Due date PM8 (2012-05-31) Delivery date 2012-08-10
EXPERIMEDIA
Dissemination Level: PU
Table of Contents
1. 2. 3. Executive summary ............................................................................................................................ 5 Introduction ........................................................................................................................................ 6 The FHW embedded experiment .................................................................................................... 8 3.1. 3.2. 3.3. 3.4. 3.5. Vision .......................................................................................................................................... 8 Experiences currently offered ............................................................................................... 10 The experience examined in the embedded experiment ................................................... 10 Experience architecture .......................................................................................................... 12 Component tear-down ........................................................................................................... 13
3.5.1. Smart devices ....................................................................................................................... 13 3.5.2. Streaming.............................................................................................................................. 13 3.5.3. Experts Toolbox ................................................................................................................ 13 4. Experiment design............................................................................................................................ 14 4.1. Goals ......................................................................................................................................... 14 4.1.1. Goal 1: Be an EXPERIMEDIA test bed ........................................................................ 14 4.1.2. Goal 2: Explore suitability of FIRE technologies for the field under examination .......................................................................................................................... 14 4.1.3. Goal 3: Measure impact of FIRE technologies .............................................................. 14 4.1.4. Goal 4: Identify parameters that affect impact ............................................................... 14 4.2. Experiment procedure............................................................................................................ 15 4.2.1. Preparatory phase................................................................................................................ 15 4.2.2. Execution phase .................................................................................................................. 15 4.2.3. Data acquisition phase ....................................................................................................... 16 4.2.4. Analysis phase...................................................................................................................... 17 4.3. 4.4. Examined parameters ............................................................................................................. 17 Experiment evaluation ........................................................................................................... 18
4.4.1. Baseline success ................................................................................................................... 18 4.4.2. Moderate success ................................................................................................................ 18 4.4.3. Success .................................................................................................................................. 19 4.4.4. Exceptional success ............................................................................................................ 19 4.5. Recruiting ................................................................................................................................. 20 4.5.1. Random visitors .................................................................................................................. 20 4.5.2. Organized groups................................................................................................................ 20 Copyright IME and other members of the EXPERIMEDIA consortium 2012 2
EXPERIMEDIA
Dissemination Level: PU
4.5.3. Summer schools and other FHW activities..................................................................... 20 4.6. 5. 5.1. 5.2. Constraints ............................................................................................................................... 20 System architecture ................................................................................................................. 22 Requirements ........................................................................................................................... 25 Experimental facility implementation............................................................................................ 22 5.1.1. The architecture in the scope of the experiment............................................................ 25 5.2.1. Requirements on the FHW facility................................................................................... 25 5.2.2. Required components by FHW (development) ............................................................. 26 5.2.3. Requirements for input from EXPERIMEDIA partners ............................................ 26 5.3. 6. 6.1. Content lifecycle ...................................................................................................................... 27 Minimum ethical principles ................................................................................................... 28 Ethics, privacy, PIA.......................................................................................................................... 28 6.1.1. Doing good .......................................................................................................................... 28 6.1.2. Doing no harm .................................................................................................................... 29 6.1.3. Risk management ................................................................................................................ 29 6.1.4. Consent................................................................................................................................. 29 6.1.5. Confidentiality ..................................................................................................................... 29 6.1.6. Data protection ................................................................................................................... 30 6.2. Ethical oversight principles ................................................................................................... 30 6.2.1. Informed consent ............................................................................................................... 30 6.2.2. Deception ............................................................................................................................. 30 6.2.3. Data collection..................................................................................................................... 30 6.2.4. Withdrawal from the investigation ................................................................................... 31 6.2.5. Observational research ....................................................................................................... 31 6.2.6. Data protection regulation................................................................................................. 31 6.2.7. Consortium partner responsibility .................................................................................... 31 6.3. 6.4. Points to consider in the FHW embedded experiment ..................................................... 31 PIA ............................................................................................................................................ 32
6.4.1. PIA phase 1.......................................................................................................................... 32 6.4.2. PIA phase 2.......................................................................................................................... 32 7. Risks.................................................................................................................................................... 33 7.1. 7.2. 8. Risks for the participants ....................................................................................................... 33 Risks for the experiment ........................................................................................................ 34
EXPERIMEDIA 8.1.
Dissemination Level: PU
Current status........................................................................................................................... 35
8.1.1. Video capture ...................................................................................................................... 35 8.1.2. Streaming.............................................................................................................................. 35 8.1.3. Experts toolbox.................................................................................................................. 35 8.1.4. Augmented reality ............................................................................................................... 35 8.2. 9. Future plans ............................................................................................................................. 36 Conclusion ......................................................................................................................................... 37
EXPERIMEDIA
Dissemination Level: PU
1. Executive summary
This deliverable outlines the design and plan for EXPERIMEDIAs embedded experiment focusing on shared, real-time, immersive and interactive cultural and educational and executed by the Foundation of the Hellenic World at its premises at Hellenic Cosmos in Athens. Starting with this executive summary and an introductory note in section 2, the remaining of this document is organized as follows: Section 3 presents the venues vision with respect to the technologies discussed in the experiment and their ways in which they will be used in order to enhance the cultural and educational experiences that the venue is able to provide to its visitors. The section closes with a presentation of the user interaction scenario that will be supported by the embedded experiment. With that in mind, Section 4 provides the technical and methodological description of the experiment. This includes the specification of the experiments goals, the procedure that will be followed, the data that will be collected and the way in which this data collection will be achieved, the way the data will be analysed and the parameters that will be studied. This follows from the corresponding scenario, as it was defined in deliverable D2.1.2, as well as from the methodological guidelines described in D2.1.1. Section 5 presents the detailed architecture of the experimental facilities that will be developed and utilized for the needs of this experiment. This follows from the architectural blueprint described in the D2.1.3. Section 6 discusses ethical and privacy concerns that may be relevant to the experiment in question. This follows from the ethical and data protection guidelines that have been established in deliverable D5.1.1. Special emphasis is given to the Privacy Impact Assessment (PIA) methodology dictated in D2.1.1. Section 7 presents the current status of the work and discusses the projected timeline for the execution of the different tasks that are related to the implementation and analysis of the experiment, following from the DoW as well as from the implementation plans of the relevant activities of other EXPERIMEDIA WPs.
EXPERIMEDIA
Dissemination Level: PU
2. Introduction
This deliverable describes the experiment design and plan for one of the three embedded experiments that use the EXPERIMEDIA facility. EXPERIMEDIA is a "FIRE" (Future Internet Research and Experimentation) project. For the project and in this context "experimentation" means that we do not know exactly beforehand what the researchers using the EXPERIMEDIA facility will want to do. The embedded experiments are used to provide requirements during facility development and to verify and validate the facility capabilities to be used by further experiments that are either funded during the open calls or unfunded. In our embedded experiment we shall develop a testing setting that will relate the FHW 3D content and the FHW real-time rendering engine with new and innovative user oriented media services. Using this setting we will explore the possibility to present this content in different device and network contexts, to single users and to user groups, in real-time, whilst supporting the live interaction between all involved user groups. We will also explore how the provided services may be enhanced with the exploitation of social media extensions. In order to achieve the above we will extend the existing 3D content provision infrastructure with capabilities such as live interaction with remote users and Augmented Reality (AR). This will allow us to offer two additional services to the ones currently available to our visitors: real time feedback by the content experts themselves and association of real objects to the ones contained in the virtual worlds and to the digital metadata that is available at FHW. Through suitable experimentation with real users we will examine on one hand the suitability of the utilized approach and future internet technologies for this type of applications and on the other hand how the technical parameters of these technologies affect the Quality of Experience (QoE) that is finally offered to the participating end users. The successful execution of this embedded experiment will provide a validation for the overall architecture and approach to experimentation followed in EXPERIMEDIA, and the experience gathered during the execution of the experiment will be vital towards the better planning for and execution of future EXPERIMEDIA experiments. The remainder of this document is organized as follows: Section 3 presents the venues vision with respect to the technologies discussed in the experiment and their ways in which they will be used in order to enhance the cultural and educational experiences that the venue is able to provide to its visitors. The section closes with a presentation of the user interaction scenario that will be supported by the embedded experiment. With that in mind, Section 4 provides the technical and methodological description of the experiment. This includes the specification of the experiments goals, the procedure that will be followed, the data that will be collected and the way in which this data collection will be achieved, the way the data will be analysed and the parameters that will be studied. This follows from the corresponding scenario, as it was defined in deliverable D2.1.2, as well as from the methodological guidelines described in D2.1.1. Copyright IME and other members of the EXPERIMEDIA consortium 2012 6
EXPERIMEDIA
Dissemination Level: PU
Section 5 presents the detailed architecture of the experimental facilities that will be developed and utilized for the needs of this experiment. This follows from the architectural blueprint described in the D2.1.3. Section 6 discusses ethical and privacy concerns that may be relevant to the experiment in question. This follows from the ethical and data protection guidelines that have been established in deliverable D5.1.1. Special emphasis is given to the Privacy Impact Assessment (PIA) methodology dictated in D2.1.1. Section 7 presents the current status of the work and discusses the projected timeline for the execution of the different tasks that are related to the implementation and analysis of the experiment, following from the DoW as well as from the implementation plans of the relevant activities of other EXPERIMEDIA WPs.
EXPERIMEDIA
Dissemination Level: PU
3.1.
Vision
Our aim at FHW has always been to exploit all sorts of capabilities that technology has to offer at any given time, in order to create and provide engaging cultural and educational experiences. As such, in this experiment we aim to cross-fertilize AR and social media with traditional virtual reality installations in order to provide a cultural and educational experience that should put visitors in touch with what is fundamentally engaging, help them build a scaffolding of the historical core concepts and motivate them to go deeper into the subject themselves. Augmented Reality (AR) brings the virtual world to the physical world. Information such as audio or computer graphics is layered over real objects in a manner that is attracting widespread attention and is being used to create applications for a range of uses. Research has suggested, for example, the potential for using augmented reality location-based games (AR/LBS games) that use wireless handheld devices to provide virtual game information in a physical environment, as an educational tool. Augmented Reality mobile applications are beginning to change the way visitors of museums, art galleries and tourist locations in general see and interact with the physical world. Augmented Reality also appeals to constructivist notions of education where students take control of their own learning, and interact with the real and virtual environments. These types of training provide opportunities for more authentic learning and appeal to multiple learning styles. AR applications can provide each student with his/her own unique discovery path, move from a rote learning concepts and an I teach-you listen method to a more active and participatory learning method Copyright IME and other members of the EXPERIMEDIA consortium 2012 8
EXPERIMEDIA
Dissemination Level: PU
where learners take responsibility for learning and are engaged participants rather than passive observers. Besides the learning experience itself, the social aspect of a museum visit should not be underestimated. People tend to visit museums in groups and have shared experiences. Thus, by interconnecting with social networks we can also provide access to one of the main methods of socializing among youth. In this manner the provided experience can foster collaboration and introduce fun during the sometimes long and tiresome waiting queues associated with VR exhibits, without alienating the visitor form the main theme of the VR exhibit they are about to experience. With the continuous advances of game consoles, VR systems simply have to keep up. Therefore they cannot afford to remain fresh from a visual only standpoint; they need to address more of the visitors senses, provide multiple and diverse stimuli, and employ a variety of new and ubiquitous technologies not only during the show, but also during the pre- and post-show of the primary VR installation. Another direction in which museums have to work hard is that of dynamic information. The type of information that is available in static exhibitions, regardless of the rarity and importance of the exhibits, can hardly be considered as exciting for the general public. Museums have realized that a long time ago, and have tried to fill the gap with the use of museum guides, i.e. with individuals who have some sort of expertise on the exhibits and provide the information in a more lively and intuitive manner to groups of visitors. Although this has been a step in the right direction, the expertise of the tour guide has always been a point of weakness, as typically tour guides are not true experts but rather individuals who have studied a predefined presentation script and have little ability to operate outside it. Of course hiring true experts as museum guides is not an option; even if they are available, as is the case with FHW, their expertise is too useful and expensive for them to be on standby at the museum in case they are needed. Furthermore, a truly intuitive and free navigation, allowing the visitor to wonder and receive information of any type and on any topic conceivable, would require the involvement of multiple experts in the guidance of each individual group, which is simply unrealistic. Streaming technology is an ideal tool when there is a need to synchronously transmit heavy streams of information to multiple remote locations. This type of technology can make it possible for multiple individuals, and more specifically for the various experts employed by FHW, to monitor the on-going visits and step in providing specialized information, when and if that is required. Such a service would allow the venue to provide less structured and therefore considerably more engaging and stimulating experiences, as the visitors could be given more control over the direction of their experience. Combining all the above, the provided immersive experience will be extended and enhanced, and the visitor will feel that his whole visit to the museum, not just the few minutes of the VR show, forms a unique experience of immersion into a digitally enhanced world.
EXPERIMEDIA
Dissemination Level: PU
3.2.
As has already been mentioned, the embedded experiment is built around the VR immersion experience offered by the Tholos. Currently, this is offered mainly as a standalone experience that is not necessarily combined with any of the other exhibits or services of Hellenic Cosmos. The typical operation of the Tholos and of the service it offers to its visitors may be graphically modeled as in Figure 1. It is easy to see that this is a mainly one-way communication system, as the museum educator controls the system, thus specifying what the Tholos system will render and project to the visitors, while at the same time commenting on it. As a sole exception to this, visitors are able to participate in electronic polls which determine the path that the Educator will follow altering in this way the flow of the presentation in real time. The main reason for this extremely structured and predefined approach is that the museum educator is working with predefined scenarios, i.e. descriptive texts prepared by the FHW experts. These texts provide information on the 3D worlds in a specific order and therefore the tour in the 3D world has to follow the same order, otherwise the museum educator would be unable to provide synchronized information.
FHW FACILITIES
Navigation Control
USERS
Museum Educator
Visitor
3.3.
Utilizing the advanced capabilities offered by the components developed by the EPXERIMEDIA technical partners this communication model will be extended in at least two directions, as shown in Figure 2. Copyright IME and other members of the EXPERIMEDIA consortium 2012 10
EXPERIMEDIA
Dissemination Level: PU
In this model visitors, upon their arrival at the premises of Hellenic Cosmos, will be given brochures will with details about the show they will be attending (a travel to ancient Miletus). Specific markers will exists inside the brochure the use of which will be described later. Furthermore the visitors will be given Smart Mobile Devices that will have the FHW mobile application installed. The visitors at this point may enter their social media credentials to the mobile application.
FHW FACILITIES
Experts Toolbox
Mobile Application
Navigation Control
USERS
Museum Educator
Museum Educator
Visitor
The next stop will be the Tholos installation where they will attend the show about ancient Miletus. As has already been mentioned, a museum educator will be guiding the experience by controlling the content that is visualized in Tholos. But, in contrast to the conventional Tholos experience, visitors will now have the opportunity to make a larger impact to their shared experiences, either by requesting that a different path is taken or simply by asking the questions that they find most interesting, using the smart devices. In addition to the museum educator, visitors will actually be interacting with FHW experts, mainly historians, archaeologists and Copyright IME and other members of the EXPERIMEDIA consortium 2012 11
EXPERIMEDIA
Dissemination Level: PU
architects. These experts will not be located at the Tholos installation, but will be virtually attending the show remotely and will be available to provide additional information when the visitors ask for something more specialized or even when simply the path taken in the 3D world is one that the museum educator is not properly trained to support. Using the smart devices the visitors will also have the opportunity to review several of the 3D models that they saw during the show. This will be achieved using the markers on the brochures they have been given. Furthermore they will either be given a guided tour of Hellenic Cosmos or they will have the opportunity to wonder around on their own. In both cases they will be able to retrieve additional information, namely related 3D content, about the various exhibits that they will examine, using the markers that will be placed next to them.
3.4.
Experience architecture
In this subsection we discuss the components that will be used in order to make possible the provision of the abovementioned experience. We refer to this as architecture of the experience rather than the experiment, as the architecture of the experiment will have to include additional components that are not related to the service offered to the visitors but rather to the experimental process itself (i.e. monitoring of the experiment, measurement of parameters, etc) As shown in Figure 2 a set of applications will be provided to the remote experts so that he can virtually attend the show and communicate with the visitors and provide details on the displayed content; we refer to this as the Experts Toolbox. The Experts Toolbox aims to allow remote experts to join and support the museum educator in providing details on the displayed content. In order to make this possible the toolbox will include the following: An application that will allow the expert to know what is displayed in the Tholos. This will be achieved through the provision of live streaming video from the Tholos. The stream will also include audio from the educator, thus allowing the expert to know not only what the visitors see, but also what it is that they have already been told. They will also be able to be asked questions directly from the museum educator. An application that will allow the expert to interact directly with the audience. This will be achieved through social media, making the interaction shared. The second extension of the operation model regards the inclusion of a smart device operated by the visitor. This will be equipped with a suitable mobile application, built to serve the needs of the abovementioned interaction model. This smart device will provide the following capabilities: The ability for the visitors to communicate with the experts during the show and ask questions related to the virtual world or to see the already available discussions and take part in them. This will be based on social media. The use of social media has the added value of making the discussions visible to all and also available even beyond the duration of a single show, offering more in-depth information on the content of the show to the interested visitors and acting as an information vault for future reference. The opportunity for the users to explore and get familiar with virtual objects related to the content of the show. This will be achieved through the use of Augmented Reality. Specifically, observing the content through the lens of the mobile device, be it something Copyright IME and other members of the EXPERIMEDIA consortium 2012 12
EXPERIMEDIA
Dissemination Level: PU
on the provided brochure or even a real object in one of the Hellenic Cosmos exhibitions, additional content will be provided by the device. The main concept of the experiment is to study first of all the meaningfulness of these extensions and, assuming a positive observation, to assess how different flavors of these extensions, i.e. extensions built using different parameters in the EXPERIMEDIA components, are perceived by the participating users.
3.5.
Component tear-down
In this section we move a step further from the abstract architecture provided above, in order to provide links not just to the technology, but rather to the specific technological components that will be used in the experimental facility. This tear down details the technologies that contribute to the facility and will be examined during the experiment, and also outlines the critical dependencies of the experiment on the work performed in the technological WPs of the project.
3.5.2. Streaming
A Video Capture Card has been acquired and successfully tested. This will be used to generate the stream that will then be forwarded to the experts. This is installed on a standard PC with Windows OS. For the actual streaming the RTMP protocol for streaming video is used (Adobe Flash Media Live Encoder) ATOS will provide the streaming service through one of its servers
13
EXPERIMEDIA
Dissemination Level: PU
4. Experiment design
In the previous section we presented the enhanced services that we aim to offer to Hellenic Cosmos visitors by using the various EXPERIMEDIA extensions that will be made available by the project partners. In this section we will focus more on the experiment itself and discuss its goals and the way these goals will be pursued.
4.1.
Goals
EXPERIMEDIA is a very complex and ambitious project. It combines the consideration of multiple emerging technologies with the execution of medium to large scale experiments with real users in real settings, and all that under tremendously tight schedules. Therefore, as is typically the rule with highly ambitious projects, it is best if goals are set (and pursued) progressively. With that in mind, we define here goals of different difficulty, all of which will of course be pursued with equal eagerness to succeed.
4.1.2. Goal 2: Explore suitability of FIRE technologies for the field under examination
Moving beyond the mere implementation and integration of the experimental facilities and into the actual execution of the experiment, there is the question of whether all, none, or just some of the considered EXPERIMEDIA technologies are meaningful. Therefore, one of the experiments goals is to examine whether the selected EXPERIMEDIA toolboxes that will be applied in Hellenic Cosmos truly have a positive impact on the services that may be offered to the visitors.
EXPERIMEDIA
Dissemination Level: PU
4.2.
Experiment procedure
The embedded experiment will take place at the premises of Hellenic World and can be semantically split in four, partially overlapping, phases: Preparatory, Execution, Data Acquisition and Analysis, as follows.
EXPERIMEDIA
Dissemination Level: PU
with different parameters are used, in order to assess how these differences affect the users perception of the facility. According to D2.1.1 the execution of an EXPERIMEDIA experiment should follow two phases: Value Opportunity Validation and Macro Modelling Phase for the engagement of lead users for small scale experimentation, on sample use cases which will be implemented via the use of focus groups that will produce the model for the assessment of the experiment roadmap Industrialisation Assessment Phase that will test the experiment model on a bigger sample On the other hand it is clear that the timing of the embedded experiments does not allow for many iterations of the experiment, as their schedule is already extremely tight, if one considers the time at which the baseline technologies are expected to become available for use and integration in the experimental settings of the venues. The two phases will be implemented of course, as dictated by D2.1.1, but their role will be slightly altered. Specifically: The first phase will go much further than merely producing the model for the core experiment and will aim towards tackling the first two goals of the experiment, therefore focusing on a rough evaluation of the experiment components. The second phase, on the other hand, will entail more extensive and detailed experimentation as it will target the latter two goals of the experiment. Clearly, both phases will provide queues as to whether the approach could be industrialized, i.e. as to whether this experimental setting could become a new service provided by Hellenic Cosmos to its paying customers. What cannot be performed in the scope of the embedded experiment is a full scalability analysis, as it is not financially possible to provide smart devices for everyone that may wish to use one during one day of operations.
16
EXPERIMEDIA
Dissemination Level: PU
of the currently offered service. The way in which their own work is affected in also of interest. Experts. Remote experts will be the only users of one of the experiments components (the Experts Toolbox), and therefore their feedback is necessary in order to assess the parameters that are involved in the operation of that component.
As far as the data acquisition itself is concerned, two methods will be used: 4.2.3.1. Questionnaires Questionnaires will be developed in order to acquire information from the participating users. The probed information ranges from simple demographic data to information that will allow us to assess the QoE of the users. 4.2.3.2. Focus groups Focus groups differ from conventional survey methods such as questionnaires because they use insight instead of rules, and have a social and not individual orientation. They also have the benefit of translating the whole experience to words and feelings instead of numbers, which allows the participating users to be much more expressive in their feedback. On the low side, the unstructured nature makes it is considerably harder to process and analyse the acquired information; therefore this approach will not be applied in an equally extensive approach as questionnaire.
4.3.
Examined parameters
As has already been explained, a first parameter that will be examined is whether each of the EPXERIMEDIA extensions is used at all. This is of course related only to the first goal that has been set for the experiment, and it is clearly expected that the experiment will confirm that the EXPERIMEDIA extensions enhance the provided service. Getting into the specifics though, and as the details of the EXPERIMEDIA baseline components are still being formed, the final list of adjustable parameters for the different Copyright IME and other members of the EXPERIMEDIA consortium 2012 17
EXPERIMEDIA
Dissemination Level: PU
EXPERIMEDIA components is not yet known. Therefore, the exact list of parameters to be assessed will have to be determined at a later stage and will be reported in detail in D4.3.2. Highly possible candidates include the quality of the transmitted streams (bandwidth/throughput, depth/analysis/frame rate for video, sampling frequency/bitrate for audio, microphone parameters etc). Less technical but equally relevant parameters include the duration of the show, the time provided for a free tour of the venue with a smart device, the ratio of smart devices per participant etc.
4.4.
Experiment evaluation
The fact that this is an explorative rather than validating experiment (in other words we are looking to see what is right and not just to confirm something that we already know), it is not possible to know beforehand which are the correct results. This of course raises the question of how does one know whether the work has been successful and to what extent. In order to tackle this, we define herein specific success criteria, based on the goals that have been set for the experiment. Moreover, given the complex and challenging nature of the work involved, we define incremental success criteria, so that we do not evaluate merely whether we have been successful but also the degree to which we have been successful. Based on these criteria we shall perform internal (i.e. by FHW personnel) but independent (i.e. by personnel that is not related to the experiment) evaluations of the success of the work at different stage; namely at the two reporting periods for the experiment, the submission times of D4.3.2 and D4.3.3. Based on the four goals mentioned earlier, the following degrees of success are defined.
EXPERIMEDIA
Dissemination Level: PU
Identify differentiation between using and not using the FIRE technologies. In other words, we need to establish that there is a substantial difference for the visitors between the conventional experience currently offered and the one that will be offered in the scope of the experiment. Classify the impact of each component as positive, negative or neutral. This is a more specific version of the previous criterion, as here it is not enough to establish that there is a difference. What is also required is a clear indication regarding whether this difference has an impact that QoE of the visitors and if so whether this impact is positive or negative.
4.4.3. Success
Related goal: Measure impact of FIRE technologies. Success criteria: Quantify and measure QoE. In other words we need to have designed a measure that quantifies QoE and we also need to have applied this to data gathered from the experiment. Correlate measured QoE to utilized FIRE technologies. Moving a step further, to meet this criterion we should be able to identify the contribution of each component in the QoE, so that strategic decisions can be made regarding the directions that warrant further examination.
19
EXPERIMEDIA
Dissemination Level: PU
4.5.
Recruiting
When planning to run experiments with real users in real setting, the recruiting of the experiment subjects is always a complex matter. Of course, Hellenic Cosmos boasts having more than 300.000 visitors per year. Yet, this does not necessarily imply that a similar number of potential experiment participants are available. First of all, for reasons that will be detailed in the following subsection, a very large portion of these visitors are not eligible to participate in EXPERIMEDIA. Furthermore, paying visitors of the venue have to be provided with the experience that has been guaranteed to them and not with an experience that, as part of an on-going experiment, could go either way. Of course this is not to say that it will be difficult to recruit subjects for the experiment. What the above mean is that it is absolutely required to develop a recruiting strategy at this early stage, so that the relevant risks are reduced. For our experiment we have the following plans:
4.6.
Constraints
The main attendants of FHW shows are children and adolescents. In the EXPERIMEDIA experiments only adults will be considered, which creates a question regarding the validity and generality of the results.
20
EXPERIMEDIA
Dissemination Level: PU
For the same reasons, and given the fact that the navigation in a virtual world is a group experience, it will not be possible to apply some monitoring techniques (for example video recording), unless if the experiment is executed only when pure groups of participants are present, i.e. when there is no one present who is either not eligible or has not agreed to participate in the experiment. Regarding the experiment timeline, since a lot of the real exhibits connected to tags in the virtual content are in open areas of the Hellenic Cosmos venue, the weather may have a crucial impact on the execution of the experiment. Therefore, an effort will be made for the actual experiment to be organised and implemented not after October 2012, as by that time the weather conditions will not be stable enough to permit for an unobstructed execution. This is also in line with the deadline foreseen for the first reporting of the experiment in November (D4.3.2). Every effort will be made to gather as much data as possible in this stage, so that the analysis required for the second stage of reporting (D4.3.3) can go ahead regardless of the weather conditions. On the other hand, the execution of the experiment is also constrained in a much more definitive way by the timeline of the development of EXPERIMEDIA components by the technological partners of the project.
21
EXPERIMEDIA
Dissemination Level: PU
5.1.
System architecture
EXPERIMEDIA FACILITIES
Monitoring Service
Social Media
FHW FACILITIES
Experts Toolbox
Navigation Control
Mobile Application
USERS
Expert
Educator
Visitor
Experimenter
22
EXPERIMEDIA
Dissemination Level: PU
In this figure we can see that the museum educator holds the navigation control, which specifies the content that should be displayed to the visitors. Based on this input, the cluster in the Tholos dome processes the loaded 3D world in order to render the according location and viewpoint and display it to the visitors in the dome. This is the part that was already supported before EXPERIMEDIA and it is in fact the typical scenario for the utilization of the Tholos. With the EXPERIMEDIA extensions, the Tholos system, in addition to the local projections, also forwards the rendered stream (actually a downsized and 2D version of it) to the video stream server, which in turn makes it available to the Experts' Toolbox. In this way the experts will be aware of the presented content in real time. The video stream from Tholos is captured from another PC through a video capture card (AVERMEDIA Game Broadcaster HD). At this PC the video is transcoded along with the audio feed from the educators microphone and are transmitted using Adobes Flash Live Encoding to ATOS Server. The Experts will be using ATOS Flash Player to visualize the video stream. The FHW mobile application will integrate the ICCS SocialAuth in order to facilitate visitors access to social media. On the other hand the experts will be accessing the social media via the standard Web interfaces. In this way their communication will be public and available to all interested parties even after the show ends. Furthermore we will integrate the Metaio Free SDK to achieve augmented reality on the mobile devices. Finally, at all times of the experiment execution various system parameters are monitored in an automated manner by the EXPERIMEDIA monitoring service.
23
EXPERIMEDIA Mobile App using the SocialAuth API Social Media Servers
Dissemination Level: PU
Visitor Question
Expert's Answer
Experts Toolbox
Web Browser
Expert's PC Screen
Educator Voice Adobe Flash Media Live Encoder Video Feed from Dome Video Capture Machine Video Streaming Server
Navigation Control
24
EXPERIMEDIA
Dissemination Level: PU
Reality
Figure 5. Flow of information when using the visitors augmented reality mobile application.
5.2.
Requirements
As can be seen from the previous section, the technological infrastructure that will be utilized in this experiment is quite complex. As a result, it is only natural that a number of requirements are associated with its successful implementation. We discuss the most important ones below.
EXPERIMEDIA
Dissemination Level: PU
cluster for the projection and rendering of the VR content respectively, and the network via which these components will interact with the remote EXPERIMEDIA components as well as with the mobile devices. In order for these to be suitable for use in the scope of the experiment, the following requirements are in place: VR content The 3D content currently available is not semantically annotated. The documentation about the content is available and it is quite extensive, but it is not aligned with the content. In other words, although extensive texts are available that discuss each element in the 3D content, there is no automated way to relate each piece of text with a specific item in the content. In order for the tagging part of the experiment to be enabled, these annotations will have to be made available. Tholos dome and cluster The Tholos system has been designed with sole purpose to render 3D worlds in real time. The designed experiment also requires the transmission of synchronized metadata and therefore the Tholos system will have to be altered. This creates the requirement for testing time on the Tholos system, as well as for the ability to support different instances of the system, as the original system will have to remain in operation during the execution of the experiments, since it constitutes a major part of the Hellenic Cosmos operation model and cannot be unavailable during Hellenic Cosmos working hours. Network Network availability is crucial and network speed needs to be sufficient for the described tasks. The specific metrics that constitute sufficient network resources will be specified based on the requirements set by the partners developing the components in question.
26
EXPERIMEDIA
Dissemination Level: PU
The board of experts as described above may consist of many individuals each of which may be located further away in a different location. Since the FHW networking facilities cannot support such bandwidth requirements, the user of a dedicated stream server is necessary. The server will receive the source stream from the FHW facilities and will be responsible for the retransmission to the experts application. The EXPERIMEDIA facility will have to provide this Server, the requirements on the format expected and all the required credentials needed for the establishment of communication. Monitoring service In order to evaluate the user experience and to be able to track detailed info about the network statistics and the quality of service, a monitoring service is mandatory.
5.3.
Content lifecycle
The FHW personnel create all the content that is presented during the show at the Tholos installation. The FHWs personnel also generate the multimedia content presented to the audiences mobile application. The video stream that is presented to the experts is generated dynamically from the FHWs software (EVS Enhanced Visualization Engine). The assets needed for the Tholos Virtual Reality show are stored at servers of FHW. The 3d models that will be presented to the mobile application will be stored on the mobile devices. The Virtual Reality application shown at Tholos is handled from FHWs software while FHWs servers transmit the content. The video stream is delivered to the experts from EXPERIMEDIAs servers.
27
EXPERIMEDIA
Dissemination Level: PU
6.1.
In D5.1.1 a set of ethical principles has been identified for the embedded experiments. They have all been considered in the design of this embedded experiment, as explained in the following.
EXPERIMEDIA
Dissemination Level: PU
As far as the participants of the experiment are concerned, they will have the opportunity to benefit from enhanced services that were previously unavailable. In the more general sense, this experiment will be a first step towards making these technologies a part of the normal operation of the Tholos, so that more people can benefit from them in the future.
6.1.4. Consent
The preparatory phase of the experiment involves the explicit communication of any relevant information to the eligible participants (i.e. what the experiment is about, what it entails, which is their role, etc). Only those eligible participants that have agreed and will sign a note of informed consent will be considered in the experiment. This consent will have a predetermined duration and will be revocable at any time.
6.1.5. Confidentiality
During the experiment only the required data will be gathered, this data will only be made available to the individuals that are needed to process that data and no part of this data will be disclosed to any third parties. Gathered data will be anonymised to the degree that this is allowed by the nature of the experiment. All data will be purged after the analysis has been completed. Copyright IME and other members of the EXPERIMEDIA consortium 2012 29
EXPERIMEDIA
Dissemination Level: PU
6.2.
D5.1.1 has also produced a more detailed set of ethical principles, more customized to the specifics of EXPERIMEDIA and the embedded experiments. These have also been considered and adopted in the design of the experiment, as seen in the following.
6.2.2. Deception
We will never intentionally deceive, mislead or withhold information from participants over the purpose and general nature of the investigation.
30
EXPERIMEDIA
Dissemination Level: PU
6.3.
Regarding the notes in D512, p13, we will not create a schedule of movies on a website, and therefore this is not a relevant concern. Also, the Tholos movie will only be available to those Copyright IME and other members of the EXPERIMEDIA consortium 2012 31
EXPERIMEDIA
Dissemination Level: PU
present and the experts via a URL known only to them. The general public will not have access to it. Regarding the note in D512, p22, FHW holds the copyright that allows for the content to be used in the experiment
6.4.
PIA
32
EXPERIMEDIA
Dissemination Level: PU
7. Risks
As the experiment design is being formulated, and later on as the experiment will be executed, risks will be constantly analysed, evaluated and treated, in the same sense as in D1.1.2. In the experiments risk register, two types of risks are identified: risks for the participants and risks for the experiment itself. In the context of ethical oversight of the experiment of course, it is the former that is of core interest. The current instance of the risk register for the participants contains the following items:
7.1.
ID P1
Description Mobile device damaged during the installation of the mobile application
Probability low
Response avoid
Comment Instead of using the participants own devices, the project will supply the mobile devices that will be used during the experiment. See point P1
P2
Mobile device not compatible with the EXPERIMEDIA software Malicious software installed on the mobile device Participants feel pressured to participate in the experiment
high
high
Execution phase
avoid
P3
low
high
avoid
See point P1
P4
low
high
avoid
This is true particularly for members of groups who visit FHW facilities as part of an agreement between. We will make it clear to all during the preparatory phase that participation is not a requirement and has not affect.
As has already been explained, it is our priority not to put participants in any risk. Therefore participant risks are treated with the AVOID option, where applicable. This often has the result of moving the risk from the participants to the experiment. For example, P1 above was avoided by not using the participants own devices. This has had as a result the generation of risk E3 bellow, where we are considering the case of damaged mobile devices that are owned by FHW and are needed in order to run the experiments. Copyright IME and other members of the EXPERIMEDIA consortium 2012 33
EXPERIMEDIA
Dissemination Level: PU
7.2.
ID E1
Description Required experiments cannot be run because 90% of thee visitors are children and adolescents and the agreed Ethical Oversight Measures state we will not be dealing with children
Probability high
Response reduce
Comment The core concern is that the eligible participants that will be visiting the Hellenic Cosmos facilities will not be sufficient in number at the time of the execution of the experiment. In order to avoid this we are already 1) organizing parallel activities that will ensure the presence of wide numbers of adults and 2) we are working closely with universities in order to organize undergraduate and graduate student visits that will be combined with the experiment. Via close coordination with the consortium partners
E2
Required EXPERIMEDIA components not available on time or not compatible with the FHW facilities Damaged mobile devices
low
high
Experiment design
reduce
E3
low
high
Execution phase
fall-back
We will run the experiment with as many working devices are available. The mobile devices only affect one part of the experiment, and therefore the feedback of participants that are not given a device is still relevant We are planning to execute the experiments at a time that the weather is typically suitable
E4
low
low
Execution phase
reduce
34
EXPERIMEDIA
Dissemination Level: PU
8.1.
Current status
Although the current document has been continuously updated, and some further elements remain to be determined at a later time (e.g. the exact parameters to be monitored) the actual implementation has started much earlier, focusing on the elements that were already clarified. As a consequence, the following results can already be reported:
8.1.2. Streaming
The connection with the ATOS server has been tested successfully. It has been determined that the Hellenic Cosmos uplink is sufficient to cover the needs of the experiment.
35
EXPERIMEDIA
Dissemination Level: PU
8.2.
Future plans
Given the heavy constraints imposed on the experiments scheduling by the predicted fall and winter Athens weather, it is crucial that the work continues to progress rapidly. Starting from the November deadline to deliver the first results of the experiment and moving backwards, our plans include: Mid November: completion of the report on the experiment. End of November: end of data analysis Mid October: completion of experiment execution required for the first report Start of October: start of data analysis, in parallel with continued experiment execution Mid September: completion of all development work, start of experiment execution Start of September: work on integration, early tests of applications Mid August: all internal and external components should be available The plans for the second stage of the experiment (the one leading to the final report) have not yet been established, as the experience from the execution of the first part will be an essential factor.
36
EXPERIMEDIA
Dissemination Level: PU
9. Conclusion
We have described in this document the current status and future plans for the driving experiment based at the Foundation for the Hellenic World in Athens. The dual-purpose of the experiment as both a mechanism for helping the EXPERIMEDIA project and as a piece of research in itself has been discussed. The projects architectural blueprint, methodological guidelines and ethical oversight principles have all been taken into account. This report will be followed by a progress report, including intermediate results, at the end of November.
37