Deliverable No: D7.3 Final Evaluation Report
Project Title:i-Treasures: Intangible Treasures – Capturing theIntangible Cultural Heritage and Learning the Rare KnowHow of Living Human TreasuresContract No:FP7-ICT-2011-9-600676Instrument:Large Scale Integrated Project (IP)Thematic Priority:ICT for access to cultural resourcesStart of project:1 February 2013Duration:48 monthsDeliverable No: D7.3Final Evaluation ReportDue date ofdeliverable:31 March 2017Actual submissiondate:31 May 2017Version:8th version of D7.3Athanasios Manitsaris (UOM), Anastasios Katos (UOM),Eleni Katsouli (UOM), Maria Lazaridou (UOM), DimitriosTheodorou (UOM), Evanthia Kalampoka (UOM), LeontiosHadjileontiadis (AUTH), Vasileios Charisis (AUTH), SteliosHadjidimitriou (AUTH)Main Authors:Project funded by the European Communityunder the 7th Framework Programme forResearch and Technological Development.
Deliverable No: D7.3 Final Evaluation Reporti-Treasures ICT-600676Project ref. numberICT-600676Project titlei-Treasures - Intangible Treasures – Capturing theIntangible Cultural Heritage and Learning the Rare KnowHow of Living Human TreasuresDeliverable titleFinal Evaluation ReportDeliverable numberD7.3Deliverable versionVersion 8Previous version(s)Version 7Contractual date of delivery31 March 2017Actual date of delivery31 May 2017Deliverable filenameD 7 3 Deliverable v8.docxNature of deliverableRDissemination levelPUNumber of pages91 (plus Appendices)WorkpackageWP 7Partner responsibleUOMAthanasios Manitsaris (UOM), Anastasios Katos (UOM),Eleni Katsouli(UOM), Maria Lazaridou (UOM), DimitriosTheodorou (UOM), Evanthia Kalampoka (UOM), ParaskeviKritopoulou(UOM), Christina Volioti(UOM), LeontiosHadjileontiadis (AUTH), Vasileios Charisis (AUTH), SteliosHadjidimitriou (AUTH), Nikos Grammalidis (CERTH),Kosmas Dimitropoulos (CERTH), Filareti Tsalakanidou(CERTH),Giannis Chantas (CERTH), Francesca MariaDagnino (CNR), Francesca Pozzi (CNR), Lise CrevierBuchman (CNRS), Pierre Roussel (CNRS), Sohaib Laraba(UMONS), Sotiris Manitsaris (ARMINES), Alina Glushkova(ARMINES), Marius Cotescu (ACAPELA)Author(s)EditorAthanasios Manitsaris (UOM)EC Project OfficerMarcel WateletAbstractThe purpose of the final evaluation report is to access thetechnical performance of the i-Treasures systems and itsmodules in laboratory conditions and to evaluate theperformance of the i-Treasures platform and itscomponents, and the use-cases in real conditions.KeywordsTechnical performance assessment, Web-Platformperformance evaluation, Use-case evaluation, Priority forcomponents improvementFilename: D7.3 deliverable v8.docxPage 2 of 91
Deliverable No: D7.3 Final Evaluation Reporti-Treasures ICT-600676SignaturesWritten byResponsibility- CompanyDateAnastasiosKatos (UOM),AthanasiosManitsaris(UOM)Responsible of D7.3 (UOM)11/5/2017AthanasiosManitsaris(UOM)Responsible of D7.3 (UOM),Task T7.3 Leader23/5/2017LeontiosHadjileontiadisWP7 leader (AUTH)23/5/2017NikosGrammaliisCoordinator (CERTH)30/5/2017KosmasDimitropoulosQuality Manager (CERTH)30/5/2017Verified byApproved byFilename: D7.3 deliverable v8.docxPage 3 of 91
Deliverable No: D7.3 Final Evaluation Reporti-Treasures ICT-600676Table of Contents1.Executive summary . 72.Introduction . 83.Methodology . 94.Laboratory Testing of Modules . 124.1Introduction . 124.2iTreasuresEvaluator tool . 124.3Laboratory testing results . 155.Technical Assessment of the system . 215.1Methodology . 215.2Overall Assessment of the system . 215.2.1Introduction . 215.2.2Collecting the data . 215.2.3Examining the data . 225.2.4Evaluation of Results . 225.3Assessment of the Main Technical Indices . 235.3.1Introduction . 235.3.2Collecting the data . 245.3.3Examining the data . 255.3.4Evaluation of Results . 256.Evaluation of the Web Platform's performance . 276.1Introduction . 276.2Methodology . 276.3Results . 286.4Evaluation of Results . 317.Use Cases Evaluation . 337.1Contemporary Music Composition use case evaluation . 337.1.1Introduction . 337.1.2Methodology . 337.1.3Results . 347.2Tsamiko dance use case evaluation. 367.2.1Introduction . 367.2.2Methodology . 377.2.3Results . 387.2.4Evaluation of results . 397.37.3.1Pottery use case evaluation . 40Introduction . 40Filename: D7.3 deliverable v8.docxPage 4 of 91
Deliverable No: D7.3 Final Evaluation Reporti-Treasures ICT-6006767.3.2Methodology . 417.3.3Results . 427.3.4Evaluation of results . 437.4Salsa dance use case evaluation . 447.4.1Introduction . 447.4.2Methodology . 457.4.3Results . 467.4.4Evaluation of Results . 477.5Walloon dance use case evaluation . 487.5.1Introduction . 487.5.2Methodology . 497.5.3Results . 507.5.4Evaluation of Results . 527.6Human Beat Box use case evaluation . 537.6.1Introduction . 537.6.2Methodology . 537.6.3Results . 547.6.4Evaluation of results . 567.7Byzantine use case evaluation . 577.7.1Introduction . 577.7.2Methodology . 577.7.3Results . 597.7.4Evaluation of Results . 607.8Canto a Tenore case evaluation. 617.8.1Introduction . 617.8.2Methodology . 627.8.3Results . 637.8.4Evaluation of results . 657.9Cantu in Paghjella use case . 677.9.1Introduction . 677.9.2Methodology . 677.9.3Results . 687.9.4Evaluation of Results . 707.10Generic Game Framework case evaluation. 717.10.1Introduction . 717.10.2Methodology . 727.10.3Results . 73Filename: D7.3 deliverable v8.docxPage 5 of 91
Deliverable No: D7.3 Final Evaluation Report7.10.4i-Treasures ICT-600676Evaluation of Results . 758.Overall conclusions and proposals for the future . 778.1Conclusions with respect to laboratory testing . 778.2Findings with respect to the Technical Evaluation . 778.2.1Overall Assessment of the System . 778.2.2Assessment of the Main Technical Indices . 788.3Findings with respect to the Web-Platform evaluation . 788.4Findings with respect to the Use-Cases evaluation . 798.4.1Contemporary music composition use case . 798.4.2Tsamiko dance use case. 808.4.3Pottery use case . 818.4.4Salsa dance use case . 818.4.5Walloon use case . 828.4.6Human Beat-Box use case. 838.4.7Byzantine Music use case . 838.4.8Canto a Tenore use case . 848.4.9Cantu in Paghjella . 858.4.10Generic Game use case. 868.5Overall Performance of i-Treasures . 868.6Limitations and further research . 888.7Conclusions . 88References. 91Filename: D7.3 deliverable v8.docxPage 6 of 91
Deliverable No: D7.3 Final Evaluation Report1.i-Treasures ICT-600676Executive summaryThe purpose of the final evaluation report is twofold. First, to assess the technicalperformance of the final version of i-Treasures systems and its modules in laboratoryconditions through a set of trials and tests. In this last report, the TechnicalAssessment of the system is also enriched with a survey based on two structuredquestionnaires, one for the non-functional and one for the functional requirements.Second to evaluate the performance of the i-Treasures platform and its componentsin real conditions with real users, both experts and learners through the organizationof trials and tests for all four ICH use cases. These activities took place duringspecial demonstration events, running in Greece, France, Italy and Belgium(described in detail in the deliverable D6.2 “Demonstration report”).For assessing the technical performance of the i-Treasures systems the fisEval toolis used as well as in the first report (D7.2), to fuse the various independent moduleassessment indices engaged and derive an overall performance index for eachmodule, which characterizes the quality of the module.For evaluating the overall technical performance, the performance of the i-Treasuresplatform and its components and the ICH use cases, the same methodology with theone applied in D.7.2 (structural equation modeling) is used. The analysis of therespondents’ results to questionnaires was based on descriptive statistics epriorityforchanges/amendments that should be imposed to the components of the i-Treasuresplatform and the use cases in order to improve their performance.Overall, the results with respect to the laboratory testing of modules, the technicalperformance, the web-platform and use cases evaluation were promising. The overallperformance indices acquired during the second phase of laboratory testingexceeded those of the first phase, and all modalities surpassed the baseline criteria.Comparing the overall web-platform performance score with the overall score of lastyear’s evaluation, we see that the overall evaluation of the system has beenincreased by 7.6 percent. This notable increase was due to the fact that the prioritiesfor improvement of components proposed in last year’s evaluation were followedduring the second development phase.Finally, it is concluded that the i-Treasures system is perceived by individuals to be ofhigh quality (score 4.15 in Table 37) in terms of Web-Preservation & Transmission ofICH, thus contributing at an empirical level.Filename: D7.3 deliverable v8.docxPage 7 of 91
Deliverable No: D7.3 Final Evaluation Report2.i-Treasures ICT-600676IntroductionWP7 “Technical Assessment and Evaluation” deals with the evaluation of theproposed system and its functionalities in terms of complying with the userrequirements and expectations. More specifically, the objectives of WP7 for the lastreporting period are as follows: Assessment of the technical performance of the i-Treasures system and itsmodules during the second development cycle. The assessment is performedin laboratory conditions through a set of trials and tests of the system andmodules functionalities. Evaluation of the performance of the final version of the i-Treasures platformand its components in real conditions with real users. Organization oftrials/tests for all four ICH use cases.Specifically, the last phase of Task 7.2 includes the assessment of the technicalperformance of the final version of i-Treasures system and its modules. Theassessment is performed in laboratory conditions through a set of trials and tests ofthe system and modules’ functionalities, based on the assessment plan presented indeliverable D7.1 using the FisEva tool, already developed in the first phase (D7.2).The laboratory testing of technical performance is critical in order for an optimizedimplementation of the technological modalities, such as sensors, processingalgorithms and interfaces, to be achieved. The modules assessed are: a) Body &Gesture Recognition (for Tsamiko, Salsa and Walloon dances), b) secase,c)Electroencephalography (EEG) Analysis, d) Body & Finger Recognition for Potteryuse case, e) Text-to-Song, f) Vocal Tract Sensing & modeling, g) Facial expressionanalysis, and g) Multimodal Data Fusion and Semantic Analysis. In this vein, varioustechnical performance categories and indices are used specifically for each moduledue to the great variance of their characteristics. Detailed description of theassessment indices can be found in the deliverable D7.1 "Assessment Plan".Furthermore, for each assessment index a benchmark value is introduced that actsas a reference level in order to assure the quality of the modules.Within the last phase of Task 7.3, evaluation of the performance of the final versionof the i-Treasures platform and its components in real conditions with real users, bothexperts and learners, took place. Tests were realized for selected sub-use cases ofthe four use cases: Salsa dance, Tsamiko dance, Walloon dance, Cantu in Paghjella,Canto a Tenore, Human Beat Box singing, Byzantine music, Contemporary MusicComposition and Pottery. Towards this direction, the innovative evaluationmethodology of structural equation modeling is used in order to trace the “good” and“poor” parts of the system. Additionally, the contribution of each component of thesystem towards the performance of other components, and, in turn, towards to theoverall performance of the system is found.Filename: D7.3 deliverable v8.docxPage 8 of 91
Deliverable No: D7.3 Final Evaluation Report3.i-Treasures ICT-600676MethodologyIn evaluating the quality of the Web Platform and the Use-Cases the StructuralEquations Model (SEM) methodology, or the Latent Variables Model, is used [8].The methodology is described in detail in the First Evaluation Report (D7.2).The instruments for collecting data were developed as structured questionnaires,based on the operational models proposed by experts and the people involved ineach case. The survey instruments for the Technical Assessment of the System arethe following structured questionnaires:1. The questionnaire for the overall Assessment of the system. (Appendix XIII)2. The questionnaire for the Assessment of the Main Technical Indices(Appendix XIV)The survey instruments for Web Platform and use cases evaluation are the followingquestionnaires:1. Questionnaire for the evaluation of the Web Platform and ContemporaryMusic Composition use case (APPENDIX III)2. Questionnaire for the evaluation of t
Deliverable title Final Evaluation Report Deliverable number D7.3 Deliverable version Version 8 Previous version(s) Version 7 Contractual date of delivery 31 March 2017 Actual date of delivery 31 May 2017 Deliverable filename D_7_ 3_Deliverable_v8.docx Nature of deliverable R Dissemination level PU Number of pages 91 (plus Appendices)
Deliverable title: Integration Test Report Due date of deliverable: M27 Actual submission date: 11-05-2012 (M29) Resubmission date: 21-08-2012 (M32) Start date of project: 1 January 2010 Duration: 30 months Organisation name of lead contractor for this deliverable: Fraunhofer HHI Name of the lead author for this deliverable: Yago Sánchez
Deliverable N D 4.1 Deliverable title ADAPTATION OF INNOVATIVE METHODS IN SCIENCE EDUCATION (incl. Annex “Teaching Materials”) Due date of deliverable: Month XII Actual submission date: November 2010 Start date of project: 01/11/2009 Duration: 45 months Name of Coordinator: Austrian Institute of Ecology
Deliverable Grant Agreement Number 600680 Full Project Title ICT Transfer Concept for Adaptation, Dissemination and Local Exploitation of European Research Results in Central Asia’s Countries Project Acronym eINTERASIA Title of Deliverable A generic IT transfer concept Deliverable Number D 2.2.2 Work‐package WP2
H2020-ICT-2016-1-732340 1 Deliverable Title: Deliverable D3.1 Report on Historical Data as Sources Deliverable Date: 30/03/2018 Version: 1.0 Project Acronym: K-PLEX Project Title: Knowledge Complexity Funding Scheme: H2020-ICT-2016-1 Grant Agreement number: 732340 Project Coordinator: Dr. Jennifer Edmond (edmondj@tcd.ie) Project Management
1 Deliverable Deliverable Number D24.6 Deliverable Title White paper on suitability of HNScienceCloud and European Open Science Cloud for synchrotron and FEL applications Lead Beneficiary PSI Authors A. Ashton (PSI), R.Dimper (ESRF), A.Götz (ESRF), D.Salvat (ALBA), F.Schlünzen (DESY)
Part 1: GIS Utility Risk Identification, which addresses project Deliverable 5A Part 2: Risk Mitigation and Risk Response Planning, which addresses project Deliverable 5B. NOTE: Part 1 (Deliverable 5A) was delivered on March 14, 2005. This report includes a finalized Deliverable 5A and th
Stage Gate Processes 1. Scope 2. Select Deliverable 1 Deliverable 2 Deliverable 3 Deliverable 4 LINKS Policies Procedures External Standards Gate Gate Gate Gate Lessons Learned Incidents Actions AAR MoC / Dispensations Templates Examples Metadata Naming Standards Publishing Archit
1 Advanced Engineering Mathematics C. Ray Wylie, Louis C. Barrett McGraw-Hill Book Co 6th Edition, 1995 2 Introductory Methods of Numerical Analysis S. S. Sastry Prentice Hall of India 4th Edition 2010 3 Higher Engineering Mathematics B.V. Ramana McGraw-Hill 11 th Edition,2010 4 A Text Book of Engineering Mathematics N. P. Bali and Manish Goyal Laxmi Publications 2014 5 Advanced Engineering .
