Music, handicap and IoT

Publié le ven 9 Oct 2020

French version

Automatic measurement of sensory disorders related to autism.

Towards an enhanced musical instrument.

Project objective: Automatic measurement of sensory disorders by connected musical instruments (IoT)

Solution for automatic measurement of sensory disorder profiles with children having an Autism Spectrum Disorder (ASD). Measurements are made using musical instruments during musical workshops held at the Annecy Regional Radiation Conservatory (CRR). We also want to define the augmented and adapted instrument for everyone, offering an interface of the musical instrument adapted to the sensory profile of the person with a handicap.

kewords – Application fieldsMéthodologyKnow-howScientific problematicResultsPartnerships – Contacts


Psychophysical or sensory measurements, IoT Internet of Things, information fusion, sensory disorders, sensory profile.

Application filds

  • Sensory profile measurement
    • measurement during music workshops at the Annecy Regional Conservatory with children with autism spectrum disorders
    • variation over time of these profiles; measurement of the effects of music workshops.
  • Technology parts:
    • music instrument instrumentation,
    • actimetry by algorithm from sensor data placed in musical instruments,
    • identification of profiles and their evolution.


  • modalities’ identification to be measured resulting from the expert knowledge in the study (practitioners, experts in the field)
  • musical instruments instrumentation by adding sensors adapted to the modalities.
  • Fusion of multi-modal, multi-source information by IoT.
  • Management of imperfect information: imprecise, uncertain, incomplete or even contradictory.
  • Determination of appropriate dashboards: practitioners, family.
  • Experimental validation.
  • Privacy and health data regulation where applicable.Instrumentation des instruments de musique par ajouts de capteurs adaptés aux modalités.

Know-how – LISTIC part

  • Identification techniques for sensory profiles based on musical activity measurements:
    • information fusion and associated theories (possibilist theory, probabilist theory, evidence theory).
    • Learning, representation, and use of high-level information
    • Uncertainty managment and data incompletness management,
  • Ability to master the main Machine Learning techniques of Artificial Intelligence, which makes it possible to identify the one adapted to the problem to be addressed
    • deep learning
    • data mining
    • supervised and unsupervised classification methods (FCM, KNN, Random Forest, SVM)
  • Rapid prototyping for sensitive objects (embedded sensors) and connected objects (IoT). Mastery of embedded tools (nanocomputers, microcontrollers).
  • Rapid prototyping of interconnection applications for connected and intelligent objects.

Scientific problematic

  • Measurement of human activity (actimetry) from sensors preferably non-supported and non-intrusive.
  • Modeling and identification of human situations and activities, management of related imperfect information
  • Explicability : how to bring semantics and meaning to the measurement.


The first results have allowed us to identify the sensory modalities involved in sensory disorders that could be measured using sensors placed in musical instruments. These results were presented at an international conference with a reading committee:

Musical instruments for the measurement of autism sensory disorders
E Benoit, S Perrin, S Donnadieu, C. Dascalu, G. Mauris, J Favory, C Dautremer
Joint IMEKO TC1-TC7-TC13-TC18 Symposium 2019, Jul 2019, Saint Petersburg, Russia


You wish to meet us and/or to study an R&D partnership:

Contacts :