Exploiting the best sensory modality for learning arithmetic and geometrical concepts based on multisensory technology and serious games.

Technology is starting to be widely used in classrooms. However, in schools, the visual channel is often the one most frequently exploited in teaching. In particular, to date no technology has exploited ICT for multisensory integration in teaching. Yet, recent results from psychophysics and developmental psychology [Gori et al. 2008, 2010, 2012, 2014] show that children have a preferential sensory channel to learn specific concepts and that the visual signal in not always the most powerful channel.

Starting from these recent advances in educational literature, weDRAW proposes that different sensory-motor signals be used (audio, haptic, visual and movement) to teach new concepts to primary school children, adopting art as a source of inspiration for developing novel multimodal, embodied and enactive teaching and learning paradigms and technologies.

The main idea behind weDRAW is to develop a technology that associates music with numeracy and drawing with geometry in new ways. In particular, weDRAW shows that it is possible to learn arithmetical concepts from rhythm and music and geometrical concepts from body movement and drawing. The project creates a new understanding of the association of music and arithmetical concepts and of drawing and geometrical concepts, and exploits such an understanding in technologies for learning in an innovative way to achieve a “deeper learning of Science and Mathematics combined with Arts” that improves the learners’ creative capacities.

The new technology, including serious games platforms, is developed in a way which involves teachers in the learning process, having the final goal, on one hand, of improving the creative capacities of children and, on the other hand, of supporting the teacher’s role in the learning process. The main idea behind the project is to create a four-phase approach in which the teacher is directly involved in the evaluation of the modules to be used for each specific learning aspect. Starting from this initial evaluation, the teacher can identify the best modality to teach the specific concept to each child in a personalised way and use this modality to teach further concepts. The interaction between students who perform the task, by using different modules (visual, audio or haptic), provides a unified multisensory concept of the circle that will help the child to internalise it better. At the end of the process, the learning is evaluated by the teacher.

A major goal and output of this project is to apply the proposed multisensory approach and technologies to two specific populations, namely: visually impaired and dyslexic children. weDRAW expects to improve the deficits of these two groups, as has already been done in previous activities with visually impaired children (e.g. see Whereas dyslexic children encounter problems with rhythm, visually impaired children have problems with space and geometry.

weDRAW develops and validates two multisensory technology prototypes and three serious games to learn numbers and geometry starting from music and drawing. By adopting such a multisensory learning approach and technology, typically developing children will exploit their preferred sensory channel complemented by the other ones, whereas visually impaired and dyslexic children will supplement the missing or impaired preferred sensory channel with the other ones.

Moreover, the adoption of such a multisensory learning ecosystem contributes to an early diagnosis of dyslexia and other learning difficulties. The two prototypes and the three serious games developed by the project provide such evidence.

Basic information

Country: France, Greece, Ireland, Italy, United Kingdom

Coordinator: Fondazione Istituto Italiano di Tecnologia, Italy

Programme: Horizon 2020

Project Acronym:

Target groups: primary school students, teachers

Topic: Computer science

Start year: 2017

End year: 2018


Contact person: Monica Gori Email Monica.Gori (at)

weDRAW is an EU-H2020-ICT-funded project focusing on multisensory technologies for teaching mathematics to primary school children. The final goal is to open a new teaching/learning channel based on multisensory interactive technology. The project represents an ideal testbed to assess the support multisensory technology can provide to learning mathematics. Psychophysical data is collected to define the best modality to perceive multisensory signals in children at different ages. Psychophysical results are associated with pedagogical data and used as a baseline for the development of new technological solutions and serious games.

Multisensory technologies are ideal for effectively supporting an embodied and enactive pedagogical approach, exploiting the best-suited sensory modality to teach a concept. This represents a great opportunity for making technologies which are both grounded in robust scientific evidence and tailored to the actual needs of teachers and students. Based on experience in technology-enhanced learning projects, weDRAW proposes five golden rules it deems important for seizing this opportunity and fully exploiting it.

  • Ground technology in pedagogical needs: Multisensory technology should provide support to teach those concepts that teachers consider the most suited for technological support. These could be concepts that are particularly difficult to understand for children, or concepts that may benefit from communication through a sensory modality other than vision. In a survey weDRAW conducted on over 200 mathematics teachers, it was surprising to find that more than 75% of teachers agreed on the same concepts.
  • Ground technology in scientific evidence: Multisensory technology should leverage the sensorial, perceptual, and cognitive capabilities children have according to scientific evidence. Concretely, for example, a technology able to detect specific motor behaviours in a target population of children (e.g., primary school) makes sense only if scientific evidence shows that children in the target population can actually display such behaviours. The same holds for feedback: multisensory technology can provide a specific feedback (e.g., based on pitch), if (i) children can perceive it (e.g., they have developed perception of pitch), and (ii) an experimentally proven association exists between feedback and the concept to be communicated (e.g., the association between pitch and size of objects).
  • Make technology flexible and customisable: This is a typical goal for technologies, but it assumes here a particular relevance. It means assessing (i) which is the preferred sensory modality for a child to learn a specific concept, and (ii) whether specific impairments require exploiting particular sensory modalities. As a side effect, technology may help with screening for behavioural problems and addressing them. For example, recent studies show that musical training can be used as a therapeutic tool for treating children with dyslexia.
  • Emphasise the role of the teacher: Technology does not replace the teacher. Rather, the teacher plays the central role of mediator. In an iterative methodology, the teacher first chooses a concept to teach; then, following an initial evaluation phase, the teacher identifies the best modality to teach it to each child, and personalises technology to exploit the selected modality; the teacher finally evaluates the outcomes of the learning process and adopts possible further actions. Moreover, design, development and evaluation of technology should obviously be carried out in the framework of a participatory design process involving both teachers and students.
  • Promote cross-fertilisation with arts and human sciences: Taking a rigorous scientific approach should not exclude the opportunity of getting inspiration from the human sciences, and in particular from the arts. Initiatives bear witness to the increased awareness of how art and science are two strongly coupled aspects of human creativity, as well as the impact of art on scientific and technological research. In the case of multisensory technology for education, the extraordinary ability art has to convey content by means of sound, music and visual media provides a significant added value.


If you know of European or national project in STEM education, please let us know. 

Submit project


Read more on what Scientix offers to science education projects

For project managers:

connect with Scientix!

For project managers:

Request support for your project from Scientix Teacher Ambassadors!