Usability assessment

ABC Project - Test with users from Trabajo Juanma on Vimeo.

The head-mounted inertial interface for users with CP has been validated using the ISO9241-9. The method is based on Fitt's law, which is an empirical model of human motor performance for aimed movements. Head motion is recorded in a series of reaching tasks. Values of R-squared above 0.9 are indicators of a strong correlation of those motion patterns with the linear model proposed by Fitts. The interface was validated for healthy subjects in a control group and the experiments were repeated in a group of volunteers with CP. The range of values of R-squared calculated for the CP group (0.855+/-0.105) are related to the heterogeneity of the user’s muscular tone (hypertonia, hypotonia and dystonia) and the presence of involuntary movements. Tests show that the correlation is poorer for certain profiles of motor impairment, mainly hypotonia. The results of this study show that the movements registered in users with the inertial technology can be modeled by Fitt's law.

Figura13

Figure 13. Measure of Throughput (TP) for the two groups. The box plots represent the values measured for each task during the work sessions of cerebral palsy (CP) and healthy subjects (HS).

As expected, healthy subjects in the control group were considerably faster and more efficient than those in the cerebral palsy group during the reaching tasks proposed. Despite the difference in the task performance, average motor performance of both groups was proven to fit the linear model proposed by Fitt's law, with values of R2 above 0.9. Four of the six members of the cerebral palsy group showed good correlation with Fitt's law. The worst match corresponded to users with hypotonic cerebral palsy. Although more specific tests should be run with users with hypotonic CP in order to establish whether their head movements follow Fitt's law, our results indicate that regardless of the task, motor performances in both control and experimental groups showed high correlation with Fitt’s law. We conclude that cursor’s movement with the head-mounted interface can be well modeled by Fitt's law in healthy participants and with small limitations (e.g. hypotonic CP) also in CP users.

Usability of the Health module

Health Monitoring System (HMS) for more severely affected children and Physical Activity Monitoring (PAM) for less affected children, have been tested in real contexts.

PAM has been used to provide quantitative data characterizing two different methods of rehabilitation for children with DCP. PAM was formed by five wireless triaxial accelerometers fixed with biadhesive tape on the skin of children on the following body segments: posterior part of forearms, posterior part of shanks, and posterior part of lower trunk in correspondence of the centre of mass (L2-L3). Acceleration time-courses were recorded during 5 continuous minutes of video-game based therapy and 5 minutes of conventional therapy, and the Root Mean Square (RMS) of acceleration was computed for each device on the signals.

Eight children with cerebral palsy (mean age: 6.50±1.60 years) have been tested in clinical settings (the Department of Child Neurorehabilitation of Fondazione Santa Lucia). The results of this study have been published on the European Journal of Physical and Rehabilitation Medicine.

Usability of the Emotion module

In the frame of the development of the affective computing components the usability of the emotion detection system and the emotion management system were tested in three rounds with users.  The emotion detection system with sensors was tested in two rounds to assess the usability and the form of the sensors and additionally to collect training data to be used to build a training model for the algorithms. In the first round 13 users were attending the experiments. We used the training data from the users and built a model based on Support Vector Machines using classes supported in Weka API. Weka is a suite of machine learning software written in Java. To evaluate the performance of the built model, 10-fold cross-validation was used with several parameters of precision, recall and percent of correctly classified. Necessary adjustment of parameters were derived from these models.

Final user trails of the emotion detection system were conducted with 15 users. The tests were conducted from different perspectives mainly in comparison to international affective databases, as the IADS-2 database, to show the usability and validity of the system. Here a compliance level of the model up to 93,3 % was calculated.

The emotion management system was as well tested on its usability. The data was collected through two groups of users 15 participants in total. The first group contained 10 users as students from several university departments with different computer skill and knowledge level, while the second group contained 5 users who were relatives of DCP user. Experiments in second case took part in Clinical Laboratory of Experimental Neurorehabilitation of project partner in Rome. Test participants had to use the system by following a predefined usage scenario. After that they had immediately to answer two questionnaires. All users could use the system without significant issues. The results of the questionnaires revealed high acceptance of the system and its functionalities in the different question groups ranging from 69% - 95%.

Usability of the Communicator

The assessment tests of ABC functional prototype were carried out with DCP people, using three different input types and two ways of doing the click: by electromyography (EMG) and by a dwelling time. Two different ways of selection have been used: automatic scan and the IMU placed in the head.

The assessment has been done in two validation stages (DCP users with information about the interfaces and communicator (first tests), and DCP users trained with the interfaces and the communicator (second tests).

Six users participated in the study: 2 men and 4 women aged between 25 and 45 years, all of them had cerebral palsy and were users of other kind of communicators.

Two  subjects were assigned to each mode of selection: automatic scanning + EMG, IMU + EMG, IMU + dwelling time.

Seven users participated in a further iteration testing : 4 men and 3 women, aged between 25 and 45 years, all of them had cerebral palsy and were users of other kind of communicators.

Condition

Participants

Participation

automatic scanning + EMG

2 men, 1 woman

Already included in first trials

IMU + EMG

1 man, 1 woman

New participants

IMU + dwelling time

1 man, 1 woman

Already included in first trials

After the second tests, we concluded that people needed also training for better performance of the system. Real Context Test

Three users participated in the real context tests with the communicator: 1 men and 2 woman, aged between 32 and 39 years, all of them had cerebral palsy and were users of other kind of communicators. One of them became stressed and decided not to continue the test.

In these tests, users had the communicator during 3 months and they were instructed for using it daily.

Besides the free use, 8 real life situations have been defined to be fulfilled by the users.

Results confirmed the interest of the functionalities of the ABC communicator for the potential users.

These are the features best ranked by the users after real test conditions:

  • The communication and emotion management capabilities.
  • Health alerts and management.
  • Free open architecture for library design, easily adaptable to the users own language (pictographic, logographic, and alphanumeric) and the possibility to design different screen configurations to optimize the space available.
  • Easy-to-use and easy-to-learn capabilities of the communicator. Concept selection is a process based on a free search of some pictograms and users found it a “natural” way of interaction..

However, in this real context test has been detected some potential improvements (Figure 14).

Figura14

Figure 14. Difficulties found during the user’s experience are related to the system assessment completed by the users.