Commercially feasible product ‘A’: IMU Head-Mounted HCI (NR1 + NR2 + NR3)

 

Description

Communication support platform formed by an adapted set of software and hardware, including a head component integrating the system. This device allows the user to control different common interactive devices (e.g. personal computers, tablets, smartphones) through the movement of his/her head.

How will it be used?

For what purpose?

As a high adaptable, easy configured and low cost device  is addressed to people with upper mobility restrictions  by improving the performance of tasks related to communication, environment control, tele-care and leisure. In this sense, the IMU solution would also contribute to improve the cognitive skills of this population group.

This solution has been conceived as one of the tools to facilitate the accessibility of users with upper mobility restrictions  to IT resources,  increasing their personal autonomy and,  their quality of life.

Current technology

Commercialized devices which are used for the same purposes can be considered as substitutes, but also supplementary products of the IMU solution if they are all employed to better fit the particular needs of the user.

Relevant substitute/supplementary products currently in the market are the following: eye-tracking, single-switch access, sip-and-puff switch, voice recognition software, mouth stick, adaptive/special keyboard, head wands, and oversized trackball mouse. The best alternatives compared to the new IMU peripheral use to be very advanced technologies, but neither widely used nor attainable because of their cost and noteworthy training required.

 

Commercially feasible product ‘B’: BCI software (NR4)

 

Description

BCI algoritms for the adaptation to DCPs the current paradigms of BCI, stressing on motor imagery and P300 paradigms.

The feature-extraction process include: (1) Data Acquisition; (2) Advanced Signal Processing; (3) Online Automated Artefact reduction (suppression); (4) Canonical Correlation Analysis (CCA) for system classification; (5) Pattern recognition of feature extraction (adaptation in real time); (6) Algorithms for facilitation of fine motor control (command interpretation). Tablet-based feedback, data storage and event logging set of tools have also been developed.

How will it be used?

For what purpose?

BCI software is based on the use of BNCI algorithms for Human Control Interface (HCI) in real out-of-the-lab contexts. EEG feature-extraction has been used for BCI control of the Communicator-based ABC System (NR11). Processing algorithms for EEG feature extraction and classification have been modified to take into account pathological patterns of CP users, such as spasticity or involuntary movements.

It allows real time adaptation and minimizes the communication latency. In addition, as a consequence of the user-centered BCI design, it has been obtained a more usable and feasible system making easy learning of BNCI use. The prominent performance of the BCI software has been achieved from the optimization of feature selection according to individual characteristics of DCP users, and from the appropriate utilization of an alternative non-EEG HCI method, i.e. the Head IMU solution.

Current technology

State-of-the-art automated artifact removal method (named LAMIC) is not able to perform the expected operation. Despite there exist available standard BCI processing tools (including phase synchronization algorithms in terms of speed), these tools are not adapted to the particular characteristics of CP users that affect the usability of EEG signals, like involuntary movements and alterations in Central Nervous System.

 

Commercially feasible product ‘C’: Affective Module (NR5 + NR6 + NR9)

Description

The Affective Module (AM) is an emotion detection system and emotion management system. The module measures physiological parameters to assess the emotional state of users, through a Support Vector Machine (SVM) classifier.

How will it be used?

For what purpose?

The module is capable to detect the affective state of DCP users, allowing them and their relatives to manage emotional conditions through the configuration of events. Namely, it includes one application for helping DCP user to express, understand and control their emotions. It is portable and detects the emotional state through physiological response and movements, distinguishing four basic moods: boredom, enjoying, frustration and stress.

The purpose of this system is to increase self-awareness of emotional state and provide some tools to help the users to manage their emotional state in different contexts and environments. Furthermore, in order to keep privacy protection of end users, they have as well total control of their personal data (e.g. users can decide to switch on/off the affective module to show their emotions).

 

Commercially feasible product ‘D’: Health Module (NR7 + NR8 + NR9)

Description

The Health Module (HM) is formed by two wearable devices connected via bluetooth to a PC-based software platform, which are capable to measure physiological parameters in real time. In particular, (1) a chest band that registers ECG, ,respiration and activity measurement; and (2) a bracelet to measure the galvanic skin response (GSR) and blood volume pulse (BVP) of the user. Three applications are possible: (a) a clinical monitoring tool; (b) a health monitoring tool; and (c) a physical activity tracking. First two are health monitoring tools for more severely affected users, while the third can be used as a supplementary follow-up tool.

How will it be used?

For what purpose?

The purpose in relation with the HM is to improve the health state management of the targeted end users. In this regard, the HM device is based on a portable tool able to:

  • Detect the main acute health problems of DCP users (such as heart failure, choking situations, etc.) and send emergency alerts.
  • Help DCP users to control their health condition, by providing feedback to them in relation to their health status inferred from the physiological sensors, which enables the system to increase user’s awareness.
  • To provide the relatives with their health status.

Current technology

DCP users have strong mobility and communication restrictions, and frequently associated health problems. For this reason, one of the main concerns of the carers is related to being aware of the user health status. This impacts both on self-management of health and stress for the relatives. The health monitoring system aims at covering both needs. As a side feature, it would allow a better health recording, potentially useful for medical doctors to provide a correct follow-up.

 

Commercially feasible product ‘E’: Library Generator application (NR10)

Description

It is a Windows application based on the Qt libraries that allow the construction of libraries to be used in the ABC Communicator. Libraries build specifically for a subject can be combined with general purpose libraries, and allows the existence of online repositories where a group of users can share their own libraries.

How will it be used?

For what purpose?

As it has been stated, the main purpose of this piece of software is allowing the creation of libraries for the Communicator. This is a service that users associations (AVAPACE) can provide to their associates and non-member (external) users. Other organisations can have interest in the libraries for the creation of specific libraries of common use or libraries of their own interest (for instance a library can be a ‘McMenu’ from McDonalds®).

Current technology

There exist different sotware-based resources to be used as communication tools of AAC languages. Currently some of the most outstanding software programmes are:

  1. Boardmaker® is a design application that lets relatives, caregivers, therapist, etc. to make and adapt educational materials for end users who need symbols; namely, it is based on PCS system.
  • Ø Pictured Communication Symbols (PCS) are a set of colour and B/W drawings comprising a core library of 5,000 symbols. Each symbol represents a concept to transmit, so users can build sentences from their combination.
  1. DeltaTalker® is a communication aid that uses the Minspeak language system for storage and retrieval of messages. It counts on user-friendly features and its design allows people with limited language abilities to develop language skills at their own pace.
  • Ø Minspeak is a way to code and organize vocabulary that uses multi-meaning icons (i.e. pictures culturally relevant and communicate a wide range of common ideas and topics) in short sequences with rule-driven patterns. The symbols can have several meanings, so users can express multiple messages with a reduced set of them.
  1. Symbol for Windows® is a comprehensive communication tool counting on large symbols databases to express a great amount of concepts. The tool consists of a number of modules, including a Bliss Editor to draw, edit, save and use Bliss symbols to produce sentences by the users.
  • Ø Bliss language was conceived as an ideographic writing system consisting of several hundred basic symbols, each representing a concept, which can be composed together to generate new symbols that represent new concepts. This language is a free open-source communication mode and allows a high communication capability, despite it is difficult to understand by non-experienced users.

The libraries created can build any of the approaches. Conceptual boards built can use proprietary symbols (such as the PCS) and free symbols such as ARASAAC or Bliss symbols.

The Communicator does not use semantic compactation as the Minspeak, this is the preferred solution according to user criteria (see D2.1), as it allows more flexibility and adaptation to the user.

One unexpected result well evaluated by users is the possibility to create long sentences and link each of them to one symbol.

 

Commercially feasible product ‘F’: Communicator-based ABC System (NR11)

Description

The core basis of the ABC System is an online SSVEP-BCI augmented alternative communicator (AAC) especially oriented to DCP users, and implemented on a user-facing Android tablet PC with an open source operating system. It integrates different modes of configuration or scanning techniques by using a specific language based on symbols and abbreviated expressions to describe common activities. The Communicator can be controlled by the BCI application and the IMU solution, even by EMG signals (but with a lower performance).

In this sense, after project finalisation the ABC System is finally composed of 6 independent and interoperable modules: (1) the IMU peripheral; (2) the BCI software; (3) the Affective Module; (4) the Health Module; (5) the Library Generator application; and (6) the Communicator of the system. The integration of the ABC System is modular to allow for independent use of each component and possible combinations between them, as well as to achieve compatibility with current systems and applications.

How will it be used?

For what purpose?

The Communicator is capable to receive information from different sources including EMG, EEG, IMUs and conventional pointer driven (or tap driven) applications, and it is also built to understand the emotional and health status of the user through the measurement of physiological signals. The Communicator pursues an easy and comfortable communication, being independent of user physical capabilities and reducing the rate of misunderstandings during the interaction between the user and the system.

Current technology

DCP users need an input interface to select and point at the desired concept. Due to their limitations, most of them actually use an scanning system combined with no comfortable section/control systems, such as pushing with their head/shoulder against a switch, or sipping-puffing air through a small straw, or finally being helped by a third person (caregivers or family).

However, in the same commercial segment of the Communicator-based ABC System, it is possible to find The Grid 2® application as the most relevant tool. This solution offers an AAC software for alternative communication, computer access and environment control, but its key disadvantage is that can be only controlled by the most common input interfaces such as mouse, touchscreen, head pointer or switches.