AI lets man with paralysis type by simply thinking about handwriting

An artificial neural network can interpret signals from the mind of a person who is imagining they are writing with a pen, and convert them into text. The device converts words accurately at 90 characters per minute, more than twice the prior record for typing with a head- or eye-tracking system.

These trackers allow persons to go a mouse cursor and slowly type messages, but Jaimie Henderson at Stanford University in California says they are all-consuming for the operator. “If you’re using eye tracking to utilize a computer then your eyes are linked with whatever you’re doing,” he says. “You can’t look up or shop around or take action else. Having that additional input channel could possibly be important.”

To solve this issue, he and his colleagues implanted two small arrays of sensors just under the top of brain of a 65-year-old man who includes a spinal-cord injury that left him paralysed below the neck since 2007. Each sensor array could discover signals from around 100 neurons – a fraction of the estimated 100 billion neurons in the mind.

As the person imagined writing letters and words on a bit of paper, the signals were fed to an artificial neural network. Team member Krishna Shenoy, also at Stanford University, says that the sensors don’t target exact neurons because plenty or millions could be involved in hand movement, but with the two arrays monitoring around 200 neurons there are enough clues within the info for the artificial neural network to develop a reliable interpreter of brain signals.

Read more: AI includes a specific Spider-Man ‘brain cell’ exactly like humans do

Ordinarily a neural network is trained with several thousand pieces of example data, which in this case would be a recording of a brain signal while writing some letter. That works fine when large data sets already exist or are given by automated systems, however in this case creating an archive that large wasn’t practical for the reason that man could have had to think about writing a large number of letters. Instead, the team took types of signals from the man’s brain while writing certain letters and made additional copies with random noise put into create a synthetic data set.

The model the team created won’t translate to another person for the reason that neural network is trained only on data in one individual, with sensors put within an unrepeatable location.

Using this technique, the man was able to type at 90 characters per minute, approaching the average of men and women his age when by using a smartphone, which is 115 characters each and every minute. The output had a 94.1 % accuracy, which risen to more than 99 % when an autocorrect tool was used.

Previous brain-computer interfaces have been able to interpret large signals, such as for example those for arm movements, but as yet haven’t been able to pick up on those for fine, dextrous movements like handwriting.

The team hopes to build on the work to make a speech decoder for use by a person who can’t speak but will probably still have the neural pathways to do so.

Journal reference: Nature , DOI: 10.1038/s41586-021-03506-2

More on these topics:

  • neuroscience
  • brain
  • neural network

Leave a Reply

Your email address will not be published. Required fields are marked *