Scientists Create AI System to Translate Thoughts into Text

A new AI system, known as a semantic decoder, has been created by scientists to convert a person’s brain activity into text, as per a peer-reviewed study published in the journal Nature Neuroscience on Monday. The non-invasive system could prove beneficial for patients who have lost their ability to communicate due to degenerative diseases, paralysis, or stroke.

A non-invasive AI system called a semantic decoder has been developed by scientists, according to a peer-reviewed study published in the journal Nature Neuroscience on Monday. Researchers at the University of Texas at Austin utilized a transformer model, which is similar to those that support Google’s chatbot Bard and OpenAI’s chatbot ChatGPT, to create the system.

The decoder was trained by study participants who listened to several hours of podcasts within an fMRI scanner, which measures brain activity. The system does not require any surgical implants and could potentially benefit patients who have lost their ability to physically communicate after suffering from a stroke, paralysis, or other degenerative diseases.

After being trained, the AI system is capable of producing a text stream while the participant listens to or imagines telling a new story. However, the generated text is not an exact transcript but is intended to capture the general thoughts or ideas. As per a news release by The University of Texas at Austin, the system produces text that matches the intended meaning of the participant’s original words around 50% of the time. The researchers did not aim for an exact transcription.

Alexander Huth, one of the study leaders, stated in a news release that the system represents a significant advancement in noninvasive methods, as it can decode continuous language with complex ideas for extended periods. This is a significant improvement from previous studies that typically focused on single words or short sentences. As an example, during an experiment, when a participant heard the phrase “I don’t have my driver’s license yet,” the system translated their thoughts into “She has not even started to learn to drive yet.”

According to a news release, participants were also shown four videos without audio while in the scanner, and the AI system was able to accurately describe “certain events” from them. However, the decoder currently cannot be used outside of a laboratory as it requires an fMRI scanner. Nevertheless, the researchers anticipate that it could eventually be used with more portable brain-imaging systems.

The study’s lead researchers have filed a PCT patent application for the technology.