Brain decoder

A study published today in journal Nature introduced a non-invasive semantic decoder based on functional magnetic resonance imaging (fMRI) and technology similar to the one powering ChatGPT.

The decoder can convert a person's brain activity when they are listening to or silently imagine telling a story into a continuous stream of text. The method created by the team from the University of Texas at Austin may enable people to speak with their minds. 

Given novel brain recordings, this decoder generates intelligible word sequences that recover the meaning of perceived speech, imagined speech and even silent videos, demonstrating that a single decoder can be applied to a range of tasks. And it can fruitfully work with multiple brain regions. This demonstrates the viability of non-invasive language brain–computer interfaces. 

Computational task was performed by Generative Pre-trained Transformer (GPT), a 12-layer neural network using multi-head self-attention to combine representations of each word in a sequence with representations of previous words. GPT was trained on a large corpus of books to predict the probability distribution over the next word n a sequence. The authors fine-tuned GPT on a corpus comprising Reddit comments (over 200 million total words) and 240 autobiographical stories from The Moth Radio Hour and Modern Love. The model was trained for 50 epochs with a maximum context length of 100. 


REFERENCES

Tang, J., LeBel, A., Jain, S. et al. Semantic reconstruction of continuous language from non-invasive brain recordings. Nat Neurosci (2023). https://doi.org/10.1038/s41593-023-01304-9

Jerry Tang, Amanda LeBel, Shailee Jain, Alexander G. Huth. Semantic reconstruction of continuous language from non-invasive brain recordings bioRxiv 2022.09.29.509744; doi: https://doi.org/10.1101/2022.09.29.509744. download pdf

Comments

Post a Comment

Popular posts from this blog

AI in Elderly Healthcare: The Promise and The Bias

Image
The potential  impact of AI technologies on elderly healthcare is very promising . AI technologies have demonstrated great potential in satisfying the unmet care needs of older adults and are expected to further develop in this area. However, as we delve deeper into this technological revolution, it's crucial to address the biases and challenges that come with it. AI, a major driver of technological development in the 21st century, has unfortunately been associated with a new form of bias - "Digital Ageism". This bias is embedded into technology and AI systems, often excluding older adults from both the development and use of technology, leading to a “physical–digital divide”. This divide exists when a group feels ostracized because they are unable to engage with the technologies being used around them.  Stereotypes and misconceptions about older adults being a homogenous group in decline, incompetent, and in need of younger people’s guidance when it comes to technology,
Read more

Neuropathological Features of Age-Related Brain Diseases

Image
As we age, the structure and function of our brains can change, which may contribute to cognitive decline.  Analysis of donated brains  from 100 centenarians and 297 nonagenarians from the 90+ Study , allowed to compare the prevalence of 10 neuropathological changes related to dementia and cognitive performance. The figure graphically represents the prevalence of the individual dichotomized neuropathological changes in nonagenarians and centenarians. Compared with nonagenarians, centenarians had equal or slightly higher but not significant prevalence of all 10 individual neuropathological changes except large infarcts. Panel B shows the prevalence of the 10 neuropathological changes in centenarians and nonagenarians  and the odds of dementia associated with these changes. Among centenarians, the most common neuropathological change was arteriolosclerosis (88%), followed by ADNC ( Alzheimer's disease neuropathological change:  75%) and ARTAG ( Age-related tau astrogliopathy:  66%).
Read more

HippoCamera for Aging Brain

Image
Memory decline is common in aging and significantly worsens quality of life, but there are few, if any, effective interventions to improve episodic memory recall in older adults. And there are no effective interventions specifically targeting autobiographical episodic memory. A smartphone app called HippoCamera allows users to create memory cues of specific events that they personally value and hope to remember. These cues consist of a short verbal description and a short video recording of the event. The approach was shown to be effective tool for improving the recall of episodic memories in older adults. Seniors were able to improve their ability to recall episodic memories, increase the differentiation of hippocampal activity, and better remember the personal past. These findings suggest that using mobile phone technologies to record and replay episodic memories may be a feasible, low- cost, and effective way to improve memory recall in older adults (Adapted from SpinnerChef:  Spinn
Read more