Artificial intelligence has gotten scary good. It can already pass major medical exams, arrange friendly meetups with other A.I. online, and when pushed hard enough can even make humans believe that its falling in love with them. A.I. can even generate original images based only on a written description, but even that may not be the limit of its potentialthe next big development in A.I. could be understanding brain signals and giving life to whats going on in your head.
Machines that can interpret whats going on in peoples heads have been a mainstay of science fiction for decades. For years now, scientists from around the world have shown that computers and algorithms can indeed understand brain waves and make visual sense out of them through functional magnetic resonance imaging (fMRI) machines, the same devices doctors use to map neural activity during a brain scan. As early as 2008, researchers were already using machine learning to capture and decode brain activity.
But in recent years A.I. researchers have turned their attention towards how artificial intelligence models can replicate whats going on in human brains and display peoples thoughts through text, and efforts to replicate thoughts through images are also underway.
A pair of researchers from Osaka University in Japan say they have created a new A.I. model that can do just that, but faster and more accurately than other attempts have been able to. The new model reportedly captures neural activity with around 80% accuracy by testing a new method that combines written and visual descriptions of images viewed by test subjects, significantly simplifying the A.I. process of reproducing thoughts.
Systems neuroscientists Yu Takagi and Shinji Nishimoto presented their findings in a preprint paper published in December that was accepted last week for presentation at this years Conference on Computer Vision and Pattern Recognition in Vancouver, one of the most influential venues for computing research. A CVPR representative confirmed to Fortune that the paper has been accepted.
The novel aspect of Takagi and Nishimotos study is that they used an algorithm called Stable Diffusion to generate images. Stable Diffusion is a deep learning text-to-image model owned by London-based Stability AI that was publicly released last year, and is a direct competitor to other A.I. text-to-image generators like DALL-E 2, which was also released last year by ChatGPT creator OpenAI.
The researchers used Stable Diffusion to bypass some of the stumbling blocks that have made previous efforts to generate images from brain scans less efficient. Previous studies have often required training new A.I. models from scratch on thousands of images, but Takagi and Nishimoto relied on Stable Diffusions large trove of data to actually create the images based on written descriptions.
The written descriptions were made with two A.I. programs created by Takagi and Nishimoto. The researchers used a publicly available data set from a 2021 University of Minnesota study that compiled the brain waves and fMRI data of four participants as each viewed around 10,000 images. That fMRI data was then fed into the two models created for the study to create written descriptions intelligible to Stable Diffusion.
When people see a photo or a picture, two different sets of lobes in the brain capture everything about the images content, including its perspective, color, and scale. Using an fMRI machine at the moment of peak neural activity can record the information generated by these lobes. Takagi and Nishimoto put the fMRI data through their two add-on models, which translated the information into text. Then Stable Diffusion turned that text into images.
Although the research is significant, you wont be able to buy an at-home A.I.-powered mind reader any time soon. Because each subjects brain waves were different, the researchers had to create new models for each of the four people who underwent the University of Minnesota experiment. That process would require multiple brain scanning sessions, and the neuroscientists noted the technology is likely not ready for applications outside of research.
But the technology still has big promise if accurate recreations of neural activity can be simplified even further, the researchers said. Nishimoto wrote on Twitter last week that A.I. could eventually be used to monitor brain activity during sleep and improve our understanding of dreams. Nishimoto told Science this week that using A.I. to reproduce brain activity could even help researchers understand more about how other species perceive their environment.
Fortunes CFO Daily newsletter is the must-read analysis every finance professional needs to get ahead. Sign up today.
