A computer program creates images based on your brain waves

Researchers at the University of Helsinki have developed a technique in which a computer models visual perception by monitoring signals from the human brain. In a way, it’s like the computer is trying to figure out what a human is thinking. As a result of this imagination, the computer is able to produce entirely new information, such as fictional images that have never been seen before.

The technique is based on a novel brain-computer interface. Previously, similar brain-computer interfaces were able to perform one-way brain-to-computer communication, such as spelling individual letters or moving a cursor.

As far as is known, the new study is the first where the computer’s presentation of information and brain signals have been modeled simultaneously using artificial intelligence methods. Images corresponding to the visual features that participants focused on were generated by the interaction between human brain responses and a generative neural network.

The study was published in the Scientific reports reviewed in September. Scientific reports is an open-access multidisciplinary online journal from the publishers of Nature.

Neuroadaptive generative modeling

The researchers call this method neuroadaptive generative modeling. A total of 31 volunteers participated in a study that evaluated the effectiveness of the technique. Participants viewed hundreds of AI-generated images of people of diverse appearance while their EEG was recorded.

Subjects were asked to focus on certain features, such as faces that looked old or smiled. While viewing a series of rapidly presented facial images, the subjects’ EEGs were fed to a neural network, which inferred whether an image was detected by the brain as matching what the subjects were looking for.

Based on this information, the neural network adapted its estimate of the type of faces people were thinking of. Finally, the computer-generated images were rated by the participants and they matched almost perfectly with the features the participants thought they were. The accuracy of the experiment was 83%.

“The technique combines natural human responses with the computer’s ability to create new information. In the experiment, participants were only asked to look at the computer-generated images. The computer, in turn , modeled the displayed images and the human reaction to the images using human brain responses. From there, the computer can create an entirely new image that matches the user’s intent,” explains Tuukka Ruotsaloresearcher at the Academy of Finland at the University of Helsinki, Finland and associate professor at the University of Copenhagen, Denmark.

Unconscious attitudes may be exhibited

Generating images of the human face is just one example of the technique’s potential uses. A practical benefit of studying may be that computers can increase human creativity.

“If you want to draw or illustrate something but can’t, the computer can help you achieve your goal. It could just observe the center of attention and predict what you’d like to create,” Ruotsalo explains. However, the researchers believe the technique can be used to better understand perception and the underlying processes in our minds.

“The technique does not recognize thoughts but rather responds to the associations we have with mental categories. So, although we are not able to uncover the identity of a specific ‘older person’ that a participant was thinking about, we can gain an understanding of what they associate with old age, so we believe this may provide a new way to better understand social, cognitive and emotional processes,” says the lead researcher. Michel Spape.

According to Spapé, this is also interesting from a psychological point of view.

“One person’s idea of ​​an older person can be very different from another’s. We are currently discovering whether our technique might expose unconscious associations, for example by looking at whether the computer still renders people elderly like, say, smiling men.


Kangassalo L, Spapé M, Ruotsalo T. Neuroadaptive modeling to generate images corresponding to perceptual categories. Scientific reports. 2020;10(1):14719. do I:10.1038/s41598-020-71287-1

This article was republished from the following materials. Note: Material may have been edited for length and content. For more information, please contact the quoted source.

Gordon K. Morehouse