Imagine that... brain scan which captures dreams

IT SOUNDS like science fiction: While volunteers watched movie clips, a scanner watched their brains.

And from the brain activity, a computer made rough reconstructions of what they viewed.

Such an approach might be able to reveal dreams and hallucinations someday. In the future, it might help stroke victims or others who have no other way to communicate, said Jack Gallant, a neuroscientist at the University of California, and co-author of the paper.

He believes such a technique could eventually reconstruct a dream or other made-up mental movie well enough to be recognisable. But the experiment dealt with scenes being viewed through the eyes at the time of scanning, and it is not clear how much of the approach would apply to scenes generated by the brain instead, he said.

People shouldn’t be worried about others secretly eavesdropping on their thoughts in the near future since the technique requires a person to spend long periods in an MRI machine.

For now, the reconstructed movie clips are only crude representations, loosely mimicking shapes and movement, but not nearly detailed enough to show that a blurry human-like figure represents the actor Steve Martin, for example.

The work was published by the journal Current Biology. The paper reports results from the brain scans of three co-authors, who were chosen because the study subjects had to be motivated enough to lie motionless in an MRI machine for hours and stay alert as they stared at a tiny dot, Gallant said. The machine was used for a technique called functional MRI, which shows brain activity.

The first task was to teach the computer how different parts of each subject’s brain responded to scenes of moving objects.

Participants stared at a dot to keep their eyes still as movie clips lasting 10 to 20 seconds unfolded in the background. That went on for two hours as the MRI machine tracked activity in their brains.

The study focused on parts of the brain that respond to simple features like shapes and movement. So it was limited to “only the most basic parts of vision”, Gallant said.

Researchers then fed the computer 18 million one-second YouTube clips that the participants had never seen. They asked the computer to predict what brain activity each of those clips would evoke.

Then they asked it to reconstruct the movie clips using the best matches it could find between the YouTube scenes and the participants’ brain activity.

The reconstructions are blends of the YouTube snippets, which makes them blurry. Some are better than others. If a human appeared in the original clip, a human form generally showed up in the reconstruction. But one clip that showed elephants walking left to right led to a reconstruction that looked like “a shambling mound”, Gallant said. The YouTube clips hadn’t shown elephants and so “we just had to make do with what we had”.