Elon Musk recently announced that the next Neuralink project will be a cortical implant to restore vision: “The resolution will be low at first, like the first Nintendo graphics, but eventually it may surpass normal human vision.”
Unfortunately, this claim is based on the misconception that neurons in the brain are like pixels on a screen. It’s no wonder that engineers often assume that “more pixels equals better vision.” That’s how monitors and phone screens work, after all.
In our recently published study, we created a computational model of human vision to simulate what kind of vision an extremely high-resolution cortical implant might provide. A movie of a cat with a resolution of 45,000 pixels is sharp and clear. A movie generated using a simplified version of the model of 45,000 cortical electrodes, each stimulating a single neuron, still has a recognizable cat, but most of the details of the scene are lost.
The reason the movie generated by the electrodes is so blurry is because neurons in the human visual cortex don’t represent tiny dots or pixels. Instead, each neuron has a specific receptive field, which is the location and pattern that a visual stimulus must have in order for that neuron to fire. Electrical stimulation of a single neuron produces a blob whose appearance is determined by that neuron’s receptive field. The smallest electrode—one that stimulates a single neuron—produces a blob about the width of your pinky held at arm’s length.
Consider what happens when you look at a single star in the night sky. Each point in space is represented by many thousands of neurons with overlapping receptive fields. A small point of light, such as a star, results in a complex pattern of firing across all of these neurons.
To create the visual experience of seeing a single star with cortical stimulation, you need to reproduce a pattern of neural responses that resembles the pattern that would be produced by natural vision.
To do this, you would obviously need thousands of electrodes. But you would also need to replicate the correct pattern of neuronal responses, which requires knowing the receptive field of each neuron. Our simulations show that knowing the location of each neuron’s receptive field in space is not enough – if you don’t also know the orientation and size of each receptive field, then the star becomes a fuzzy mess.
So even a single star – a single, bright pixel – generates an immensely complex neural response in the visual cortex. Imagine the even more complex pattern of cortical stimulation required to accurately reproduce natural vision.
Some scientists have suggested that by stimulating just the right combination of electrodes, it might be possible to produce natural vision. Unfortunately, no one has yet proposed a sensible way to determine the receptive field of each individual neuron in a specific blind patient. Without that information, there is no way to see the stars. Vision from cortical implants remains grainy and imperfect, regardless of the number of electrodes.
Restoring vision is not just a technical problem. To predict what kind of vision a device will provide, you need to know how the technology fits into the complexity of the human brain.
How we created our virtual patients
In our work as computational neuroscientists, we develop simulations that predict the perceptual experience of patients seeking vision restoration.
We previously created a model to predict the perceptual experience of retinal implant patients. To create a virtual patient to predict what cortical implant patients would see, we simulated the neurophysiological architecture of the brain region involved in the first stage of visual processing. Our model approximates how receptive fields increase in size from central to peripheral vision and the fact that each neuron has a unique receptive field.
Our model successfully predicted data describing the perceptual experience of participants in a wide range of studies of cortical stimulation in humans. After confirming that our model could predict existing data, we used it to make predictions about the quality of vision that potential future cortical implants might provide.
Models like ours are an example of virtual prototyping, which uses computer systems to improve product design. These models can facilitate the development of new technology and evaluate the performance of devices. Our study shows that they can also provide more realistic expectations about what kind of vision bionic eyes can provide.
First do no harm
In our nearly 20 years of research into bionic eyes, we’ve seen the complexity of the human brain defeat one company after another. Patients pay the price when these devices fail, leaving them with orphaned technologies in their eyes or brains.
The Food and Drug Administration could require companies developing vision-restoration technologies to develop contingency plans that minimize harm to patients if technologies fail. Options include requiring companies that implant neuroelectronic devices in patients to participate in technology escrow agreements and to carry insurance to ensure continued medical care and technology support if they go bankrupt.
If cortical implants come anywhere near the resolution of our simulations, it would still be an achievement worth celebrating. Grainy, imperfect vision would be life-changing for thousands of people currently suffering from terminal blindness. But this is a time for cautious rather than blind optimism.
This article is republished from The Conversation, a nonprofit, independent news organization that brings you facts and reliable analysis to help you understand our complex world. It was written by: Ione Fine, University of Washington and Geoffrey Boynton, University of Washington
Read more:
Ione Fine receives funding from NIH National Eye Institute Grants R01EY014645
Geoffrey Boynton receives funding from grant EY R01 EY014645 from the National Institute of Health