Mary Lou Jepsen: Could future devices read images from our brains?


I had brain surgery 18 years ago, and since that time, brain science has become a personal passion of mine. I’m actually an engineer. And first let me say, I recently joined Google’s Moonshot group, where I had a division, the display division in Google X, and the brain science work I’m speaking about today is work I did before I joined Google and on the side outside of Google. So that said, there’s a stigma when you have brain surgery. Are you still smart or not? And if not, can you make yourself smart again? After my neurosurgery, part of my brain was missing, and I had to deal with that. It wasn’t the grey matter, but it
was the gooey part dead center that makes key hormones and neurotransmitters. Immediately after my surgery, I had to decide what amounts of each of over a dozen powerful chemicals to take each day, because if I just took nothing, I would die within hours. Every day now for 18 years — every single day — I’ve had to try to decide the combinations and mixtures of chemicals, and try to get them, to stay alive. There have been several close calls. But luckily, I’m an experimentalist at heart, so I decided I would experiment to try to find more optimal dosages because there really isn’t a clear road map on this that’s detailed. I began to try different mixtures, and I was blown away by how tiny changes in dosages dramatically changed my sense of self, my sense of who I was, my thinking, my behavior towards people. One particularly dramatic case: for a couple months I actually tried dosages and chemicals typical of a man in his early 20s, and I was blown away by how my thoughts changed. (Laughter) I was angry all the time, I thought about sex constantly, and I thought I was the smartest person in the entire world, and —(Laughter)— of course over the years I’d
met guys kind of like that, or maybe kind of toned-down versions of that. I was kind of extreme. But to me, the surprise was, I wasn’t trying to be arrogant. I was actually trying, with a little bit of insecurity, to actually fix a problem in front of me, and it just didn’t come out that way. So I couldn’t handle it. I changed my dosages. But that experience, I think, gave me a new appreciation for men and what they might walk through, and I’ve gotten along with men a lot better since then. What I was trying to do with tuning these hormones and neurotransmitters and so forth was to try to get my intelligence back after my illness and surgery, my creative thought, my idea flow. And I think mostly in images, and so for me that became a key metric — how to get these mental images that I use as a way of rapid prototyping, if you will, my ideas, trying on different new ideas for size, playing out scenarios. This kind of thinking isn’t new. Philiosophers like Hume and Descartes and Hobbes saw things similarly. They thought that mental images and ideas were actually the same thing. There are those today that dispute that, and lots of debates about how the mind works, but for me it’s simple: Mental images, for most of us, are central in inventive and creative thinking. So after several years, I tuned myself up and I have lots of great, really vivid mental images with a lot of sophistication and the analytical backbone behind them. And so now I’m working on, how can I get these mental images in my mind out to my computer screen faster? Can you imagine, if you will, a movie director being able to use her imagination alone to
direct the world in front of her? Or a musician to get the music out of his head? There are incredible possibilities with this as a way for creative people to share at light speed. And the truth is, the remaining bottleneck in being able to do this is just upping the resolution of brain scan systems. So let me show you why I think
we’re pretty close to getting there by sharing with you two recent experiments from two top neuroscience groups. Both used fMRI technology — functional magnetic resonance imaging technology — to image the brain, and here is a brain scan set from Giorgio Ganis and his colleagues at Harvard. And the left-hand column shows a brain scan of a person looking at an image. The middle column shows the brainscan of that same individual imagining, seeing that same image. And the right column was created by subtracting the middle
column from the left column, showing the difference to be nearly zero. This was repeated on lots of different individuals with lots of different images, always with a similar result. The difference between seeing an image and imagining seeing that same image is next to nothing. Next let me share with you one other experiment, this from Jack Gallant’s lab at Cal Berkeley. They’ve been able to decode brainwaves into recognizable visual fields. So let me set this up for you. In this experiment, individuals were shown hundreds of hours of YouTube videos while scans were made of their brains to create a large library of their brain reacting to video sequences. Then a new movie was shown with new images, new people, new animals in it, and a new scan set was recorded. The computer, using brain scan data alone, decoded that new brain scan to show what it thought the
individual was actually seeing. On the right-hand side, you
see the computer’s guess, and on the left-hand side, the presented clip. This is the jaw-dropper. We are so close to being able to do this. We just need to up the resolution. And now remember that when you see an image versus when you imagine that same image, it creates the same brain scan. So this was done with the highest-resolution brain scan systems available today, and their resolution has increased really about a thousandfold in the last several years. Next we need to increase the resolution another thousandfold to get a deeper glimpse. How do we do that? There’s a lot of techniques in this approach. One way is to crack open your
skull and put in electrodes. I’m not for that. There’s a lot of new imaging techniques being proposed, some even by me, but given the recent success of MRI, first we need to ask the question, is it the end of the road with this technology? Conventional wisdom says the only way to get higher resolution is with bigger magnets, but at this point bigger magnets only offer incremental resolution improvements, not the thousandfold we need. I’m putting forward an idea: instead of bigger magnets, let’s make better magnets. There’s some new technology breakthroughs in nanoscience when applied to magnetic structures that have created a whole new class of magnets, and with these magnets, we can lay down very fine detailed magnetic field patterns throughout the brain, and using those, we can actually create holographic-like interference structures to get precision control over many patterns, as is shown here by shifting things. We can create much more complicated structures with slightly different arrangements, kind of like making Spirograph. So why does that matter? A lot of effort in MRI over the years has gone into making really big, really huge magnets, right? But yet most of the recent advances in resolution have actually come from ingeniously clever encoding and decoding solutions in the F.M. radio frequency transmitters and receivers in the MRI systems. Let’s also, instead of a uniform magnetic field, put down structured magnetic patterns in addition to the F.M. radio frequencies. So by combining the magnetics patterns with the patterns in the F.M. radio frequencies processing which can massively increase the information that we can extract in a single scan. And on top of that, we can then layer our ever-growing knowledge
of brain structure and memory to create a thousandfold increase that we need. And using fMRI, we should be able to measure not just oxygenated blood flow, but the hormones and neurotransmitters
I’ve talked about and maybe even the direct neural activity, which is the dream. We’re going to be able to dump our ideas directly to digital media. Could you imagine if we could leapfrog language and communicate directly with human thought? What would we be capable of then? And how will we learn to deal with the truths of unfiltered human thought? You think the Internet was big. These are huge questions. It might be irresistible as a tool to amplify our thinking and communication skills. And indeed, this very same tool may prove to lead to the cure for Alzheimer’s and similar diseases. We have little option but to open this door. Regardless, pick a year — will it happen in five years or 15 years? It’s hard to imagine it taking much longer. We need to learn how to take this step together. Thank you. (Applause)

76 Replies to “Mary Lou Jepsen: Could future devices read images from our brains?

  1. I believe computers will eventually interface with our brains.
    Personally, I'm enthusiastic.  I imagine a time when I'll be able to share my "visualizations" with others.

  2. So what, all she can propose is some other math algorithm to decode mri scan data.
    And all those futuristic mind reading stuff is just a make up.
    We know from internet what human is, it's 90% sex, the rest is chatting surfing and playing.

  3. C'mon Why does everybody feel like the need for a "story of my life" thing? The actual video starts after 5 minutes

  4. In what other ways could technology help us to better leverage the potential of our brains? What are the most promising developments at the intersection of neuroscience and tech?

  5. There is a difference between memory and imagination. When people look at images and then remember them they are not using their active imagination they are remembering. With memory we recreate the past, with imagination we make something new, something that hasn't existed before. A computer cannot and never will be able to do this. A computer can only record something that has already happened or, at best, make a guess based on past experience. It can never come up with anything new. The active imagination is perhaps our least understood faculty. Apart from the question of who might want to "see" what we are thinking, the real issue this talk raised for me is why would we want to avoid the creative process of communication our 'visions' by having a computer do it for us?

  6. Great, another way for the government to invade our privacy. Just imagine a Christian government -thought crime can become a reality in this life!

  7. Amazing. Wow, makes me want to live a few hundred years just to see what this technology would be like once it's perfected.

  8. this is absolutely incredible, some of its potential ramifications are also absolutely terrifying

  9. Work on stuff like this should be banned forever. Ittwould be the Frankenstein monster of the 21st century. If they can pull a thought out, they can put one in, or how about shutting them off altogether? what gives them the right to enter our minds? And dont say they won't because you know they will and they will find a way to tax us on it as well.  but the elite will have blockers on theres so no one can harm them.Then they will label you with every fleeting thought you have and you will not have a second of privacy .people would hack your brain like a cheap pc. Truth be told I dont think the mind would ever handle it, you would go insane.if not from the massive amount of info going through your brain then from the voices in your head that arent yours.

  10. Once the Government and market maniacs can see into your head? Game over. I'm sure this may be a fascinating problem to work on but it's not welcome in today's  climate of throwaway privacy. (or to give it it's proper nightmare fuel dues, it's TOO WELCOME for my liking!).

  11. My main problem with this is the experiment of reducing brain activity to an image. What methodology do they use to do that? Do they effectively claim that the subjective experience of imagery is somehow reflected say three-dimensionally in the brain?

  12. Ban all technology that can record or implant thoughts. As civilization is in this current era, we're far too immature to handle this technology. It'll easily get abused, and something worse than the Orwellian nightmare unfolding today will come to fruition. Every thought will be recorded, the bad will be ones pruned from our minds, and the acceptable thoughts from oppressive governments will be put into our heads the moment we think negatively about them. Some would say we're livestock right now, but we still have some freedoms. What little we have will be taken away as technologies improve. If you think the present is bleak when we have para-military police, drones that can bomb anyone in the country, and the entire internet is recorded, just wait till they can auto-censor any dissenting opinion on the web, inject happy thoughts in your brain, and crush any critic with force.

    The world is nice today. It's that storm in the distance I'm worried about.

  13. This technology isn't to do with "injecting* thoughts. You conspiracy theorists jump to your guns so quickly. Before you start putting on your tin foil hats, this is dealing with reading mental images in the most literal form, not reading your complete thoughts (ex. I'm primarily a verbal thinker rather than visual). I'm sure it's going to take much much longer to decode linguistic data from our brains as it spans connections in so many different regions of associativity, not to mention all the other tangled niceties of language itself (audio, semantics, specific language, rhetoric, idioms, just to name a few).

  14. Soon they scann our subconsius as well and now moor about us then we do.

    All this is bad and it is bad that that people accept it.

  15. At some point, if technology progresses, we will have to see ourselves not as selves but as part of a whole network of minds. 
    Not as we do today, but in that we will need to give up many of the things that we today consider personal and private in order to become something far greater.
    Just as we gave traded some rural freedoms in exchange for urban life with it's greater opportunity and health, we will trade much of the privacy we cherish today for the wealth of knowledge and capability of massively paralleled human minds communicating instantaneously.
    It will happen, so we had better start planning for it today.

  16. Transferring a thought directly to another person or brain is almost always impossible not because of lack of technology but because of what thoughts are. That is patterns of activity in specific arrangements of neural connections unique to that person. So in order to communicate it directly, the recipient must have an essentially identical brain down to the lowest level, the sensory input. Otherwise you'd need to communicate most of your life experience together with every thought.

    In this regard, language is a really ingenious tool of separating the high-level concept of one's thought in a form receivable by another mind with a vastly different experience. Of course, even that only works if the experiences of those minds have been similar enough to have created a similar set of intermediate concepts.

  17. An amazing and inspiring video by one of the greatest minds of today! Imagine what we could accomplish by being able to communicate thoughts directly. The implications are so vast to comprehend, yet awe-inspiring at the same time!
    Keep up the good work!

  18. Take note: The explanation of the youtube clips…where the actual clip is presented on the left, and the computer predicted image is on the right, I'm assuming that the computer predicted images are not painted from the computer, but it uses a similar technique to Google Image Search, where Google looks for similar images. I'm guessing then that the computer looks for previously viewed clips in its archives that stimulated the brain in a similar way to the new clip does. Pretty cool, but the technique should be explained because it's far easier to misinterpret what is actually happening. If I'm wrong, correct me.

  19. This kind of reminds me a bit of the film The Final Cut that starred Robin Williams.  In it he would review all the memories of a persons life and condense it down into a feature length film when they have died.

    Anyway, I'm sure this is going to be one of those controversial technologies people will be afraid of because it's just so different to what they are used to.

  20. Idk there's something pretty hokey about this presentation. I don't believe for a second the computer derived an image through an mri scan unless it already had the image on file and used a best guess through repeated electrical brain patterns. I think this talk is pure sci-fi. Nothing except the very superficial is known about brain function and probably will never be known in reality. Makes a good story though. ; )   

  21. I can't see the Intelligence or Law Enforcement Communities letting that go.  They'll have watched this, thought about the interrogation potential and needed to rush out of the room for a change of underwear.
      It will be impossible for anyone to keep any secret.

  22. Enter the exciting world of future to get sneak peak about what others thinking through scanning and decoding with high-resolution smart scan technology. Here idea may now look like a far-fetched sci-fi dream But the possibilities are immense. Subsequently this kind of technical possibilities will raise several moral and ethical questions. How are going to answer or solve those maters would be interesting for the human race. Good Speech. Debatable, still worth viewing.

  23. They canť do edge recognition. They have 50 years of failure in text-to-speech. Now, they recognize brain syntax in a 0.1 mm voxel. Those bros have 40 years of language failure. Now, they decode the brain. I hear they have been able to separate the noise of a car in a 2 by 2m room using a good chunck of a petaflop. I see, I smell fraud. Maybe they like to lie? Maybe they like to pick their results? Some nice little dry labbing? Maybe they like to lie because they are liars?

  24. Can't wait to hear about what the NSA is doing in 10 years. Time to get your tinfoil hats ready gentlemen. 

  25. Of CoUrSE right?

    We have polygraphs so reading brain images is obviously next! And there's nothing wrong wrong with the polygraph…

    The brain is so simple, we just get it….

  26. Thousands and thousands of hours of calibration for each person who's thoughts are to be read, and still you'll only be able to recognize "a human", "a cat", "a text"—some wide class of objects, because the result is weight mean of what they shown you while recording your reaction. Or some celebrities at best, if they were exposed during the calibration. It can have some applications, but nothing to be excited with…. And definitelty nothing to do with magnets.

  27. I guess it's lucky she didn't change her hormones to be a 20 year old female, or she'd have immediately gotten pregnant with someone she met a few hours ago and would spend the next few years looking after an accidental child.

    Stereotyping is fun.

  28. This would be the ultimate tool for the police and we would not need any trials. One scan and we know if you are guilty or not.

  29. I've always thought about something like this, creating content straight from your head for everyone to see.

    That is real privacy of what a person actually thinks and experience mentally.

    Call Nvidia or AMD to help with the resolution problem XD
    , I kinda wonder if the Vector Based Codec can help with this problem..

    Even though Vector Based Codec  is at least 5 years away.

  30. Quite peculiar how beneficial this can be, yet still be aware of limitations of viewing what unlimited thoughts could look like when interpreting into visual/audio/ect..
    I think it could be a brilliant idea.

  31. The Solid Snake Simulation (S3) Plan is almost complete. Now we only need to address countering anti-mind reading measures such as implantation of fake memories, Prior disruptors and anti-mind reading protection cranial implants. Our reptilian overlord plans are almost complete.

  32. Okay someone confirm the process to me because my brain just broke: they scan the patterns in the brain of the people watching videos, and when they're watching a new video, the computer puts together the video extracts that correspond to the brain patterns exhibited by the subjects?

  33. This was inspirational! Suffering from brain injury i used chemicals to train my hand to move agen! I use youtube videos to enhance my creativity! I study frequencies sound can heal! This is what i miss about mtv in the 80s it was all about the music and how it connects to the meter! From personal experience i have gone deep in the rabbit hole, not until recently that i have tuned my frequency and mathematics! Symbols don't lie, i have tapped into a beautiful world! I do belong in this universe, society will accept me for who i am! I do have a voice, this is amazing! Thank you for the words for this is uplifting! Ted talks means the world to me! Thank you for everything you post, its deep love it!

  34. My addiction to benzodiazepines and my eventual success in getting off of them gave me a new appreciation for the power of modern medicine to contribute tremendously to our mental well-being.  The benzos drastically affected my mental stability, and the right drug to counteract them did more to settle me down in three months than rigorous meditation and yoga could do in three years!

    I still think both practices are very valuable, and I still think we are more than our brains, but I've since realized that the future of the mind probably has a lot more to do with neuroscience and medical innovations.

Leave a Reply

Your email address will not be published. Required fields are marked *