CTRL-labs lets you control machines with your mind

No, this isn’t science fiction! CTRL-labs is using neural signals and AI to build neural interfaces. Adam Berenzweig, from CTRL-labs R&D, joins us to explain how this works and how they have made it practical. Sponsors: DigitalOcean https://do.co/changelog – Check out DigitalOcean’s dedicated vCPU Droplets with dedicated vCPU threads. Get started for free with a $100 credit. Learn more at do.co/changelog https://do.co/changelog. O'Reilly Open Source Software Conference https://oreil.ly/2V8rrMH – OSCON has been ground zero for the open source community for 20 years. This year they’ve expanded to become a “software development conference” — because in 2019, software development IS open source. The program covers everything from open source, AI, infrastructure, blockchain, edge computing, architecture, and emerging languages. Use the code CHANGELOG20 to get 20% off Bronze, Silver, and Gold passes. Fastly https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship. GitPrime https://www.pluralsight.com/product/flow – Download GitPrime’s 20 Patterns book https://resources.gitprime.com/books/20-patterns/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog(q1), a field guide to help engineering managers recognize achievement, spot bottlenecks, and debug development processes with data. Featuring: Adam Berenzweig – Website https://adam.berenzweig.com/, LinkedIn https://www.linkedin.com/in/madadam, X https://x.com/madadamChris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: O’Reilly AI keynote from CTRL-labs https://youtu.be/5Z5aZK2C3ewCTRL-labs website https://www.ctrl-labs.com/CTRL-kit https://www.ctrl-labs.com/ctrl-kit/TED talk “The real reason for brains” https://www.ted.com/talks/daniel_wolpert_the_real_reason_for_brains?language=enIBM healthcare Practical AI episode https://changelog.com/practicalai/27Upcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Being with your change log is provided by Fastly. Learn more at fastly.com. We move fast and fix things here at Change Law because of Rollbar. Check them out at rollbar.com. And we're hosted on Leno cloud servers. Head to leno.com slash change log.

• 0:16Speaker 00

  This episode is brought to you by Digital Ocean, the simplest cloud platform out there. And we're excited to share they now offer dedicated virtual droplets. And unlike standard droplets, which use shared virtual CPU threads, their two performance plans, general purpose and CPU optimized. They have dedicated virtual CPU threads. This translates to higher performance and increased consistency during CPU intensive processes. So if you have build boxes, C I C D, video encoding, machine learning, ad serving, game servers, databases, batch processing, data mining, application servers, or active front end web servers that need to be full duty CPU all day, every day. Then check out Digital Ocean's dedicated virtual CPU droplets. Pricing is very competitive, starting at 40 bucks a month.

• 1:01Speaker 00

  Learn more and get started for free with a $100 credit at do.co slash change log. Again, do.co slash change log.

• 1:21Speaker 00

  Welcome to Practical AI, a weekly podcast about making artificial intelligence practical, productive, and accessible to everyone. This is where conversations around AI, machine learning, and data science happen. Join the community and slay with us around various topics of the show at changelog.com slash community. Follow us on Twitter. We're at Practical AI FM. And now into the show.

• 1:46Daniel Whitenack?

  Welcome to Practical AI. This is Daniel Whitenack. I'm a data scientist with S I L International. And I've got Chris, my co-host on the line, who's chief AI strategist at Lockheed Martin. How are you doing, Chris?

• 2:01Chris Benson?

  Doing great. How's it going, Daniel?

• 2:02Daniel Whitenack?

  It's going, it's going well. You just came back from O'Reilly AI, right?

• 2:06Chris Benson?

  Yep. I spent last week in New York City at the O'Reilly AI conference, had a lot of fantastic conversations, did a little bit of recording, and so maybe some of that's to come down the road.

• 2:17Daniel Whitenack?

  Awesome. I can't wait to hear that. Any highlights?

• 2:22Chris Benson?

  I really was talking, I talked to quite a few people about TensorFlow 2. Everyone seems to love going from 1 to 2 and truly adopting Keras as the blessed way. And so I'm probably biased because I've been pushing into TensorFlow 2 myself, trying it out a bit. So yeah, I had a lot of great conversations, met a lot of great people there.

• 2:43Daniel Whitenack?

  Awesome. I can't wait to hear more about it. Well, today on the show, I think it's going to be a lot of fun because we're going to talk about something that might seem a little bit sci-fi-like to certain people. But I think we're going to try to bring it down and make it practical because after all, this is practical AI. I'm really excited because we've got Adam Berenzweig from Control Labs with us. Welcome, Adam.

• 3:12Speaker 03

  Thanks. I'm excited to be here.

• 3:13Daniel Whitenack?

  Yeah, Adam's going to help us dig into a little bit of what Control Labs is doing specifically around neural interfaces or processing our brain signals to control machines. So I'm pretty excited about this. Adam, do you want to start by just giving us a little bit of your background and how you got into AI and ended up at Control Labs?

• 3:37Speaker 03

  Yeah, absolutely. So academically, I studied speech recognition. I have a PhD in speech and signal processing, which is kind of, if you think about it, it's the branch of machine learning that's closest to kind of a human-computer interface application area. In general, if you think about machine learning as, you know, one big aspect of machine learning anyway is artificial perception, sight and sound and hearing. You know, a big part of the world that we as humans need to perceive is communication from other humans. And so speech recognition is, I guess, the branch of machine learning that's always been focused on understanding the output of human brains in the form of speech. So maybe that's going to set some of the context for what we'll talk about later with what we're doing at Control Labs.

• 4:22Speaker 03

  But so after my PhD, I actually sort of, in the middle of my PhD, I went to Google as an intern and decided it was too amazing and I stayed. I did go back and finish my degree later, which was, I consider, one of the most heroic things I've ever done is to actually leave Google for a bit and go back and

• 4:41Daniel Whitenack?

  Yeah, bold.

• 4:42Speaker 03

  and finish. And so at Google, I was there for 10 years. And I worked, in general, I'd say my beat was kind of building products around machine learning. So I was no longer really doing hardcore, kind of deep research, but taking what was coming out of the lab, taking stuff that was being worked on in academia and of course within Google, and thinking about how to apply that to products. And that was across a number of domains. I worked on text with Google News and the recommendation algorithm there. I worked on Google Music. My PhD was actually, you know, I was talking about speech before, but I was particularly interested at the time was music and non-speech audio and understanding just the world of sound around us, computational auditory signal processing, sometimes it's called.

• 5:24Speaker 03

  And this was kind of post-Napster, it was late 90s. And so I was also a little bit kind of, you know, idealistically, politically motivated to come up with technological solutions that would allow musicians to connect directly to their fans without these intermediary giant corporations controlling all the intellectual property in the middle. So, you know, I thought, as many people did at the time, that recommendation algorithms are going to be this great equalizer. You know, we'd already had a great way to distribute music and we had a great way to produce music cheaply, you know, people could make little recording studios at home. But in order for people to find music, we had to, you know, solve that problem. So I was interested in doing recommendation, but from the audio signals, actually analyzing

• 6:06Speaker 03

  the content of music from the audio and then using that to understand it and make recommendations to people. So then that turned into Google Music and I helped both the music recommender for Google Play. And then eventually I started to work on images and I worked on goggles, which was one of the early image recognition, mobile phone apps. You could point your camera at something in the world and we would tell you what it was. This was all pre-deep learning, actually, while I was working on that project. And so the project at the time I joined was, the app had already been launched and it was kind of an amazingly cool toy, but it wasn't quite yet like a really useful tool for people. So the obvious thing to do next was to put this into, you know, to run this technology, run recognition on. People's photos. At the time, Google didn't have a Photos project. It had, there was

• 6:51Speaker 03

  the Android camera, there was photo visual search on the web, you could upload photos. There was Picasso, the desktop app. So at the time, the Photos project at Google was kind of in its nascent form, but that was clearly where the technology wanted to have its home. So I worked on images for a while with that. Then I left and I helped start a company called Clarify, which was an early deep learning image recognition company. So yeah, in between the time when I joined Goggles and when I left Google, the deep learning, this is about 2012, 2013. So working on Goggles, I saw some of the early, you know, large convnets that were coming out of research. And there was an intern at Google, Wojciech, who, you know, basically was the first, you know, deep convet I'd ever seen.

• 7:34Speaker 03

  And it was doing things that, you know, we'd never seen possible from an image recognizer. Before that, it was really clear that, oh my God, this is really going to change everything. And it was a technology that was ready for prime time.

• 7:45Daniel Whitenack?

  Yeah. So for some of us, I mean, for me, I guess, in particular, I feel like a lot of what I've been exposed to is kind of like post the deep learning revolution. So for someone that was involved in machine learning for quite a while before that, how did you perceive all of these things coming in around, around deep learning and the acceleration that happened?

• 8:08Speaker 03

  Oh, it was magic. It was incredible. I mean, you know, the field had been making advances, sort of incremental advances for 15 years at that point. And, you know, SVMs were kind of still the hot things, statistical machine learning. And we were, you know, we could do, you know, a lot of the technologies were okay and to pretty good for, you know, a narrow range of things and a narrow range of classes. You could get a binary classifier that would work fairly well. But the whole idea of having a single network that could recognize a thousand things at once, or, you know, eventually 10,000 and upward, was enormously It just changed a lot. And similarly, on the speech side, the results that were coming

• 8:52Speaker 03

  from the early deep learning-based speech recognizers were just blowing everything away. It was pretty wild.

• 8:57Chris Benson?

  So at what point did you, I know that you were saying that you had gotten to clarify around 2012, 2013. Was there anything else you did prior to jumping into Control Labs?

• 9:09Speaker 03

  Yeah, so I worked for a year on an idea for a startup, which I still hope somebody will go and do. It was actually quite different than what I'm doing now, but I was just thinking about the problem of teams collaborating together and how machine learning can wrangle all of the data that sits between people on a team. And probably you have this problem, you've got data scattered across Google Docs and Slack and Trello and who knows other places and where is all this stuff and who knows where things are.

• 9:37Chris Benson?

  You know me so well.

• 9:40Speaker 03

  Yeah. So I worked on that for a bit. We had some good ideas, but eventually I got sucked into Control Labs because Reardon and crew reached out to me at some point while I was doing that. And it was really just up my alley. It touched on some ideas that I'd been thinking about for 20 years at that point.

• 9:56Chris Benson?

  So can you kind of tell us all about what Control Labs is and how it came about and what you're doing there and kind of just give us an intro to it?

• 10:02Speaker 03

  Absolutely. So Control Labs, we're building a neural interface. And maybe a bit surprisingly for what people think of when you hear that phrase, the device that we build is worn on the arm. And I'll get into what that means in a second. But essentially, the goal of the company is to become the new interface technology that allows people to connect and control machines. And so you can think of this as eventually perhaps a replacement for everything you use to control machines right now, keyboards, mice, touchscreens, joysticks, all of that. Because when you boil it down, all of those physical mechanical controllers are just a machine that is designed to convert signals coming out of your brain that get transmitted into muscle movement that you move your hand to touch something,

• 10:47Speaker 03

  which eventually gets transduced into a signal that goes into the computer to control something in software. And so we're just trying to cut out all of that middle stuff and go straight from the signal coming out of your brain, through your muscle, decode it from your muscle, and go from there. So Reardon and Patrick started the company about three years ago. At this point, they met at the neuroscience department. At Columbia, Reardon's background is quite interesting. He was at Microsoft back in the day and basically started the Internet Explorer project and saw that through and at some point got sick of tech, decided he was going to go back and study Latin and Greek and got a classics BA and then eventually started dabbling in neuroscience and got a neuroscience PhD and so he met Patrick.

• 11:28Daniel Whitenack?

  And now he's back.

• 11:29Speaker 03

  Now he's back in tech. He's back and he's, yeah.

• 11:32Daniel Whitenack?

  And this is the CEO, is that right?

• 11:34Speaker 03

  Correct.

• 11:35Daniel Whitenack?

  Okay. Yeah. So you mentioned that what most people might kind of think about when they first think about like a brain machine interface, like the first thing that I think about is like a bunch of wires coming out of my brain or like, you know, someone drilling a hole into my skull and inserting something and then like connecting it to a big server or something like that. And then I, you know, take over the world. I, you know, I've seen too many sci-fi movies, maybe. Yeah, bad imagery, man. This is, uh, this is not the case with control labs, right?

• 12:08Speaker 03

  Yeah, we're not going for the plug in the back of the skull just yet. I mean, and so there's actually some really interesting reasons why we believe that that's actually never going to be the right way to do this. If what you're interested in is control, and I'll talk about what that means, but if what you're interested in is having some effect in the world, then the motor nervous system is the part of your brain that is specifically, you know, evolved to do that. So let's maybe just talk a little bit about the neurophysiology and the neuroanatomy to kind of back up what like is behind that belief of ours. Um, you know, not to mention the fact, of course, that it's not so, you know, user friendly to like drill holes in your skull in order to be able to play video games, but.

• 12:49Daniel Whitenack?

  Thank goodness.

• 12:50Speaker 03

  Besides that, yeah. So, you know, maybe a good way to start is that, um, so one of our science advisors, Daniel Wolpert, who's just incredibly brilliant neuroscientist and his expertise is in the motor nervous system. He's got a great TED Talk that, um, that you should totally, everyone should go and watch. Uh, I think it's called The Real Reason For Brains. And, you know, he makes this super surprising but really cogent argument that from an evolutionary perspective, the only reason that brains exist, they're not there for thinking and feeling, even though that's like, obviously what we think of as, you know, our human experience of having a brain. But as far as animal evolution goes, brains are there for one purpose, and that's to move muscles to have some effect in the world because from a survival evolutionary perspective, like, nothing else that you do has any purpose in the world. With the

• 13:35Speaker 03

  one exception that he makes is sweating. It's the only thing you could do to affect the world besides move a muscle, you know, breathing, talking, everything involves your motor nervous system. And he also gives this really cute example of, um, there's a sea squirt, which is like, you know, this primitive form of life, and it is an animal. It's born with a brain and it swims around for a while at the early part of its life. And then at some point, it, you know, settles down, finds a nice little rock, and then it digests its own brain, uh, because it no longer needs it because it's not moving anymore and it doesn't, you know, need this extra baggage.

• 14:08Chris Benson?

  Yep. My wife claimed I did that years ago.

• 14:10Speaker 03

  Well, I, I have some evidence that you still have a brain, but I'm not going to argue with your wife.

• 14:16Chris Benson?

  Go ahead. I'm sorry.

• 14:17Speaker 03

  So, yeah, so, you know, if you think about it from that perspective, all of the, the cortical BMI, in other words, the brain machine interface that is focused on getting signals out of the cortex, if what you're trying to do is figure out, okay, so let's talk about control. So, you know, our name of the company is Control Labs. And from our perspective, what we think of control is, it's the process of turning intention in the mind into action in the world. So you've got some desire to have some change happen in the world. That's, that's kind of our loose definition of intention. And you, and then to turn that into action, actually make something happen in the world, the part of the brain that does that, you might have a lot of like swirling unconscious thoughts and all the stuff that's, you know, responsible for regulating your internal

• 14:59Speaker 03

  organs and whatnot. But the moment you go to do something in the world and that intention becomes action, that's your motor cortex. That's the part of your brain that is like what was called the output port of the brain. And in fact, even people who are doing, you know, invasive cortical BMI, or the, you know, opening up the skull and trying to plant electrodes in there. They're planting those electrodes that are targeting the motor cortex because that's the part of the brain that is involved for this stuff. You know, you specifically don't want information. You don't like all the rest of the processing that's happening in the brain is essentially background noise that you want to kind of get rid of. So that's the first principle, kind of from a neuroscience perspective, why we're interested in the motor nervous system.

• 15:36Daniel Whitenack?

  Okay.

• 15:37Speaker 03

  The second thing is just like a matter of scale. Like, you know, there's billions of neurons in the brain. When where we are focused on the arm and the forearm, and the reason why we're there is because it controls the hand and the hand is super important for humans. And in fact, about a third of the motor cortex is dedicated to the hand. Another third goes towards the speech production system and then the rest of your body, like all of your body only gets like the remainder. So, you know, there's on the order of 10,000 neurons, motor neurons in the arm. And so like, you know, we just have a much simpler problem in terms of trying to decode signal from noise being outside the body. And then the third thing is that there's just a signal, signal to noise ratio question. And the really great thing about the neuroanatomy, when we talk about surface EMG, so the technology

• 16:19Speaker 03

  we use to decode the signal, I should say, is surface electromyography. So that's basically taking the electrical voltage changes when your muscles contract and decoding that with electrodes that sit on the surface of your skin, you know, that are embedded in a bracelet or eventually a watch. And they're just reading, you know, voltage potential differences as your nerves make your muscles fire. And the critical thing to understand and why we call this a neural interface is that, you know, nerves communicate in this kind of quasi-digital way with these spikes. So even though it's an analog signal in the sense that there's a voltage potential wave that goes goes down the axon of a neuron and also through the muscles, we'll talk about the shape of that wave and in fact, the size of it, the amplitude is extremely rigid for given kind of neuron.

• 17:04Speaker 03

  It basically always looks the same. And so information is encoded in the nervous system through essentially the firing rate of these spikes. It's almost like a digital, binary, you know, ones and zeros sort of information flow. So when the brain is trying to move a muscle, a spike train, a series of these very fixed shaped spikes will be sent from the motor cortex in the brain. One neuron will carry that signal down the spinal cord. Second neuron will carry it from the spinal cord out to the muscle. So there's only two neuron hops from there. And then when it hits the muscle, it's the same exact shape of the signal is propagated through the muscle as the muscle contracts. So the signal that we get is essentially the exact same information that's coming out of the motor cortex.

• 17:45Daniel Whitenack?

  That's cool.

• 17:45Speaker 03

  So this was like, you know, the whole insight that Brendan and Patrick, you know, as neuroscientists understood is like, why go into the brain when the signal that you actually want, as far as control, is like available so much more easily on the surface of the skin than in the muscle. And there's one more point to make, which is that the voltage signal, as it's traveling, the size of the voltage in these signals as they're traveling in the nerves is like in the order of microvolts. But what happens in the muscle, once a single nerve, motor neuron will come and innervate, meaning, you know, attached to a nerve, a whole sort of bundle of muscle cells, or these muscle fibers. And that whole thing together is called the motor unit. So one nerve and all the muscle fibers that it innervates is called the motor unit.

• 18:28Speaker 03

  And each of those muscle fibers in the motor unit only gets input from that one signal. So it's kind of like a one to many mapping. And that whole thing, when a spike comes down the nerve, it all fires at once and it's basically an amplifier. So from an electrical perspective, we've got this amazingly nice built-in amplifier that takes the tiny little microvolt signal that'd be traveling in a neuron and pumps it up to millivolts that's readable through all the tissues of the arm and through the skin and not worry about hair and all that stuff.

• 18:54Daniel Whitenack?

  All right, Adam, so maybe it's just because I don't want to have a hole drilled into my head, but you've convinced me that that's not the way to go. And, you know, there's this really great signal that we can tap into through the surface EMG. Maybe you could kind of switch directions a little bit here and kind of motivate why neural machine interaction or a neural interface, why is it important that we pursue this sort of interaction and don't just, you know, stick with the kind of mechanical interfaces that we're familiar with?

• 19:31Speaker 03

  Yeah, thanks. That's a great question. So I guess maybe one way to get into this is think about the transition that seems to be happening in personal computing. So a lot of people who pay attention to technology believe that we're on the cusp of a giant paradigm shift. What is going to be the personal computing platform that is going to replace the phone in your pocket? So maybe let's talk a little bit about the history of HCI and the kind of paradigm shifts as I see it over the last 40 or 50 years. So in the beginning was the command line. By the way, that's a really great essay by, I think it's Neil Stephenson, about just how cool it was when you used to compute, just using terminals and text input. But it's also interesting, and then it's perspective about the transition from

• 20:13Speaker 03

  that to GUI-based, Windows, what do they call it, WIMP, Windows icons, menus and pointers, like the desktop computing paradigm that we all know and love, maybe love, that came out of the research at Xerox PARC and Doug Engelbart's "Mother of All Demos." And so that was a second phase. And then there was touchscreens and mobile computing. And so it seems like with VR, AR, XR, wearable computing, and the whole idea that computing is getting cheaper and smaller and will start to become embedded in the things around us and maybe just embedded into things that we can wear and a multitude of sensors and a multitude of processors around us. And so for a long time, HCI researchers have been anticipating

• 20:54Speaker 03

  this transition. And I guess ubiquitous computing is probably the best umbrella term for this. Some people are now calling it spatial computing, wearable computing. But so the vision is that once the display becomes mounted in your eyeglasses or your contact lenses, then you no longer have to be holding anything. So how do you do input? You're not walking around with your magic AR glasses in the future, holding an Xbox controller so that you can navigate your email. And if you think about it, like every one of these big HCI paradigm shifts really comes with like a pair of technologies that play the input and the output role that together make up the new interface paradigm. So when the shift from

• 21:38Speaker 03

  console to computing to GUI, computing... Like the mouse is the input and then like a cheap rasterized display technology was was the output. And with touchscreens, it's kind of interesting because the output and the input are fused together into the single device. But they have two very different roles that they play and they're actually completely separable. You could have a touchscreen that's only input if you wanted. I mean, and if you go back and read the writing, so I just reread some of Jarry Lanier. You know, he's one of the early VR pioneers and I guess he's credited as coining the term VR. And he was so focused in the beginning on the input as much as the output. You know, they did all these experiments with data gloves and all kinds of other motion capture technologies. Because they recognized that if you want to have a truly immersive experience, then it's not it's not enough just to have this visually

• 22:23Speaker 03

  visual immersion, like, you know, the input side. You also have to understand like what is the person trying to do and let them interact with the world. And so that was a big focus in the beginning. And I think what happened around the time when, you know, when Palmer was was making the hardware work on the on the display side, that like raced ahead and cheap and, you know, good enough display technology with good head tracking happened before there was another big shift on the input side. So that kind of leapt ahead and that leads us to where we are right now, where there's this kind of a big gap. And if you use VR systems today, it's really beautiful. Like you feel visually immersed, you look around, it's amazing. And then you just have these sticks where your hands should be, you know, with these kind of like game controllers stuck onto a positional tracker.

• 23:08Speaker 03

  And it's very kind of disappointing from a from an interactions perspective. You don't have the the naturalness, the expressivity, the like incredible, you know, dexterity and agility and skillfulness that you have with your hands. So that's like, I think, the big picture that we're thinking that, you know, this technology is going to be the thing that is, you know, in a device that you just are wearing all day long and is the way that you interact on a minute to minute basis with all the devices in your life. You know, trolling the volume in your car and, you know, typing an email while you're walking down the street or you know, playing a video game, whatever it is, it's, you know, hands are the best controllers. And so all you need is hands. So we're just building a device that will just allow you to plug in directly to the signal that's controlling your hands.

• 23:50Chris Benson?

  Gotcha. So, one of the things I'm kind of wondering, and that was a great explanation, by the way, but if you're in a very practical sense, I'm imagining that person that's starting to utilize this technology going forward, and I have seen the AI conference video of the presentation, and so I have a sense of being able to utilize the technology where you're starting to remove the hardware between. But we also have other HCI, which for listeners, if you don't know, is human computer interface forums, like voice interfaces are so popular. I have a kind of a two-part question. How would you imagine what you're doing at Control

• 24:30Chris Benson?

  Labs interfacing with these other types of HCI, such as doing this chat, which is so popular now, and we're already kind of programming that in APIs, and the second half to this is, when you're looking at that combined world of utilizing all of these methods together, I know you guys have ControlKit that you've released, and it has what is essentially, I guess you're describing as an API for the brain. How do you realize that in an API, considering that you have different modes of communication?

• 25:02Speaker 03

  I think there will be a lot of things used at the same time together. I think that there's going to be a multimodal approach, and maybe it'll shake the two things of voice input and this idea of a neural interface that's getting the output from your hands. And the way that those things can be used together is really fascinating, and they have different strengths and weaknesses. Obviously, there's privacy concerns, or just kind of social awkwardness. You don't always want to be speaking out loud in public to control your computer, or to do input in terms of text input and communicate. You might be sitting in a cafe, and you're writing a sensitive email, or you just don't want to bother your neighbors, and so people still type. On the other hand, there might be circumstances

• 25:43Speaker 03

  where your hands are full and you can't use them, and speaking might be the most natural thing. Although, we've been exploring some really interesting ways that you can still do quite a lot while you're holding objects, because in some sense, we didn't really talk about how sensitive the device is, but tiny little twitches that are essentially invisible to the naked eye are enough to do control with this technology. So you could be holding your coffee cup and just doing little presses or wiggles with your fingers and be typing. And then, of course, there's the, there's a combined impact, right? If you have, you know, all channels available at once, if you're in your office and you can speak out loud, but you can also use your hands, you know, maybe we'll just get that we can use that to boost up the bandwidth. If you think about this technology, it's just a way to increase the output bandwidth from the brain to the world.

• 26:27Speaker 03

  We have so much bandwidth coming in, right? High resolution video displays and, and great audio. And the, the bandwidth coming out from the brain is just lagged behind for 50 years. I mean, typing on a keyboard, you know, is one way to kind of estimate what the, what's the maximum, just in terms of pure bits per, per second that you can get out of the brain. Um, you know, the fastest type is in the world doing like 150 words a minute. That's something like 30 bits a second. And, you know, compare that to like gigabytes of information coming in. So yeah, I think this multimodal thing might be a way that it goes. And another thing I was, point I was going to make is when I was working on speech recognition, it always seemed to me, especially when I was at Google and the first really great ASR

• 27:10Speaker 03

  systems were coming out and were put into Android in terms of, you know, speech input on Android. It always seemed to me a waste that I couldn't also use my hands while I was doing speech. But I'd be holding the phone and it would be making all these errors and, you know, hopefully not too many, but it still, still makes errors sometimes. And then you kind of have to slowly go back and correct things with your hands in this old school typing way. Uh, it just seemed to me that there was like, had to be a better way that you could combine the input of your hands with speech. So that's one of the interesting areas that we're working on.

• 27:47Speaker 00

  This episode is brought to you by the O'Reilly Open Source Software Conference in Portland, Oregon, July 15th to the 18th. OSGCON, the O'Reilly Open Source Software Conference is where you go to understand what's shaping software development from AI and cloud technology to distributed computing and learn how to put it to work for you and your team. Whether you're looking to understand where software development is heading or machine learning can make or break your code, OSGCON is where you'll find your answers. Hear from industry leaders like Pete Skomaroch, Holden Karu, Alison McCully, and Sam Charrington. Passes start at 796 when you register with the code CHANGELOG20 before June 7th. Again, before June 7th, use the code CHANGELOG20 and head to OSGCON.com/CHANGELOG to learn more and register.

• 28:44Daniel Whitenack?

  You know, because this is a podcast, it might be hard to imagine the sort of interface that you're talking about. And I know Chris mentioned that you've released this kind of control kit developer. I don't know what you, I guess kit is a good word. You already picked the best word. But could you just describe to our listeners what the interface or the device looks like and kind of how you wear it and then how it connects to another device, let's say a computer you're wanting to kind of pipe with your muscles, I guess, in how you're describing it. So could you kind of just describe for our listeners how that looks and how it connects?

• 29:26Speaker 03

  Yeah, it looks kind of like a chunky sort of cyberpunk bracelet right now. We're currently where it kind of midway up the arm in the forearm, but the goal is to get this down at the wrist and have it be able to be integrated with a watch. We have devices that work on the wrist already. It's actually, the biggest challenge to getting to the wrist is really just the, your tendons move a lot. If you kind of, you know, look at your wrist if, as you're kind of moving, like, look at the space where you would wear a watch as you move your wrist around. There's usually all these kind of divots and the topography of your, just like the mechanical challenges of keeping the electrodes in contact with your wrist as you move around is the major thing keeping us from being at the wrist. But yeah, it, the electrodes that make contact with your skin are on the inside of the device,

• 30:11Speaker 03

  obviously, against the skin. They're pretty cool. They're actually gold plated right now and they-

• 30:17Daniel Whitenack?

  It's got some bling.

• 30:18Speaker 03

  It's got some serious bling and the whole, the whole effect is this sort of like, you know, cyberpunk meets like studded biker leather look that- Oh, I'm into this now. Yeah.

• 30:28Daniel Whitenack?

  Chris, do you think I could pull it off?

• 30:29Speaker 03

  Ooh, absolutely, Daniel. So that's where we are now. And of course, you know, this is, we're just beginning. This is the first device that we have that is not just like a total research prototype. You know, our previous generations of device were like, literally, we started with sweatbands with electrodes sewn into them and then kind of ratcheted up the ladder of hardware productionization. So, yeah, so it's Bluetooth. It communicates to a host computer over Bluetooth. And currently with the developer kit as we're shipping it right now, you need a like a fairly, you can't connect directly to a phone. You need a host computer that can run the other side of the pipeline. It's actually doing the machine learning inference

• 31:10Speaker 03

  and the and the processing, signal processing mostly happens off board device right now. But you know, as we as we go through the iterations of hardware, we're going to move more and more processing on onboard so that you can connect directly to a phone, let's say.

• 31:23Chris Benson?

  And can you, since you've kind of segued into it, can you tell us a little bit about what you're doing in terms of machine learning for this? Where is it being used? What kind of problems are you solving? And what are some of the choices if you're able to share that you're using to tackle this?

• 31:38Speaker 03

  Yeah. So maybe one thing to say about the machine learning side of this from the beginning is, I think the most exciting thing that we're doing kind of from a machine learning perspective is, this is an entirely new domain, a signal domain to work on. You know, if you think of, actually, you had a guest on a couple of weeks ago from IBM Healthcare who

• 31:58Chris Benson?

  Yep.

• 31:59Speaker 03

  gives you a mo. And he said something about, you know, you guys are talking about processing speech as a way to diagnose psychiatric disorders and stuff like that. And again, I think, you know, the idea of some kind of brainwaves came up. And he said, your speech is a brainwave. And we very much feel the same way about the signal that we're working with. You know, it is this bio signal, a signal generated by the brain that is extremely information rich, carrying all this content about what people want to do. And it's as rich as speech, really. And it's new. It's like, you know, nobody has had access to this kind of signal before. I mean, not outside of, you know, very narrow clinical settings and neuroscience research. So that's just really cool. It's like, you know, it's

• 32:42Speaker 03

  like as if we just invented a microphone and we can start doing speech recognition. Except instead of, you know, like the microphone, the time distance between the microphone was invented and when machine learning was good enough to do, you know, speech recognition was like, you know, almost 100 years. And for us, it's like, we're there already. So we've got this access to a new signal that's extremely interesting and already have great machine learning tools to work on it. And the parallels to speech actually go pretty deeply because it's a, you know, multi-channel, time varying, continuous signal that has frequency content and some of the information is there, but it's got this particular structure and the way that it was generated. And then there's like all these layers that you can, you sort of start by parsing the kind of lower levels, understanding how to do the signal processing and get rid

• 33:27Speaker 03

  of the noise and figure out where the information is. And then as you go up the stack of processing, you get into meaning and semantics and what is the person trying to do and actually recognizing people's intention.

• 33:38Daniel Whitenack?

  All right. So you kind of alluded to part of this a little bit ago where you were talking about, you know, where machine learning fits into this pipeline and there's kind of two sides of it. So is my understanding right that kind of the device that you're wearing on your arm, this super cool looking

• 33:56Speaker 03

  Cyberpunk.

• 33:56Daniel Whitenack?

  Bracelet, armband, yeah, cyberpunk armband, it's sending these continuous neural signals to a computer or to a host computer. And then you're running a model on that host computer that's kind of making inferences based on those signals. Is that, is that kind of the overall pipeline? Or did I get something wrong there?

• 34:19Speaker 03

  No, that's exactly right. And the kind of differences that we make are dependent on what, what the application is and, and what we're interested in doing. So the basic capabilities of ControlKit, the API, as it is right now, include things like reconstructing the position of the hand. So, so the canonical classic demo we have is essentially like you'd see something from like Elite Motion. It's essentially a motion capture experience where you just see, you know, virtual rendering rendition of your hand. And what we're doing is we're predicting all the joint angles, you know, the positions of your fingers and your wrist as you, as you move your hand around. And we can also detect things like how strongly your muscles are contracting. And this is one of the kind of primary advantages

• 35:03Speaker 03

  that this technology has compared to a camera based hand tracking system. So most of the effort and investment over the past five or 10 years in terms of hand tracking for the purposes of, you know, VR and XR have gone into camera based systems, inside out tracking and outside in tracking. And, you know, with any camera based system, you always have occlusion problems. What happens if your, your hands are not in view of the camera, right? So you can either try to solve that by just placing cameras everywhere. Or you can have a technology that doesn't have any occlusion problems at all, which is what we're providing. So we sometimes have been talking about this as maybe like inside in tracking, where because you're sort of wearing this device that is looking inward at your own muscle and your and your own neural signal, there's no possibility of any occlusion problems.

• 35:47Speaker 03

  So the device works as well with your hands in your pockets or in a glove or in a space suit or whatever, you know, could be used in the future. So you don't have occlusion problems. And since we sense the strength of your muscles contracting is an enormous signal for us. And that's a really another interesting difference between camera based tracking systems. You know, if you kind of make a fist and now make a really strong fist, visually, there's really not that much difference there. But in terms of the signal that we see, the firing rate of the neurons that are controlling these muscles, it's like huge. It's like an enormous event. So the other part of API is recognizing stuff like forces of pinches on each finger and your fist and other interesting ways that you can kind of tense or relax your muscles.

• 36:32Speaker 03

  And that I think is going to be really important part of the design problem around this, which is like how to use that signal. It feels extremely natural and expressive when you when you can kind of, you know, use muscle tension and how light or hard you're squeezing to be able to control software. But it's not how we've used. It's not how we're used to thinking about designing interfaces. So that's going to be a big challenge for us and for designers who are going to be working with this technology. Let me think if I covered everything in the API. So the hand skeleton tracking poses and pinches, you know, also recognizing just hand pose, the different finger pointing and shapes. And then we're working on turning that stuff into usable, reliable controls, the equivalent of point and click.

• 37:17Speaker 03

  And we are working on the extremely interesting, to me personally, project of of typing or figure out how to do text input with this technology.

• 37:26Chris Benson?

  Gotcha. So, given the fairly unique type of signal that you're pulling off with the glove and passing over to the computer, what kinds of machine learning architectures are you trying to utilize and apply to these types of problems? How are you approaching? I know you mentioned machine vision, which is typically done with convolutional neural networks. Would this be like with a recurrent neural network of some sort or something like that?

• 37:53Speaker 03

  Yeah, we do use RNNs of certain flavors and we use ConvNets in other places. It's actually a kind of grab bag of approaches and different things work better for different kinds of inference we're trying to do. One of the other interesting things is that, so our current device has 16 channels. So, there are 16 pairs of electrodes that go around your arm to kind of cover all of the area and pick up lots of different muscles. And the way to combine those signals from neighboring electrodes and there's actually, it parallels a branch of signal processing that's going to think about beam forming and multi-microphone array technology.

• 38:34Speaker 03

  This is popular in audio processing if you're trying to isolate one signal from a crowd of background noise or you've got a room of people talking and trying to extract one signal out of many. So, there's some of that kind of work as well. But yeah, we use RNNs, we use ConvNets, we're writing the machine learning stuff in TensorFlow. Our research team works mostly in Python and then we have kind of the production system, which is in C++ and we've actually got a really nice way of blending those things together. I think that's kind of just on that kind of practical side. One of the challenges that we've always, in any machine learning based team that I've always worked on is like there's usually a big gulf between what researchers are comfortable doing and then getting code into production can be like this big gulf.

• 39:16Speaker 03

  So, we were very focused on making that hop very, very short and allowing researchers to work in an environment that's extremely close to what the production systems are like and having, they natively can work in the environment that they prefer and have it be a quick hop to production.

• 39:31Daniel Whitenack?

  So I'm curious, I think, so I may be misremembering things, but I think you said that something about the signals that you're getting off the muscles are fairly consistent in how they're shaped. I was wondering, as far as like shipping the developer kit, is this a thing where like you train kind of models on your end, on the control lab side, with a ton of data that you've collected for different tasks, and then you kind of ship the models with the control kit to just be used for inferencing? Or do you have to do any sort of like transfer learning or fine tuning for a certain person's, like, are my signals like different enough that you need to do some fine tuning for my, my

• 40:16Daniel Whitenack?

  signals versus like signals you've seen before?

• 40:19Speaker 03

  I've pinpointed one of the most probably critical problems from a research perspective that we're working on right now. So we've come at it from both ways, right? Everyone is different, everyone's anatomy is different, the signal does look quite a bit different across people. And not just across people, but across sessions. You take the device off, you put it back on, the electrodes enter a different place, they're going to see a slightly different view of the underlying neuroanatomy and the signals will be different. So you have kind of cross-session generalization, that's, you know, take the device off, put it back on, and you also have cross-user generalization. I want to train something on people that we have available here in the office, or, you know, getting people to come in and do data collection, and then have it work on someone in the world who we've never seen before, and we've never seen their signal before. So there's, there's

• 41:04Speaker 03

  two basic approaches, right? We can try to collect a lot of data from a lot of people and build models that will just generalize out of the box, and we do that. And then we also can go the other way, where we come up with ways that can learn very quickly on least data possible to get good performance with data from that one specific user. So in that case, you know, you have an experience where there'd be some kind of onboarding, calibration, training period where, you know, you would do some movements or play a little game or something like that, and it would be learning about you and your own particular Signal as you're doing it. And then there's hybrid approaches where you kind of get as far as you can with generalization and then have a transfer learning or domain adaptation, as you were mentioning, approach

• 41:45Speaker 03

  where you you try to build systems that are designed to be customized or personalized. So they might not be, the goal of them isn't necessarily to get the best out of the box performance as possible, but the goal of them is to learn about the kind of space of of signal that you could see across people and and learn how to adapt very quickly to a new user's signal when you do get a little bit of data from that user.

• 42:09Chris Benson?

  So as we've been talking, there's something in the back of my mind that I've kind of been wondering about, and that is, this is such a specialized domain that you're that you're addressing. Is there a role in this for biologists or doctors in the in the development process to kind of capture that domain knowledge? Or is this something that you guys have been able to just to master on your own and move forward with the technology?

• 42:31Speaker 03

  Uh, the company was founded by neuroscientists and it and it's pretty important. There is a lot of domain knowledge that goes in. And even just at the very basic level of understanding what the signal that we are looking for ought to be and ought to look like. You know, the essential idea of trying to use EMG as a consumer device and, you know, for controlling computers goes back, you know, a ways. Actually, I remember when I was in grad school and we talked a little bit before about my interest in music, but I was also interested in, you know, new musical instruments and musical performance systems. And I remember reading about people who were hooking up electrodes to their body in various ways and then using that signal as a as a musical instrument.

• 43:15Speaker 03

  And so the ideas are floating around. I think it does require some deep understanding of the neurophysiology to to under. I think one of the key things was like knowing when to keep iterating on the hardware and when to stop. So Meirden and Patrick knew that if they could get the hardware to be good enough to recognize, you know, individual spikes on single motor units, then that essentially was all the information that there was to be had, and they didn't have to keep iterating after that. But they shouldn't stop making the hardware better until that point was achieved. So I think that was key and that differentiated, you know, our attempt at this from from earlier ones. We do have, we've got MDs, we have MD PhDs on staff. And so, you know, some understanding of the anatomy is important.

• 44:01Speaker 03

  But there is some level at which machine learning is machine learning and the signal is what it is. And kind of, I mean, one of the cool things about working in machine learning, I'm sure you guys have appreciated this, is that like, it's just a chance to become like a kind of mini expert on a particular domain that you happen to be working on a new problem. You know, you might come into knowing nothing about, you know, phonemes and phonetics and the way that language is constructed, but you start working on speech recognition. You learn a lot about it really quickly. Or, you know, if you're analyzing stock market data, you suddenly learn a lot about, you know, financial signals. And so I think that's part of the fun of doing machine learning and probably, you know, if anybody could come in and work at this company and learn a lot about that stuff, it's important. But, you know, it's not critical prerequisites.

• 44:42Daniel Whitenack?

  Yeah, so I appreciate you sharing that. That's definitely one of the things that I really appreciated about this industry and working in this area is all of the things that you constantly get to learn. Speaking of kind of people working in this space, I know I'm already kind of, all of these ideas for utilizing the Control Kit and this technology are already popping into my mind. Like you were mentioning text input and I'm thinking about like over 350 sign languages that, you know, are known in the world. And typically, you know, people have not very good access to information that speak those sign languages. And being able to have one of these devices on each hand and document those languages

• 45:27Daniel Whitenack?

  and get resources to these people, like, there's already all sorts of stuff popping into my mind. So if I want to like pursue something like that with Control Kit, you know, what's the rollout plan and what's the current access to Control Kit for developers?

• 45:41Speaker 03

  Yeah, we really want to hear about all the, you know, amazing applications that people have in mind when they hear about this technology. And we want people to start building with it. That's absolutely where we are as a company. We've got, you know, some basic stuff proved out. We really believe that this is a technology that's going to be. Super important in the future and we want people to start playing with it and help us figure out how it can be used and what's it good for. So the first step would just be, you know, go to our website. It's Control Labs. That's C T R L dash labs dot com. And we've got a sign up there for the waitlist on our, on our dev kit. We're starting slow. You know, we're, we're a startup. We're a small company. We don't have the, you know, bandwidth quite yet

• 46:23Speaker 03

  to support the hordes of people who, who may want to work with this stuff. But sign up and, um, you know, kind of rolling out over the, over the, over the next year, starting slow with, with developers, kind of handpicked for a couple of applications that we think are really interesting. And as we go, we're just kind of going to sort of pick up the pace and, and see what people want to build with it. The, uh, the, the sign language thing, by the way, is, it's super interesting. That's something I've been, you know, studying a lot about recently. And sign language is, I mean, this, I don't want to go into a super deep dive on it, but the, the grammar of sign language, the phonology of sign language, a natural sign language is extremely interesting. It's obviously, you know, as, as complicated as spoken languages, but it's, you know, uses

• 47:06Speaker 03

  the articulators and that's, you know, the technical term for like the part of your body that makes the signal go from the brain to the world. And so I think we have a lot to learn from it because it's like the main proven use case right now where we have like super high bandwidth output coming from people's brains in a communication channel using the muscles of the hand. And so it's very interesting.

• 47:26Daniel Whitenack?

  Yeah. Well, I'm, uh, I know I'm gonna be headed to the website after this to fill out the application and describe something about sign language and see if I can get one of these uh, super, super dope bracelets on my arm.

• 47:40Speaker 03

  Awesome. I, I know, I know someone who works there. You might have an in if you.

• 47:44Daniel Whitenack?

  Okay. Sounds great. Yeah. Well, I really appreciate you, um, sharing so much of the great context around this technology and also kind of bringing it down to make it practical and help people understand that there is a practical route towards kind of processing these signals and getting them into a computer to utilize them. So it's really, really been useful for me and I appreciate you taking the time to walk us, uh, walk us through that. We'll definitely put the links to the website and the control kit and the TED talk and all of those things that we talked about in our show notes, but thank you so much, Adam. It's, it's been great to talk.

• 48:21Speaker 03

  Thanks. It was really fun.

• 48:23Chris Benson?

  It was super cool, man. Thanks.

• 48:26Speaker 00

  All right. Thank you for tuning into this episode of Practical AI. If you enjoyed the show, do us a favor, go on iTunes and give us a rating, go in your podcast app and favorite it. If you are on Twitter or social networks, share a link with a friend, whatever you gotta do, share the show with a friend if you enjoyed it. And bandwidth for change log is provided by Fastly, learn more at fastly.com. And we catch our errors before users do here at change law because of roll bar. Check them out at robot.com slash change log. And we're hosted on Lino cloud servers. Head to lino.com slash change law. Check them out. Support this show. This episode is hosted by Daniel Whitenack and Chris Benson. The music is by Breakmaster Cylinder. And you can find more shows just like this at change law.com. When you go there, pop in your email address, get our weekly email, keeping you up to date

• 49:11Speaker 00

  with the news and podcasts for developers in your inbox every single week. Thanks for tuning in. We'll see you next week.

• 49:27Speaker 00

  Because you've listened all the way to the end of the show, got a little preview here for you of our upcoming podcast called Brain Science. This podcast is for the curious and explores the inner workings of the human brain to understand behavior change, how about formation, mental health, and the human condition. This show is hosted by myself, Adam Stokoviac, and my good friend, Mary O'Ree, a doctor in clinical psychology. It's Brain Science Applied. Not just how does the brain work, but how do we apply what we know about the brain to better our lives. Here we go.

• 49:58Speaker 04

  That applied brain science really stood out to me because I want, I don't want it to just be data. I want you to go, how can this fit? What can I take away? Now, how am I going to change? And that that sort of is where you come in more. And even some of the questions like, so like, I want to ask you, what are some of the most challenging things working in the tech world when it comes to relationships?

• 50:18Speaker 00

  Probably the most important one is isolation. More and more of the world and companies are being, for good reasons, they're being okay with what they call distributed teams.

• 50:28Speaker 04

  Yeah.

• 50:28Speaker 00

  And that means that you and I, we work for the same company, but you work from your home office, I work from my home office. I might go into the office a couple of times a week if I live local, but even if I live in San Francisco, I'm still probably a remote worker, even though. I can hop in an Uber or hop on, you know, the train or whatever and go into the office and be there in a half hour, but why waste the time?

• 50:49Speaker 04

  You know, and this is where I would revisit what I, what I want to talk about with resonance. And that whenever we're, we're learning, no matter what thing, it's really helpful when we get feedback that's both immediate and specific. And so when you're by yourself and you don't have any in an interaction with other people, how can you get any feedback? I mean, you're losing most of the nonverbal communication and you also don't have all of the voice inflections or facial expression. Have you ever tried to, you know, be sad, feel sad and smile at the same time? Try it.

• 51:29Speaker 00

  It's, it's pretty hard.

• 51:31Speaker 04

  Right, because facial expression is exactly what's involved when it comes to empathy, which is relationships. I was reading a research article recently and it talked about, you know, how couples who are together a really long time end up sort of looking like each other. I've heard that's, yeah. And so what they've looked at is when we actually empathize with other people, facial expression is really key within that. And so when you empathize with the partner you're with over and over and over again, your face begins to make the same creases and facial expression as it relates to where somebody else is emotionally.

• 52:13Speaker 00

  Wow.

• 52:14Speaker 04

  Right?

• 52:15Speaker 00

  Say it is. So that's, that's goofy.

• 52:18Speaker 04

  Well, they've, again, this is sort of the hotbed when it comes to neuroscience these days is mirror neurons. And these mirror neurons are what are involved with empathy. And so mirroring, meaning I, I get another person's emotional world.

• 52:36Speaker 04

  And so one of the research studies looked at Botox. And what they found is that Botox, because it it actually assists in paralyzing facial muscles.

• 52:48Speaker 00

  Right.

• 52:48Speaker 04

  But then then you can't contort your face, so you don't get wrinkles. But actually, levels of empathy go down.

• 52:55Speaker 00

  Uh uh.

• 52:57Speaker 04

  Right.

• 52:57Speaker 00

  Because your physical appearance can't reflect your inner appearance.

• 53:00Speaker 04

  You're, yeah, you got it. And so when you're working in these remote locations, it it might facilitate better work or more focus. And it allows people to be distributed and to capitalize on the talents across the country, right?

• 53:15Speaker 00

  Yeah, wow. So that's, see, that's like a treasure trove, in my opinion. Talking about in a scientific way, you know, not just like, hey, this is my opinion.

• 53:25Speaker 04

  Yeah.

• 53:25Speaker 00

  About all the cons of that. Because I think what we can do is still have remote work, but do it in more healthy ways. Because I'm I'm fully, I mean, I've been self-employed remote worker since 2006. Now, I'm a unique animal. And I know, I know that. My wife knows that.

• 53:42Speaker 04

  Right.

• 53:42Speaker 00

  And I'm fine with it. I'm a good human being, but I've got some flaws. And I'm willing to accept and share those to some degree. And I think the problem is, is we just, we just lack more, maybe a more purposeful or intentional feedback loop.

• 53:57Speaker 04

  Yeah.

• 53:57Speaker 00

  Which I think is is super important to being able to operate in this world in just good ways. And in a healthy ways, this is probably the the best way to use in this show context is healthy ways.

• 54:09Speaker 04

  One of the things that's fundamental, I would say, to being human is change. Right? And so sometimes people come in and are really key in our life for a period of time. And then things change, either we grow or they grow, or they change in a different direction. And then the relationship changes or that feedback loop gets modified in some way. That isn't always a bad thing. It's just going, my sense of choice actually is a critical component when it comes to feeling good about my life. If I feel like everything is sort of outside of me and I don't have any charge over it, like I didn't choose to work. In a more remote location, or I didn't choose to go to school, or I didn't choose

• 54:52Speaker 04

  this person, then it feels far more oppressive as opposed to I actually participated in the outcome that I'm actually experiencing. So I then also have more charge over whether or not I want to change it.

• 55:07Speaker 00

  I think this feedback loop process that we're talking about here is is super common to to developers, you know, from people who write code to people who plan and to engineer and to manage and lead. Like, there's no one in the software process that doesn't understand the the feedback loop. And the reason, the reason why is because in product development, they have this concept of agile. And basically, it means you produce something, you put it out there, and you expect the feedback loop to happen in order to gain insights and course correction to then release another version of it that that continually and iteratively becomes more and more improved. So this whole process in day to day work in software is

• 55:52Speaker 00

  normal. And I think it's interesting how it can apply to their lives and people's lives, you know, to take the same importance of a feedback loop, for example, and apply it.

• 56:01Speaker 04

  Right. Well, so this is very much how it goes in relationship, which is why there is an important, when it comes to sort of things resonating, you ever walk into a room or an interaction with a couple of other people and like something just feels wonky or off? You're like, I can't put my finger on it.

• 56:20Speaker 00

  Definitely been there.

• 56:22Speaker 04

  Right? Well, and so to be able to identify that in relationships and even go, wow, I need to, I'm experiencing this person in my world with the limited interactions that I have with them. It hasn't really resonated with me. And so I don't get good feedback. So now I'm going to be more defensive because I feel as though there's a threat. It doesn't necessarily mean the person is threatening. However, my brain is going to tell me, hey, we need to be more protective. We need to do some strategies so that you're not fully exposed.

• 56:56Speaker 00

  You know, one way I look at scenarios like this, I would say as of late. Is because if you've ever watched a TV show or a movie where the, you know, the narration, the storytelling part of it, they expose a character in a certain light. And you may dislike that. They may be a villain or villainous, right?

• 57:16Speaker 04

  Sure.

• 57:17Speaker 00

  But the moment they turn the story to their backstory and why they are the way they are, or why they're acting the way they're acting.

• 57:25Speaker 04

  Yeah.

• 57:26Speaker 00

  Then kind of fall in love with them and you're almost rooting for them.

• 57:29Speaker 04

  Right.

• 57:29Speaker 00

  I feel like that's the same thing that happens day to day to our lives is that, you know, there are people who seem villainous or not for us. But we don't understand their backstory and why they are the way they are for us to have and employ that empathy that's required to have this, this dance, as you say, this iteration of relationship, you know?

• 57:49Speaker 04

  Right.

• 57:50Speaker 00

  We just assume they are who they are and we project, you know, our worst fears onto them and they become true.

• 57:56Speaker 04

  Yes. You got it. This is why in the absence of, you know, a face, I, I don't really get to engage with people in the same sort of humanness that we are all in. And so you're exactly right. I, I mean, over and over and over again, because you can identify and go, oh, that's why they're harsh. Or, you know, I recently had an interaction I had shared with someone that I, I was a competitive gymnastics coach for a number of years. And so somebody thought that my response to them when they were really struggling was kind of harsh, but they remembered that I had told them I was a coach for so long.

• 58:38Speaker 04

  And they're like, oh, this is just another side of her coming out.

• 58:42Speaker 00

  Right.

• 58:42Speaker 04

  And I'm not sure I prefer it, but I get it. And then it switched for their reaction. Because then they're like, oh, wait, we're on the same team. She's not trying to like oppress me or fight back against me. She actually is helping me trying to get me to where I want to go.

• 58:59Speaker 00

  My wife and I have learned this, this concept of goodwill, right?

• 59:03Speaker 04

  Yeah.

• 59:04Speaker 00

  I can take your feedback or your criticisms in a different light if, if I know that you have goodwill for me.

• 59:12Speaker 04

  Yep.

• 59:12Speaker 00

  Meaning that you're not trying to harm me, that you are for me, not against me. And sometimes change, as we all know, is painful and can be painful. So sometimes the necessary feedback and or criticism that can influence that change can also be painful, but I can accept it differently if I know.

• 59:30Speaker 04

  Right.

• 59:30Speaker 00

  That she or they or whomever is in the scenario with me has goodwill for me, you know, whereas if you know that they're not for you, then you obviously take it a whole different way and that's, that's an okay thing. But we often are, you know, in relationship with people that are giving us crucial feedback and we need to have that kind of, that lens, like it was significant in our marriage to understand, hey, I know there are times when you give me feedback, I am not happy about it, but, but I know you have goodwill for me. So therefore I calm down, I listen, I, you know, I take that in and I process it, whatever, but I take it in a different way because I know that she's for me and not against me.

• 1:00:13Speaker 04

  Yep. One of the key things when it comes to change is a sense of openness and even relationally, like of going, I need to be able to see some, how somebody else responds or how they're feeling as based on their perspective of what they're going through and not just my perspective of their perspective. And so this goodwill is like, I believe that we're on the same side and that you're not trying to make it harder for me, but so I can understand if I were sitting where you were sitting, had the background that you had, why you would have taken it in that way. And then I can provide an opportunity to clarify or create more connection, even when it doesn't feel good.

• 1:00:57Speaker 04

  And I honestly think this is so much of what's missing in people's relationships. If I look at relational interactions through the notion of conditioning, wherein I get a sort of hit of dopamine, feel-good feelings because I went to a person, I had a conversation that didn't necessarily feel good, but there was openness on both parties to hear one another's perspective, that it actually then reinforces like, oh, when I go and I have this exchange with people, I feel better. So now I'm going to go and engage with other people and get the feedback, even if I might not like the feedback, because now I'm buffered,

• 1:01:41Speaker 04

  and I'm not alone in this, and somebody else sees my world.

• 1:01:47Speaker 00

  That's a preview of Brain Science. If you love where we're going with this, send us an email to get on the list to be notified the very moment this show gets released. Email us at [email protected]. In the subject line, put in all caps, "Brain Science", with a couple bangs if you're really excited. You can also subscribe to our master feed to get all of our shows in one single feed. Head to changelog.com/master or search in your podcast app for Change All Master, you'll find it. Subscribe, get all of our shows, and even those that only hit the master feed. Again, changelog.com/master.