Can Robots Feel? With Robert Shepherd and Ilayda Samilgil

Can you give a robot a sense of touch? Neil deGrasse Tyson, Chuck Nice and Gary O’Reilly, learn about soft robotics, sensors, and understanding human physiology with co-founders of Organic Robotics Corp, Professor Robert Shepherd and Ilayda Samilgil.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:01Neil deGrasse Tyson?

  Welcome to StarTalk. Your place in the universe where science and pop culture collide. StarTalk begins right now. This is StarTalk Sports Edition. We're gonna talk about censors in this episode. More on that in just a moment. First, my co-host, Chuck Nice. Chuck?

• 0:25Chuck Nice?

  Hey, what's happening?

• 0:26Neil deGrasse Tyson?

  Professional stand-up comedian. You do stand-up clubs, right? Pretty regularly.

• 0:31Chuck Nice?

  Yeah, I mean, uh, that's the only way you can be a comedian.

• 0:34Neil deGrasse Tyson?

  Okay, I guess so. That is how you earn that.

• 0:40Neil deGrasse Tyson?

  Also, Gary O'Reilly. Gary, former soccer pro and soccer commentator. We're here, my co-hosts. All right, Gary, you, you, you, this is one of yours again. I, I just know the title of the show, "Censors", but you're gonna have to take us into it and tell us where you come, where it's coming from, and where's it going. So start us off.

• 0:59Gary O'Reilly?

  So, the show begins in space with a suit that has been designed for the planetary exploration of the moon and Mars.

• 1:08Chuck Nice?

  In a galaxy far, far away, a time long, long ago? Perhaps?

• 1:13Gary O'Reilly?

  Perhaps. Um, we meet someone who not only worked on the space suit in question, but who wants to give robots a sense of touch. Now, this is gonna press a button for you, Chuck, but...

• 1:22Chuck Nice?

  Oh my, I was about to say...

• 1:24Gary O'Reilly?

  Plenty of show, plenty of show to go.

• 1:25Chuck Nice?

  I was about to say, who the hell idea was it to do this show?

• 1:29Gary O'Reilly?

  This means us getting into soft robotic technology. We will explore smart garments that are fiber optic based, and therefore more photonic than electronic.

• 1:42Chuck Nice?

  That I don't have a problem with. That's more haptic.

• 1:44Gary O'Reilly?

  All right.

• 1:45Chuck Nice?

  See, that's, that's, that's more haptic augmentation than it is giving a robot a sense of touch.

• 1:52Gary O'Reilly?

  Okay.

• 1:53Chuck Nice?

  You know, that's helping me be able to touch things in a better fashion. I'm cool with that 'cause it helps me, a human being!

• 2:02Gary O'Reilly?

  Okay, so I'm gonna try and give you a soft landing here Chuck. We'll see if you get a few bumps along the way. Right, we also meet another co-founder who is part of this team that is driving forward this particular project in so many different directions. The particular technology has multiple medical applications. Think computer gaming, wearables, they'll go there. It's a fashion item, and yes, you guessed it, it has sports applications as well. In particular, it'll be technique, analytics, injury risk reduction, biodata feedback, such as breathing analytics and much, much more. And you're gonna love this Chuck, the inner house husband that you are, it's also washable. Oh, how thoughtful.

• 2:47Gary O'Reilly?

  Exactly, right, let me introduce our two guests. Elida is CEO and co-founder of the Organic Robotics Corporation, graduate of Cornell University in mechanical engineering. We'll also meet Dr. Rob Shepherd, Associate Professor at the Sibley School of Mechanical and Aerospace Engineering at Cornell University, studied at Harvard in George Whiteside's Department of Chemistry and Chemical Biology, where of all things, he realized he was going into robotics. And no surprise to see that he is CTO and co-founder of the Organic Robotics Corporation.

• 3:24Chuck Nice?

  Awesome.

• 3:24Neil deGrasse Tyson?

  Welcome guys.

• 3:26Chuck Nice?

  Elida, you should say your name so that we can all just know it.

• 3:29Speaker 03

  Elida.

• 3:29Gary O'Reilly?

  Elida.

• 3:31Chuck Nice?

  And your last name, please, go ahead.

• 3:33Speaker 03

  Shad Miguel.

• 3:34Chuck Nice?

  Shad Miguel.

• 3:35Neil deGrasse Tyson?

  Shad Miguel, okay. So what is the smart suit thing? What is a smart suit? We all have these images of Neil and Buzz on the moon with these big bulky suits as they're skipping along, but they don't look very nimble and they don't look as comfortable as perhaps they could be, I don't know. So what, so our image of being in space is that, all right? So how are you gonna change this?

• 3:58Speaker 04

  Well, I think I have a collaborator, Anna Diaz Artias, at Texas A&M. And she makes, she's into bio-astronautics. She wants to make space suits that astronauts want to wear that they can maybe perform more like athletes here on Earth. And what Buzz had to do was fight against pressure in the suit to bend his arm. And one of the things she wants to do is make mechanical counter-pressure suits, which is more like what you see on the Star Trek movies, where it's conformal stuff and it's squeezing. Your your skin to keep that pressure differential from, you know, popping you a little bit. And but.

• 4:37Neil deGrasse Tyson?

  So I hadn't really thought about that because if I have if because I'm in basically zero pressure atmosphere, right? So you have to put pressure back on me so that my blood doesn't boil or the weird things don't happen, right? And so with that pressure, I don't have I don't have I can't move it in my joints, right? Because there's pressure everywhere. So you need some kind of mechanism to get through all the joint parts, not only my elbow, but my shoulder, my fingers.

• 5:04Speaker 04

  Yeah, so there's two ways to do it. One is to use the same suits you're being used today and augment the force so that you can get help fighting against that pressure. Or you can remove the internally pressurized suit altogether and just have like a compression garment that squeezes you. And then your joints are less hindered to move.

• 5:24Chuck Nice?

  Like in a flight suit, when you uh that squeezes your lower extremities so that the blood isn't draining down and it pushes everything up so you don't pass out.

• 5:34Speaker 04

  Yeah, it's pretty similar to that. Like so a giraffe actually has something like an evolved G suit. So they have a high pressure from their heart to feed blood to their brains. But when they lean down to drink water, their heads don't explode because they have, you know, fascia and skin.

• 5:52Neil deGrasse Tyson?

  That's good. That's good.

• 5:53Speaker 04

  That's squeezing it.

• 5:55Neil deGrasse Tyson?

  I think giraffes 101, their heads kept exploding, right?

• 5:58Chuck Nice?

  By the way, that is a great name for a bar, The Exploding Giraffe Head. Come drink it, our word a hole.

• 6:09Speaker 04

  So the more you can be like giraffe skin, the better.

• 6:11Neil deGrasse Tyson?

  So, Elida, what does organic have to do with any of this? When I think of material science and I think of robotics, biology doesn't ever enter my head here. So what is the organic part of this?

• 6:23Speaker 03

  Well, I think that's actually a better question for Rob because we just took his lab's name and then added Corporation instead of Lab to it. So that's why we're called Organic Robotics Corporation instead of Organic Robotics Lab. But, you know, we're now going for Light Lace, which is the name of the technology, which we can talk later. But you can talk more about the organic stuff. It's not food. Everyone's like, oh, is it food? Organic means-

• 6:48Speaker 04

  USD is certified?

• 6:49Neil deGrasse Tyson?

  Yeah, you have organic robots in the organic aisle of Whole Foods, right? That's what's gonna be next.

• 6:54Speaker 04

  It's just organic chemistry, using chemistry to make robots and now people better than, you know, machined aluminum and bolts and stuff like that.

• 7:06Gary O'Reilly?

  So whoever wants to answer this, I'll have two of you. Soft robotic technology, what does it actually mean? And am I right in saying it's kind of like bio mimicry here? Or am I on the wrong path?

• 7:20Speaker 04

  Yeah, I'm glad you gave me an opportunity to do that because I think we're evolving towards where the company started and, you know, my lab sort of has taken its own direction. So I started off at Cornell in mechanical engineering, taking what I learned at Harvard, making soft robot machines. And what they are is just continuum deformations, smooth deformations that don't look like hinged robots from the, you know, that you think of from the 50s. So they're made up of materials like animals. They move more like animals. But everything we made at Harvard was what's called feed forward in a robot. Like you issue a command and then it just does it, but it can't respond to the environment because it doesn't have any sensing.

• 8:03Speaker 04

  So what we did in my lab here at Cornell was to add soft sensors to the system, touch sensors, so it could feel what it's interacting with and then respond to it and make a truly feedback-controlled soft robot.

• 8:18Speaker 04

  And so then it can feel and respond and look like an animal. And we use optical light guides instead of electrical sensors for a variety of reasons. And Alida was in my lab making these sensors for robots and then decided, why don't we just use these on people instead? And then that's when the company started and that's, the lab continues to make stuff for robots. The company makes stuff for people.

• 8:47Neil deGrasse Tyson?

  Okay, by the way, that's the plot of a disaster movie, right? Where the brilliant engineers say, let's do this on people.

• 8:55Speaker 04

  Right.

• 8:57Neil deGrasse Tyson?

  And everything turns for the worse.

• 9:00Gary O'Reilly?

  So how do you scientifically go from knowing as our fingers can touch and sense to being able to interpret, touch and replicate it for one of your technologies?

• 9:14Speaker 04

  It's very basic. We don't, we cannot do it nearly as well as organisms do. There's a variety of mechanoreceptors in our skin that respond to different pressures and different frequencies of pressure, temperature. And this is a lot of information that we have to encode and then send to our brains. And we only have a limited space to do that with. So we encode this multitude of information into electrical signals spiked through our neurons. And then our brain interprets that, interprets that information and then sends commands out. One of the reasons we use optical systems in my group instead of electrical ones, you can encode more information optically than you can electrically.

• 9:59Neil deGrasse Tyson?

  Hence the urge to lay fiber optics for media communications.

• 10:05Speaker 04

  Right. You can pulse at high frequencies, but you also have phase information, color information and intensity. All of them can represent a different mode of touch feedback. And you can encode that into one signal and then send that to the computer, which then sends the response.

• 10:24Neil deGrasse Tyson?

  Using light instead of electricity, does that mean you're immune to the electromagnetic pulse that the aliens will send down and cripple our civilization?

• 10:34Speaker 04

  It is a big, it actually is a, you know, it's a very important point you bring up. We're immune to electromagnetic interference. So you can, you could potentially use these things in like MRIs. Um, or other places where magnetic fields can penetrate or electric fields. So yeah, where they're in a cockpit, there's a lot of places where you might want that.

• 10:54Neil deGrasse Tyson?

  And be resistant to aliens.

• 10:58Neil deGrasse Tyson?

  Every time the aliens come, they shut off your electricity, right?

• 11:01Chuck Nice?

  Yeah, with the EMP.

• 11:02Neil deGrasse Tyson?

  Yeah, everybody knows that.

• 11:04Chuck Nice?

  Everybody knows they're going, first, first, first line of attack is the EMP.

• 11:07Neil deGrasse Tyson?

  That's right.

• 11:08Chuck Nice?

  And then they're just like, clearly they're using some type of optic system.

• 11:15Chuck Nice?

  We underestimated them.

• 11:17Speaker 04

  Unfortunately, we still have to convert it to electrical signal. So there is.

• 11:21Neil deGrasse Tyson?

  In the end, yes.

• 11:22Speaker 04

  Yeah.

• 11:23Chuck Nice?

  Right, right, right.

• 11:24Speaker 04

  Yeah.

• 11:25Gary O'Reilly?

  So how, how accurate is this fiber optic ability? So, because I'm, I, I, it's, as Neem said, it's media, it's, it's TV, it's, it's a telephone connection. But you're using it in a totally different way.

• 11:41Speaker 04

  Yeah.

• 11:41Gary O'Reilly?

  So you're obviously using, how are you interpreting how that light changes? In terms of the results that you then decode and interpret?

• 11:50Speaker 04

  We use color and intensity right now. And we make our sensors so that they can change color when they're stretched and they can change intensity when they're stretched. Um, stretching can be linear or pressing or something like that. But, um, we're not as sensitive as something called a fiber brad grading optical wave guide. Um, which are, they're sensitive to, um,

• 12:15Speaker 04

  Hundreds of nanometers of movement. Um, we are sensitive to tens of microns of movement. That's like a human hair diameter. But we think that's sufficient for robots and people. And by making that trade off, we can make our systems much cheaper. Um, and, and much smaller.

• 12:31Neil deGrasse Tyson?

  It would seem overkill if you had nanometer sensitivities. That's, there's nothing we do in our life that.

• 12:36Chuck Nice?

  That, when do we ever, yeah, where, where you like that. I think that was a nanometer off.

• 12:43Chuck Nice?

  Oh, no wonder I missed that pool shot.

• 12:46Neil deGrasse Tyson?

  Yeah, just think people want to, an enemy is a billionth of a meter. So, yeah, we're not, but a human hair, that's something we can see. We, that means something to us. We can feel that.

• 12:54Speaker 04

  We decided that was a good target. Um, because yes, exactly what you said, you can feel that, you can't feel an anometer.

• 13:00Neil deGrasse Tyson?

  Right, right, of course. We're going to take a quick break, but when we come back, more of the secrets of this emergent technology of these space suits that will basically replace humans and will all die. I'm pretty sure. Is that where this show is going, Gary?

• 13:16Gary O'Reilly?

  No, I, I didn't because you were happier note, but.

• 13:19Neil deGrasse Tyson?

  Okay. All right. So we'll be right back. StarTalk Sports Edition.

• 13:37Neil deGrasse Tyson?

  We're back. StarTalk Sports Edition. We're talking about sort of robotic suits for all manner of applications. Not only that we've thought of, but maybe stuff we haven't even dreamt of yet. We've got two mechanical engineers turned roboticists here. We've got Ilayda Shamigil and Rob Shepard as our guests. So let me start off. Uh, Ilayda, tell me, in, in my notes here, we say something about light lace and fiber optic tech. What, what, what precisely is that?

• 14:09Speaker 03

  So Light Lace is the name of our technology. It's a fiber optic sensor. But unlike other fiber optic sensors, it's soft and stretchable. And that's why we're applying it to humans. Because if you're wearing a wearable device, you don't want to have bulky or uncomfortable fibers or electronics on you. So that's our technology. It's a soft and stretchable fiber optic sensor. And it measures things like motion. It measures your muscle fatigue. It can measure your chest expansion, respiration, heart rate. So basically, it is a way to track your vitals as well as your biomechanics while being comfortable.

• 14:44Neil deGrasse Tyson?

  Whoa, because you know, this reminds me. I haven't heard this reference lately. But Chuck and Gary might remember. There's an expression that says, it was all over you like a cheap suit. All right. And a cheap suit doesn't fit you well. But it needs to in order to look good on you. But if it doesn't fit you well, you move one arm and other pieces of the cloth move and it climbs up your neck. And so, so what you're saying, not to put words in your mouth, is that previous suits were wearing you.

• 15:17Neil deGrasse Tyson?

  Whereas now you get to wear the suit. Is that a fair characterization of this?

• 15:23Speaker 03

  I think we can say that. And I think one of our selling points is that you wear compression garments or like tight fitting clothing to exercise anyways. So it's just gonna feel the same way, but now we're making them smarter.

• 15:38Neil deGrasse Tyson?

  Right, so everything in Lululemon is snug fitting on everybody's body who buys clothing there. So now it's gonna know all about you, is what you're saying.

• 15:47Chuck Nice?

  How is the data retrieved? Like, I mean, what is this? How is this speaking to whatever it's speaking to? So that we get all this information?

• 15:57Speaker 03

  Yeah, it's pretty similar to other wearable devices. We have a phone app, like an iOS app you can download, and it speaks to the device via Bluetooth. So you can just look at your measurements real time or stop your exercise and just log in and see how you did.

• 16:13Gary O'Reilly?

  So when you've put the fibers through fabric? Is there an optimal pattern for those fibers fibers to be woven into? Or is it basically a simple grid system?

• 16:24Neil deGrasse Tyson?

  Um.

• 16:25Speaker 03

  There is an optimal pattern because our fibers are made so that only parts of the fibers are sensing. And we want the sensing mechanism to be where we want to sense. So if you want to sense your tricep, the fiber has to be, the sensing part of the fiber has to be woven into or integrated into the garment in that specific area. If you're sensing.

• 16:47Neil deGrasse Tyson?

  A lot of, I wanna monitor my.

• 16:49Speaker 03

  Yeah.

• 16:49Neil deGrasse Tyson?

  I wanna monitor my belly fat, so would you put sensors?

• 16:53Speaker 03

  We've, we've done something where we place our sensors around the abdominal area to measure how you're stiffening your muscles when you're doing certain exercises. So what we could do is you could wear this every day and it will eventually tell you if your belly got larger in size or it got um smaller in size or it's stiffer when you touch it. You know, you could, we could also measure um the response back when you touch it. So if it's soft, we can measure that. If it's no longer soft, we can measure that. So definitely.

• 17:25Neil deGrasse Tyson?

  Um um yeah, okay.

• 17:26Gary O'Reilly?

  Are you able to weave this into fabrics that are very slight, high performance? Say like for a dancer, a performer, an athlete? Or do the fibers, and you said they're the sort of diameter of a human hair? Are they necessary to be in thicker, more heavyweight fabric? Or can you really go into lightweight?

• 17:46Speaker 03

  We can't go into lightweight, but just a clarification, the fibers we currently make are not as thin as human hair. It is possible. Currently, they're around eight hundred microns um in diameter. And we prefer actually thinner fabric for them to go into because like, especially if we're gonna wear it outside than summer, you don't want to wear something thick, but we could do either.

• 18:10Chuck Nice?

  So right now, we're talking about diagnostic. And if I'm getting ahead, Gary, then.

• 18:15Gary O'Reilly?

  No, no, go ahead. Go for it.

• 18:17Chuck Nice?

  So instead of listening to the body, which is what you're doing, is there an opportunity to tell the body what to do?

• 18:25Speaker 03

  So we can tell the human that they need to do something about it. But we're not trainers ourselves. So we're gonna have to rely on a trainer to look at the data and say, oh, OK, when you do this training five times in a row, 10 minutes intervals, this is how your body reacts. So maybe you should change this. And then they can take more data and see how that change affected that.

• 18:48Neil deGrasse Tyson?

  And you get real time data, multi dimensional data there. That's excellent. OK.

• 18:53Speaker 04

  I think Chuck wants to know about actuation and how you can modulate the force. But sensors are hard enough, so we're gonna do that first.

• 19:03Chuck Nice?

  There you go. You're like, OK, I got you. You're like, you're in the sci fi mode there, Chuck.

• 19:09Speaker 04

  There are people working on it. It's very hard. My lab is working on it, but it's for company. It's very difficult.

• 19:13Neil deGrasse Tyson?

  But if you could, then it could be like, like the force. You can go on your app and just change what the body, the human body is doing in response to your wants and needs.

• 19:24Speaker 04

  Yeah, that's why that's one of the things the bioastronautics application is for. It's the sensors for measuring the environment. And they're also built in actuators into the mechanical counterpressure suit to help them perform better.

• 19:35Neil deGrasse Tyson?

  Right, right, right.

• 19:36Chuck Nice?

  Wow, that is crazy. So back to what Elida just mentioned. When you're talking about, we have this tranche of data. And then we offer this reservoir of data. Is it possible at one point, you'll be putting personal trainers out of work? Because that data will be uploaded to our AI cloud, and it will make those determinations for you.

• 20:01Neil deGrasse Tyson?

  Oh, yeah, I like that. I like that.

• 20:03Speaker 03

  I mean, I think this can go, yeah, we can say that for many other jobs, like a lot of technologies have the potential to take over or complement it. We want to be on the side of complementing the physical, uh, personal trainer or a physical therapist or a coach, but we're not trying to take anyone's job.

• 20:21Chuck Nice?

  I see what you're doing there. I see what you're doing there, Elida, because you're like, look, these personal trainers, they're pretty large people and they're very, very fit.

• 20:31Chuck Nice?

  We don't want to piss them off. We don't want to piss these guys off.

• 20:36Gary O'Reilly?

  So the thing here is, major, major league sport has already taken note of Light Lace. And you've, congratulations, by the way, uh, were awarded an NFL award for $50,000, first and future award for your product. Um, what was it that they saw in Light Lace that made them so enamored? And you've kind of shined a light, pun intended, on that for us.

• 21:04Speaker 03

  We get a lot of light puns, for sure. You know, there are a lot of selling points of our product. I think one being we can measure different things at the same time. Instead of relying on like three devices, you can just use one shirt that has Light Lace and it can measure both your respiration, your muscle fatigue and your motion. So if you want to do this currently, you would need three different devices. So maybe like a strap that does respiration, you would need sensors that go in your muscles to do that. And then you'd need camera-based systems for motion tracking. We combine all of it. I think there is a lot of value in that. And because we're an optical-based technology, we have higher sampling rates. So if there's inconsistency between each movement, we're more likely to measure that compared to any other sensor.

• 21:49Speaker 03

  So I think that was another reason why they were interested. And lastly, I'd also add this, this is a shirt, like we have some shorts behind me, but it doesn't have to be a shirt. And for NFL, for example, leggings, lower body, those are also very important. And it's very, very simple for us to just integrate the sensor into leggings or lower body garments and measure that as well. So there's a potential to measure full body, gloves, shirts, leggings, socks, shoes. It doesn't, depending on what the person wants to know. We don't really have a lot of limitations when it comes to the form factor.

• 22:26Gary O'Reilly?

  How how well do you think, Rob, your light lace will perform in an NFL game when it comes to contact? Because you have a pressure sensors here.

• 22:34Chuck Nice?

  Well, yeah, and that it's contact and weather conditions as well because you're you're playing in every single conceivable weather condition in the NFL.

• 22:42Neil deGrasse Tyson?

  Plus, you're sweating.

• 22:44Chuck Nice?

  And you're sweating.

• 22:45Neil deGrasse Tyson?

  On top of your fiber optics, yeah.

• 22:46Chuck Nice?

  And you're having a car collision on every single play. So you put that all together.

• 22:50Neil deGrasse Tyson?

  As they describe it, yeah, they described the the impact of every tackle as though you're in a car accident every single time.

• 22:56Speaker 04

  There are rapid decelerations for sure. Um, we can capture those. As Alada mentioned, we we can sample really, really fast. Think of this as a wearable ultra-high-speed motion capture system that can also measure pressure interactions. It has to be worn close to the body, so the temperature variations aren't that big a deal, because your body's going to sort of maintain that.

• 23:20Gary O'Reilly?

  It's own body temperature, right?

• 23:21Speaker 04

  It's not electrical, so you can get it wet. Um, there's not really that many environmental sensitivities for using sports, and they can handle high pressures there.

• 23:31Speaker 04

  They're soft, but they're very tough, and toughness means you could absorb a lot of energy before you break.

• 23:35Neil deGrasse Tyson?

  So you know what this sounds like with if you if if you have a full-body lace, everything, right? Neck, everything, and then you go up with your with your iOS app. That's a tricorder, isn't it?

• 23:53Chuck Nice?

  That sounds great.

• 23:55Neil deGrasse Tyson?

  Isn't that a tricorder? The tricorder measures everything that's going on in the body without cutting you open. That's kind of what you've done there.

• 24:04Speaker 04

  We don't have to just use our sensor. We can fuse lots of different sensor information.

• 24:08Chuck Nice?

  Right.

• 24:08Speaker 04

  Um, your Apple Watch uses photoplasmography, which shines light in, measures the reflected light out. And from that, they can, you can get a lot of things. You can get systolic pressure.

• 24:19Chuck Nice?

  Yeah.

• 24:19Speaker 04

  Um, you can get respiration rate.

• 24:23Speaker 04

  But what what those systems are missing is

• 24:25Neil deGrasse Tyson?

  Also, you get your ox numbers too, right? I mean,

• 24:28Speaker 04

  Yeah.

• 24:28Neil deGrasse Tyson?

  Pulse oximeter works, yeah. The pulse oximeter, right, right?

• 24:32Speaker 04

  The amount of signal processing that goes on to get all that information from that little bit of blood flow is incredible.

• 24:38Neil deGrasse Tyson?

  Yeah.

• 24:38Speaker 04

  Um, what is not there is the the amount of air inhaled, um, which is extremely important.

• 24:45Neil deGrasse Tyson?

  You can you can calibrate that by seeing the expansion and contraction of the chest cavity, can't you?

• 24:50Speaker 04

  We can.

• 24:51Neil deGrasse Tyson?

  Right. So, yeah, that's part of the So this is better than a tricorder. Without a doubt. You have exceeded 25th century technology right here. Look at you.

• 25:01Gary O'Reilly?

  If we take a step back, you we take a step back, you said this is more accurate than the high speed cameras. Now, we've done shows in the past discussing high speed cameras, and it was exclusively on baseball and in particular pitchers. Are you are you now being co-opted to bring this product to baseball, be it college, be it minor league or major league? And are you working with the pitchers? If you can, if you can outgun a high speed camera, um, you've got to be in there, surely.

• 25:33Speaker 03

  Yeah, we're definitely, baseball is going to be our beachhead market. We think there's a lot of value there because they move really fast, especially pitchers. And even with human eye or motion capture camera based motion capture systems, it's hard to capture that because everything happens in less than two seconds. So if you take a camera based system and you get maybe like four data points throughout the pitch, we can give you, let's say, 400 data points. And that's

• 26:01Neil deGrasse Tyson?

  Right.

• 26:02Speaker 03

  So you don't have to interpolate between the pitch. You can just look at the pitch and you'll be able to see, oh, pitch one versus pitch 10. At this millisecond, you behave differently. And maybe that's why you're not pitching as fast, or maybe that's why you're going to get injured.

• 26:16Neil deGrasse Tyson?

  Well, more importantly, well, not more importantly.

• 26:20Chuck Nice?

  Just as important is form for everything. Yes. Form is all of sports. Yep. It's all about having that form become a non-conscious second nature.

• 26:35Neil deGrasse Tyson?

  Muscle memory.

• 26:36Chuck Nice?

  Muscle, thank you. Yeah. Muscle memory. You guys are actually able to teach muscle memory. Yeah.

• 26:43Speaker 04

  And so there's a study in 2015 by the first author, I think is Wilk. And what they showed is that after they sampled 300 baseball players, the ones that had the greater shoulder mobility were four times less likely to need Tommy John surgery. So if you can help them increase their external, internal rotation, degree of freedom, then they can increase their lifetime with some reasonable probability. So teaching form is really important. And these things, a pitch takes two seconds, but the initial accelerations take milliseconds. And it's obscured.

• 27:26Speaker 04

  So if you are trying to capture it with external cameras, those cameras need, you really, and these are, we're talking about 10,000 degrees per second shoulder rotations. If you're capturing at 1,000 samples per second, you're under sampling and you really need to be sampling at, you know, maybe 10 times the rate for a non-periodic motion. There's something called the Nyquist criterion, which says twice, but that's for a periodic motion. For something aperiodic like this, it needs to be even more than that.

• 27:57Neil deGrasse Tyson?

  Just to clarify, so if you have something that repeats and you want to measure it, you can say, how many data points on this repeating feature do I need to characterize exactly what's going on? And that's your Nyquist frequency you're describing. But if it doesn't repeat, you don't, you can't.

• 28:16Chuck Nice?

  All bets are off. You can't compare what doesn't repeat. No. That's it.

• 28:21Speaker 04

  And that's an important point. We think that pitching inconsistency is a huge variable that cannot be measured right now to what matters. This is a hypothesis and we need to validate it. But we think by measuring faster, you will reveal that the accelerations are faster than are being measured right now. And then if you can't, if you don't know it and you can't repeat it, you can't get these consistencies. And further, the torques generated in this, we think are being radically under-reported. And that can result in injuries. And you wonder, why can't people tell when these injuries are going to happen? You should know the soft tissue loading conditions that are viable, but pitchers injure themselves all the time.

• 29:03Chuck Nice?

  Coaches are going to love this. You know, this is even outside of sports. Yeah. I'm just thinking about these applications for rehabilitation, not of athletes, but I'm talking people who have catastrophic injuries. Yeah. Where they have to learn again how to walk. Everybody thinks that we walk. You learned how to walk. You actually learned how to do all the things that you do, that you take for granted. Your brain is running, you know, a process that lets you do these things. And if you damage your brain, then it doesn't work anymore. You have to relearn it. This kind of technology could greatly reduce the amount of time it takes to do that.

• 29:45Speaker 03

  Yeah, definitely. There's rehab applications, physical therapy applications. It's just a way to get feedback. If, you know, if your brain can't get that from your sensors, you can use external sensors like LightLase and see that feedback on an app and then try to behave that way and see how change of behavior, change of the way you're moving, change of your motion is affecting that LightLase reading.

• 30:09Speaker 04

  One of the themes here is that we have sensors. We can feel ourselves, but we have imperfect memories and biased memories. And so being able to digitally record.

• 30:21Chuck Nice?

  You think?

• 30:24Speaker 04

  I constantly think I'm 6'2". So, but anyway, we have, uh, we have these, if we can digitally record this information, then we can go back, not just with ourselves, but with other sports scientists and physiologists and help you with less bias, interpret what's happening. And, and physical therapy with great application where I'm horrible at doing physical therapy. I've never successfully completed a physical therapy regimen. And I think if I was reporting my information to the doctor, um, they would be able to call me out on not actually doing it, you know, and help me improve faster.

• 30:56Neil deGrasse Tyson?

  And I just want to highlight for something, cause you just said it, and I want to make it clear. When you refer to the acceleration in a pitcher, just what's going on, the ball is going from zero miles an hour to a hundred miles an hour in a fraction of a second. All right? And, you know, from, you know, the, the pitcher gets ready and then they cock their arm and as their hand goes forward, they're accelerating the ball from zero to a hundred miles an hour. There's nothing we do in life that accelerates that fast. Right? In a car, your head would snap off and roll backwards. So, but there's not only that, there's also, there's some rotation of the wrist. There's a lot of sort of joint

• 31:41Neil deGrasse Tyson?

  action going on there and you capture all of that. I'm very impressed by what this is and what it can, can do for us.

• 31:47Speaker 04

  Thanks for saying that. And it's really a lot of who, who identified that beachhead market and I've been fascinated with it. And yes, in two seconds, a hundred miles an hour, but how fast is it, does it get, you know, in the first 80% of that velocity, how does it, what time duration does it achieve that? And we think it's, and this is backed up by some other data, um, that it happens in like very soon in the, in the pitch. And then this is also why people with longer arms are viewed as better pitchers because they can apply that acceleration for longer periods of time. Or they can do the same speed by accelerating at a slower rate over a longer period of time, which can preserve their joints for longer.

• 32:29Speaker 04

  But we have examples of pitchers who defy that. And then how do they defy that?

• 32:35Neil deGrasse Tyson?

  Like Pedro Martinez was not a tall pitcher for the Red Sox. I think he might have been sub six feet even.

• 32:41Speaker 04

  Tim Linsencom was 5'10", I believe. And if you look at his biomechanics, he extends his most pitchers will pitch over something like 85% of their wingspan, they'll move that. But he did it like 125% or something like that. You know, there's all kinds of stuff to go in for biomechanics and injury reduction. And these, it's been, it's great that systems like, what are they? Trackman and?

• 33:08Speaker 03

  Hall guy, Trackman.

• 33:10Speaker 04

  All of these motion capture systems that are out there now and some that are, some are even getting higher in sampling rates are providing all this information. But being able to have that unobscured at rates, I mean, there's really nothing limiting it. We could potentially do millions of samples per second.

• 33:27Chuck Nice?

  And they all use cameras. They all use cameras.

• 33:30Gary O'Reilly?

  See, the thing is.

• 33:31Chuck Nice?

  You're at the actual source.

• 33:32Neil deGrasse Tyson?

  Yeah, exactly. Exactly.

• 33:34Gary O'Reilly?

  There is not a sport that will embrace detailed, detailed data like this, like baseball. The coaches and the GMs are going to love you.

• 33:41Neil deGrasse Tyson?

  They'll eat it up. They'll eat it up. We got to take a quick break. When we come back, we're going to find out what the future of all of this can be in the third segment of StarTalk Sports Edition when we return.

• 34:02Neil deGrasse Tyson?

  We're back, StarTalk Sports Edition, talking about the future of monitoring what your body is doing on the inside, but you're doing it from the outside. This magic material that our two guests have pioneered and I think I perfected your names here. So you have Ilayda?

• 34:25Speaker 03

  Ilayda.

• 34:27Neil deGrasse Tyson?

  Ilayda?

• 34:28Speaker 03

  That's good.

• 34:30Neil deGrasse Tyson?

  Yeah, oh, good. Iliad Shamilgi.

• 34:32Speaker 03

  You moved to an A minus from a B plus.

• 34:35Neil deGrasse Tyson?

  Okay, thank you. And Rob Shepard. You guys are mechanical engineers and, but you're into robotics, you're into robotic monitoring, the human condition.

• 34:48Neil deGrasse Tyson?

  And sports has a huge benefit from this, particularly which we're spending time on, which we spent time in the last segment. The, the rate at which you can gather information on something that is moving fast and is non-repetitive is without precedent here. So what are, what are your best applications today? And what do you see coming down the pike tomorrow?

• 35:13Speaker 03

  Yeah, I think as we briefly discussed earlier, the best application, Brighton, uh, would be baseball and pitchers specifically because of the high speed motion they go through and how often they go through that motion, which usually results in injuries. It's very common to have a Tommy John surgery there. But whether we see this in the future, like I said, we want to make full body garments, not just shirts or not just straps, but also leggings, shoes, socks, really anything you want to measure or any area you want to measure. And we want to apply not only to sports, but also provide medical benefits like clinical application, physical therapy. We've looked into robotics, of course, and even car seats.

• 35:54Speaker 03

  You know, we had an interest in measuring comfort levels, driver attentiveness. Did you forget your child? It happens a lot in a car.

• 36:02Chuck Nice?

  Hey, hey, hey, it only happened twice.

• 36:08Neil deGrasse Tyson?

  And the third time it was on purpose.

• 36:10Speaker 03

  No judgment.

• 36:11Gary O'Reilly?

  No judgment. Yeah, look, Lada, when you discuss those applications outside of the sporting realm, I'm sitting here wondering, and I, I'm not a computer gamer, but this sounds like it is absolutely set to walk itself into VR and AR. Now, uh, uh, has that universe come to you?

• 36:31Speaker 03

  Yeah, definitely. I'm, I'm surprised I forgot to mention that, but the why is there are two different routes we can go for when it comes to AR, VR training. We can do training. So instead of just seeing a reconstruction of your hands or your body in the VR environment, you'd also receive feedback on how much force you're applying. So if you're learning the medical training, you need to know how much exactly to push when you're doing a surgery and you could get that measurement with our sensors if you were wearing them. And then gaming, the same thing. Instead of just seeing your body in the VR environment, you can. Get feedback on how much you're pushing, how much you're pulling something, what that results in, like in the gaming environment.

• 37:16Speaker 03

  So it makes it more real.

• 37:18Chuck Nice?

  Yeah. So now, this is information flowing in one direction. Do you have the ability to send information back? So, you know, if I'm, I'm talking for corrective purposes. So if I'm throwing and your fabric is saying, "Hey man, what you're doing right now is you're stressing right here in this part of your rotator, boom." And if you continue to do that, we know what's gonna happen. But at the point where it's happening, is there something, is there a signal that can be sent that lets me know, "Oh, I just did that." And then if I don't feel that, "Oh, that's, I did it right." That type of deal.

• 38:00Speaker 03

  Yeah, there are a couple of different ways we can do this. One of them being we can send the information back via a different light pattern. So our sensors glow. We can make them not glow, or we can make them glow in different patterns. So maybe having them glow three times or having them glow red could mean you are about to get injured. You should stop. We could have them glow green to mean something else. And the other way to go with this is have the app tell you. So if you have your phone or iPad or the device in front of you, it could also just tell you there.

• 38:33Speaker 04

  It is great feedback truck. We'll build that in there. Another thing we do is put a little vibro tactile thing in the pod. Like I said, this has an electrical portion too, which is located, a small pod located.

• 38:45Gary O'Reilly?

  You're gonna make garments glow and change color. You're gonna sweep up, you're gonna sweep up the teen market. You're gonna get the 20-somethings. You're gonna get the 30-somethings and the 40-somethings who still think they're 20-something.

• 38:57Chuck Nice?

  Not to mention all the ravers that will go into a club and get rid of their light sticks so that they can just use their shoulders.

• 39:08Gary O'Reilly?

  That's it. That's what I'm saying. So look, I gotta, I gotta wash you now. And what you're doing is incredible. But now, is there a dialogue between your sort of...

• 39:21Neil deGrasse Tyson?

  Gary, I find what you're doing entirely credible. It's not incredible, it's entirely credible.

• 39:27Speaker 04

  Thank you, that's great.

• 39:28Neil deGrasse Tyson?

  Yeah.

• 39:29Gary O'Reilly?

  I stand corrected, or sit corrected. The work that biomedical engineers are doing, trying to map and code signaling, the neural signals, how long before you guys talk to them and you come up with something together?

• 39:41Speaker 04

  I mean, from my, so my lab, we, this is where we both work on wearables and robotics, which is prosthetics. And so measuring signals from the outside world and transmitting it to our body through electrical impulses is something we have worked on in the past and are continuing to work on, but it's such a long slog and...

• 40:03Neil deGrasse Tyson?

  Well, electrical impulses that you feel on your skin or that you somehow impart into the brain?

• 40:08Speaker 04

  That you would impart into sensors in your skin, which then would send that information to the brain.

• 40:14Neil deGrasse Tyson?

  Right.

• 40:15Speaker 04

  Yeah.

• 40:16Neil deGrasse Tyson?

  It's almost the same thing, right? Because your brain is sensing the rest of your body.

• 40:19Speaker 04

  Yeah, and we worked on the hardware part, which is making a prosthetic hand that actually is pretty good and it's lightweight, high force, it's quick, feedback controlled and everything. And it's quite simple, at least for us, to add the capability for that hardware to output electrical impulses that could be then used.

• 40:43Gary O'Reilly?

  So something I saw that you've been working on is embodied energy systems. And I've seen a little tiny little video of your soft aquatic robot, which I would call a fish. But it's been called a soft aquatic robot. So talk to me about embodied energy systems, because that's, I think that's going to talk Chuck out.

• 41:02Speaker 04

  Okay.

• 41:05Speaker 04

  So if you look at the best example of the general purpose robot today, I think is Spot from Boston Dynamics, that yellow quadruped that walks around.

• 41:14Gary O'Reilly?

  Mm-hmm, yes.

• 41:15Speaker 04

  So we can do that for about, at least the last time I heard is 90 minutes.

• 41:20Neil deGrasse Tyson?

  Is that the one that opened the door? Two of them opened the door and the other one walked through?

• 41:25Speaker 04

  Yeah, there's appendages now that have hands and so it can, yeah, it's really useful. Fed robots for a long time, but they're very specific. This one, you can tune to do lots of different things, but it can only do it for 90 minutes. Where if you look at something, I like to use a walrus as an example. A walrus has a ton of fat on it, but that fat is multifunctional that you can, a walrus can operate without recharging for, you know, weeks at a time. And it can just sit still for, you know, theoretically months at a time. But it can do, has all kinds of agile things. It can, it could outrun you, definitely me. They're fast on land, they can swim, they do all kinds of things. And they can do it because they're using their energy in a multifunctional way.

• 42:08Speaker 04

  It's not just a battery pack that isn't adding anything else to the system. So what we decided to-

• 42:13Neil deGrasse Tyson?

  If a walrus is chasing me on the, on the land,

• 42:19Neil deGrasse Tyson?

  it's not catching me.

• 42:22Neil deGrasse Tyson?

  I would never live that one down.

• 42:25Neil deGrasse Tyson?

  This one is not gonna catch.

• 42:27Speaker 04

  Yeah, they can charge really fast.

• 42:28Chuck Nice?

  They're pretty fast. They're like, they're like, no, think of a hippo in the water. You would think big fat ass hippo will never catch me. But you put a hippo in the water, those suckers are moving like a little motorboat.

• 42:39Neil deGrasse Tyson?

  I know, I'm just saying, I would never live it down the next day.

• 42:44Speaker 04

  Your willpower would-

• 42:45Neil deGrasse Tyson?

  Chase me down on land, okay?

• 42:49Speaker 04

  So we've decided to make liquid that could be used as hydraulic fluids for moving robots around also as the battery. So the liquid has electrical potential that we use to power the robot. So we've called it robot blood. And now we recently published something called the robot heart, which is a stretchable pump that behaves, it's electrical. It's not the same way as our body works. There's no muscle.

• 43:13Chuck Nice?

  Our heart is electrical too, though, Rob.

• 43:14Speaker 04

  That's a good point. Thank you for, yeah, that's great. I would start using that. Stretchable soft pump that it also pumps the same blood that powers the heart and then gets, gets everything moving. So the energy is doing a lot more than just powering the system.

• 43:30Chuck Nice?

  You're creeping me out.

• 43:33Chuck Nice?

  You're creeping me out because now you got robots that have a heart and they got blood and they feel and they and they don't need to be recharged and

• 43:46Neil deGrasse Tyson?

  Chuck Chuck, if it bleeds, we can kill it.

• 43:49Chuck Nice?

  If it bleeds, we can kill it.

• 43:51Speaker 04

  Yeah.

• 43:52Gary O'Reilly?

  You've built a synthetic vascular system into your robots as well. So if it if this fluid is energy dense, it's not just as a hydraulic, is it creating? How is it creating?

• 44:07Chuck Nice?

  It's gotta be a conductor and hydraulic.

• 44:09Gary O'Reilly?

  Is it just kinetic? Is it kinetic or is it chemical? Is it both?

• 44:12Speaker 04

  It's electrochemical. So the hydraulic liquid has electrical potential in it and then we pass it by electrodes, which then turn that electrical potential into electricity.

• 44:23Neil deGrasse Tyson?

  That's very clever.

• 44:24Speaker 04

  Thanks.

• 44:25Neil deGrasse Tyson?

  And Chuck, any any liquid will have a have that property, not the electrical properties, but it'll have the the pressure properties. So

• 44:33Chuck Nice?

  Right, yeah.

• 44:34Neil deGrasse Tyson?

  He's just being clever and double dipping.

• 44:36Chuck Nice?

  That's what I'm saying. He's using, yeah. It's like our circulatory system, you know, it does many different things.

• 44:43Speaker 04

  Yeah, that's right.

• 44:44Chuck Nice?

  It does many different things. So, you know, it's making use of one thing in several different ways, which is

• 44:51Speaker 04

  That's what nature does.

• 44:51Chuck Nice?

  Yes. Yeah. So, yeah, that's very, very clever.

• 44:54Speaker 04

  Thank you.

• 44:55Chuck Nice?

  And by the way, stop it.

• 45:00Speaker 04

  All right, we'll stop.

• 45:05Speaker 04

  I'll tell the students stop.

• 45:06Gary O'Reilly?

  Is there anything you could tell us about where you like to look at going in the future? What your aim is, where your aim is?

• 45:15Speaker 04

  Well, we my lab, we want general purpose robots. And for that, they need to be adaptive on planet environments. Space exploration is a great place for that. Operate for long periods of time without needing to be recharged. Also, ocean exploration. So we're focused on ocean exploration robots and space exploration robots. But to be adaptive, you need feedback control. It's not just vision. Like, you know, there's I don't think there's an example of an animal that only uses vision. I mean, touch is an important part of adaptivity. So that's why we spent a lot of time making our robots adaptive with these stretchable fiber optic touch sensors.

• 45:54Chuck Nice?

  Yeah, that makes sense because once again, talking about nature, you know, like so many animals in nature use sensors that we would never even begin to understand. Well, we would never understand them for our use because we couldn't do it. You know, even like something as simple as a snake sticking out its tongue.

• 46:12Gary O'Reilly?

  Hmm.

• 46:13Chuck Nice?

  You know?

• 46:14Gary O'Reilly?

  Yeah.

• 46:14Chuck Nice?

  Or a dog, you know, sniffing another dog's butt, you know?

• 46:18Gary O'Reilly?

  Really? You had to go there?

• 46:20Chuck Nice?

  No, come on, let's be honest, they're not just perverts. They're taking a lot of information.

• 46:26Neil deGrasse Tyson?

  No, no, no, Chuck, Chuck, the question for the ages is, if dogs can smell things like miles away.

• 46:33Gary O'Reilly?

  Yeah.

• 46:34Neil deGrasse Tyson?

  Why do they have to get within a quarter inch of a butt to smell it?

• 46:38Chuck Nice?

  Okay, I take it back, they are perverts.

• 46:41Neil deGrasse Tyson?

  Doesn't that mean?

• 46:42Chuck Nice?

  Yeah. It's like, hey, you know what, Jim, I knew it was you from a block away, but eh, what the hell?

• 46:49Neil deGrasse Tyson?

  I'm gonna double check at a quarter of an inch.

• 46:51Chuck Nice?

  I figured I'd get up in there anyway.

• 46:55Neil deGrasse Tyson?

  I knew it. So guys, we gotta land this plane.

• 46:59Gary O'Reilly?

  Oh, before we do, Neil, I need a practical answer. Just how washable is it? Are we talking one wash? Two washes? Can I get a year out of this?

• 47:08Neil deGrasse Tyson?

  Can I iron it? Can I, can I wring it dry?

• 47:11Speaker 03

  We need more tests to be able to say that, but we've done tons of tests and they survive. The electronics part is fully removable, so you can just slide it out and then put it in the washer and dryer. But to be able to say if it will last a year, we need to test it for a year, which we haven't yet.

• 47:29Gary O'Reilly?

  All right. Good. As long as you don't mind me asking.

• 47:32Neil deGrasse Tyson?

  Cool. And one last point I just want to verify, and I think it's true. Rob, you were describing the limits of the pitching

• 47:44Neil deGrasse Tyson?

  at the pitching motion that would put a pitcher at risk of requiring Tommy John surgery. That presumably you don't know any of this in advance. You have to teach.

• 47:57Neil deGrasse Tyson?

  The system, what will and will not be the consequences of what someone does in the system, to then have a baseline of data, so that you can advise a next generation of people on what the causes and effects are of their problems, isn't that correct? You need, don't you need actual people to try this, get the Tommy John surgery and say, oh, here's why you needed the Tommy John surgery, that's after the fact, so that everybody that comes later, they can know it before the fact?

• 48:24Speaker 04

  Yeah, that's, that's a, we need to provide a benefit, that is not just on injury prevention, to get people wearing these, so that we can develop the probabilistic models that will allow us to predict when these injuries will happen. But there's also the possibility that we can analytically predict this, based off of, again, we think that accelerations are being radically under-predicted, based off of motion capture data, and if we can say that the acceleration is actually, you know, more than 500 degrees per second squared, it's 700 in the first couple of milliseconds, and that'll greatly, that'll change.

• 49:02Neil deGrasse Tyson?

  Just to be clear, that's an angular acceleration.

• 49:05Speaker 04

  Correct, yes.

• 49:06Neil deGrasse Tyson?

  When you say degrees, right.

• 49:08Speaker 04

  And that could end up with torques that are, you know, 40% more than what people think, and like, no wonder you're, you're, you're tearing your, your shoulder musculature, ligamenture, and, you know, not just the elbow, although the UCL is what fails mostly. But then, that's because that acceleration's being transferred into that elbow snap. So, we think that, yes, we want a probabilistic model, but maybe along the way, we can capture information at high enough frame rates, that we can actually understand, just from a fundamental level, of course, you're, you're breaking your ligaments, because you're applying torques that are beyond what they can handle. Wow.

• 49:49Neil deGrasse Tyson?

  Take make. All right, guys, it's been a delight to have you as guests on StarTalk Sports Edition. I will be monitoring your space going forward, and if you make new developments, be sure to come back on and talk about them, right here on StarTalk Sports Edition. Thank you. All right, excellent. Chuck, Gary, always good to have you there, man. Pleasure, it's a pleasure. As my co-host, this has been StarTalk Sports Edition, all about the future of stuff, figuring out what you're doing inside your own skin, for better or for worse, on that one, okay?

• 50:21Neil deGrasse Tyson?

  I'm Neil deGrasse Tyson, your personal astrophysicist. Keep looking out.