IBM's AI for detecting neurological state

Ajay Royyuru and Guillermo Cecchi from IBM Healthcare join Chris and Daniel to discuss the emerging field of computational psychiatry. They talk about how researchers at IBM are applying AI to measure mental and neurological health based on speech, and they give us their perspectives on things like bias in healthcare data, AI augmentation for doctors, and encodings of language structure. Sponsors: Fastly https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship. Rollbar https://rollbar.com/changelog – We catch our errors before our users do because of Rollbar. Resolve errors in minutes, and deploy your code with confidence. Learn more at rollbar.com/changelog https://rollbar.com/changelog. Linode https://linode.com/changelog – Our cloud server of choice. Deploy a fast, efficient, native SSD cloud server for only $5/month. Get 4 months free using the code changelog2018. Start your server - head to linode.com/changelog https://linode.com/changelogAlgolia https://www.algolia.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship – Our search partner. Algolia’s full suite search APIs enable teams to develop unique search and discovery experiences across all platforms and devices. We’re using Algolia to power our site search here at Changelog.com. Get started for free and learn more at algolia.com https://www.algolia.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship. Featuring: Ajay Royyuru – Website https://www.linkedin.com/in/ajayroyyuru/Guillermo Cecchi – Chris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: IBM 5 in 5: With AI, our words will be a window into our mental health https://www.ibm.com/blogs/research/2017/01/ibm-5-in-5-our-words-will-be-the-windows-to-our-mental-health/Predicting Cognitive Impairments with a Mobile Application http://www.scitepress.org/Papers/2018/67340/67340.pdfAutomated analysis of recent-onset and prodromal schizophrenia https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5887598/Prediction of psychosis across protocols and risk cohorts using automated language analysis https://onlinelibrary.wiley.com/doi/full/10.1002/wps.20491Upcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Beta for change log is provided by Fastly. Learn more at fastly.com. We move fast and fix things here at change law because of roll bar. Check them out at rollbar.com and we're hosted on Lino servers. Head to linode.com slash change log.

• 0:21Speaker 00

  Welcome to Practical AI, a weekly podcast about making artificial intelligence, practical, productive and accessible to everyone. This is where conversations around AI, machine learning and data science happen. Join the community and slay with us around various topics of the show at change.com slash community. Follow us on Twitter. We're at Practical AI FM. And now into the show.

• 0:47Daniel Whitenack?

  Well, hey, Chris. How you doing?

• 0:49Chris Benson?

  I'm doing fine. How's it going today, Daniel?

• 0:51Daniel Whitenack?

  It's going really well. I'm still in the midst of grading for my Purdue class, but I'm, I see the finish line. So how about, how about with you?

• 0:58Chris Benson?

  Just started the new job at Lockheed Martin a couple of weeks ago and have been heads down in that. And obviously the holiday season's coming up with the family. So great time of year.

• 1:08Daniel Whitenack?

  Yeah, definitely. So today we actually have two guests from IBM Healthcare. I'm really excited that Ajay Royyuru and Gishar Mo Chekki are joining us. Welcome, guys.

• 1:20Speaker 03

  Hey, hi. Hi, Chris. Hello, Daniel.

• 1:22Daniel Whitenack?

  So as I mentioned, they're both with IBM Healthcare. So Ajay is a VP of IBM Healthcare and Life Sciences Research. Gishar Mo is a principal researcher of computational psychiatry and neuroimaging. And so I'm really excited to hear about what they have to tell us here on Practical AI today and how AI is related to healthcare and psychiatry and mental health. It's going to be a really exciting show. But before we jump into those things, I'd love to give our guests a chance to introduce themselves and give us a little bit of background about how they eventually got to this place of integrating AI and healthcare and psychiatry. So Ajay, do you want to start us out?

• 2:05Speaker 03

  Sure. Thanks for the opportunity to chat. This is Ajay. I am leading our healthcare and life science research portfolio at IBM. I just completed 20 years working at IBM.

• 2:16Daniel Whitenack?

  Congratulations.

• 2:18Speaker 03

  Thank you. My background is in molecular structural biology. Prior to coming to IBM, I was a postdoctoral scientist at Memorial Sloan Kettering Cancer Center. But that was a while ago. And in moving to IBM, a lot of my research interest has become entirely computational. So the work that I do now is actually at the intersection of healthcare biology and all things information technology.

• 2:43Daniel Whitenack?

  It's really interesting how you've kind of gone through that path and eventually landed at all of these integrations of computation and IT. I'm excited to hear more. Gishermo, do you want to give us a brief intro? How did you get eventually into this world of computational psychiatry?

• 3:00Speaker 04

  Well, my background is in physics and neuroscience, but I was always interested in philosophy. And then after completing my PhD, I did a fellowship in psychiatry before coming to IBM. So naturally, mental health became very clear, clearly for me, an intersection between all of my interests, right? So this is what I'm doing now, just trying to understand how we put together mental health with AI.

• 3:35Daniel Whitenack?

  And so how did you really decide that mental health was a good target to start using AI technologies on?

• 3:42Speaker 04

  Well, one clear reason is that mental health needs it, right? So if you look at the daily practice of mental health, it's very constrained by the fact that you have neurologists, psychiatrists, healthcare providers that need to make judgments about the mental state of a patient or a prospective possible patient. And the way it's done today relies to a large extent, among other things, on the interaction between the patient and the person who, the physician who evaluates them. That interaction is to a very large

• 4:26Speaker 04

  extent determined by, you know, language patterns, right? So how

• 4:31Speaker 04

  The patient is speaking to the clinician. And outside of mental health, we have an incredible wealth of tools to study language. And that at the moment, unfortunately, are not being used for the purpose of helping clinicians during the evaluation, actually, in the end, helping patients to have better healthcare. And so it's a really dire need of help from mental health practitioners. And that's perhaps the main motivation.

• 5:04Daniel Whitenack?

  Yeah, it's interesting that you've brought up the idea of analyzing language. Because actually, when this topic was first brought up to me, I guess it wasn't the first thing that came to my mind. I was thinking, oh, we're studying like mental health computationally, maybe we're studying like brainwaves or something like that. But from what you've said, is the motivation to combine, like you said, these NLP techniques and AI and with language as related to mental health, is that really spawning from the patterns that you've seen in clinics? That they're using language as a primary means to measure and identify mental health issues? Is that the primary motivation?

• 5:47Daniel Whitenack?

  Or was it because maybe you also are able to get data more easily than some other ways or something?

• 5:53Speaker 04

  Well, yes, of course, it's in principle easier to get speech data and language data in general, because that's, we don't need any special machines to do that. But fundamentally, you know, you were talking about brainwaves. Well, speech is a brainwave. And it's very important because it is important for our behavior, right? It's one of the most essential tools that we humans use to interact with the world and with each other. And it's a very clear way in which most psychiatric conditions, but even neurological conditions are expressed. So disrupted patterns of behavior go hand in hand with.

• 6:40Speaker 04

  The disrupted patterns of of language, right? So in some cases, it's it's obvious like in psychosis. You know, it's directly mapped to language. But we see that even in conditions such as Parkinson's, there is a clear trace of of the disease in in the language patterns that are produced by the patients. And in other cases, even the language patterns that can be or cannot be processed by the patients. So it's it's more than just availability of data. It's really at the core of what defines a mental dysfunction.

• 7:21Chris Benson?

  So AJ, could you tell us kind of how you're tying together this process, these techniques of using NLP for speech into kind of a practical? I mean, what is your goal here? What are you actually trying to produce in terms of of usability?

• 7:37Speaker 03

  Yeah, so we should really talk about how this becomes very practical. But, you know, just examine the context first. The clinical encounter that used to occur entirely in the clinic, where the individual, let's say a patient, is actually coming with a scheduled appointment, is meeting with an expert practitioner, and they're having a dialogue or a clinical exam. And a clinical evidence is gathered in the course of that discussion. It may be through a physical exam or a psychiatric evaluation, as Gishar Mo is explaining. So that's a typical clinical encounter. It used to be that that was the only way in which you you, the practitioner, would know something about the patient. But what has occurred in the last decade or so

• 8:20Speaker 03

  is with the availability of many, many different forms of technology, including audio recording of speech, we are actually able to take the evidence gathering from the clinic. To something that is of similar quality but outside the clinic. And it allows the observations to move from an episodic encounter in the clinic to possibly a more continuous measurement that is occurring in addition outside the clinic as well, in the life of the person.

• 8:49Chris Benson?

  So this is not necessarily a clinician that is using this this mobile app that you're talking about. So this is used outside the clinic by non-medical personnel, non-medical people between clinic visits? Is that accurate?

• 9:03Speaker 03

  It is being used by a clinician, let's say, to do a research study involving human subjects. But instead of just observing and recording while in the clinic, the clinician is actually able to use technologies like a speech recording device on a phone to actually observe outside the clinic as well.

• 9:22Chris Benson?

  So it's still under the direction of the clinician there essentially. But is it fair to say that the the person who is being measured is also using it outside the clinic's environment between sessions?

• 9:34Speaker 03

  Right, the sessions are, could be any time during the day, could be initiated by the subject. And a conversation is happening. It could be a monologue or a dialogue that is getting recorded and then being analyzed by the techniques that Giselle will describe. But it extends the observation window from the 20 minutes or 30 minutes in the clinic to the entire day. From in the clinic premises to wherever the subject is. And as we all know, when you have a mental health condition, sometimes even showing up for an appointment in the clinic is not something that will achieve 100% compliance. So extending the observation, physical location as well as time window allows better participation.

• 10:19Speaker 03

  And of course, the mental health status of the individual is not constant, right? So let's say you have the opportunity to initiate a conversation with the mobile app and record it. You would do it in instances where you want that to be captured. And that when it is not in the clinic, doing it in this manner actually allows the subject to actually provide more information about their condition that may or may not always be reproduced in the clinic.

• 10:45Daniel Whitenack?

  Yeah, I think you've brought up a few really good points here. I know that in previous shows and in my conversations outside of the show, when I'm talking about AI in healthcare, a number of things come up, the first being like, well, we don't want people, you know, just using a smartphone app to diagnose themselves and not going to a doctor. So we don't want to kind of get rid of doctors or automate them away. But there's also, you know, privacy concerns. So it sounds like that, you know, in your case, you're not just like having a recording of all conversations at all point to to to improve diagnosis, but they are kind of like clinical sessions, but you're recording them at the participants indication throughout, you know, between clinical visits, but then also it's being

• 11:30Daniel Whitenack?

  reviewed by a doctor, right? Do you view this as kind of like an augmentation to the to the doctor's current workflow or, you know, or something kind of that that couldn't turn into a completely different workflow for for helping diagnose and treat and measure mental health?

• 11:49Speaker 03

  You're right. It is actually very thought out, deployed really as an augmentation to how the clinician observes and and makes decisions for the patient or subject. It is with informed consent and it is with the ability of the participant to turn the observation on or off, right? So it is not always on and the participant is actually deciding when they want to actually allow the observation to take place. And after the observation is done, so let's say, you know, the conversation you and I are having, if I had subjected this conversation through that consent, after I am done speaking, me

• 12:30Speaker 03

  as a subject, I'll get to review what it is that has actually been observed from this. And then I choose whether the clinician is now being provided this input or not. So every every session is therefore has that rigor of consent.

• 12:47Daniel Whitenack?

  So that's fascinating to me, just kind of as I'm trying to imagine if I had this app on my device going around through daily life, I'm curious, how do people choose to turn it on and off, you know, in terms of the, if you're looking at lots of different use cases, do people tend to have it on most of the time, kind of knowing that that's recording? Does it make them nervous? Does it change their behavior? I'm trying to imagine if I was that patient, how I would react to having this tool.

• 13:16Speaker 03

  Right, so, you know, we have done some analysis with retrospective data, that means sessions that have previously been recorded already in a clinician's office, for example, and built the analysis methodology based on such retrospective data. And then we moved into the very carefully constructed prospective studies that you're asking about. In the prospective studies, not only is the individual first informed what it is that every session will be about and how they have to participate, but for each session, they are actually taking some steps. For example, in one study, the technology is actually deployed as an app on the phone, and they are actually starting the app, the app will actually prompt them with certain

• 14:01Speaker 03

  questions, then Gisemmo can walk you through actually what the example questions are and what an example session is like. So it's initiated by the individual, and, you know, they go through it, it may be a few minutes, five minutes, about 10 minutes, and then they conclude the session, and that's the information that then gets used to analyze.

• 14:21Daniel Whitenack?

  Yeah, I'd love to turn to Gisemmo, actually, on that same point, I was already thinking of kind of a follow-up to this in terms of, on the technical side, the people that are actually, you know, implementing the technical, you know, the models and the interaction of the models with the app and all of that, it sounds like there's a real importance between those technical people and the doctor's expertise, like you just mentioned, kind of developing this question and answer session. Could you speak more to that interaction and the importance of that, Gisemmo?

• 14:56Speaker 04

  Yeah, it's a great point. And it's something that we have developed very carefully. In all the studies that we have published and we are conducting, we are working very close to clinicians, psychiatrists and neurologists. And that's very important, both because we want to eventually, what we develop, be adopted by the field of mental health, but also because we are interacting in a very productive way. So, and I mean, we can think of this in two parallel avenues. One is the typical AI, big data, science approach, right?

• 15:39Speaker 04

  So we try to create features of all colors and shapes and throw them against the wall and see what sticks. But at the same time, of course, you know, the space of features is, for all practical purposes, infinite. So you always need knowledge, right? So at the same time, what we are doing is, is by interacting with clinicians and biomedical researchers, we are trying to, you know, open up their minds and trying to understand how the features and the symptoms that they have found to be most relevant can be turned into algorithms, right?

• 16:21Speaker 04

  So I can give you a very concrete example of both, right? In the first case, when we create features, we have results showing that we can discriminate Parkinsonian patients that are on the medication, Levodopa, or off the medication, using features that include frequency components of the voice that are not detectable by the human ear, but they are still there because the drug affects, it's psychoactive, so affects your nervous system, and of course, trivially, affects your voice. On the other side, we study psychosis, and one essential component of what defines a psychotic state of

• 17:06Speaker 04

  A person is what psychiatrists call flight of ideas. And that is the notion that these patients may be talking about something and very dramatically jump the topic to something completely unrelated. So what we did there was to, using NLP techniques, create an algorithm that will detect those jumps using what's known as a technique called semantic embedding that is very commonly used in NLP. So this is one way in which we interact between the both worlds. So learn and formalize as much as possible, you know, decades or even centuries of knowledge

• 17:49Speaker 04

  in psychiatry, psychology, neurology, and at the same time trying to leverage all the power of AI, NLP, signal processing, and computer science in general. So I hope that that gives you an idea.

• 18:03Daniel Whitenack?

  Yeah, definitely. So following up on what you were just saying, Gajermod, it sounds like a ton of different knowledge from psychiatry that you're trying to kind of infuse in these algorithms and these techniques. It sounds like there's a bunch of different applicable NLP techniques, like you were just talking about semantic embedding and other things. I was wondering if you could just walk us through like what the data is like that you're actually gathering as far as both the features you're using for inferences and also the training. For example, if you're getting audio, does that mean you're kind of gathering the audio in this question and answer sort of session? And then converting, doing kind of speech to text?

• 18:47Daniel Whitenack?

  So using a first model to get the text? And then the text is input features to other models that do like the semantic embeddings or other things? Could you give us a little bit of a sense of that data flow and the structure and type of data?

• 18:59Speaker 04

  Absolutely. Yeah, so we work, as Adi was saying, with either clinical interviews or speech samples that are gathered, having a clinical evaluation in mind. So we have a monologue speech samples. We have written text in some cases. And we also have dialogue in other cases. And the context is that we either have semi-structured clinical interviews that seem to be the most effective. And by semi-structure, I mean it's not following a very precise flow

• 19:41Speaker 04

  of a structured flow of questions and follow-up, but not trying to nudge the patient into talking about something and expressing themselves. In other cases, we have monologue with anchor subjects. In some cases, it can be very short. And we typically target naturalistic samples. So, for instance, we ask the patients to talk about typical day in their life, or how their week was, or where they would like to go for vacation. Because the idea is that with those type of prompts, we can reuse them, as Ajay was saying,

• 20:23Speaker 04

  on a weekly or even daily basis, so we can monitor their state. Then, what we do with the data is, yeah, of course, we have, in the case of speech, we have the audio files. And we process them as such. We extract voice features that are very well established in the field of voice processing. We extract features related to, for instance, the pause distribution, between words, the phoneme structure, something that's called the vowel space. It's how you pronounce your vowels. That might be different, for instance, across different accents,

• 21:07Speaker 04

  even in the same language. And then, on the lexical side, we extract the expected low-level features, so we can parse sentences into their grammatical components. So we can understand how verbs and nouns and adjectives are used and where in the sentence. And that has shown to be important in center conditions. We also extract, as I was saying, the idea of semantic embedding. So that allows us to take a word or a sentence and have a notion of how similar

• 21:48Speaker 04

  that word is to other words. We can use target words that are of interest for the particular condition and understand how the patient is in their discourse is getting closer in meaning or farther in meaning for certain concepts that are relevant. And then we also extract higher level features and those are more aligned with, as I was saying, concepts from psychiatry. Just to give you an example, we have algorithms that can measure how metaphorical the content of a phrase is and that is relevant in psychosis

• 22:31Speaker 04

  because one of the symptoms of psychosis is disruption of your appreciation of metaphors, both in terms of how you understand them and how you produce them. So that gives you an idea of the full spectrum of features that we analyze. We study from the audio and from the text side of language.

• 22:59Chris Benson?

  So, Gishermo, that is quite a list of features that you're extracting, kind of going from the phoneme structure, vowel pronunciation, accents, a lot of the lexical stuff you just covered. Are there certain patterns that you have found through the data that have been more relevant than others, that you're noticing seem to be weighted heavier in your analysis through NLP? Are there things that are sticking out as particularly important or has that been established?

• 23:29Speaker 04

  Well, what I would say is that language and even more speech production is such a complex phenomenon, right? It's so, you know, we know from computer science how difficult it is to deal with it, how difficult it is to produce coherent language. It's, it comes natural for us humans to do it, but any, you know, any disruption in, in the health of, of your brain, will have immediately an effect in language. So, like I said, even for conditions that are considered, traditionally have been considered

• 24:11Speaker 04

  motor disorders or Parkinson's, we know and we found, and we are not the only ones, we found very clear effects in language and even in content, right? So even if you have something that supposedly is a motor dysfunction, the content of what you're producing as you speak is affected. So, yes, we can talk about comfor, maybe, of features that seem to be popping up often, and one is like the one I mentioned, that we originally developed for psychosis, the idea of measuring flight of ideas as semantic coherence.

• 24:50Speaker 04

  That seems to be useful to analyze different conditions and even situations in which, for instance, a patient may take a psychoactive drug like ecstasy or methamphetamine. But if I had to answer your question, I would say that every single aspect of language is affected, or differently, but it's affected because, again, language is, is a very complex phenomenon that involves many, many different aspects of your brain function.

• 25:27Speaker 04

  So any tiny disruption will have an effect.

• 25:31Speaker 03

  Another interesting thing that, you know, Gisemmo focused on early enough and been very instructive for us is to really emphasize the spontaneous production of speech. So basically not go in a direction of some rote answer, but rather have the individual, you know, create an answer, pre-existing context and answer doesn't exist in that person's mind yet. So that spontaneous production is actually eliciting some of these features that he is describing. You know, it's enhancing the visibility of those features quite well. So Gisemmo, maybe you want to describe actually the picture test, which really is a very nice spontaneous production.

• 26:09Speaker 04

  Yes, yeah, that's a very good example. So we are studying actually a different, a number of conditions using this approach that was initially developed decades ago to study cognitive decline. And you can look it up, it's called the cookie theft task, and there are variations of that. And essentially you're shown a picture, it's a drawing, it's a hand drawing of a typical, you know, 1940s, 1950s Americana household situation.

• 26:47Speaker 04

  There is someone who seems to be a mother doing the dishes, but she seems to be absent-minded. And there are two kids, a girl and a boy, and the boy is standing on a stool trying to get a cookie from a jar. So the task is just to describe that in your own words. It's something that takes two, three minutes at most, it's very natural. And variations of that can be used to be repeated, you know, very often, so you don't get bored.

• 27:23Speaker 04

  And what happens is that when you analyze the content of that description of the task, what you say, what type of words you use. But also the structure, even the syntax of what you are saying, how you're constructing the sentences and how fluid or how simplified your speech is, that contains a huge amount of information about your cognitive state. And that has been used by manual raters, like I said, over decades to have an estimate of your cognitive state. But now we can do

• 28:05Speaker 04

  that in a completely automated way. And we have shown that we can infer the clinical scales that are produced by the human evaluators with a very high accuracy with the advantage that we can do this remotely. And like I said, we can do this at a very high frequency and without having to bring the patient to the hospital or the clinician to the house of the patient. And it has value that goes even beyond the idea of measuring or estimating cognitive decline, because it can be applied to many other conditions. Because as I was

• 28:49Speaker 04

  saying, even something that on the surface looks so natural as it's coming in such a picture, requires a huge amount of brain real estate. And any failure will leave an imprint in the way that you perform these tests.

• 29:05Daniel Whitenack?

  I think that leads into a question that's been kind of in the back of my mind through this whole conversation. I mean, you've mentioned that the way in which you gather data and kind of the spontaneity of it is really important. And that immediately kind of leads me to think about bias in data, both in terms of the way that you gather it. But you've also already mentioned like accents and language variety and that sort of thing. And we've already seen kind of, you know, disasters in healthcare scenarios where maybe you're trying to like, you know, diagnose skin lesions or something and your data only has data from like light skinned people or something. And I would guess that the same sorts of things exist in language in the sense that

• 29:47Daniel Whitenack?

  Like both education level, maybe, but also regional accents, you know, second language, speaking people not speaking in their first language, all of those things kind of come into play when we start thinking about language. And I know IBM has also done a lot of work around fairness and bias. I was wondering if that has entered into this work yet, or is it something that you want to kind of probe further in the future?

• 30:12Speaker 04

  So, yes, of course, that we take that into consideration and we try to account for those, I don't know to call them biases, but there's the context of the person, right, is the the personal context and and even maybe the group context. Now, we have several cases in which we can track the patient over time. And for those, we have the best way of accounting for various issues because we have the history of the patient. So in in some of the studies that we have conducted, we we know that if we didn't have the story, the context of the person, we

• 30:56Speaker 04

  could not get any results. Trivially speaking, for instance, if you don't know that the person is a male or female, the acoustic content would be confounded, right? So when possible, we try to precisely have studies that track the individual and and that accounts to a large extent for those biases, as you mentioned. But also, it's really part of one of the goals that we are pursuing is that the possibility of personalizing the evaluation and eventually the treatment for a person, right? So just being able to track someone on on a daily basis that

• 31:41Speaker 04

  is taking a certain medication or following following a certain treatment, it's one of the ultimate goals that we want to do. And in those cases, we have ways to account for the biases. This is much easier to account for the individual biases.

• 31:56Chris Benson?

  So Ajay, curious, can you kind of describe what the output looks like here? Are we really talking about, you know, is there one diagnosis? Or do you have multiple diagnosis as an output? And what is your, what do your models look like to support that output? Is it different models for each diagnosis? Or one model to rule them all, as you might say? What is that output? And how are you structuring your models to get to that output?

• 32:20Speaker 03

  Sure, yeah. Actually, just to continue for a second on the issue of language and bias. You know, the retrospective work that Kishore Mo has done, he already looked at several different languages and people speaking in their native languages, English versus Spanish versus Portuguese and so on. So I think that's very important to actually think of this science, as well as its eventual use, as being close to what the person already experiences and not actually take the person into some new territory where, you know, that distortion or bias is actually more pronounced, right? So I think that's a research goal that we have to maintain is to actually make the technology work for the person and not the other way around.

• 33:05Speaker 03

  So that is a, is a quest that we are continuing on. But the retrospective work already shows us that that actually is, is possible. So we are encouraged by the fact that we should be able to bring these technologies into different languages. Okay, so to your question of, you know, how does results actually get reported? And exactly what are we describing in that report? First, I would say that this is not really diagnosis. There's nothing clinically diagnosis-like that is being generated here. Rather, what we are doing is surfacing features that the clinician is already trained to look for and make sure that those features are actually visible to the clinician. So the diagnosis and possible help to the patient,

• 33:47Speaker 03

  whether it is in terms of diagnosing or in terms of treating, is being done by the licensed expert practitioner. So all that we are doing is, using this tool, we are making sure that the patient's own experience is being captured sufficiently well. Features that are clinically relevant, like the ones that Kishore Mo is describing, are actually being captured and surfaced. And it is on the basis of those features that a trained practitioner would actually then be prompted to do what they are trained to and licensed to do already. Right. So this is augmentation is not, you know, attempting to do what the practitioner does already, which is diagnose and treat. So the report actually has, you know, both graphical as well as, as well as numerical

• 34:32Speaker 03

  and textual form of these features being surfaced. Whether they are in a, in a, in a graph, so the, you know, disjoint thoughts that Giselle Mo is talking of can typically be presented in a graph form. And so you either have disjoint graphs or extremely complex graph that is actually demonstrating the complexity of the word choices and the, and the context that the person is talking about. And a trained psychiatrist is actually then able to look at that and as they're accustomed to, use those features to actually then be able to, to make better decisions. For, so the most easy way in which this might get used is for screening purposes. So you're actually getting a psychiatrist in the future might actually be getting this

• 35:16Speaker 03

  kind of report just to keep tabs on what is happening in life of a person who needs to be watched. And you're using that to actually just watch and screen. And when, you know, it gets to a threshold of some concern, you're actually indeed intervening, the practitioner at that point intervening and doing what they're trained to do. But now you actually have extended the observation to life of the person and you're able to observe more thoroughly and act upon it before it becomes catastrophic. Right. So I think that's the more likely usage here. Diagnosing and treating by itself is actually the hardest problem and that we really need to have practitioners do.

• 35:56Daniel Whitenack?

  Yeah, yeah, I think that that brings things together really well and gives us a lot of great context for the use. And as we kind of wrap up here for, for the episode, I'd love to first off, just thank you guys for working on some application of AI that, that really is making a positive difference for people. That's something that, that Chris and I always, um, you know, want to promote as much As we can, but I'd love to just get get you guys to share as we kind of wrap up what you're excited about as far as either results that you have now or maybe next steps that you're going to. And then also for listeners who are maybe more interested about this subject, either on the NLP side or the or the application side.

• 36:40Daniel Whitenack?

  Where can they find out more about your work or the techniques that you're using? So if you guys could give us a little bit of that perspective, that'd be great.

• 36:48Speaker 04

  Well, I have to say that what is keeping me up at night with excitement is work that we are developing around doing something similar to what we were describing. But in the context of of therapy, right, in therapy sessions with again, the same idea of expanding and providing additional tools to the therapist to track the evolution of a patient that is undergoing some type of of of therapy. And being able to integrate information from different sources that are relevant to the particular individual

• 37:32Speaker 04

  that is undergoing this this therapy. I think this is one of our next frontiers and it's challenging, but at the same time very exciting.

• 37:43Speaker 03

  And you know what what excites me the most about this is a lot of the mental health as well as neurological conditions that individuals experience has really been either not attended to or been misdiagnosed and not the right kind of help provided or provided only in bursts and in acute situations, but not really more continuously. What we are witnessing through this work as well as all other things that are happening with Internet of Things and and how technology is intersecting with our daily lives is a change that we are seeing and experiencing where the technology actually allows us to do things in a different way. But in this case, you know, going from episodic encounters in the clinic to a continuous measurement

• 38:29Speaker 03

  done in the convenience of your home and and in your daily routine. What that does is actually brings attention and allows allows practitioners to actually address real issues that are beyond what is happening in the clinic. So it might extend the reach of help that people get. And that is a change. That is such a huge change for the positive. Because the unmet needs for mental health are huge. And, you know, using these kinds of technologies, one is actually able to hopefully increase the aperture through which these needs are addressed. So that change, I think, is very much for the positive. And, you know, people who do experience these conditions, whether it is anxiety, depression, cognitive decline, they need that help.

• 39:14Speaker 03

  And we are conceivably moving in a direction where that that becomes possible for them.

• 39:19Chris Benson?

  Balaji and Gijeramo, thank you very, very much for coming on to this episode. We will definitely put links to your papers out in the show notes so that our listeners can access those. If people want to find out more or reach out to you, how would you like them to reach out to you?

• 39:36Speaker 03

  So one place to get to is the IBM Healthcare and Life Science Research website, which features a lot of our breaking scientific news, including the work that Gijeramo is talking of. And that's just at research.ibm.com/healthcareandlifesciences. And that's a good place to go.

• 39:56Chris Benson?

  Fantastic. Thank you both for coming on the show. Wish you very well in this work. Goodbye.

• 40:01Speaker 03

  Thank you. It's a pleasure talking to you.

• 40:06Speaker 00

  All right. Thank you for tuning into this episode of Practically AI. If you enjoyed the show, do us a favor, go on iTunes, give us a rating, go in your podcast app and favorite it. If you are on Twitter or a social network, share a link with a friend, whatever you gotta do, share the show with a friend if you enjoyed it. And bandwidth for changelog is provided by Fastly. Learn more at fastly.com. And we catch our airs before users do here at changelog because of Rollbar. Check them out at rollbar.com/changelog. And we're hosted on Linode cloud servers. Head to linode.com/changelog, check them out, support this show. This episode is hosted by Daniel Whitenack and Chris Benson. Editing is done by Tim Smith. The music is by Breakmaster Cylinder. And you can find more shows just like this at changelog.com.

• 40:48Speaker 00

  When you go there, pop in your email address, get our weekly email, keeping you up to date with the news and podcasts for developers in your inbox every single week. Thanks for tuning in. We'll see you next week.

• 41:04Speaker 05

  I'm Tim Smith and my show Away From Keyboard explores the human side of creative work. You'll hear stories, sometimes deeply personal, about the triumphs and struggles of doing what you love. Jumping off into the abyss is kind of my skill.

• 41:20Speaker 05

  And so I'm not saying that it's not scary. I'm saying that perhaps my skill is just not being able to estimate how scary it will be. New episodes premiere every other Wednesday. Find the show at changelog.com/afk or wherever you listen to podcasts.