Intelligent systems and knowledge graphs

There’s a lot of hype about knowledge graphs and AI-methods for building or using them, but what exactly is a knowledge graph? How is it different from a database or other data store? How can I build my own knowledge graph? James Fletcher from Grakn Labs helps us understand knowledge graphs in general and some practical steps towards creating your own. He also discusses graph neural networks and the future of graph-augmented methods. Sponsors: DigitalOcean https://do.co/changelog – The simplest cloud platform for developers and teams Whether you’re running one virtual machine or ten thousand, makes managing your infrastructure too easy. Get started for free with a $50 credit. Learn more at do.co/changelog https://do.co/changelog. AI Demystified (FREE five-day mini-course) http://learnai.infinite.red/ – Get an introduction to the most important concepts, types, and business applications for AI and Machine Learning. This course is 100% free. The Brave Browser https://brave.com/cha913 – Browse the web up to 8x faster than Chrome and Safari, block ads and trackers by default, and reward your favorite creators with the built-in Basic Attention Token https://basicattentiontoken.org. Download Brave https://brave.com/cha913 for free and give tipping a try right here on changelog.com https://changelog.com. Featuring: James Fletcher – LinkedIn https://www.linkedin.com/in/james-fletcher-9289156a, X https://x.com/jmsfltchrChris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: Grakn Labs https://grakn.ai/GraphSAGE http://snap.stanford.edu/graphsage/Prologue https://en.wikipedia.org/wiki/PrologGraph Nets (TensorFlow) https://github.com/deepmind/graph_netsGrakn KGLIB https://github.com/graknlabs/kglibGrakn example projects and tutorials https://github.com/graknlabs/examplesGrakn Blog https://blog.grakn.ai/Upcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Being with your change log is provided by Fastly. Learn more at fastly.com. We move fast and fix things here at Change Law because of Rollbar. Check them out at rollbar.com. And we're hosted on Leno cloud servers. Head to leno.com slash change log.

• 0:16Speaker 00

  This episode is brought to you by Digital Ocean, the simplest cloud platform out there. And we're excited to share they now offer dedicated virtual droplets. And unlike standard droplets, which use shared virtual CPU threads, their two performance plans, general purpose and CPU optimized. They have dedicated virtual CPU threads. This translates to higher performance and increased consistency during CPU intensive processes. So if you have build boxes, C I C D, video encoding, machine learning, ad serving, game servers, databases, batch processing, data mining, application servers, or active front end web servers that need to be full duty CPU all day, every day. Then check out Digital Ocean's dedicated virtual CPU droplets. Pricing is very competitive, starting at 40 bucks a month.

• 1:01Speaker 00

  Learn more and get started for free with a $100 credit at do.co slash change log. Again, do.co slash change log.

• 1:21Speaker 00

  Welcome to Practical AI, a weekly podcast about making artificial intelligence practical, productive, and accessible to everyone. This is where conversations around AI, machine learning, and data science happen. Join the community and slay with us around various topics of the show at change.com slash community. Follow us on Twitter. We're at Practical AI FM. And now into the show.

• 1:47Chris Benson?

  Welcome to another episode of the Practical AI Podcast, where we make artificial intelligence practical, productive, and accessible to everyone. I am one of your co-hosts, Chris Benson. I am Principal AI Strategist at Lockheed Martin. And with me today as usual is my co-host, Daniel Whitenack, who is a data scientist at S I L International. How's it going today, Daniel? It's going great.

• 2:10Daniel Whitenack?

  It seems like the past week or so has been the week of messy data for me. So I've been been dealing with a bunch of missing rows and weird data issues. It seems like for the for the past week, which maybe that's like typical for every person in AI. And everyone's like, oh, that's my week every week, but seems particularly to have hit me this this last week. But what about you? You're at GTC, right?

• 2:36Chris Benson?

  I am. I'm at Nvidia GTC, which is their GPU Technology Conference in Washington DC. It's going on now, although right now I'm hanging out in the hotel room so we can do this. But a lot of fun. I came to Washington at the beginning of this weekend for the Alpha Pilot race. And you know, we've had a recent episode on Alpha Pilot and that was the second of four. Super cool doing that and had a lot of fun, did some various things on stage. And then today at GTC, I've got a session coming up that I'm leading. It's kind of a fireside chat where I'm kind of both moderator and panelist together with a couple of other really, really smart people.

• 3:12Daniel Whitenack?

  Yes, that sounds great. I hope that maybe some of that will be available at some point where people can access it.

• 3:19Chris Benson?

  Yep, I think they put it all online afterwards.

• 3:21Daniel Whitenack?

  Awesome. If you want to follow up on that or interested in in other things related to Nvidia, you can definitely connect with us on our Slack channel. If you go to changelog.com/community, you can join us on a public Slack and/or on LinkedIn and ask some of those questions and follow up on guests and all of those different things.

• 3:44Chris Benson?

  Well, today we got a treat. We have a guest by the name of James Fletcher, who is Principal Scientist at Grakken Labs. And I think we're gonna talk all about intelligent systems and knowledge graphs in the in the minutes ahead. Welcome to the show, James.

• 3:59Speaker 03

  Hi, guys. Thanks very much for having me along.

• 4:02Chris Benson?

  So I noticed on your LinkedIn as we were prepping for the show, it said a couple of things and one of them is a little bit, but the first one it said is it says that you're presently leading research on machine intelligence and cognition at grakn.ai. But it also, and anyone that listens to the show much knows I'm an animal nut. I just own that moniker. It says that you are an entrepreneur with a background in computer vision for automated veterinary diagnostics. And I just, before we got into the, the main topic, I just wanted to ask you about that. If you could take just a second as a tangent and, and tell us what that means.

• 4:32Speaker 03

  Yeah, absolutely. So that was quite a fun project. And that was my first foray into machine vision, which actually started when I was studying. I was studying general engineering at university and ended up in this specialization in machine vision. And I really didn't see that coming. I always thought I was going to head towards mechanical engineering or something like that. And then when I saw the capabilities that were coming out in machine learning at the time, I was like, okay, wow, this is really good stuff. This is disruptive, right? You can really do something new with this and no one's using this. This is clear in industry. I'm studying under Professor Andrew Zissman at the time,

• 5:14Speaker 03

  who's quite a big name in computer vision. And we got on well and coming out of that course, I said to him, you know, is it okay if I look at actually commercializing some of these algorithms? This stuff is clearly enough to warrant a whole company around it. And so off I went and started doing that. That was actually a family business. My dad is also an engineer. And so the two of us decided, you know what, actually let's give this thing a shot.

• 5:41Daniel Whitenack?

  How was it? Cause I know like the transition of research out of university into the commercial world can be kind of an interesting journey. Was that awkward in trying to convince the right people?

• 5:54Speaker 03

  That's a good summary of the journey.

• 5:57Daniel Whitenack?

  Awkward, you mean?

• 5:58Speaker 03

  Well, no, I wouldn't say, no, I wouldn't say it was awkward, but we weren't knowledgeable on IP and all of that kind of thing. But I mean, at the end of the day, it was released open source by the university. That was actually really pretty trivial. No, so, but that actually formed, that was an interesting conversation also because it had been implemented and released open source in MATLAB. But you know, that wasn't actually commercially useful to us. So that was a rewrite job from the start to put it into Python so that we could actually, you know, productionize that. And then it was really happenstance and things that put a lot of things together for us. We, we had these generic algorithms, so we wanted to find a place to use them. And as a family, actually, there's a hobby farm involved here, which my parents have. And we happen to have connections with the veterinary college nearby.

• 6:42Speaker 03

  So we went to them and we said, you know, we need a vertical, we need a special, a specific task that we can hone in on. To actually, you know, prove the usefulness of these algorithms and what they can do. And so we were looking at veterinary science and they said, yeah, that's exactly what we need. We don't have anyone who's actually being able to help us at the university do this stuff at the moment. So we launched this whole research effort with them. What was interesting actually as that developed was, and this is a lesson in being an entrepreneur, I guess. Is that the core value of the business actually moved sideways from the AI algorithms that we were working with, from the machine vision. And into the actual hardware and robotics that we needed to actually fully automate the process.

• 7:25Speaker 03

  Because it's all very well having a machine vision algorithm that automates, you know, the skill of looking through a microscope. But if you don't have a machine that puts the, the microscope slide on the microscope essentially, right? I mean, that's, I'm really simplifying it, but I'm sure you got the idea. Then, you know, how many samples can you actually run? Like, what's the actual improvement you get to that whole system? And so actually, that was the area that was much harder. Once you have an image on a computer, you're kind of laughing. But getting to that point was actually a little bit, a little bit more tricky. But yeah, the end goal was actually trying to control parasite burdens in animals, particularly grazing livestock. But the, but that translates sideways actually into human health.

• 8:07Speaker 03

  Because rough statistic is that two billion of the world's population actually has this parasitic worm infection. There's a number of different reasons why you might want to work on this particular problem.

• 8:18Daniel Whitenack?

  And there's a lot of samples to run.

• 8:21Speaker 03

  There's a lot of samples to run. Exactly, exactly. You hit it in a nutshell.

• 8:26Chris Benson?

  Well, that's pretty fascinating. And I just, as a way to close that off, I run an American nonprofit charity called the Animal Institute, which brings technology like AI and computer vision and such to solve problems in animal welfare. So if you ever have any interest in discussing these topics further, I definitely have a playground to play in.

• 8:43Speaker 03

  Well, absolutely. Sounds like we should definitely go there.

• 8:46Daniel Whitenack?

  I was just thinking while you were talking about it, I mean, the application is definitely interesting and valuable, but I also think it illustrates, I get asked all the time, and maybe you do as well, like, what should I start working on to get into machine learning or get into AI? What kind of problems should I start looking at? And I think like, the best thing that you can do is start working in an area where you have some connection or where you're passionate about. So for you, this was kind of a connection between what you studied at university and worked on in research, along with your family, and engineering, along with like this hobby farm and the connections you had with the veterinary school. So it made a lot of sense to go into that vertical. So yeah,

• 9:27Daniel Whitenack?

  that's what I think, you know, people should consider is trying, just try something out that you're passionate about, because those are usually the things that you would stick with long enough to learn and to experiment and to level up.

• 9:41Speaker 03

  I totally agree with that. I think that's a really good point, because what you're really saying there is that you will exceed yourself better in things where you are motivated, right?

• 9:50Daniel Whitenack?

  Yeah, yeah, definitely.

• 9:51Speaker 03

  Not just learning machine learning, but everything. So if you've got that motivation, the more motivation you can summon and put into one place, then like, absolutely, you'll you'll double down on it, right? The passion will get you through the hard times, right? When you're missing all those rows in your data set, right?

• 10:06Daniel Whitenack?

  Yeah, yeah, for sure. Thanks for the extra motivation this week.

• 10:09Chris Benson?

  I was gonna say, this has turned into completely a motivational show, totally unexpected in this area. And we haven't even hit the main stuff we were expecting to talk about, so.

• 10:17Speaker 03

  No, there you go.

• 10:18Daniel Whitenack?

  Well, speaking about that, I mean, like, how do you get from robotics and microscope slides to knowledge graphs? What's that kind of journey like?

• 10:28Speaker 03

  Yeah, well, unfortunately, I don't have some twisting roller coaster to tell you, only that what I wanted to move out of doing the technical work on that project, I was looking around for the next challenge. I suppose one of the things that I really like to be is sort of like impact driven in terms of the choice of where I wanted to work. I wanted to see something where you can, you know, where you get that value actually disposed. And so you could see that project with the same, like you had like, you could see where you were gonna actually make some impact. And looked around all the roles and had this really great conversation with Haikal Prabali, the CEO here at Grakken. And we were a really over excited conversation when we first met where he was explaining to me

• 11:11Speaker 03

  all of the ethos about Grakken and the vision that the company has. And I was pretty sold to work here straight off the bat from that conversation. So really just a pivot. His ethos is to take on people that have demonstrated themselves within the scope of what they do, not necessarily that they have to be people who've worked on, you know, knowledge graphs or or graphs in, in, at all in the past, right? So he's very open minded about which field you're coming from, coming from robotics himself, actually. So there's a bit of resonance there.

• 11:42Daniel Whitenack?

  Cool. Well, maybe you could just define. So if I go to like the Grakken website, which is grakn.ai, we'll put it in the show notes. You talk about a couple of things which you've already mentioned, and I think it'd be great to kind of dig into those terms a little bit more. So one of the things you mentioned is intelligent systems on the website. And then you just mentioned knowledge graphs. So maybe you could start out by just kind of sharing what Grakken means by intelligent systems and what sorts of intelligent systems people are developing out there.

• 12:14Speaker 03

  Yeah, absolutely. So the terminology that's being used at the moment is an interesting and kind of hot topic of its own. And naturally, you're gonna get a Grakken biased spin while you're talking to me. But the general ethos, I think it's better to start with knowledge graph.

• 12:29Daniel Whitenack?

  Okay.

• 12:30Speaker 03

  It's good if we also start with what we, how we describe Grakken and what that does for people, right? So Grakken itself is a database. Right? And typically, when you're talking about knowledge graphs, that's what you're talking about. You're talking about some sort of actually large store of knowledge. Now, knowledge graph itself is essentially totally synonymous with knowledge base, which would be like the mathematically correct terminology that's been abused on the web a lot for for other things. So we tend to go with knowledge graph. It's a little bit sexier. Uh, and also immediately gives someone with without experience in knowledge bases, an idea of the shape of the data, which is a, you know, a graph in the computer science sense.

• 13:12Speaker 03

  So what do we actually mean by like knowledge graph as opposed to just graph? So there's all sorts of different graph types of format all over the place. But what we're trying to build here is a system which takes you from, you want to make that leap from a graph full of data to a graph full of knowledge.

• 13:31Daniel Whitenack?

  Yeah, I was just gonna jump in and say, I think that's maybe the part where I struggle. I think a lot of people have dealt with databases. And maybe some people are familiar with graph structured data like, oh, I've got this node, which is a person and another node, which is another person. And they're connected by, I think the terminology is some edge that like is like this person is friends with this person or, you know, something like that. When does like a database or graph data go from being just a database to being a knowledge graph? What's the idea around that?

• 14:06Speaker 03

  Yeah, so the idea is that the way the way that we built the system up is how can we capture all of these different kinds of knowledge? Right? And so what we have is we built a knowledge representation system. Right? So Graken itself is actually, everything that's in Graken is actually built on top of a graph database. That's actually the start of the innovation. I think that helps people understand what we're doing. So we started, if you start with a clean slate and you're gonna build a project, we started with a graph database. And then we built other things on top of that. Right?

• 14:38Chris Benson?

  Can you talk a little bit about what the different, when most people probably think database, they're probably thinking of a relational database, kind of more the classical Postgres and those kind of databases. As you explain here, could you differentiate between what a graph database and a relational database are so that people can, if they're not already familiar, they can kind of make that jump?

• 14:59Speaker 03

  Yeah, exactly. So as we were already talking about, right? Like, uh, so we've got a graph in the in the computer science sense as opposed to in that like the X Y plot sense in that we've got nodes and edges interconnected. Right? So in a typical graph, a node might represent anything. For instance, I like your example. From one node, which is a person, to another node, which is a person you'd has, like has friends as the label of the edge in between those two nodes, right? So what we can do is rather than a relational database forces you to store everything in tables. Right? That's what you've got. You've got a set of filing cabinets. And each file in those respective cabinets may have a reference written on it that links you to a file in another cabinet. Right? That's the kind of structure of the data that you've got available to you.

• 15:43Speaker 03

  But what we find is that, well, as soon as we're dealing with data that's more representative of a network, then dealing with it in those kind of tables gets really messy, really, really fast. Because as soon as you've got like one thing which is connected to eight other things and eight different file cabinets, and all of those are also connected to eight different things, you know, you get into a big mess with that starting structure.

• 16:02Chris Benson?

  Doesn't scale well there across laterally.

• 16:04Speaker 03

  Exactly. And well, the idea is that when you're actually trying to build some kind of application with those things, the complexity that you as the user of the database has is enormous, right? Suddenly you have to try and control this structure that wasn't really designed for the data that you have. So then you go a layer up and you say, okay, now I need a graph structure to actually more naturally represent my data. Right? And so that's where graph databases are kind of born.

• 16:33Chris Benson?

  And when you say kind of more naturally, other than that it reflects the data, the relationships between the data very accurately, is are there any other advantages? But going graph, if somebody is trying to make that decision today and they're looking at that, maybe they're looking at Gracen. What are the benefits of going graph database versus relational database?

• 16:51Speaker 03

  I mean, I think you kind of say it in a nutshell in that the idea is to be able to naturally represent a network data as it is.

• 16:59Chris Benson?

  Is it easier to get to the data, though, in that way and not having to do giant SQL, classical SQL?

• 17:05Speaker 03

  Exactly, right? And we go a level more natural again when we actually come to the knowledge graph element that Gracen builds on top, right? So once you've got your data in like a graph form. Now you want to be able to concisely refer to and search your data and reference what you're looking for, right? So the the major innovation, there's, I would say, there's two major parts that you need to understand to figure out what Gracen is and why it helps you. The first thing is you've got, we've got this knowledge representation system. And we have this flexible model. I don't think we we want to talk in like technical depth on on all of the intricacies of that. Yeah, yeah. You can basically make entities, relations and attributes. We make these three things, these three kind of characters, right, that you have in the

• 17:50Speaker 03

  story of building a Gracen schema. And the entities are, you know, things like people, things like companies, even things like abstract concepts in the world, right? But then when when someone references an entity, you immediately know roughly what they're talking about. Relations are the kind of glue that sit in between these things, right? So that's what you would use as edges in the graph that we were talking about before, right? But relations are probably the the most standout concept in terms of what we do. Because these relations allow you a huge, huge volume of flexibility. They say that not only can I have a friendship between two people, right, and say that person A is friends with person B, but I can say that they're also friends with person

• 18:35Speaker 03

  C, person D, person E. I can do that with one relationship. We used to know that as an edge. So in this case, what we're saying are these relations are hyper edges, right? And you can see there, so immediately we're starting to introduce like big concepts at the low level of the structure that the that we define, right? We say basically, we want to upgrade how you can represent your domain. We want to give you this toolbox, which we're calling the schema in Gracen, that lets you model your domain. In all of the complexity that it has, right? And that then means that you've now got this format, this structure that can govern your data, that can look after your data for you.

• 19:20Speaker 03

  It can make sure that you haven't done anything that's logically invalid. It can make sure that everything is cohesive within your database. So when you start adding facts, right? You now know also what the context of those facts is because we heavily label all of the, all of the elements that go into the graph. For instance, you could insert a company, a charity and a university. All of them, all of those types that we've described, that we can describe, have inherited from organization, right? What that now means is that when I want to search my data, I can search for either companies, for charities or for universities, and I could search for those individually.

• 20:04Speaker 03

  Or I can just ask more generic questions and I can say, just tell me about organizations in my data, right? And so what we're trying to do there is to get this really natural way to actually interact with your data, so that you're using your own domain terminology to actually access what you're looking for, rather than having to say, to sort of imagine what are my nodes and what are my edges in my graph and how do they fit together, right? Instead, we try and bring that to the user and reduce the burden on them when it comes to assessing what's going on in their knowledge graph.

• 20:47Speaker 00

  What is up practically, I listeners? We're working with Infinite Red to promote their free AI mini course. It's called AI Demystified. Learn more and enroll at learnai infinited dot red. This free five day mini course is a great introduction to the most important concepts, types and business applications for AI and machine learning. Each day of the course includes a lesson, a quiz and an assignment to submit your learning. And after you've completed the course, you'll also get a certificate of completion for your LinkedIn profile or for your. If you've been feeling lost in the world of AI and hearing lots of buzzwords, then by the end of this mini course, you'll be able to speak intelligently about AI and machine learning and their practical business applications. Again, this course is completely free.

• 21:32Speaker 00

  Learn more and enroll at learnai infinited red. Again, learnai dot infinite dot red.

• 21:51Daniel Whitenack?

  So James, I appreciate kind of where the conversation has landed in that there's natural ways of representing your data and that can be modeled well on top of a graph. I've tried kind of graph databases in certain scenarios with more or less success and some have been really useful. But something I always find is like, it seems really hard to build a quote unquote, knowledge graph in the sense of kind of developing your schema can be hard because you may know what entities you have, but not like might be multiple ways to represent them, or you may have just like a bunch of unstructured data and you're not totally sure

• 22:32Daniel Whitenack?

  what entities to choose. So like, how do you recommend if people are interested in creating this sort of representation of knowledge, where should they maybe start thinking about the data that they have and how to develop a schema?

• 22:49Speaker 03

  So that's a really great question. I don't have a short answer, but essentially, that has been a huge part of what I've been doing here at Grakken and what we do overall with members of the Grakken community, where we try and help people to actually understand the principles of what is an entity, a relation and an attribute, how do they best fit together? And actually, what's super interesting about that is that that's a really great meeting of philosophy and technology. Which I found incredibly interesting and that essentially my thoughts on this is that we now see knowledge engineering and knowledge representation as entire careers

• 23:32Speaker 03

  that are actually coming around now, right? That you actually have someone who's, you know, a specialist, an ontologist, I've also heard them called, right? The body of knowledge of the best way to do this is not yet set upon. And we have our own ways of doing that here at Grakken and those ways and how we think that that things should be done informs the design decisions that we make in the language that we provide for the knowledge graph. So at the moment, it's actually been on my to-do list a long time to actually write some best practice for knowledge representation and building your schema in Grakken. We have snippets here and there and we have examples here and there. And it's very difficult to give really generic guidance, but we do have some that we would

• 24:16Speaker 03

  give out. That's a little bit long-winded for here, but maybe we can link to that in the future.

• 24:20Chris Benson?

  Yeah, no worries. I would actually want you to extend that just a little bit. I'm kind of curious, what can you do with a knowledge graph that you would not be able to do if you didn't have one, as you're talking about kind of design and thinking about, you know, what best practices are? What comes to mind?

• 24:35Speaker 03

  So the main thing that anyone who's interacted with me in a professional context will know is that what I harp on about is trying to get to the point of true to domain modeling, right? What I really want is to see people building a knowledge graph where they start with a schema where one person who builds the schema could show it to their colleague and their colleague will immediately understand what elements of data are where in the knowledge graph, right?

• 25:06Chris Benson?

  That makes sense.

• 25:07Daniel Whitenack?

  Yeah, and just to clarify, to make it super clear for listeners, when you're talking about the schema, you would basically, like, we gave the example before of like, Person is friend with person. So like there's a person type entity in this knowledge graph, but there could also be like country type entities or organizations or like different metrics, websites, resources, all sorts of things. That's the sort of schema or ontology that you're talking about, right? The definition of what things are we going to put in our knowledge graph and how are we going to label them? Is that the best way to think about the schema?

• 25:43Speaker 03

  That is absolutely correct. And what I think is also really nice is to make some analogies to OOP, so your object oriented programming, right? So, so anyone who's familiar with OOP, and there's a lot of people out there, I imagine you have quite a lot of listeners who are familiar with OOP, then what we're saying here is we're defining the class. We define a class, right? And those are our schema elements. And then when we actually insert data, we're inserting like instances or instantiating objects of that class.

• 26:10Daniel Whitenack?

  And just a quick interjection for those who don't know what OOP is, he's talking about object oriented programming. It's a technique for representing real world concepts in code as well. Just keep going. I just wanted to let anyone know that didn't know that.

• 26:23Speaker 03

  Yeah, yeah, yeah, absolutely. So, so the idea is that all of the elements that we would have, as you say, we have this like this schema and you can, you can update that, that over time, but that is the map for your data, right? That tells you what things are present in our knowledge graph and how can they be connected to one another? So for instance, we can immediately say in that example where you had like a person entity and also an organization entity, we can then also define the friendship relation that you talked about, right? And we can say, okay, a person can be in a friendship with other people. That makes sense. Can be, can a person be in a friendship with an organization? Now, maybe that's philosophically debatable, but I would probably say the answer is no.

• 27:07Speaker 03

  In which case, that should not be permitted by your schema and you should write a schema that disallows that. And what that means is that takes some weight off your shoulders because when someone tries to add some piece of data inadvertently that says that there's a friendship between a person and an organization, then you, then Grakken can automatically reject it and say, no, that's rubbish. That can't exist.

• 27:26Daniel Whitenack?

  So I think maybe there's a bit of a misconception and maybe parts of time that I've been thinking about knowledge graphs and maybe other people too, where there's kind of this sense that like when you hear about, oh, Google's knowledge graph or something, it's just like information is all over the internet and like, if you create a knowledge graph, then all that, you just suck in all that information and then you automatically know a bunch of stuff. But there is actually a lot of work in terms of developing a schema that represents the types of things that you're interested, the types of knowledge that you're interested in. It's not just like automated thing where you just like crawl a bunch of websites and then you have a knowledge graph on a certain subject. Would that be accurate?

• 28:11Speaker 03

  Yeah, I mean, absolutely. You can go in any number of ways that you want to. So you can start trying to scrape information from the internet, but you know, the quality of the information that you get may not be that high in terms of, you know, can I ensure validity of the kind of facts that I've pulled from that? And there's plenty of people that are trying to do that. So that would be automatic like entity recognition and this kind of thing. Our focus is more on building these things from the ground up so that, you know, if you've, someone's got proprietary data or they've got a particular data set, that actually they can realize an enormous amount of extra benefit from just managing the data that they have very carefully,

• 28:54Speaker 03

  rather than maybe trying to augment it with just all data from the internet. You know, probably you can take a more targeted approach and just bring in elements where you're fairly aware of what that information even is, right?

• 29:05Chris Benson?

  So wanted to kind of delve into a different area, given that we're an AI podcast. And so I wanted to ask, you know, how is artificial intelligence related to knowledge graphs? And are knowledge graphs a source of data that might be available for AI models? Or is there some other connection there?

• 29:22Speaker 03

  Yeah, I mean, where to start? So I mean, the way we see it is that Knowledge graphs are going to be central to the effort towards, well, intelligent systems, as we put earlier, right? So that's our nice way of trying to avoid using AI. To make systems more intelligent than they are today, we want to empower them with as much as we can. And so the idea here is to, you know, much of the world is still using relational databases. And as we talked about before, you know, structurally, they present themselves, you know, present us with some challenges when that format isn't natural. So instead, what we want to do is we want to actually be able to capture the full complexity of the world, right?

• 30:06Speaker 03

  Actually capture all of our knowledge in one place and then be able to present that to, for instance, learning models for them to learn over it. But what we also provide is actually the artificial intelligence of the eighties. And that is automated reasoning. So what we have at Grakken built into the open source core product is a automated reasoner that allows you to infer new data based on the data that you already have and sets of logical rules that you know must be true. So this is super interesting, right?

• 30:46Speaker 03

  Because in the day to day, we all use our deductive logical skills any number of times and we essentially just don't notice because it's so second nature to us. But if you actually try to point to any tools that anyone technical is using right now, about the only thing that people have heard of and they did like a week on it at uni or something is Prolog. That's about the only tool out there for logical programming, right? And it sounds like something computers should be able to do easily, right? Like a small set of facts and figuring out a new fact based on a rule just sounds like an if else blocks, right? Sure. But when you actually try and scale that and make that work and be able to have

• 31:29Speaker 03

  any number of possible rules that you might want to be able to write and bring that into the database level, that's when things start to get a bit interesting there. Because now we can say when A and B and C are true, then D is true. And what's nice about this is that your database then, whenever you ask for something that fits the bill for D. It's gonna give you that regardless of whether or not you ever even stored that in the database.

• 31:54Chris Benson?

  So I just had, it's almost a tangent of a question. Would you, talking about Prolog and using automated reasoning, which was kind of before the days of machine learning as we know it today. And I just want to ask, is there any tie in maybe today? I know you were saying that you're kind of including that in your approach. But today, I guess if we were going to tackle that with the current set of technologies, we'd probably use things like generative adversarial networks and along with natural language processing to try to create things new from what you already had. Is there any tie into that? And just as a random side question, is there any similarity maybe in the two?

• 32:33Speaker 03

  Well, great question. So I think our ethos is when you have facts, if you can write a rule that definitively tells you that a new fact must be true based on what you have, like that's absolutely fundamental. When you can use that, then you should use that because why is that true? Well, because firstly, it generalizes perfectly, right? Any new set of A, B, and C, and you know that D will be true. And secondly, it's explainable that when you see D, then you can say, well, why did I see D? And the database can tell you, well, that because A, B, and C. Now, what's really interesting, and this is the crossover space that's happening right now, is as you said, how do we see that complementing the other tools that we want to use?

• 33:18Speaker 03

  How do we see that complementing any other machine learning approach? And so essentially the border for me is to describe it as well. You're either, if you were a human approached with a particular problem, you would probably decide whether to use one of kind of two major skillsets that you have, either how you deduce things in your logic or your intuition.

• 33:43Speaker 03

  And so essentially what we need is we need to start figuring out, okay, when do we need to deduce things logically versus when do we need to use a machine learning approach, which gives us some kind of intuition based on experience, right? And so that's actually the center of my work here at Grakken is how do we actually build learners on top of a logical reasoner, on top of a knowledge graph, in order to like get to the next level of intelligence of our machines, right? How do we make an iterative process between those two that ingest new facts that have been learned and then reasons over them? Or how do we reason over facts and then learn from them, right? So this is very much an unsolved region and it's super invigorating at the moment to be in that space.

• 34:27Daniel Whitenack?

  And what do you think are the sorts of tasks that are kind of low hanging fruit for learning on top of a knowledge graph? For example, one thing that comes to mind is question answering, sort of tasks or something like that. Are there other tasks that have been explored in AI, maybe in a non-knowledge graph way that you think are particularly relevant to explore on on top of a knowledge graph?

• 34:55Speaker 03

  Absolutely. I mean, as I said, that's actually kind of the whole remit of of the research division here at Grakken is to try and fulfill those end user problems. And what are they? Well, I actually wrote a whole blog post on all the kinds of problems that we see there. So you're absolutely right, question answer systems is, I mean, that's what that eighties logical reasoning AI systems were all about was were building expert systems, but they didn't really work because you had to hand code everything. Well, now we can maybe use machine learning to derive some of it automatically, right? And we do question answer systems. And you see that with Google's knowledge graph and this sidebar that they have, right? When you type in a search, it may just directly find the thing that you're interested in,

• 35:37Speaker 03

  not just links. But then besides that, we see a lot of applications in, for instance, well, we can talk about knowledge graph completion. So that's maybe I want to find new links in between elements of my graph that I'm interested in. So for instance, if I ingest a lot of biomedical data, then maybe I want to try and predict new links between a drug and a disease, right? Want to infer new treatments. Or maybe I want to, you know, uh, enrich my whole graph before I try and make those as well. So I can, you know, find other relations, interactions between genes, proteins, et cetera, right?

• 36:15Speaker 03

  But then there's other tasks on a totally different spectrum. So what about NLP systems and computer vision systems? When you apply background knowledge to them, right? Well, as humans, when we approach understanding a person who says a sentence, we have behind us however many years we've been on the planet of experience of hearing people say sentences. We often don't really bring that, but we also have, more than that, we also have our knowledge of the world, right? We often hear someone say something and we, we mishear what they say and what they said sounded ridiculous, given our knowledge of the world. And so we correct ourselves or we nudge them and you say, did you just really say that? Because that, that doesn't like align with my understanding of the world.

• 37:01Speaker 03

  That's what we hope that then the knowledge graph can do. And we've got, you know, I've had a number of conversations with people who want to improve, for instance, uh, their company's customer service platforms. Where they know the body of knowledge, they know quite a lot about a customer. They know a lot about their products and the kind of things that they offer. And, you know, if a customer says my connection's broken, can we immediately infer what they're talking about? Because we naturally know products that's that customer has. Okay. They have a home broadband connection with us. So they're probably talking about that, right? In machine vision, as we've already talked about a little bit from my past, then

• 37:41Speaker 03

  often we just present a learner with a, with a flat image. We try and get it to guess what's in the image based just on the pixels. But, you know, again, if the learner starts to see things that are nonsensical in the image or things that go, that are often seen together, that would be a, you know, a big help for it to be able to understand and identify when it might be wildly wrong, based on the other things, the other context, the surrounding context of the, of the problem that it's trying to solve.

• 38:21Speaker 04

  This episode is brought to you by Brave. Big news from the Brave team. Version 1.0 is official. That means our favorite open source, privacy-focused, blazing-fast browser is ready for primetime. Their brand-new iOS app landed just in time for the announcement, and the Brave team is celebrating by granting 8 million basic attention tokens to the community. That means when you download the iOS app, you get 20 bat absolutely free. Put it to good use by heading to changelog.com, hitting the triangle icon in the upper right-hand corner, and flipping us a tip.

• 39:08Daniel Whitenack?

  So you started to get into a little bit of the details of where you think certain tasks like computer vision or other things could be augmented by a knowledge graph. And it seemed like in some of those cases, it was a matter of like, okay, you have the image, and you have this other information that goes along with the image that helps you reason about the image or predict something. Is that where you see kind of the near term of knowledge graph augmented AI? I don't know what the proper term for that is, but is that kind of where you see the near term? I know that there's also people exploring or doing AI with graph-structured data itself, rather than just kind of extracting features from the graph as new features in a model,

• 39:53Daniel Whitenack?

  but actually using graph-structured features or subgraphs or other things in AI models. Are you familiar with that at all? Or how do you see, maybe as a person who says, okay, well, this sounds cool. I'd love to try to augment some of my AI systems with knowledge from a graph. Where might they start looking in terms of methods and next steps?

• 40:18Speaker 03

  Right, I mean, great question. So I totally agree. What we don't wanna do is just stick with the status quo of sort of taking essentially sort of square shaped data as inputs to machine learning pipelines, right? That's like the status quo at the moment, right? Is we have, our data is stored in these filing cabinets. And so what do we put into our machine learning model while it's data that looks like filing cabinets? Right? And what do we get out? Surprise, surprise, right?

• 40:47Daniel Whitenack?

  Yeah, and I think it's probably confusing to people sometimes, it has been for me where like TensorFlow talks about a graph, right? So it's not a graph of the data. It's more of a graph of the computation and how it's executed on a certain architecture or the logic of that computation. Whereas what we're talking about here is actually data that is structured like a graph being processed through one of these systems as a graph would be different than just putting a tensor in, right?

• 41:17Speaker 03

  That's absolutely true, yeah. So that's one of the fundamentals that makes learning over, well, anything except just like a matrix or vector representation difficult is that all the frameworks are set up to take those things in. And as you say, in the case of these pipelines, the shape of the processing is a graph, but we don't really need to worry about that compared to the input and output. And as you say, what we're here, we're saying is, what do we do? How do we move from these square inputs to something else? So that's actually a big body of work that I've been doing over the last year. It's been looking at one of the approaches that have been done around that.

• 41:57Speaker 03

  And some of the first approaches, which is still quite common, is to do, for instance, walks through the graph. Like, I'm interested in some particular entity in my graph. So why don't I start there within my graph and then just walk randomly and see what I encounter? Record what I encounter and then maybe I use that as like a row in a vector or something and feed that into my model. That's one way of doing it, right? But but you're kind of hoping for some serendipity there. You're kind of hoping that I'm gonna encounter things in my graph that are important. Right? Because I'm just sort of walking around randomly, literally randomly walking is what is the approach through the graph. So, okay, so the next thing. And so from this, there was a really nice piece of research came out of Stanford.

• 42:41Speaker 03

  They called their paper Graph Sage, or at least their approach was called Graph Sage. And we actually implemented that here over the knowledge graph. And the idea of that was to essentially not just take these what these single walks, but to actually look at all of your neighbors, take a subset, a random subset of all of your neighbors. But then also look at their neighbors and their neighbors and their neighbors and sort of have this more like spider web shape of the graph that you would analyze, right? And sort of in some way, without going into all of the technical detail, that basically roll that information inwards towards the port, the entity that you are interested in. So you kind of gain some information as you move from that, like the outer radius of a s- like

• 43:24Speaker 03

  outer circumference of a circle, like inwards. And that's also really nice, right? So what that's also doing is still kind of putting your data into a box shape because you're still dealing with a tree now. So we've gone from, from a line, which was the walk, to then a tree. And we still didn't find what, what was really difficult about this, so we tried using this. But what it doesn't manage to capture, say, say we are trying to do something really difficult. We're trying to find a new drug for a di- to treat a disease. Now, if we try and do this, if we just look at like generally what does a drug look like and what's nearby to it? And also generally do what, what does a disease look like and what's near to it? When we then try and match those two things,

• 44:08Speaker 03

  we haven't actually looked at any of the common connections that exist between a drug, that drug and that disease specifically. Like we haven't actually figured out what are the, like logically, what are the paths that actually connect these things? We should probably be interested in those. Those are probably like the most important features in this graph. Instead, we've just looked at roughly what they look like. And then you end up with just like some generic answer like paracetamol treats lots of diseases because lots of diseases exhibit pain, right? So what we want is again, a more targeted approach and that leads us to, no, we have to do the hard thing. We actually have to learn over a graph shape.

• 44:47Speaker 03

  Right, we actually have to take in graph data.

• 44:50Daniel Whitenack?

  Yeah, so I I'm kind of thinking about natural language processing because that's the world I live in. And, you know, some of what we've learned recently is that, you know, it's very useful to have your algorithm learn the proper representation of text, taking into the context of, you know, context around just like a single token, for example, in order to actually learn a good representation of text for a certain task. It sounds like what you're saying is it would be useful to do similar things for graphs in that we need to learn how to represent graph structured data in a neural network because it might not be

• 45:32Daniel Whitenack?

  like if we just take all of the nearest neighbors and put them in kind of standard row structure and use that as a representation, then we might miss that actually the predictive thing is beyond the nearest neighbors, right? And like a bunch of links away, even though it's not a nearest neighbor, that's like the thing that's indicative of the thing that we're trying to predict. Is that kind of along the right track?

• 45:56Speaker 03

  Absolutely. What I see you describing there in NLP is is definitely what we're aiming for here, right? And not just in graphs, but I think the industry in general is where we're now seeing like beyond curve fitting, it's called, right? And like, how do we move beyond where we are right now to a point where the machine is actually understanding, it actually learns to understand what's going on? So like, we already talked about that with like NLP based on a knowledge graph, it understands the context. You were just talking about that there, context in a machine vision problem, also understanding the context of what's actually in the image. All of these things mean that the learner can not just sort of learn by rote or learn by exact examples, but can actually understand what's going on. What's really interesting in graph is that you have exactly that. You might have like one

• 46:40Speaker 03

  particular feature that you find, like if I see some particular thing that's in some particular way related to what I'm interested in, that's a huge indicator. But you might also just see a general structure that occurs, that when the, you know, I have these five elements, these five entities all connected together in a particular way, they all have particular types, that is a very typical structure for a really effective drug.

• 47:07Speaker 03

  Right, but those combinations come up again and again. But in like a generic sense, and maybe we want to learn that, we want to learn some kind of structure. So then what we were faced with was we were faced with the problem of, okay, no, we actually need to learn over graphs. And to our luck, um, we're not, you know, we don't have the, the budget to do like to, and, and the manpower to do these huge research efforts ourselves. But our neighbors over here in London, DeepMind, released a paper last year. And they also released a library to support what they were doing, where they've generified a lot of the concepts of graph learning and how to do learning over graphs um in this really neat way. Given they were acquired by Google, I mean, it makes sense that they also figured out how to do this in TensorFlow.

• 47:51Speaker 03

  So what they've got there is a pipeline that now actually lets you input a graph into TensorFlow as the data. And get that same graph back out as an output, but with updates made to every element of that graph. So that means that essentially what we can use is we can use that as a little toolbox. That allows us to perform any number of different tasks over a graph structure. And obviously we've tailored that here at Grakken to work over the knowledge graph. But what we can do is we can just carefully frame the kind of problem that we have. So that this toolbox can help us to solve that.

• 48:35Daniel Whitenack?

  And is that the GraphNet's library?

• 48:37Speaker 03

  That's exactly the one. Yeah, that's the one.

• 48:39Daniel Whitenack?

  Okay. Yeah, we'll definitely link that in the show notes as well, because it seems like they have a good, you know, usage example and notebooks and such that the people can play with that.

• 48:48Chris Benson?

  So you've totally won me over and I'm looking forward to, to jumping in and playing with this and I know Daniel is too. Could you start walking us through what it is like to actually build a knowledge graph with Grakken? And you know, what, what do you need? What languages do you need to know? And also I noticed on the website, you, you talk about, uh, is it GracuL? Am I, am I pronouncing that right?

• 49:09Speaker 03

  So that's Gracal, that's Gracal.

• 49:11Chris Benson?

  Gracal, I'm sorry.

• 49:12Speaker 03

  No, no, no, no worries. So yeah, so yeah, I can give you the whole like overview of what you would do, right?

• 49:18Chris Benson?

  Fantastic.

• 49:19Speaker 03

  Actually, to close down what we were talking about just there, the whole learning approach that we've been building and all of the research that we do on top of knowledge graphs, right, I'll emphasize that, is all, we released all of that as code available via our GitHub. Specifically, we have a library called KGLib. So that's our knowledge graph library for machine learning. So KGLib is the center of those projects and the main one that we're running right now is knowledge graph convolutional networks. So that's how do we apply these learners on top of both the reasoner and the knowledge graph shaped data. The starting point is how do you actually get a knowledge graph, right? How do I actually get my knowledge graph together?

• 50:01Speaker 03

  Now, the components that you have there, as you pointed out, is something we should start with. So you have Grakken itself, right? So Grakken Core is released open source. You can download that from GitHub or install it with a package manager. And that's a database which is going to run. You could, you know, install that on your local machine and get it up and running or put it in the cloud. And so you need that backend service running. Now, when it comes to actually accessing that, we have three officially supported drivers at the moment. We have Python, Node.js and Java. So we make sure that all of those are up to date and working with the latest Grakken. What's really interesting there actually is the communication protocol between those clients and

• 50:42Speaker 03

  and Grakken is called GRPC. So that's something from Google, Google's Remote Procedural Call, that has replaced using REST services. So what's really nice about this, the actual end goal that that gets you to, is it means that when I'm accessing the database in Python, with Python, I get to actually use native Python functions. All I have to do is import the package that talks to Grakken. Import the Grakken client in Python at the top of my script, right? And then I can just instantiate a communicator that that will talk to Grakken and make queries to the database just out of my native Python. I can just launch them straight from my application. And it doesn't feel like you're talking to a database anymore.

• 51:27Speaker 03

  Right? It just feels like you're making function calls, which comes back with information that's pertinent to your knowledge graph.

• 51:34Daniel Whitenack?

  That's great. And would you use that? So would you use that client tool to help you build your knowledge base? Like, let's say that I have a bunch of text data and I'm like pulling entities out of it that I or or classifying that in a certain way to store it as a certain type of entity. Would I kind of be doing that in Python and then push that to Grakken via the the Python library? Or they're like bulk upload techniques or like ways to get data, let's say, from relational to graph? What's the sort of range of what people do?

• 52:10Speaker 03

  Yeah, absolutely. Great question. So basically, you're absolutely on the money. The idea is that we give the users these clients in their native language, because that's their strength, right? We already know that they know how to speak that. And they get all of the freedom that that language offers. And then the way that you're actually interacting with Grakken is through Grakken's query language, Gracal. Right? You can probably see where the name comes from, right? So the Grakken's got this query language called Gracal. And the idea is that that's this really concise, really expressive language. But then what you would do is that is your one stop shop for how you actually talk to the knowledge graph in terms of what your intentions. So if I want to either retrieve something, then I make what we call a match query.

• 52:54Speaker 03

  If I want to insert something, then I use an insert query. And and if I want to to wherever I see a particular pattern, insert something that's a match insert. I'm sure you get the idea, right? So you have all of these different ways that you can read and write from the database. And you do all of them in the same way through your application. You just, you know, you call ask the client, you say dot query, right? And make a make this query. And then the response you get back will be the answer, right? Either you inserted something or read. Now then, what we've got, we've got a number of, we've got a repository of examples so that people can have a look on there. You know, very typically, people are, as you say, they're migrating from either SQL data or from CSV data, in which case it's a matter of just writing a, what we call an

• 53:38Speaker 03

  an ETL pipeline, so something that will just traverse over all of that data that you have and make the appropriate queries in Grakal to get that data shifted over into Grakon itself. Now, one of the questions that people ask me really often and definitely comes in on our community Slack quite often, is can I like automatically build my knowledge graphs? And we kind of talked about that a bit earlier in the call. The problem is that, like, it's possible to automatically ingest a relational database into a knowledge graph, but the problem is you just end up with the same structure that you had in your relational database, but in the knowledge graph, you know, you still end up with something broken because you need to apply that human understanding that you have

• 54:23Speaker 03

  of the data that you have in these table formats. You need to say, what's that actually mean? What does my domain look like? So what you do is you first, well, it's an iterative process, of course, like a lot of engineering, but you're going to start out by saying, here's my schema, here's what I think my domain looks like. Okay, now when I make it, when I go over this file, what parts of that schema can I infer from the particular row I'm dealing with right now?

• 54:49Chris Benson?

  So I guess if somebody wants to get into this, I know we're both very excited about it. And I've learned a lot that I didn't know before the conversation. Where can they go and learn more and actually start digging into using Grakon themselves and Grakal? Any specific links that you want to recommend?

• 55:06Speaker 03

  Well, we have the docs available on our website, people seem to think those are quite fun. And we also have, so there's also some in-depth examples there on, for instance, how to do data migration into Grakon so that you've got, you know, get that knowledge graph up and started, so you've got something to play with. We then have an examples repository on our GitHub. And also, for those who really like to jump in at the deep end, then the KGLib repo is quite a good place to, if you want to see immediately from the top, how you're going to then do the machine learning over it. And then I suppose the other thing to majorly encourage is to check out our blog. So that's blog.grakn.ai. So we have a lot of stuff there that will give people an idea and give them a flavor of

• 55:48Speaker 03

  What you can achieve with the knowledge graph and how succinct it could be to get you motivated to actually move your data over and give it a try.

• 55:56Chris Benson?

  Well, James, thank you very, very much for coming on the show and just kind of schooling us in all this. It's been really fascinating and we appreciate it. So thank you and we'll talk to you soon.

• 56:07Speaker 03

  Thank you very much for having me, both of you.

• 56:12Speaker 00

  All right. Thank you for tuning into this episode of Practical AI. If you enjoyed the show, do us a favor. Go on iTunes, give us ratings, go in your podcast app and favorite it. If you are on Twitter or social network, share a link with a friend, whatever you gotta do, share the show with a friend if you enjoyed it. And bandwidth for change log is provided by Fastly. Learn more at fastly.com. And we catch our airs before users do hear change law because of roll bar. Check them out at rollbar.com slash change log and we're hosted on Linode cloud servers. Head to linode.com slash change log. Check them out. Support this show. This episode is hosted by Daniel Whitenack and Chris Benson. The music is by Breakmaster Cylinder and you can find more shows just like this at changelog.com. When you go there, pop in your email address, get our weekly email, keeping you up to date with the news

• 56:57Speaker 00

  and podcasts for developers in your inbox every single week. Thanks for tuning in. We'll see you next week.