Federated learning in production (part 2)

Chong Shen from Flower Labs joins us to discuss what it really takes to build production-ready federated learning systems that work across data silos. We talk about the Flower framework and it's architecture (supernodes, superlinks, etc.), and what makes it both "friendly" and ready for real enterprise environments. We also explore how the generative Generative AI boom is reshaping Flower’s roadmap. Featuring: Chong Shen Ng – LinkedIn https://www.linkedin.com/in/chongshenng/Chris Benson – Website https://chrisbenson.com/, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaEpisode links: The future of AI training is federated https://pretalx.com/pyconde-pydata-2025/talk/9Y9DM8/DeepLearning.ai short course on Federated Learning with Flower https://www.deeplearning.ai/short-courses/intro-to-federated-learning/Flower Monthly https://flower.ai/events/flower-monthly/Federated Learning in Automotive https://flower.ai/industries/automotive/Federated AI in Finance https://flower.ai/industries/finance/Federated Learning in Healthcare https://flower.ai/industries/healthcare/Federated AI on IoT Systems https://flower.ai/industries/iot/FlowerTune LLM Leaderboard https://flower.ai/benchmarks/llm-leaderboard/Flower Intelligence https://flower.ai/intelligence/GitHub https://github.com/adap/flowerSlack https://friendly-flower.slack.com/join/shared_invite/zt-35epydsx3-_e~KjYPEcyevkJZ4Ja3XkA#/shared-invite/emailFlower Discuss https://discuss.flower.ai/Check out upcoming webinars https://practicalai.fm/webinars! Sponsors: NordLayer is toggle-ready network security built for modern businesses—combining VPN, access control, and threat protection in one platform that deploys in under 10 minutes with no hardware required. It's built on Zero Trust architecture with granular access controls, so only the right people access the right resources, and it scales effortlessly as your team grows. Get up to 32% off yearly plans with code practically-10 at nordlayer.com/practicalai http://nordlayer.com/practicalai - 14-day money-back guarantee included.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:03Speaker 00

  Welcome to Practical AI, the podcast that makes artificial intelligence practical, productive, and accessible to all. If you like this show, you will love The Change Log. It's news on Mondays, deep, technical interviews on Wednesdays, and on Fridays, an awesome talk show for your weekend enjoyment. Find us by searching for The Change Log wherever you get your podcasts. Thanks to our partners at fly.io. Launch your AI apps in five minutes or less. Learn how at fly.io.

• 0:44Daniel Whitenack?

  Welcome to another episode of the Practical AI Podcast. This is Daniel Whitenack. I am CEO at Prediction Guard, and I'm joined, as always, by my co-host, Chris Benson, who is a Principal AI Research Engineer at Lockheed Martin. How are you doing, Chris?

• 1:01Speaker 00

  Doing great today, Daniel. It's a beautiful spring day here in Atlanta, Georgia, and I gotta say, the flowers are coming out. It's a

• 1:09Daniel Whitenack?

  Yes.

• 1:09Speaker 00

  Nice day to talk.

• 1:11Daniel Whitenack?

  They're probably distributed all over the various lawns.

• 1:15Speaker 00

  Everywhere, federated even.

• 1:17Daniel Whitenack?

  Yes, yes. Well, Chris, this reminds me, last week, we had a kind of Part One intro to federated learning and some details about that with Patrick from Intel. He mentioned recently that he was at the Flower Labs conference and the flower framework around federated learning, he mentioned quite a few times. Well, we're privileged to kind of carry on the conversation around federated learning into a kind of Part Two on the subject, because we've got Chong Shen with us, who is a Research Engineer at Flower Labs. Welcome, Chong. How are you doing?

• 2:00Speaker 02

  Hi, I'm doing very nice, very well. Thanks for having me.

• 2:04Daniel Whitenack?

  Yeah, yeah. And actually, we were talking before the show. This is the the second time that we've got to chat about Flower on the podcast back in 2021. So even, you know, before AI was invented with ChatGPT, apparently, we were having conversations about AI. And one of those was with Daniel from Flower. That's episode 160, titled Friendly Federated Learning. It took me a second to to say that one. But I'm sure a lot has changed and updated and advanced in in that time, of course. Maybe just to start things out,

• 2:46Daniel Whitenack?

  Chong, could you give us a little bit of a context of of your background and how you got kind of introduced to this idea of federated learning and eventually ended up kind of working, working with Flower?

• 2:59Speaker 02

  Yeah, absolutely. Well, thanks again for having me. So my background is in computational physics. So I spent many years working, doing research in the computational physics field, both my PhD and a postdoc. So I worked a lot on parallel computing, on supercomputing classes. I was also very interested in machine learning and deep learning in general. So when I pivoted away from academia to go into what I call industry, there was this space where you have distributed learning. So that was in 2021. So when I started my career back then, it started as a sort of a data science consulting business, but specializing

• 3:44Speaker 02

  in federated learning. And I saw lots of projects that were very interested to adopt federated learning or this distributed learning approach to solve some specific problems that they have. But I also came across the Flower framework. And open source development is a big passion of mine. So being able to develop

• 4:07Speaker 02

  A framework that is used effectively with a very permissible license, I think it's a pretty cool thing to do. So that's why I decided to join Flower Labs and become a core contributor to the framework itself.

• 4:23Daniel Whitenack?

  Yeah, and I feel already already connected with you because my background is in is in physics as well. It's always good to have other physicists on the on the show that have somehow migrated into into the AI world. I'm wondering in that transition, like, you know, one of you mentioned this transition kind of academic to industry, you were getting into even consulting around federated learning. Was that idea of federation or distributed, you know, computing or however you thought about that, was that kind of a key piece of what you were doing in in academia, which led you kind of into that interest? Or was

• 5:07Daniel Whitenack?

  it something else that kind of sparked the the desire to really dig in there as you were kind of going into to quote industry as you as you mentioned?

• 5:16Speaker 02

  Yeah, it wasn't something I came across in in academia, surprisingly. But somehow, when I stepped into the data science world, I came across people who are looking into it, and that that became an approach that back then we sort of adopted to try and solve some problems. So we saw that federating could be a way to solve it. And then it's off. It's very coincidental. Okay, it's distributed learning, so distributed computing. So it resonated with me quite quite strongly.

• 5:44Daniel Whitenack?

  Yeah. And was that related to working with sensitive data or in regulated industries or something like in those, you know, consulting projects or, you know, just, yeah, interested in kind of that progression?

• 5:58Speaker 02

  Yeah, actually, there are, I would say, two broad categories. One where the data is incredibly sensitive, and we usually refer to them as really siloed data, you know, data that should not absolutely leave. The boundaries of where it was generated. And then the second group or second cluster is the problems where the data sources are so massive, the point at which the data is generated, generate so much data every second of the day that they just can't do any useful or meaningful analysis on this kind of raw data. And you have to do a lot of down sampling.

• 6:40Speaker 02

  So they try to look into pushing competition to the edge and try to see if they could apply some sort of machine learning approach or deep learning approaches on this sort of massively generated data without needing to down sample them.

• 6:52Daniel Whitenack?

  That makes sense. And yeah, if I guess I should explicitly mention as well that in the kind of part one of this two parter with Patrick, Patrick did provide a kind of detailed introduction to the idea of federated learning. And we discussed that at length. So if people want to go back to the previous episode and listen through that, that may provide some context. But it probably would be worthwhile just to give, you know, your sort of 30 second or couple minute view on federated learning and how you would describe it at a high level. And then maybe we can jump into some other things.

• 7:34Speaker 02

  Sure, absolutely. The easiest way to think about it is looking at your classical machine learning approach. Right. Classically, you need to bring all the data into a single location. Think of a database or on disk. And then you train your model on that data. But sometimes it's not so easy to actually bring all the data into one location just because of the privacy reasons about moving your data, some geopolitical considerations surrounding it, and also the data volume that's being generated. So instead of bringing all the data to one spot and training a machine learning model on that, what you do is you move the machine learning models

• 8:17Speaker 02

  to the point at which the data is generated. And then you train these local machine learning models at these sources.

• 8:26Speaker 02

  Then instead of moving the data across, you move the model weights to a central server, which is much, much smaller. You can then aggregate the model weights to learn from these various data sources. And then over time, as you repeat many, many rounds of this, you end up with a globally aggregated model that has learned from this variety of data sources without needing to move the data source across. That's the essence of federated learning.

• 8:54Chris Benson?

  I'm curious, as you've, as you guys have worked on the framework and, and you know, you have new users coming into it. What, what usually prompts a typical user from your perspective to move into federated learning? Like what, like, you know, before they're really like fully into it and they understand the benefits and, and they're sold on it, if you will. What's usually in your experience, the impetus that kind of gets them into that mindset and, and kind of drives them in that direction initially? What, what, what causes the change in the way they're thinking? So they go, I definitely need to get into federated learning and go use Flower specifically.

• 9:32Speaker 02

  Yeah, absolutely. I think from my experience, the biggest driver is when they realize they can't move their data. Right? But when they speak to all the parties involved, they say, oh, I have this data set. Oh, you have this data set, but I don't really want to share them. And then, okay, this is where, you know, federated learning or FL comes to the picture. And, and they said, okay, we really need to do this. This is one, one aspect of it. And the other aspect is when there's this big company who has, you know, as I say, many, many data sources. They say like, okay, it's super difficult to coordinate all our databases together so that we can have a cohesive way to train the machine learning model. And this is also when you can, when you try to look for all distributed machine learning systems, then you realize, oh, they, they come across federated learning.

• 10:16Speaker 02

  So there's, there's these two, so these two vectors that drive the typical use cases.

• 10:22Chris Benson?

  Curious if I can follow up on that because I have a, I have a personal curiosity. I happen to work for one of these big companies that has data in lots of different places. And, and in addition to that, and we kind of in the previous, in last week when we were talking, we talked a little bit about some of the privacy issues as well. I'm curious what you think about this. Yeah, like in our case, and we're not the only one, lots of that data is stored at different levels of security and privacy. There are different enclaves, if you will, where you're trying to do that. And how does that ramp up the challenge of federated learning when you have different, you know, different security concerns around the different data enclaves that you're trying to bring together through federated learning?

• 11:06Chris Benson?

  How, how does one go like, instead of saying all different locations for distributed data are equal, when you're dealing with different security concerns, do you have any, any ways of starting to think about that? Because as, as I come into it as a newbie on this, that seems like quite a challenge for me. And do you have any advice or guidance on how to think about it?

• 11:26Speaker 02

  Yeah, yeah. I think this is from my experience is that the complexity of the solution scales with a number of data stakeholders involved. And when you mentioned about different levels of the enclaves, that to me sort of signals that there are many data owners who manage their data a bit differently. So the primary, the key to solving that is to harmonize the, the data standards first, be able to get on to a federation. And then from then onward, the implementation becomes much, much easier.

• 12:11Speaker 02

  Maybe it's one of the key things that I've seen.

• 12:12Daniel Whitenack?

  And we've kind of talked about your background, the sort of introduction to federated learning, some of those motivations. Maybe, you know, before we get into flower specifically and some of the more production use, you know, from your perspective as kind of being a central, you know, you're at a kind of central place within the ecosystem of federated learning, I guess. What at a, you know, just very honestly, sort of, because we had that last episode in 2021, from 2021 till now, like what, how is the state of adoption

• 12:54Daniel Whitenack?

  of federated learning kind of in industry different maybe now than before? How has that grown? Or how has that matured as a, as a kind of ecosystem, I guess?

• 13:08Speaker 02

  Yeah, it's a very, yeah, it's a very good question. If I want to, if I were to put a number to it, I, and this is really arbitrary, I think there's a hundred x difference from, you know, 2021, when the flower framework sort of existed and now. And one of the key changes in the usage of federated learning is the ability to train foundational models and large language models. And this has been a significant change and driving force. So previously, when we talked about using the flower framework, you may be confined to models that are not super large, you know, small by today's standards of the order of

• 13:52Speaker 02

  hundred, you know, millions of model parameters. But these days, when we're talking about making use of text data, image data for these foundational models, you are thinking about models at an order of billions of parameters. And there is a fundamental change in also how we have structured the architecture of our framework.

• 14:15Speaker 02

  And also to increase the ability to stream large model weights. So all these things are happening right now as I speak and there's some exciting new progress. Hopefully we release a new version in a couple of weeks. And for the users, the usage is identical. Nothing has changed, but what has been unlocked is the ability to then train very large models. So all of these really increases appeal of using federated learning or the flower framework for a larger variety of use cases.

• 15:03Speaker 04

  You know what's beautiful about good code? It just works. No fuss, no five hour debugging sessions at 2am. That's exactly what Nordlayer brings to business security. While you're busy shipping features and fixing bugs, Nordlayer handles your network security in the background. It's like having a senior DevOps engineer who never sleeps, never takes vacation and never accidentally deletes the production database. Zero trust architecture? Check. VPN that doesn't make your team want to work from the coffee shop instead? Double check. Deploy in under 10 minutes with no hardware to rack and stack? Triple check. Built on the same foundation as NordVPN, but designed for teams who need granular

• 15:45Speaker 04

  access control and compliance reporting because apparently it works on my machine is not sufficient for the auditors. The good news is our friends get up to 22% off plans plus an additional 10% with the code practically10. That's practically10. That's less than your monthly GitHub Copilot subscription, but infinitely more useful when the security team comes knocking. Check it out at nordlayer.com-practicalai. Again, nordlayer.com-practicalai.

• 16:20Chris Benson?

  Well Chung, as we've kind of dived into the show and we've already started making reference to Flower quite a bit, but we haven't actually really described specifically what Flower is in detail as a framework and what it brings and such as that. Could you, could you take a moment and we probably should have done this before, but maybe kind of, kind of express, you know, exactly what Flower is, what the components are and how it kind of helps the user begin to federate their data in terms of their, you know, what their workflow is. Could you talk a little about the kind of the basics of it?

• 16:58Speaker 02

  Yeah, absolutely. So the Flower framework is our flagship open source code that's built on the Apache 2.0 license. And this framework allows users, any data practitioners, to build a federated learning solution. So with the framework, what this means is they are able to, I guess in code terms, install a basic Python distribution of Flower and to build the different apps that allows you to construct the fundamental federating architecture. So what this means is to be able to spin up your server, which aggregates the model

• 17:42Speaker 02

  parameters, and to write the code to also do your training on the clients. The structure that we provide within the framework allows users to follow the same reproducible way to perform their federated learning. So I think at the essence, this is, this is what it is. What I also wanted to say that in the moment, the appeals of Flower for me personally is that we really emphasize the user experience. This is why we always say Flower is the friendly federated learning framework. We Want we prioritize the experience of all our users. We support them on Slack. We also have a discourse channel called Flower Discuss,

• 18:25Speaker 02

  where we actively answer any of the questions from users. And we also have a fantastic community that has contributed a lot of code improvements to the call framework as well. So we are completely open, we're built transparently and really accountable for every single line of code that we commit to it, you know, to the highest standards.

• 18:47Daniel Whitenack?

  Yeah, and I I can testify personally. We we at Prediction Guard, we work at a work with a number of students over time at Purdue University. They have like capstone projects, we're in the same town. So it's natural that we would, you know, work with work with some of those students. We've done that a couple of times now. And one of those student groups that we had, I believe it was was last year, actually did this sort of capstone project related to federated learning and training language models, translation models and trying various things.

• 19:25Daniel Whitenack?

  And they they evaluated a bunch of different things, but I think ended up using Flower for the reasons that you mentioned. So they they were newbies into into this world of federated learning. Obviously, very smart students, no doubt there. But they but they definitely gravitated to the user experience with with Flower because, you know, they had programmed in Python and and it just sort of came came naturally to them. So, yeah, I'm sure that's a common experience that that maybe you all hear from from others. That's sort of natural, Pythonic way to kind of approach these topics.

• 20:08Speaker 02

  Yeah, yeah, we do, absolutely. I'm very happy that you shared that experience. This is good to always hear feedback from my community. But yes, Python being the really.

• 20:21Speaker 02

  Driving language behind machine learning models and deep learning models right now. So it's a really natural way to to provide a Python SDK, you know, we support it from day one. And we will continue to support it for a long time.

• 20:34Chris Benson?

  I'm curious with the kind of extending that just a little bit. Beyond being in the language, I like I like the notion of the friendly language, you know. The word friendly appeals to me in terms of that user experience. Could you talk a little bit more about kind of why you're branding around friendly? And what that means from a user experience standpoint? You know what other aspects of it make it friendly? There are so many things out there that are not friendly that that definitely. That definitely grabs my attention.

• 21:08Speaker 02

  Yeah, absolutely. I can, I think what would be nice to explain is for the past 10 releases, we have dramatically improved the friendliness of our framework. Hopefully, I hope I hope that's the that's the experience that people will get out of this. The main point is to reduce the cognitive load of any developers who want to use our framework. So I'll give one concrete example. We introduced the flower CLIs a couple of releases ago, I think probably late last year. And what this does is with a simple FLWR space new command, NEW.

• 21:49Speaker 02

  A user is able to navigate options through the command line and immediately have a templated project to work with for federated learning and it runs out of the box. After flower new, the user goes through this, just follows the steps. And then you do FLWR space run and it runs out of the box. And we have, you know, the core templates that is necessary for users to build on. We have the PyTorch, TensorFlow, typical ones and the more exotic ones, you have Jax. And those who want it, you can use NumPy as well. All all of these provides the boiler code for a user to get started with. And it reduces so much startup time.

• 22:29Speaker 02

  Then with that, once a user has built all their applications, the user can also really monitor their runs. We also introduce commands like F L R space L S. It's like really like L S in your terminal to just see what runs, what Flower runs are running at the moment. And also others like F L R space log to see the logs of your of your code. So all of these really simple CLI tools really help a user navigate and work with running code much more easily. Previously, I would say, you know, twenty twenty one, twenty twenty two, early twenty twenty two, the Flower framework was in a different place.

• 23:12Speaker 02

  What how it worked back then was it was still friendly, but the way that it a user would need to start the federation would be to start three Python scripts. And this is not as intuitive or natural if you want to scale up or put into production. So with the introduction of the Flower CLI and a different way of deploying the architecture which drives the federation, it really makes it so much easier for users to start building and then deploy the deploy their code.

• 23:47Daniel Whitenack?

  Well, you you were kind of leading into maybe what was going to be my next question. You mentioned kind of taking things into production. So some people might hear kind of friendly framework, which is a good thing, as Chris mentioned, but they might associate that with, you know, prototyping and and learning and that sort of thing, not necessarily production usage. So I I'd love if you could kind of help us understand what does a if if I'm implementing a federated learning system with Flower, what what does a production federated learning system look like? I'm sure there's different sorts of.

• 24:30Daniel Whitenack?

  You know, ways that that could manifest, but certainly you've seen a lot of use cases. Maybe you could just highlight some examples for us. What does that production federated learning system look like? And what are some of the considerations that you have to think about going from kind of a toy prototype of like this might work to a full scale production rollout?

• 24:53Speaker 02

  Yeah, absolutely. I think it is a nice segue between the fairness aspect and moving to production. Because what I just want to mention here is that I walked through a very simplified workflow of how a user would build out an FL solution. Okay. With a flower framework, you could build and write the apps that you need for your aggregation, your server aggregation, and also for the clients, which actually train the models at the data sources. In the first iteration, a user might actually run it in what we call the simulation runtime. So without worrying about the actual data sources or to work out the data engineering

• 25:38Speaker 02

  aspect of it, you could test the implementation of the basic architecture in the simulation runtime using data sets that are obtained from Hugging Face, for example, or from data sets that you could just create artificially just for testing purposes. With the same code that you use to train the models and the clients and to aggregate, you can then point the code to a different runtime and then execute it in what we call the deployment runtime. And this brings us one step closer to production. So once you have this mode of execution, the clients would then be tapped into the data sources and you can then start training your actual federated model.

• 26:24Speaker 02

  So what does it take to deploy a production system? So firstly, there is a nice acronym that I like to use from the Tiny ML community. It's Blurb, I'm not sure if you've come across that before. Have you come across that before?

• 26:37Daniel Whitenack?

  Yeah.

• 26:37Speaker 02

  Just out of curiosity.

• 26:38Daniel Whitenack?

  Yeah. But go ahead and, yeah, go ahead and explain, yeah.

• 26:42Speaker 02

  Yeah, yeah, so the Tiny ML community talks about bandwidth, latency, efficiency, reliability and privacy. If I'm not mistaken, I could be wrong with the last one. But in the production grid system, what you really want is the reliability of the deployed solution to do, to do the full computation, right? Actually, it doesn't have to be federated learning, but systems in general. So with the current version of the Flower framework, we have separated what we call the application layer, where users will build apps and this other one that users will modify. And then we also have the infrastructure, infrastructure layer, which lies, which underpins this system.

• 27:24Speaker 02

  This infrastructure layer is responsible for receiving the Flower commands from a user and then to distribute all the necessary code to the, to the clients, for this, for the clients to actually perform the training. So in Flower terms, you come across it, what we call is the super link to actually host the server. And the supernodes, supernodes are the long-running services which basically orchestrate the clients. So these two components are long-running. So with these two components, because they are long-running, the users can then run multiple,

• 28:06Speaker 02

  execute multiple federations across other systems without worrying about any of these components failing. So this is where the reliability comes into, comes to the picture. Because the connections are also established, we also handle the bandwidth and the connection. So we try and reduce the latencies between the supernodes and the superlink as well. So the infrastructure is something that's being deployed once and that will persist for the lifetime of the project. And this makes it much easier for users to continue to work with the production grid system. So it's always there waiting for you. Anytime a user wants to go in and execute a run and look at the results, it's always there without worrying about any component failing and stopping the run.

• 29:09Daniel Whitenack?

  Chris and I are so happy that you are joining us each week to hear from amazing guests and listen to some of our own thoughts about what's happening in the AI world. But the things are just moving so quickly and we recognize and want to see you all participate in the conversation around these topics. That's why I'm really happy to share that we're going to start posting some webinar events on our website where you can join virtually and actually participate, ask questions, get involved in the discussion and really deep dive into various topics related to either various verticals or technologies or tools or models.

• 29:54Daniel Whitenack?

  The first or next of these that's coming up is June 11th at 11:00 AM Eastern Time. It's going to happen virtually via Zoom and you can find out information about that at practicalai.fm/webinars. This is going to be a discussion around on-prem and air-gapped AI, in particular as how that relates to manufacturing and advanced logistics. I've seen personally as we work with customers in this area, just the transformative power. Of this technology there from monitoring machine-generated events to providing supply chain advice and so much more. But there's a lot of struggle in terms of

• 30:38Daniel Whitenack?

  deployment of this technology in those air gapped or in those on-prem environments. So that's what we're going to talk through on June 11th. I really hope to see you all there for this discussion. Again, June 11th, 11 a.m. Eastern Time. It's going to be virtual via Zoom and you can find out more information at practicalai.fm/webinars.

• 31:27Daniel Whitenack?

  So Chiang, I love this idea of the sort of super nodes and super links. And my thought is I'm trying to work out in my head, kind of, if I was, you know, let's say I'm working in the healthcare space and my sort of nodes are maybe different hospitals or different facilities in a network or something like that. And I have a central place where I have my my super link and I'm doing the aggregation. Just from a practical standpoint, as I think Chris mentioned before, you have these different facilities, you have different maybe stakeholders with different data. What do I need to do as like, let's say I'm

• 32:08Daniel Whitenack?

  the person that's in charge of running the experiment, training the model. What do I need to do on the setup side to sort of connect in these super nodes or wherever the clients are? What sort of needs to exist there? How do I kind of register them and that setup process to really get going before I'm able to, you know, go in like you say, and from a user perspective, run experiments or perform runs, training runs and that sort of thing?

• 32:41Speaker 02

  Absolutely. So there are many ways to go about it. But I think the cleanest way is to think about to Groups of roles, one is the administrative role. Um. And they are responsible for deploying, deploying the supernodes in each of these, let's say, healthcare, uh, facilities, healthcare cent- centers. Um. Now they are responsible for, uh, making sure the, um, correct user is registered on to the, um, um, to the Superlink or the Federation, and also to, you know, um, coordinate any, um, uh, monitor, basically monitor the usage of the Superlink itself. So that's the admini- admin- administrative role.

• 33:24Speaker 02

  And then there is the user role, or data practitioners, data scientists would then write their apps, you know, their server apps and their client apps, and then run these apps on the Superlink, on the Federation that the administrator has deployed. So I think this clear distinction would be, um, an easy way to, uh, to think about it. Um. So as a start, an administrator would say, say there are five, uh, um, hospitals want to form a Federation, and administrator or, or administrators can go in, um, and deploy the supernode with the template, for example, uh, if you're using, um, communities or Docker containers, you can have Helm charts that can deploy the supernodes

• 34:09Speaker 02

  in each of these, um, five hospitals. The Superlink can be hosted by a trusted third-party server, or it can also be hosted by Flower Labs, for example, can host a Superlink for you because it's just a simple service. And then the users would register or be authenticated on the Superlink. So they need to be, uh, both authenticated and have the authorization to run the flower commands on the, on the Superlink. And that way, uh, you can get like a production system up and running, um, in a cross-silo setting.

• 34:42Chris Benson?

  I'm curious, as we're kind of talking through it, and I'm learning a lot from you as you're describing it. You've kind of made reference to admin roles and client and server apps and, you know, superlinks and super nodes and stuff. Which, you know, kind of in the context of federated, there's networking and stuff like that. So I guess I have a generalized question around that. And that is, is there any set of knowledge or skills that a user can kind of ramp up into? Or needs to know to use Flower effectively? Like particular, for instance, you know, that maybe they're coming from more of a kind of the data science or kind of, you know, deep learning role.

• 35:26Chris Benson?

  And maybe they haven't done a lot of networking and stuff like that. Do they need, are there skills that they need to be able to ramp up into to be most effective at using Flower? That you would recommend? Or, you know, what would the expectation on the user be at that capacity?

• 35:44Speaker 02

  Yeah, that's a good question, actually. It's a fair question as well. In my opinion, what we're trying to convey is that users do not need to think about the communication aspect of it at all. That everything is handled by the infrastructure. Of course, if a user starts to be more, starts to run into, when the federated learning solution becomes a bit more complicated and run through very special cases. And this is where some understanding of the communication protocols and how this is set up, this could help as well.

• 36:27Speaker 02

  And I think for users who are stepping more into sort of administrative role and want to deploy the super nodes or work with the infrastructures, basically the superlinks and super nodes. There are the questions of infrastructure slash DevOps. You have to have some familiarity with deploying this in containers or working with pods, things like that. But fundamentally, when you first start to work with the framework, you can get started with a vanilla production system without worrying too much about the communication or needing to know too much about it. And then as you, you know. Get your feet wetter, then you can learn more along the way.

• 37:08Daniel Whitenack?

  Well, yeah, that that line of thought, along with something that you earlier said about kind of how, you know, large language models, generative, have have pushed the boundaries of how you do communicate data, you know, weights back and forth, how you can handle, you know, larger models with the more recent versions of Flower, and you're releasing the new version in a couple of weeks, you know, even with with more. I'm wondering generally how, you know, certainly that's one aspect of how this sort of boom in generative AI has probably influenced your, you know, roadmap and how you're thinking about things, what people are wanting to do with Flower.

• 37:51Daniel Whitenack?

  I imagine there may be a variety of ways that that's impacting Flower. I was even thinking while you were talking about that, I was like, wow, it'd be cool if there was a, you know, MCP server or something, or, you know, helpers on top of on top of Flower that I could just, you know, type in natural language. And, you know, that would be a friendly, friendly interface to to set up my experiments and that sort of thing. So, yeah, I mean, as a, as kind of one of the the folks, you know, core folks working on the framework, how have you seen this kind of boom in interest around generative AI influence kind of the the roadmap and what you're thinking about at Flower?

• 38:33Daniel Whitenack?

  What what you're, you know, maybe envision for the future of of the framework, that sort of thing?

• 38:40Speaker 02

  Well, when you brought up the model context protocols, I had a smile on my face, but there's definitely been some interesting conversations recently as well. We in the team are looking to that. Yeah, but the impact of generative generative models or large language models slash multimodal model, multimodal models.

• 38:59Speaker 02

  There's been a, it's one of the driving forces for the flower framework as well. We really believe that this state of the art L L M's as we speak, yeah, they're running out of data to train. You know, back in, you know, in December last year, you were co-founder of OpenAI, you were saying that data is running up, no, data has run up to train these L L M's. And yes, that's exactly the sentiment that we feel as well. It's the tip of the iceberg. There are tons of data locked in silos that could benefit from having large language models, either pre-trained or fine-tuned on in order to be useful, to be made useful.

• 39:44Speaker 02

  And the way to achieve it is through federated learning. I think this is one of the key technologies that is driving the framework.

• 39:54Chris Benson?

  I'm curious, kind of to extend that notion a little bit. As we're, you know, we've been so into kind of the generative AI hype cycle for the last couple of years and stuff. And, and now we're that, that's kind of moving into combining models in different ways and agentic, you know, focus and ultimately physical, you know, models going out there in terms of, of interaction. And so, and I, I know I'm, what I'm seeing out there involves instead of just having one model, you know, people are now putting lots of different combinations of models together to get jobs done.

• 40:34Chris Benson?

  Does that in any way change kind of how you should think about, about using federated learning? Is, is like every model that you might have in a solution, just its own one-off flower implementation? Or is there any ways that you guys are thinking about combining models together? If they're all using data from, you know, different resources and stuff like that? Like how, as we're moving into my solution has many models in it. Does that change in any way how users should think about using flower or architecting a flower-based solution?

• 41:07Speaker 02

  Hmm, it's a, it's a very deep question. I, I feel that.

• 41:15Speaker 02

  There are a couple of possible futures here. There is a future where these agentic workflows, where you have models that are sort of chained together to achieve a certain task, could also be used eventually in concept with federated learning. So I, I see a future where there is a possibility about that as well. But there needs to be some intermediary steps there. And then the reason is because these models, when you use them for agentic workflows, they need to be really optimized for the agentic workflows, right? They have to have, they need to be trained on a certain type of structure and also be optimized for it. There needs to be some proper evaluations for that.

• 42:01Speaker 02

  So sort of the missing, I see the future where, you know, if, if these two sort of pathways of agentic workflows and federated learning come together, yeah, it will be that people should think about like having strong evals for this kind of workflows and then knowing that there is a limit to them. Once you're able to quantify them, then to look for ways you can improve it through distributed learning, such as federated learning. And this is how you rationalize an improvement over agentic workflows.

• 42:32Daniel Whitenack?

  Well, Chong, it's been fascinating to hear some of your, your perspective on, you know, especially production use of, of federated learning and, and Flower. As we kind of draw to a close here, I imagine we'll have Flower back on the podcast here in another couple of years or, or before, hopefully this, this does, uh, does be a recurring one. But as you look to that, this sort of next season of, you know, either what you're working on or just the ecosystem sort of more broadly, what's, what's very, what's exciting for you, interesting for you that kind of is always top of mind or is, is most, uh, most there?

• 43:17Daniel Whitenack?

  When you, when you go to, you know, you're going back from work in the evening? What, what's on your mind as you look forward?

• 43:24Speaker 02

  Yeah, absolutely. I think I'm very keen to, to think about this F Foundation LM that is purely trained on FL, on federated learning and has been shown to be both like privacy preserving and also state of the art. I think if the viewers and also yourselves, if you check out, we are collaborating with VANA as well in the US, they are looking into data Dallas and we are very much working on that. So I'm really looking forward to seeing the first element of world that is training in FL way with sort of standards.

• 44:03Daniel Whitenack?

  Awesome. Well, yeah, we look forward to that as well. Certainly come on the show and give us your comments on it when it, when it happens. But thank you so much for taking time, Chong, to talk with us. Really appreciate your perspectives and please pass along our, our thanks to the Flower team and their continued work, you know, as a team on a great addition to the, to the ecosystem.

• 44:27Speaker 02

  I will. Thank you, Daniel and Chris. Thanks for having me, having you on the podcast.

• 44:38Speaker 00

  All right. That is our show for this week. If you haven't checked out our change log newsletter, head to changelog.com/news. There you'll find 29 reasons. Yes. 29 reasons why you should subscribe. I'll tell you reason number 17, you might actually start looking forward to Mondays.

• 44:58Speaker 05

  Sounds like somebody's got a case of the Mondays.

• 45:01Speaker 00

  28 more reasons are waiting for you at changelog.com/news. Thanks again to our partners at fly.io, to Breakmaster Cylinder for the Beats and to you for listening. That is all for now, but we'll talk to you again next time.