Written by a model from the transcript; not yet reviewed.
LinkedIn recently re-engineered its newsfeed to treat content recommendation as a sequence modeling problem, moving away from independent deep learning scoring architectures. Tim Yurka, VP of Engineering at LinkedIn, joins host Matt Merrill to discuss how the team combines large language models with traditional signals, manages inference costs at massive scale across 1.3 billion members, and uses natural language policies and agentic tooling to steer content quality and combat low-quality output. The discussion explores the engineering realities of running AI systems at scale, highlighting the necessity of hybrid architectures, CPU offloading, custom attention kernels, and deep system design over simple model dumping.
Matt Merrill introduces the episode and outlines the focus on LinkedIn’s feed overhaul using LLMs.
Tim explains his 13‑year journey at LinkedIn, from AI engineer to VP of Engineering overseeing consumer products.
Tim discusses the challenges of scaling from 690M to 1.3B members and delivering relevant content instantly for new users.
Tim details the iterative training pipeline, A/B testing, and policy‑driven evaluation to ensure high‑quality, timely content.
The conversation covers the multidisciplinary pods, generalists, and specialized roles that enable rapid iteration and deployment.
Tim describes the GPU‑centric architecture, batching, custom attention kernels, and CPU‑offloading strategies for cost‑effective inference.
The team explains how they combine LLM embeddings with quantitative popularity signals and other features to maintain performance and relevance.
The discussion turns to new sleep‑inducing problems, coordination across teams, and the need for continuous improvement in a rapidly evolving AI landscape.
Tim outlines LinkedIn’s responsible AI commitments, including demographic‑free signals, audit checks, and trust mechanisms like verified profiles.
LinkedIn explains how they guard against bots that post unoriginal material, ensuring authentic user contributions.
The host likens LLM technology to a filter that cleans user input before it enters the system, preventing future issues.
The conversation highlights the importance of matching tool capabilities to tasks, avoiding a one-size-fits-all approach.
Scaling to 1.3 Billion Membersdata
LinkedIn grew from 690 million members in 2020 to 1.3 billion members, with seven new members joining every second, creating massive scale and zero-shot onboarding challenges.
Sequence Modeling for Contentinsight
Instead of scoring items independently using feature crossing MLPs, LinkedIn treats content recommendations like language sequence prediction to map out where users go next.
Natural Language Evaluation Policiesstrategy
Product managers define content quality policies that are evaluated using tastemakers and automated via GitHub Copilot CLI in an active learning loop.
Custom CUDA Kernel Optimizationstrategy
To maintain throughput without buying excessive Nvidia GPUs, infrastructure engineers optimize custom attention kernels down to the CUDA level for inference time.
Numeric Data Misinterpretationstory
Early LLMs treated numeric data and article counts like plain text and missed popularity signals, requiring the team to combine LLMs with traditional counting systems.
Compounding Agentic Error Ratesinsight
If each step of a 20-stage agentic workflow operates at 99.5 percent success, the end-to-end workflow becomes completely useless.
“LinkedIn recently rebuilt its newsfeed to treat content recommendation as a sequence modeling problem.”
“If you have an agentic system where each step of the agentic system works 99.5% of the time, and it's like a 20-stage agentic workflow, then like the end to end workflow is useless.”
“We run kind of regular audits to make sure that the kind of feed distribution, feed quality is equitable across different audiences and creators using like a series of statistical checks.”
“But protective of your APIs, right? Like, as far as I know, like you do a lot to prevent bots from just like putting out AI slop content.”
“We want to make sure that, obviously, members should use these tools, but we want them to use the tools to bring the kind of authentic, original perspectives to LinkedIn, not like super repetitive stuff through automation.”
“Good system design, knowing when to use the right tool for the right job, when to use the right size tool for the right job, input sanitization.”
News feeds and recommendation systems have long relied on deep learning architectures that score each candidate item independently. As LLMs have matured, they have opened up a fundamentally different approach, where a system can reason about content the way it reasons about language. However, that power comes with a fresh set of engineering challenges around cost, scale, and evaluation. LinkedIn https://www.linkedin.com/ recently rebuilt its news feed to treat content recommendation as a sequence modeling problem. The general approach is to predict what a user will want next, much like an LLM predicts the next token in a sentence. Tim Jurka https://www.linkedin.com/in/timjurka has worked at LinkedIn for 13 years and is currently a VP of Engineering. In this episode, Tim joins Matt Merrill to discuss how LinkedIn re-engineered its feed, how the team combines LLMs with traditional signals, managing inference costs at massive scale, steering content quality using natural language policies, and more. Sponsorship inquiries: [email protected] mailto:[email protected] The post How LLMs Are Reshaping Recommendation Systems https://softwareengineeringdaily.com/podcasts/how-llms-are-reshaping-recommendation-systems/ appeared first on Software Engineering Daily https://softwareengineeringdaily.com.
9,897 words · 4 speakers
Page updated · transcript and summary are machine-generated