Pod Insights Software Engineering Daily Inside China’s Great Firewall with Jackson Sippe About this episode China’s Great Firewall is often spoken about but is rarely understood. It is one of the most sophisticated and opaque censorship systems on the planet, and it shapes how over a billion people interact with the global internet, influences the design of privacy and proxy tools worldwide, and continues to evolve in ways that challenge researchers, developers, and policymakers alike.
Jackson Sippe https://www.linkedin.com/in/jackson-sippe/ is a PhD researcher at the University of Colorado Boulder whose work focuses on uncovering how national-scale censorship systems operate. Jackson recently helped lead a groundbreaking study analyzing a previously undocumented GFW technique that quietly broke fully encrypted proxy protocols across China for more than a year.
In this episode, Jackson joins Gregor Vand to discuss how the Great Firewall works at a technical level, the 2021–2023 blocking event, the popcount-based detection algorithm his team reverse-engineered, the cat-and-mouse ecosystem of censorship circumvention, and what these findings mean for the future of the open internet.
https://i0.wp.com/softwareengineeringdaily.com/wp-content/uploads/2023/11/Gregor-Vand.jpegGregor Vand is a security-focused technologist, having previously been a CTO across cybersecurity, cyber insurance and general software engineering companies. He is based in Singapore and can be found via his profile at vand.hk or on LinkedIn.
Please click here to see the transcript of this episode. http://softwareengineeringdaily.com/wp-content/uploads/2026/02/SED1892-Great-FIrewall.txt
Sponsorship inquiries: [email protected] https://cms.megaphone.fm/organizations/619b680e-d813-11ea-9750-e7ecac406436/podcasts/e60108fe-e328-11ea-b2b6-1348249f900a/episodes/3913370a-bc44-11ed-ace6-8b1f96d82bba/[email protected]
The post Inside China’s Great Firewall with Jackson Sippe https://softwareengineeringdaily.com/podcasts/hacking-chinas-great-firewall-with-jackson-sippe/ appeared first on Software Engineering Daily https://softwareengineeringdaily.com.
Transcript 0:00 Speaker 00 China's Great Firewall, or GFW, is often spoken about but is rarely understood. It is one of the most sophisticated and opaque censorship systems on the planet, and it shapes how over a billion people interact with the global internet, influences the design of privacy and proxy tools worldwide, and continues to evolve in ways that challenge researchers, developers, and policymakers alike. Jackson Sip is a PhD researcher at the University of Colorado Boulder, whose work focuses on uncovering how national-scale censorship systems operate. Jackson recently helped to lead a groundbreaking study analyzing a previously undocumented GFW technique
0:41 Speaker 00 that quietly broke fully encrypted proxy protocols across China for more than a year. In this episode, Jackson joins Gregor Van to discuss how the Great Firewall works at a technical level: the 2021-2023 blocking event, the pop-count-based detection algorithm his team reverse-engineered, the cat-and-mouse ecosystem of censorship circumvention, and what these findings mean for the future of the open internet. Gregor Van is a security-focused technologist, having previously been a CTO across cybersecurity, cyber insurance, and general software engineering companies. He is based in Singapore and can be found via his profile at van.hk or on LinkedIn.
1:37 Speaker 01 Hello and welcome to Software Engineering Daily. My guest today is Jackson Sip.
1:42 Speaker 02 Hey, happy to be here.
1:44 Speaker 01 Yeah, so we're going to be getting into a pretty interesting topic with Jackson today around what is called the GFW, so the Great Firewall. And we'll get into what even that is. But also, Jackson's done a lot of research into how it operates, and actually, sort of how it operates through the years because it's been changing as well. So Before we get into all that, as we like to do, maybe just sort of, Jackson, what is your background? How did you get into this kind of research in the first place?
2:12 Speaker 02 Yeah, sure. So, yeah, I am currently a PhD student at the University of Colorado Boulder, and I've been here for about five years now. I'm advised by Dr. Eric Wistro, who's been doing this work for always longer than that. And as I first started my PhD, I got involved with an organization called the GFW Report that has focused on censorship within China. And so through working with them, I've done a number of different projects, sort of exploring the relationship between China and censorship.
2:45 Speaker 01 Awesome. So maybe let's just get into what is... I always kind of get the acronym mixed up because I think there should be a C in there somewhere because it's sort of like the Great Chinese Firewall perhaps. But yeah, maybe what is the GFW?
2:59 Speaker 02 So the GFW, as the Great Firewall as it's known, tends to refer to the censorship mechanism in China. That acronym has sort of expanded as it's gained popularity. Sometimes people refer to Iran's GFW, so we often refer to it as China's GFW now. And it's a collection of different techniques and deployments that are spread around China in order to filter citizens in China's access to the internet. It affects a number of different protocols. We see it on DNS traffic, on TLS traffic, on quick traffic these days. And it's also been used almost as a weapon against organizations or services that haven't complied with the censorship desire of the nation.
3:44 Speaker 02 So one of my favorite facets of the GFW is a tool called the Great Canon, which is sort of uncommon, something that not many people have heard of. But it was this technique that they deployed in 2015 against GitHub, actually. So GitHub was allowing for proxies to sort of run through GitHub pages, I think it was. And the idea was, is China couldn't get them to comply with, "Hey, we don't want you to be this proxy source." And so what they did instead is they looked at all of the HTTP traffic, unencrypted traffic that they could going to and from China.
4:23 Speaker 02 And they would inject a JavaScript request in that traffic that would then make a request to GitHub. Right. So effectively, they designed the largest denial of service attack ever by injecting these JavaScript requests into every HTTP request that went across the GFW.
4:42 Speaker 01 Wow. Okay. I think I had heard of GitHub playing a part somewhere, but yeah, that's new to me as well. So I guess like looking at the fact that you've done a lot of research in this space, again, before we get into what the research sort of turned up. Why, again, just for the audience, you know, some people in the audience may have been to China, may understand roughly what happens from the internet perspective. But equally, I think let's just assume zero knowledge. So like, why is this kind of research difficult in the first place?
5:13 Speaker 02 Sure. So the goal of the GFW is to restrict Chinese citizens, specifically their access to information outside of China that they deem undesirable. Right. So this might be like Western entertainment that maybe goes against the goals of the CCP, the Chinese Communist Party. And what we really struggle with is finding ways to detect that censorship not being citizens located in China. Right. So one of the biggest kind of logistical issues we have is getting access to vantage points.
5:52 Speaker 02 Right. Places in China that we can connect to and then send requests that may or may not get blocked by the GFW. Additionally, because we aren't part of the CCP or the Chinese government, right? We don't necessarily know when we've reached ground truth, right? At the end of the day, this is sort of a black box system and we can, to some extent, only speculate, right? It takes a number of experiments to really determine whether or not what we have observed is what we think it is or just some other effect of the network.
6:25 Speaker 01 Okay, so before we kind of get into what the research was, there's also a kind of like a timeframe thing here, which is helpful. So something kind of happened in, I believe it was November twenty twenty one. Let's start there. What happened then? And maybe did that kind of almost kick off why the research was then undertaken? Or you can help us out?
6:47 Speaker 02 Absolutely. Yeah, sure. So I think that the incident you're referring to in November twenty twenty one was the deployment of a new technique by the GFW to detect and then block fully encrypted protocols. And I think to understand that, we need to sort of get back to how Chinese citizens have responded to the GFW, right? Which is, there are plenty of people in China who disagree with it, right? They would like to be able to access maybe whatever is latest on Netflix or something else. And so they will use proxies in order to circumvent this censorship and connect out to whatever content they otherwise wouldn't be able to access, right?
7:28 Speaker 02 And these proxies use a number of different techniques to be censorship resistant, right? If you think about a proxy that you've accessed before, or maybe like a VPN that you've connected to before, that traffic, it's obvious, right? We can look at it in tools like Wireshark, right? Or observe it on the network somehow and see that that is clearly a VPN connection, right? Or some type of standard proxy connection. So the GFW has designed ways of detecting and blocking that, right? And so proxy developers kind of had to get more creative and they had to develop protocols that would be less obvious to some type of network observer like the GFW. And these tools took two different forms.
8:11 Speaker 02 The first of which being mimicry, right? We are going to try and make our proxy look as close to all the other traffic on the internet. And if you aren't familiar, almost all the traffic on the internet looks like TLS because everybody's doing something that involves a website, right? And so the first thing they did is tried to mimic TLS, right? And look as close to normal TLS as possible. But with TLS kind of having sharp edges, there were always these issues that would exist. And the censor could then block that traffic because they would see that, say, like, the encryption scheme was wrong, right? They picked the wrong cipher suite, something like that. And so this led to a different approach. If we can't look exactly like everything else,
8:54 Speaker 02 let's try to look like nothing. Let's try and blend into the background. And that led to these protocols that we've referred to as fully encrypted. And a fully encrypted protocol involves some type of key exchange between the client and the server. And then after that key exchange has occurred, every single byte that is sent from the client to the server and vice versa is encrypted. There's no protocol header. There's no exchange of information about size or anything like that, right? It is just going to be encrypted bytes as the payload of like a TCP connection. And the idea was, okay, this just sort of blends in. There's nothing to worry about. They won't be looking for this type of traffic. But these tools became pretty prolific. They became the standard, right?
9:39 Speaker 02 And every proxy had sort of implemented their own version of this fully encrypted protocol. And so in November of 2021, users all of a sudden just couldn't access their proxies, right? And it wasn't just one, right? It wasn't just one particular implementation, but it was widespread. It was all of these different proxy developers, right? Shadowsocks and V2Ray are two names that are pretty popular in the Chinese censorship circumvention space. And both of their services had sort of fallen over as a result of this attack. And this was alarming because this had been a very stable approach for years. And users were trying to understand what had happened.
10:19 Speaker 02 And there was no clear answer of, oh, this one developer has decided to pull the site, things like that.
10:25 Speaker 01 Yeah, this is very interesting because on the layman side, I do remember there was just like a point in time when, so I was living in Hong Kong. Yeah, I guess in November 2021 and beforehand, and there was this sort of watershed moment where people started saying, oh, my VPN just doesn't work in China anymore. And it was sort of like, make sure you get maybe the quote correct VPN before you get there because you've got no chance of like getting it whilst you're there, that kind of thing.
10:53 Speaker 01 So it kind of became this which one works and some people, I've even seen like posts today where like, I think people still go into mainland China and haven't fully anticipated the kind of tools, if you want to call it that, that they'll need to just be able to access things the way that they would hope to be accessing them.
11:13 Speaker 02 Absolutely.
11:14 Speaker 01 Yeah, great. So let's move to the research that you and your colleagues did. I think this was like fairly groundbreaking research into sort of how this all worked. Yeah, let's just like talk through that. I mean, we've, I believe it was a sort of six month setup that you had to put in place. But I mean, like, let's kind of start there. Like, where did the research begin and how did you go about it?
11:37 Speaker 02 Yeah, so as I said, I've worked extensively with this organization called GFW Report. They were sort of the leads on this project. They were the first people to be notified of what was going on. And there was no explanation to start, right? It was sort of this widespread event where no one knew exactly what tools were working, what tools weren't. And it started with effectively just seeing, okay, which of these tools have all of a sudden stopped working and what techniques were they using, right? And that led to this realization that, okay, it is in fact these encrypted protocols that seem to be the issue, right?
12:14 Speaker 02 If we send a regular TLS connection or TLS style proxy connection, there doesn't seem to be a problem, but the fully encrypted traffic is an issue. And so what we ended up doing is setting up a number of VPSs in China, primarily in the Tencent and Baidu clouds. And then we set up some sync servers. In the US, right? So we had a number of different university partners, Boulder, as well as University of Maryland and the University of Massachusetts Amherst. And we set up servers at all of these universities that would just sort of listen to any TCP traffic that came to them. Nothing particularly interesting on that end.
12:56 Speaker 02 But what it allowed us to do is we could now send traffic from inside of China outside to the US, right? Crossing that border and effectively triggering any censorship that may occur there. And while this sort of set up our architecture, we then sort of had to go into, okay, how are we going to determine what exactly is fully encrypted traffic, right? This is a hard question. And so what is encrypted traffic and how did we figure it out? Well, the first thing we did is started sending in these TCP payloads, just random bytes, right? See that this is getting blocked? Okay, that makes sense. What if we send all zero bytes or all one bytes, right? And I'm sitting in my advisor's office, late at night, we're trying to figure this situation
13:41 Speaker 02 out, and we see that certain bytes would get by when others wouldn't, right? When we just sent a long string of the same byte. And ultimately, what we found was that they were determining the entropy of the payload based off of the number of bits that had been set in the total payload, right? So they were doing a count, what we refer to as a POP count, on each byte. And if it was roughly 50% of bits set zero and one, then they would consider that to be high entropy traffic, and then they would ultimately block those connections.
14:16 Speaker 01 So we're going to get into POP count shortly. Just kind of just sticking on, I guess, the setup here. Were there any kind of risks with this? I don't know. Did you ever think that, like, effectively within China, something would get triggered and sort of there might be any risks associated with this? Or is that just something that comes along with the research and you have to just accept that that may be the case?
14:40 Speaker 02 Yeah, that's a great question. So the risks with this particular work were relatively low in our sort of threat model, if you will. We try to reduce the amount of information that we leave on these servers, right? We assume that at any point in time, they could get flagged as being owned or controlled by us and just shut down, and we'd lose access to whatever was on them. So we are very careful in how we access those machines, making sure that all the data that we're collecting is removed quickly so that we don't, of course, have setbacks, as well as reducing what may be accessible to an adversarial nation, like China, were they to do, conduct forensic on one of these servers.
15:19 Speaker 02 But aside from the sort of technical risks there, there's also kind of these risks to the people doing the research, right? In this particular work, again, there wasn't much of a concern. We were just doing this sort of observation of what was occurring. But in some of our other work, we have really struggled with these concerns and even struggled to have people accept the research that we have done due to the legal risks associated with it. Some people refer to some of our work as being an offensive attack against these services that are deployed in China.
15:51 Speaker 01 Which is very interesting, because at least from what you described so far, you're simply allowing traffic to leave China and go, in this case, I guess, into the US. There is no sort of threat data there. It's not like you're doing a kind of reverse pen test or something. It's like, it's just information.
16:10 Speaker 02 Exactly. You're exactly right, in this case. And now, some of our other research, one particular attack, we found a memory leak in their DNS censorship that allowed us to leak vast amounts of memory off of these servers. That was one of the attacks that we were maybe a little more concerned about, but it sort of goes to show how just observing this traffic or sending traffic back and forth isn't something to be as concerned about.
16:35 Speaker 01 Yeah. So you mentioned pop count. That is an interesting concept, as you've just touched on. It refers to sort of the density, well, the bit density in the payloads. But let's talk a bit more about that because it seems like that was such a critical heuristic for how the GFW now operates.
16:57 Speaker 02 Yeah, absolutely. So pop count is kind of a technical term or an implementation of hamming weight, right? So with hamming weight, we're concerned with like the number of non-zero symbols in some type of data. And with pop count, we are just counting the number of non-zero bits in some form of data, right? So for each byte, we simply count, okay, how many bits have been set? And, you know, eight bits in a byte. So if we see four of those bits set, we can assume that this is relatively high entropy, right? The closer we are to 50%, the higher likelihood that this is a packet containing high entropy and likely encrypted. Data will cluster around that number, right? Because if we have to flip a coin eight times, right?
17:41 Speaker 02 We would expect four heads and four tails. Same thing applies when setting bits in a byte. And ultimately, this sort of speaks to how the GFW wants to execute this censorship, right? They are doing the best they can to come up with crude but effective techniques, right? So this is a relatively simple thing to do, right? We're just counting bits. But that's the sort of technique that they have to deploy in order to stay relatively optimized and keep up with the vast amounts of traffic that they are constantly adjusting.
18:16 Speaker 01 Okay. And just to back up for a second, the sort of overall thing that was kind of discovered here was the idea that rather than selectively blocking specific things, it's kind of the other way around now, which is that everything is blocked unless, I guess, in this case, like the bit density can be determined to be, well, I'm looking at the notes here, which is it has to fall within a very specific range, is that right? For it to be
18:42 Speaker 02 Yes. So it's funny, the range was exactly 3.4 to 4.6 was the count. Within that threshold, they would consider that traffic to be high entropy and they would ultimately block it. How they came up with this number, we're not exactly sure, right? That was just in our test. That's what we observed is the exact threshold for which they would consider it to be high entropy.
19:05 Speaker 01 Right. And so then for the GFW to sort of accept that this data is random and should pass effectively, that as you touched on, that sort of clusters around like the 4.0 of bit density. Is that correct?
19:17 Speaker 02 Exactly, exactly. So count all the bits that you have in the packet and see how many are set to one. And then so divide the numbers set to one by the total number of bits, right? And if that value shows up between 3.4 and 4.6, then you can say, okay, this is clearly an encrypted payload and we're going to block it. Otherwise, we allow it.
19:40 Speaker 01 Gotcha. Yeah. So then there are kind of like other, I guess, exemption rules that I think you discovered as well around ASCII and protocol fingerprinting. So maybe if we could just like dive into those as well.
19:54 Speaker 02 Absolutely. And so I think what's important to understand here is that that entropy count actually came last. Right? So we refer to our rules as exemptions, right? So we're trying to find a way to exempt traffic from blocking. Sort of like you said, block everything and then decide what not to block. And when we think about POP count, right? We have to look at every single bit in that payload in order to conduct that entropy test, right? And that's, that can be considerably resource intensive when you're considering the scale of traffic they're observing. So they tried to identify other tests that they could run first to sort of filter off that traffic. And there were two different ways that they did this.
20:39 Speaker 02 The first being that they would look for large amounts of ASCII. So if they saw a continuous string of ASCII characters, encoded ASCII characters, they would exempt that traffic. If it started with ASCII characters, they would exempt that traffic. And that was effectively just saying, okay, this is unlikely to be encrypted data, right? So we can, even though we don't know what this protocol is, we can allow it, right? The other approach they took to filter off traffic before they reached that entropy measurement was through protocol fingerprinting. It was really a crude protocol fingerprint, but effectively they chose a handful of protocols.
21:17 Speaker 02 That they knew they saw a lot of, for example, TLS, looked at what the first few bytes of a TLS connection were. And then if they saw those bytes at the start of any packet, they would say this isn't going to be considered a fully encrypted packet, right? And that did two things. In the case of TLS, you're able to filter off the vast majority of your traffic. And the measurements that we conduct here in that regard, it would be about eighty percent of traffic will get exempt right there with looking for TLS fingerprints. But in addition to that, if you think about TLS packet, right, we've got a couple of header bytes at the top, and then we have an encrypted payload. So if you aren't filtering off that traffic,
21:59 Speaker 02 you will have an insanely high false positive rate because so much of the internet is in fact encrypted. It's just encrypted in a way that we're okay with, right? As long as it's TLS, that's not a problem. Or as long as it's SSH, that's not a problem.
22:12 Speaker 01 Yeah, I wanted to then touch on false positive effectively, sort of, you call it sort of collateral damage to like legitimate traffic, if you like. How, I think there was a, I believe in the research was estimated around, was it zero point six percent of like the normal internet traffic would be false positives? How did you, I guess, arrive at that validation?
22:33 Speaker 02 Absolutely. So the false positive rate we concluded was zero point six percent. You're exactly right. And the way that we did that is by taking these rules that we had determined and testing them against benign traffic that we had here at the University of Colorado. So we are fortunate to have a resource here that allows us to see all of the network traffic going to and from the university. And we rolled out these rules and basically looked at the traffic that wouldn't be blocked, or that would be blocked if it was conducted here at CU. And what that allowed us to do is say, okay, no one at the university in the United States is going to be using censorship circumvention tools or proxies, right? Because there's no blocking occurring, why would they go to the trouble?
23:19 Speaker 02 And so we could sort of infer that this may be the false positive rate. And that allowed us to identify a couple of other exemptions that they were doing. We took some of the bytes that we observed in those packets that would have gotten blocked here at CU. And we found some other TLS resumption headers that they were allow listing. And we found that the large majority of the traffic that would have gotten blocked here at CU ultimately belonged to torrent services, those specific protocols for torrenting. And so we believe that there may have been some reason that they still wanted to block that traffic anyway. So their false positive rate might have been seen as even lower based off of that.
24:03 Speaker 01 Yeah, very interesting. I guess that's an interesting piece of doing it within a university. There's obviously a lot of, just a lot of traffic generally. So, yeah. I mean, was there any sort of, like, if you were to do this again, for example, like, would there be any benefit to doing it in a sort of more siloed environment? If you like, or do you think that would not really affect the results?
24:24 Speaker 02 Do you mean for calculating the false positive rate?
24:26 Speaker 01 Yeah, exactly.
24:27 Speaker 02 Well, I don't think so. And the reason being is that having access to this sort of network that has just a large amount of random traffic should be, in theory, representative of the type of traffic that would be coming and going across that national border. Right? And so our hope is that we're modeling that as close as possible.
24:46 Speaker 01 Yeah, that's a really good point. Okay, so if we look at the idea of pop count manipulation. So I think I'm going to let you take us through this. So, I mean, you are a programmer and I believe a lot of this was done in Rust. So I know we've got a lot of Rust listeners as well on the podcast. So could you maybe just take us through sort of what even is pop count manipulation? And why was that part of the research?
25:12 Speaker 02 Absolutely. So, once we had sort of determined, okay, this is how they're doing this blocking, right? What can we do to help proxy developers get around this blocking? Right? This is one of the most common techniques for censorship circumvention. We've got to get it working again. And our solution was, we knew that they didn't have some fancy metric for determining whether or not it was encrypted, right? It was just this pop count. So what if we stuffed the payload full of additional bits, right? We can check the pop count of our own packet that's about to go across the network. And if we see that payload is highly entropic, which of course it's going to be because it's encrypted,
25:54 Speaker 02 we can instead add some ones or some zeros, depending on which way we need to go, and get outside of that encrypted threshold range, right? And so that was one of the more sophisticated techniques that we deployed in order to stop this censorship from being effective was we would add bits to the payload using a scheme based off of the key for the encryption, so that it was pseudo-random, right? There wasn't just a bunch of ones at the beginning or a bunch of zeros at the beginning or something. And then we would add a few bytes at the end that would tell you exactly how many bits you need to remove, right?
26:36 Speaker 02 And so then when the server or the other host receives this packet, they can decrypt how many bits they need to remove, and then they can determine which bits are supposed to be removed based off of that key. And ultimately, this got implemented in ShadowSocks's Rust implementation, as well as the ShadowSocks Android implementation, and has been working to this day, I believe.
27:01 Speaker 01 Wow. And like, what kind of performance hit or overhead would you say this adds, if any?
27:08 Speaker 02 Sure. So we estimate that the overhead is about seventeen percent. So if you assume a worst-case scenario of you have a pop count of exactly four, right? Meaning we have four zeros, four ones, and every single byte on average. What we need to do is add just enough bits, in this case we'd be adding ones, right? To exceed the pop count of four point six. And what that worked out to is about seventeen point six percent overhead, which we find to be very tolerable, right? Given the overhead that's already required for doing this sort of multi-layered proxy, we believe that seventeen percent overhead is something that you could apply to every single packet
27:52 Speaker 02 and not necessarily just the first few packets of the connection or to get around the censorship.
27:56 Speaker 01 So is this like ShadowSocks Rust, to your knowledge, is this being used within VPNs, like commercially available VPNs? Or like, how is this sort of been taken up beyond the sort of research stage?
28:11 Speaker 02 Yeah, so ShadowSocks Android is an app that you can download, right? And you can use as a client, right? ShadowSocks Rust is a, they provide like a command line implementation, but it's also designed as a library that can be added into other VPNs. Off the top of my head, I don't know the names of like the commercial ones that do use it, but it is heavily used by more user-friendly applications.
28:37 Speaker 01 Yeah, this is something I'm probably, for sure, gonna go and do some extra research on, you know, where certain, I wouldn't sort of name names, but like certain VPNs have like, said, "Oh, we've got this like new protocol and it's faster, better, can get through anything else that might not have been working." So I'm curious if maybe this is what actually they've been referring to, building into their commercial products. So that's a little note for me to go on afterwards. I think you also came up with other sort of circumvention methods. One is, it sounds very simple, but I mean, there must be a lot more to it, like adding an HTTP or TLS header. How does that even work?
29:18 Speaker 02 You're exactly right. It is as simple as it sounds. And so we found that there were these exemption, basically byte strings, right? That would be the first four bytes of a TLS connection, right? And so we said, "Okay, can we just add those four bytes to the start of a fully encrypted or fully random payload?" And the answer was yes. And we were like, "Whoa, well, that was really simple." And so what we ultimately ended up doing is we provided this information immediately to the proxy developers, right? We said, "Hey, guys, we're working on this. We're still not exactly sure how it works, but we do know that if you add these couple of bytes to the start of every payload, your traffic will get through no problem." And so we sort of used this as a...
30:00 Speaker 02 Emergency solution, right? A little Band-Aid to give out that would allow people to get their services back up and running while we developed the kind of more sophisticated approach behind the POP count manipulation.
30:13 Speaker 01 I mean, is there any evidence that the GFW is actively probing suspected proxies like this? Or?
30:20 Speaker 02 Absolutely. So, active probing was a technique that was around long before this particular incident. And it's one of the primary concerns of proxy servers, right? Is that the GFW will see a connection to an IP address, right? And they'll be like, that connection looks a little weird. We think it might be this proxy implementation. But we don't want it to go block an IP address willy-nilly. So what we're going to do is we're going to make a couple of connections to that IP address and see if we can get it to talk to us in a way that is representative of one of these proxies, right? And so this is a common technique that they were using. And it was actually, they were doing active probing with sort of this entropy check prior
31:04 Speaker 02 to the passive detection that was going on in this paper. And I do think that that's an important distinction to make, is that this was a passive attack where they were just blocking any traffic that met these parameters and it wasn't affecting specific IPs. There were some that were specifically targeted, but it wasn't like this specific IP is a proxy and we're not going to allow traffic to it.
31:29 Speaker 01 And like, I mean, is there any sort of defense against this? Or is this just kind of part of the landscape and it's going to be, for anyone that's like trying to do this, is this just sort of a cat and mouse thing?
31:41 Speaker 02 Well, and cat and mouse is the term that we frequently use, right? Is how are we going to get just one step ahead this week? And there are a couple of different defenses or strategies that we can use to prevent active probing. There are techniques like trying to behave the same for every single connection, right? So you might try to have a standard web server, right? And it will receive standard TLS connections, standard HTTP requests, right? It will respond to those. But when it gets special connections, right? Proxy connections, it will handle those differently. And the hope is, is that by using some type of cryptographic key that's sent by the client... The sensor will never be able to determine that it is in fact a proxy, right?
32:24 Speaker 02 They won't have access to that key in order to get that host to respond in a specific way. There are other approaches, kind of similar vein, in which you use some sort of other application as a front. Right? So the naive proxy is known to basically use Chromium as its sort of front application and then the proxy sort of runs on top of Chromium, right? So everything looks like it's a standard Google Chrome TLS connection, but in reality, it's a tunnel for naive proxy. The third approach that we also use here, and has been a large chunk of our research, is refraction networking. And in this case, what we do is we say,
33:06 Speaker 02 "Okay, you can send a connection to any IP in this range that we can observe, right? Say like the University of Colorado's IP range. And using this like network tap that we have access to, we'll just look for connections that look like they belong to us. And when we see one, we'll pick that connection up. We will start our proxy tunnel with that connection and redirect all that traffic to its intended destination, right? And why that's so advantageous and sort of defeats the cat and mouse is now the sensor has to decide, "Okay, in order to block this proxy, we have to block every single IP address associated with this American university." And so we really raise the stakes of the false positives associated with blocking that sort of traffic.
33:52 Speaker 01 Interesting. Okay. So if we look at deployment, like, once this was kind of all discovered, if you like, did you have to do like responsible disclosure? And like, who do you even make that to? And how does that then, I guess, filter through into sort of actual non-academic users, I guess?
34:15 Speaker 02 Yeah, sure. So responsible disclosure is a funny subject to bring up when you're when you're attacking a system like the GFW, right? There are kind of two sides to that coin, right? We are disclosing our findings to the circumvention tool developers. But then there's also this idea of, "Well, do we disclose something to the GFW? Like, is that something that needs to happen?" And in this particular case, no, right? We were just reverse engineering this technique that they had. And it's not usually something that we're particularly interested in doing. It's helping them figure out what we've learned. But we do work closely with the circumvention tool developers, right? So like I said, in regards to the sort of propelling bytes that seems very simple,
34:57 Speaker 02 that was information that went out very early in this process. Like January of 2022 is when we ultimately sort of found that and started giving that information out to these developers so they could start patching their tools as early as possible. And that sort of gets into what you were asking about, how do we filter this out of the academic world and into the hands of, you know, people on the ground? And it's those same relationships, right? Where we already work closely with the developers of these tools and when we find something out, we can let them know before it goes public and they can try and get to their users as quickly as possible.
35:33 Speaker 01 Yeah, I do find the whole, yeah, as you say, responsible disclosure and then thinking, would you sort of disclose this to what, the CCP? It's a strange one to sort of think about. That said, do you have any understanding that this research has been taken in on that side and any lines of communication around it or sort of implicit or explicit changes that you've seen as a result?
35:58 Speaker 02 So with this particular research, no. There wasn't really anything we told them, right? And there wasn't really anything that we could observe. We'll get into some more of the timeline, I think, in a little bit. But on other works, like I mentioned, we had this offensive attack in their DNS injector. And we did try to disclose that to them. We sent emails to C Insert, that was who we decided was the best point of contact. We never heard back, but what we did find is that there was patching of this vulnerability after we had sort of made this disclosure, right? So we do believe that they, to some extent, are listening and some extent are paying attention to what we're doing, but it's still that black box where you don't have 100% confirmation.
36:43 Speaker 01 And yeah, as you alluded to, there was sort of another timeline marker here, which is, I think it's March 2023, that the GFW actually stopped dynamic blocking. Could you talk to us about what you understand from that?
36:59 Speaker 02 Yeah. And, you know, again, this is one of those parts where we can speculate a lot, and I can give you some of my thoughts on why, but the facts are, we don't know, right? What we do know is that, you know, the blocking began on November 6th of 2021 and ended on March 15th of 2023. In that timeframe, there were a number of political things going on in China. Xi Jinping was running for reelection in the beginning of March of 2023. I believe he was like reelected two or three days before the blocking stopped. And we've seen that pattern occur, not just in China, but in a number of other countries, where when politically sensitive events are occurring, they will sort of ramp up their censorship.
37:40 Speaker 02 And then when those events sort of settle down, they tend to turn off their censorship or ramp it back down. What's interesting with China is usually when they find a technique that they like, they stick with it. They keep it going kind of ongoing. And so what was different about this technique is unclear. We continue to do longitudinal measurements to see if this censorship has been turned back on. We still haven't seen any evidence that it has been re-enabled. But we can only speculate that, you know, potentially it was too computationally intensive or that the vantage points required were too valuable to be using for this technique.
38:19 Speaker 01 Yeah, and I'm thinking of the dates again, so 2021 and 2023. I mean, do you think the intertwines with politics, as you've touched on, but the pandemic, that was obviously quite a sort of sensitive time for China from an information standpoint, because it seemed like, at least, so I was living in Hong Kong at the time, which had a, people often refer to Hong Kong and the lockdowns, and I can say categorically, there were no lockdowns in Hong Kong, but there were for sure lockdowns in China, as sort of everyone's being able to leave your house and so forth. And I think it was quite sensitive. As to people within China being able to see what the rest of the world was doing, especially towards kind of maybe towards the end of the rest of the world would call
39:02 Speaker 01 the end of the pandemic, because that seemed to be kind of then the worst part for China. And as you say, it was ramping up into a reelection, if you want to call it that, of Xi Jinping. So anything there that you think could be related as well?
39:15 Speaker 02 I think it's very possible. Again, that's what really, one of the things that makes this research the most challenging, though, is you never really know. You don't get copies of the memos sent around the government that say, hey, turn on this censorship on this day for this reason, right? And so you're exactly right. It could have been certain events going on, like, like sort of the tail end of the pandemic that was causing the censorship to ramp up. It could have been, you know, the political timing. It's it's really hard to say.
39:43 Speaker 01 Yeah, China in a nutshell.
39:46 Speaker 02 Exactly.
39:46 Speaker 01 Yeah. And like today, you know, here we are in 2025. And just like, what is is the current status, the same effectively that it's off? Or have you seen anything change?
39:56 Speaker 02 Yeah. In regards to this specific technique, we haven't seen any activity, right? That's not to say the GFW is turned off, right? This is one small piece of the GFW, and they're still deploying a number of other techniques. But the idea of passively observing and blocking fully encrypted traffic, we are not seeing any signs of today.
40:17 Speaker 01 Looking at just speculation on how this was actually operating, even though, as you've just touched on it, this specific part is probably turned off. But like, general GFW architecture and just a speculation on how that all works, like, what do you think it is?
40:34 Speaker 02 Yeah. So from a technical perspective, there are a couple of different vantage points that they could have access to, right? And the two that sort of apply here is what we would refer to as like an in-path sensor or vantage point versus an on-path vantage point, right? And an in-path vantage point sits quite literally in the path. It is one of the hops that your traffic takes between the two hosts, right? And what's special about that vantage point is that it has the ability To tamper with that connection, right? It could change the payload or it could go so far as to just start dropping the packets. And because of the blocking technique that we observed in this research, we believe that they were using these in-path vantage points
41:17 Speaker 02 because they were just dropping the packets afterwards. The other side of that is what's called an on-path vantage point, right? This is a less sophisticated or less valuable vantage point where they get a copy of all of your traffic and they can even send packets back to the hosts. But they aren't sitting in the path where they can manipulate and potentially drop that traffic, right? And it's an important distinction because those in-path vantage points, if something goes wrong, right? That could lead to internet outages, right? And you could have major issues there. So I believe that those are potentially sensitive assets and they are very careful of what they allow run on those.
41:57 Speaker 02 Most of the censorship that we observe ongoing tends to be more of an on-path attack.
42:04 Speaker 01 And do you think there's any ML, machine learning, kind of going on there? Or is it kind of more of a static, I guess, implementation?
42:13 Speaker 02 Yeah, so I think that for specifically what we observed, they had to have, you know, deterministic code that was running, right? They had if statements that were checking those bytes, right? And checking that entropy. Because, you know, ML inference is just too slow to be doing it at line rate. However, it could be possible that they were using ML to try and determine some of those heuristics, right? It's hard to say. Though we have confirmed that the GFW is using machine learning in other aspects. There's been a massive leak of information in the past couple of months coming out of a Chinese corporation called Gedge and a research lab called Mesa.
42:57 Speaker 02 And we have found evidence in those repositories that they have been using machine learning to classify data and determine what traffic is coming from a sensor and not. However, I think that most of that research is being used in those active attacks, right? Those attacks where they're making the connections because to try and do it passively at line rate is just unrealistic.
43:18 Speaker 01 Gotcha. So something that kind of comes up in my line of work is, again, I sit in Singapore and so I'm working currently with products that do get used within China. And you know, people sometimes come to me and say like, oh, do you think there's going to be a problem with this team using, when I say problem, I mean like a performance or just access problem with this team using the product like within China. I give a very kind of, this point, lame an answer. The last time I went into mainland China is probably 10 years ago at this point. I used to go quite often and just stopped. But my, my understanding is always just that there is this massive speed, the massive latency around traffic that goes in and out of the perimeter,
44:01 Speaker 01 if you want to call it that. And that's usually my kind of, again, standard piece of advice is just if this is not something that has infra based within the GFW. So that's usually why people strike up deals. I believe Cloudflare has like a product that's kind of like a deal where they've had to, you know, JV, joint venture with, with a Chinese partner so that like that's a way to enable your product to get over this latency. I think you sent me a little piece on this before we chatted today called, and it's like called The Great Bottleneck. So yeah, maybe could you just talk to us about that? Because I think I would also love to like learn a little bit more around what actually is this, rather than me just saying, oh, there's just like a latency problem and that's what you should assume.
44:43 Speaker 02 Sure. Yeah, we've got to get more creative with our naming. I mean.
44:47 Speaker 01 The GD something. Yeah.
44:49 Speaker 02 Yeah, yeah, exactly. It's always got to be the great.
44:51 Speaker 01 Yeah.
44:53 Speaker 02 But yeah, so this has been an issue for, it seems like a long time. And it's an issue that affects, you know, more people than just the citizens of China, right, who are trying to access things they aren't supposed to, right? The Great Bottleneck is this sort of widespread idea that traffic going outside of China is going to be slow, or it's going to have some type of network issue, right? And. The research that you're referring to, I think, found that was primarily on the download side, right? So when we are pulling traffic from other places, we have an issue or pulling data from others outside of the country and we have an issue. The upload doesn't seem to be as big of an issue. I think that this primarily isn't necessarily a censorship
45:34 Speaker 02 technique, right? I think that if it was, if it was a result of the censorship that we've observed, if they didn't have the compute resources to effectively censor this traffic, we would see it more on that upload side, right? When that HTTP request gets made, that's when you would see the slowdown because that's the traffic that you are trying to observe for potential censorship things. So I think it is truly just a lack of international infrastructure in China. I think that, and I think the reason that they are doing that is because they sort of have this goal of creating this isolated ecosystem, right? They want for their users or their citizens to primarily be accessing Chinese-based sources,
46:18 Speaker 02 right? Chinese tools, Chinese services, what have you. And so I think that even though they could probably afford to spend the money to improve that international infrastructure, build some more cables, under RC cables or what have you, they are choosing not to, almost as a way to incentivize both companies and users to use national services. And so, yeah, I don't think it's necessarily malicious, but it's more pushing towards this goal of isolation.
46:47 Speaker 01 Yeah, the cloud providers are sort of another proxy battle, I guess, where you've got, you know, like Alibaba Cloud and others within China and then obviously the big three that we know in the US and the rest of the world. So I think that's an interesting take on it, which is, yeah, this probably has like as much of a commercial leaning as it does. I think that's another thing where, you know, people go into China and I asked a friend who goes more often than I do, I said, oh, my family member going to China, like, what should I advise them? And he just said, oh, you must download like this specific app. Yes, it's half in Chinese, but like this is how you
47:26 Speaker 01 Be able to like do stuff, you know, like call a cab or, I mean, that's like, maybe it seems obvious, you know, you go to, you go to another country and okay, like in Singapore, say, hey, get the Grab app like, cause Uber is not a thing here, but this is like a whole layer extra, which is just sort of like, you're going to be able to achieve nothing unless you actually download this super app and has, you know, maps and transport and food and all this kind of stuff. Everything else just isn't going to help you out, basically.
47:52 Speaker 02 I think you're exactly right. This isn't something that's unique to China. I mean, that sort of super app seems an extreme example, but just like you said, there tend to be ride sharing apps that exist in certain regional areas, or even like the US trying to block TikTok unless it gets sold to an American entity is really the same thing, right? It's this kind of idea of we are going to prioritize national services before international services.
48:20 Speaker 01 Yeah, absolutely. And yeah, I mean, I just, I carve out with this is that Hong Kong still doesn't really experience this bottleneck effect. At least that was my, when I was last there, I think last year, there is no sort of GFW per se within Hong Kong. And is it used as any kind of like proxy or something? I think we were touching on this before we started recording.
48:42 Speaker 02 Yeah. So there is sort of this idea that Hong Kong tends to have a freer internet in our perspective of things. I would definitely defer to you on any lived experience, but we do see that a number of proxies will kind of use that as their first hop. And there's even some speculation on if there are gray market links that exist between China and Hong Kong, specifically for the transit of proxy traffic that's ultimately going to go other places.
49:11 Speaker 02 Some people even believe that a lot of the proxy traffic exists simply as a way to get around the bottleneck, right? They might not be trying to access undesirable content. They just have some type of maybe business need to get outside of the country. They aren't like at the enterprise level where they can afford these contracts with a cloud player. And so they use these proxies as an alternative means.
49:34 Speaker 01 Yeah. And this crosses into like the realms of like dark fiber, I think.
49:39 Speaker 02 Yeah, yeah, yeah.
49:40 Speaker 01 So I mean, my anecdote on this is just like, you know, I could be sitting in a Starbucks here and I'm overhearing a conversation of someone who's clearly come from Hong Kong to sell a dark fiber contract to someone in Singapore. Could you maybe just touch on that briefly? Like, how does that cross into this?
49:55 Speaker 02 I'm not much of an expert on that. I've only heard sort of, like you said, these anecdotal experiences of somebody has access to some type of link that goes from mainland China to Hong Kong and then they're selling access to that to people in China. But I would love to know more.
50:14 Speaker 01 This looked, I mean, at least to me, it kind of looked legit. You know, this is, but equally, I don't know, why is it being held in a Starbucks and not in someone's office? That's maybe, that's all we need to know. If we sort of then just look at any other countries that you at least believe may have adopted these techniques. I mean, maybe there's some obvious ones that people are already thinking of, but like just from where you sit, like which other places have you maybe seen adopt the GFW approach?
50:40 Speaker 02 Yeah, so there are a couple of different things to consider there. Specifically for the GFW approach, I think it's interesting or pertinent to get back to that sort of leak that I was referring to. And you may have heard about this, it sort of hit some of the mainstream news sources and that this massive trove of data got out of this Chinese corporation called Gedge. And this corporation's goal is effectively to commercialize censorship software and hardware and then distribute it to other nations. And in that leak, they found that Kazakhstan, Ethiopia, Pakistan and Myanmar are all confirmed
51:20 Speaker 02 clients or customers of this organization that is Gedge. And so we know that they, to some extent, have the capabilities of the GFW through these, these business dealings. However, those aren't the countries that we typically think of, right? When we think of other nations that are doing censorship. I think one of the big ones that comes to mind is Iran, right? We don't know if there's any relationship between Iran and China's censorship infrastructure. Iran, they all tend to have their own sort of character, if you will. Iran is known to be very dynamic. They are constantly changing what's getting censored, how it's getting censored, where it's getting censored. And two neighboring ISPs in Iran can have completely different censorship experiences.
52:01 Speaker 02 And then another one is Russia. I mean, Russia, again, has sort of its own interesting culture where They have more of a capitalist spin, if you will, on their infrastructure, right? In China, you kind of have two or three major ISPs. They tend to have heavy involvement from the government. And so deploying that censorship infrastructure is very much a state activity. But when you look at Russia, the censorship is basically the state saying to all of these sort of independent ISPs, hey, look, you're going to block this list of sites. We don't care how you do it, figure it out, right? And so Russia has its own way of determining, or you see these weird characteristics where one ISP will look different from another,
52:45 Speaker 02 and they seem to be doing almost this bare minimum approach, right? Where it's very easy to get around and it's very patchwork. So yeah, the censorship landscape is fascinating, how much of a shift you see between all of these different countries.
52:57 Speaker 01 Yeah, for sure. And I guess just looking ahead, if you want to call it that, any predictions where things will go? I mean, we've touched on like a lot of, the whole idea here is that things just seem to keep changing and there might not be any rhyme or reason, at least from the outside, as to why. But yeah, just if you were to kind of think ahead, anything that you suspect might change or happen?
53:21 Speaker 02 It's hard to say, it is a dynamic landscape. But I do believe that censorship is going to become a little easier for the censors and it's going to become harder to circumvent. We're seeing, especially in China, this shift of middle boxes closer and closer to the end user, where due to sort of a scarcity of IP addresses, they're implementing these NATs to where a local residential address won't have a dedicated IP, right? And so as we sort of see more and more middle box infrastructure getting closer and closer to that end user, it will become easier for them to deploy more of a distributed censorship network that can kind of implement more advanced techniques because it's taking on less load.
54:04 Speaker 01 So just as we sort of start to wrap up here, what about your own research? Are you doing more into this kind of going forwards? Or is this, are you kind of leaving this here and moving on to like a different part of the censorship landscape? Or yeah, where are you taking things from here?
54:20 Speaker 02 Yeah, so I've been looking extensively into that leaked information I referred to. That's a relatively new finding for us. And so we've been looking at that to sort of be that ground truth that I said doesn't exist, right? Confirm some of these statements that we have made in all of the research we've done so far, as well as trying to understand better what role machine learning can play in this space, right? So a lot of my research is focused around that network tap here at CU and having access to this massive corpus of internet traffic that, you know, is real and finding ways that we can better classify that traffic or even generate synthetic traffic is sort of the direction I'm headed in next.
55:02 Speaker 01 Awesome. Well, that all sounds fascinating. I hope maybe we get to catch up in a couple of years, hear what you've been up to since then. And just want to really thank you for your time, Jackson. And like, I've learned a ton today. I'm sure the audience has as well. So obviously you're doing great work. It's very important that this information is understood, at least outside of China. So yeah, thanks so much.
55:25 Speaker 02 Absolutely. Thank you so much for having me.
Previous Optimizing Agent Behavior in Production with Gideon Mendels Next Engineering AI Systems for Autonomy and Resilience with Krishna Sai Page updated November 6, 2026 · transcript and summary are machine-generated