Podcast: China's hidden AI economy
A conversation with Zilan Qian at the Oxford China Policy Lab
Watch or listen to the High Capacity podcast on:
How do users in China get access to Claude and ChatGPT through the gray market? What’s really driving China’s apparent “AI optimism”? How is China trying to deal with deepfakes and AI-generated content?
In this episode, I speak with Zilan Qian, a researcher at the Oxford China Policy Lab about the hidden side of China’s AI boom, including:
How Chinese users use “transfer stations” to access American AI models like Claude and ChatGPT
Why Chinese users still want to use Claude and ChatGPT even though there are strong Chinese models
Why China’s apparent AI optimism is really driven by anxiety about falling behind
China’s push to label AI-generated content—and the problems with watermarks, detection, and false positives
Links:
Zilan’s Oxford page
Zilan on Substack and Twitter / X
Transcript
Kyle Chan (00:00)
Welcome to the High Capacity Podcast. I’m your host, Kyle Chan, a fellow at Brookings. I’m thrilled to be joined today by my guest, Zilan Qian, a researcher at the Oxford China Policy Lab and an incredibly insightful analyst of China’s AI ecosystem. Welcome, Zilan and thanks for coming on the show.
Zilan Qian (00:18)
Thanks for having me.
Kyle Chan (00:19)
So I wanted to dive first into the topic of transfer stations. These are part of a gray market economy for accessing American AI models from China. So I was wondering, what are these transfer stations that you’ve written so much about? How do they work? And yeah, can you walk me through how a user in China might get access to, say, Anthropic’s Claude models through this sort of underground economy?
Zilan Qian (00:50)
Yeah, so I think the Chinese term for transfer station is zhongzhuanzhan (中转站) and it’s used vaguely to refer something as an AI API gateway. So essentially, you can imagine it as a middleman sits between the users and frontier models like Claude or ChatGPT. So instead of paying the model providers, you pay the transfer station, and instead of sending your request directly to Frontier AI models, you go through this transfer station, and the transfer station routes your request to the model, and then the model’s response comes back through the transfer station. It’s more like setting up an OpenRouter in China.
Kyle Chan (01:36)
Hm.
Zilan Qian (01:36)
The question is really that all these frontier U.S. AI models are not operating in China. So having the kind of transfer station in China is essentially about accessing frontier U.S. models as well as accessing them at a very cheap token price.
Kyle Chan (01:56)
Yeah, so as it stands right now, ChatGPT is blocked in China, but then Anthropic blocks Claude or something like that, right?
Zilan Qian (02:05)
Yeah, I think there’s clear news from China, I think in 2023, that the government blocks ChatGPT explicitly. So even now when you’re looking at the censorship on different platforms, for example, Taobao, the e-commerce platform in China, you can find that the keyword ChatGPT and anything related was censored the most heavily. Whereas simply putting in Gemini or Claude seems to be more fine. So it’s an interesting dynamic. But because China has this AI registry system that all the kind of public-facing models serving the public need to be filed within the Chinese government system. So obviously Anthropic, Google, their models are not filed through the system. So technically they also cannot serve the Chinese users. But of course from all these frontier U.S. AI labs, they have various policies to geoblock Chinese users.
Kyle Chan (03:08)
Right. So specifically that means that Anthropic will try to identify users of Claude, and if they identify that user is based in China, they will restrict access.
Zilan Qian (03:21)
Yeah.
Kyle Chan (03:21)
Okay. So then if I am a user in China, and it seems like Claude is still very popular, How do I get access? Where do I find these transfer stations? And, can I just do a search online? Are there like marketplaces for finding these services? And what does that all look like?
Zilan Qian (03:42)
Yeah, so I think interestingly to just use Claude in China, you don’t necessarily need to go through the transfer-station process. I think some people are using a VPN to access Claude accounts, especially if they are just using AI as a chatbot. I think transfer stations are more for when you can use a variety of U.S. frontier models and you really care about the price. So with the rise of AI agents it seems that transfer stations are getting popular because token usage is surging as well as with all the harnesses, it seems the underlying infrastructure can be changed more flexibly. So I think all these are the reasons that transfer stations are becoming more and more popular.
Kyle Chan (04:31)
So let’s say I do want to get cheaper access to Claude. how do I find these transfer stations? where do I go online to look them up and compare prices?
Zilan Qian (04:43)
Yeah, so I think it’s just surprisingly easy to actually get all this information. my entry point was Taobao because I was searching how to buy Claude and Gemini models on Taobao. And there’s sellers that sell direct accounts to you, but there’s also the ones providing transfer-station service and then you talk to the sellers and then they will point you to specific platforms. There’s also different discussions on various open forums in China about comparing the quality of different transfer stations because it can vary a lot based on stability and then the prices they offer and even on GitHub you can find the kind of public leaderboards of how they are ranking through community rankings of which transfer station is more reliable. And it’s somehow fascinating that Anthropic usually block Chinese users in waves. So one day you will wake up and find that a lot of accounts get blocked and sometimes Chinese users tweet it on Twitter And somehow like you can also find under those tweets there will be like transfer-station sellers on Twitter. It’s like, hey my transfer station is still there, So use mine.
Kyle Chan (06:04)
Yeah.
Zilan Qian (06:05)
I think it’s just like it’s very interesting how this kind of information gets passed around across the internet.
Kyle Chan (06:12)
Yeah. So there’s sort of this whack-a-mole dynamic where, when Anthropic tries to block a bunch of accounts, new ones spring up and they’re quickly, sort of in the replies, being like, You can switch over to this one.
Zilan Qian (06:26)
Yeah.
Kyle Chan (06:26)
It’s this endless battle, I guess. Well, how do the mechanics of these transfer stations work? from reading your work, it seems like many of them kind of bundle access and are they based in China, these transfer station services, or are they based outside of China or do they connect to people or other organizations outside of China?
Zilan Qian (06:50)
I think you can think of the supply chain as three layers. So on the upper layer you have upstream, which tries to get access to these kind of frontier models. So there you will have the people who have for example foreign billing addresses and foreign phone numbers. And they can get access to those frontier models through various means. So I think those services sometimes they are based abroad. And then in the middle layer you’re aggregating these accounts and routing them to different users. And that part is like it’s like somewhat technical, but there’s various open-source projects on GitHub that you can set up fairly easily to do all these kinds of things. And then downstream you have users based in China who really want to use all these models in the US. So that’s how the supply chain forms. And also, in the middle, there’s various techniques you can use to try to lower the price of the token. I think it’s when I say you can sometimes buy it at 90% off compared to the official price, people seem to be very surprised.
Kyle Chan (08:04)
Yeah.
Zilan Qian (08:05)
So there’s various things you can do. For example, at the minimum you can exploit a kind of account discount. For example, back in those days I think a lot of the frontier labs used to offer enterprise and educational discounts. So people who are not using those accounts can sell them to other users. You can also pool expensive subscriptions across many users, which means that sometimes 10-ish users will share one Claude Max account. Because actually when you’re trying to maximize in the tokens, the two hundred dollars can be split into various means of usage. And they call it pinche (拼车), which is carpooling. There’s also ways of exploiting the regional price difference. It was rumored that Bolivia’s Claude account is much cheaper to buy if you’re using Google Pay. And I also found several public leaderboards showing different prices of Claude accounts in different regions. So it’s very interesting. And there’s other, less savory ways to do that. For example, you use stolen credit cards. There used to also be a mechanism because Anthropic used to refund all the money back to the account when they blocked an account. So some people will bulk-register the account, and then They use it for several days and then they intentionally input some sensitive information so Anthropic would block them and issue a refund. So that’s level one of
Kyle Chan (09:43)
Wow.
Zilan Qian (09:46)
It’s level one of exploiting these existing features. And then level two is like getting grayer is that they call it chanshui (掺水), which is swapping the models. So instead of, say, serving Opus 4.8 as stated on the surface. They will intentionally swap it to Opus 4.6. some people do it in a very aggressive way that they will sometimes swap, for example, DeepSeek for Claude or Doubao for GPT and it’s like sometimes it’s very easy
Kyle Chan (10:18)
Ha.
Zilan Qian (10:21)
To tell. So users also complain on a lot of the public forum. It’s, “My model suddenly has very low intelligence.” It’s the jiangzhi (降智) moment.
Kyle Chan (10:32)
Ha.
Zilan Qian (10:33)
Yeah. So and then the kind of level three, which is like the worst part, is selling users’ data, or sometimes it can connect it to distillation. Because a lot of the kind of users are engineers and AI researchers, so their prompts and responses are very valuable and can be used as teachers for smaller models. So if transfer stations can actually clean all the data, because when they’re routing like the users’ requests and the models’ responses all the time, so they can actually have the access to all these data. And so if they can clean them and put all these together, it’s a very valuable resource and can be sold on the black market. And there are even public transfer-station platforms explicitly saying that some of the models they serve are very cheap, but they explicitly say that this service will be used for distillation. And also I think just like A lot of the logs may contain private information that the users try to put there. So it can also be used to blackmail people. plus there are some recent cases where transfer stations seem to be engaged in prompt injection into the user’s agentic workflow in order to get some sensitive information on their local computer. So I think the level three is the part that’s like the grayest in the whole economy.
Kyle Chan (12:07)
Wow. So there are data or even cybersecurity risks potentially that you are
Zilan Qian (12:12)
Yeah.
Kyle Chan (12:13)
Exposing yourself to potentially when you get so that’s why it’s super important in terms of getting the quality right, actually getting the model that you think you’re paying for, and then, like knowing that your data is not going to be used, or maybe if it is, that you are at least aware of that, like all these variables and then You have to kind of figure that out and then there’s all these discussion forums trying to give feedback and share information.
Zilan Qian (12:40)
Help people, yeah.
Kyle Chan (12:42)
Yeah. But it’s like constantly changing, right?
Zilan Qian (12:45)
Yeah, it’s constantly changing and I think it requires a lot of trust. So sometimes you will see like messages of friends recommending to friends about using this transfer station instead of that. And then I think reputation is really important in the whole economy. And specifically on the vulnerability And data privacy side, we even see like Chinese government getting very concerned about the transfer station and the operating model. There’s several cases of some people operating transfer stations being summoned by the government. And then we also see certain kind of government-linked WeChat account publishing articles warning users about the risks from transfer stations.
Kyle Chan (13:31)
So this is like on the government’s radar. It’s not
Zilan Qian (13:33)
Yeah.
Kyle Chan (13:34)
Just something where they turn the other way and just let this happen. Yeah,
Zilan Qian (13:38)
Mm.
Kyle Chan (13:38)
Yeah. Yeah. It’s it’s so fascinating how, like I don’t know how else to put this, But how creative all these strategies
Zilan Qian (13:47)
Yeah.
Kyle Chan (13:47)
Are. I mean, it’s like any user of a subscription in the U.S. will always think about, yeah, I pay, this monthly fee for Claude or even for Netflix, but do I get the full value of it? Maybe not, and sometimes people share subscriptions, right? That’s very common anywhere in the world. But to then turn that into a full-blown industry is like
Zilan Qian (14:08)
Ha.
Kyle Chan (14:09)
Another level entirely.
Zilan Qian (14:11)
Yeah, no, I think you can even see like earlier trends even before AI. existing for example on Taobao, there’s just like so many kind of reselling of Duolingo account and then Netflix account, YouTube
Kyle Chan (14:24)
Hmm. yeah.
Zilan Qian (14:25)
Account. So all these are kind of signs that like in China I think because of excess and because people are unwilling to pay a lot for certain kind of services, so there are various creative ways people come up to navigate the whole situation.
Kyle Chan (14:44)
Right. Yeah. So even before AI there was this ongoing gray-market economy.
Zilan Qian (14:51)
Yeah.
Kyle Chan (14:51)
I like the way that you point out that, it’s like on sites like Taobao, which is like a very popular e-commerce, marketplace, basically. It’s like going to Amazon and finding these. So it’s super out in the open in terms of users finding these services, but then the actual supply-chain mechanics it’s like harder to figure out.
Zilan Qian (15:14)
Yeah, definitely.
Kyle Chan (15:16)
How about efforts by, say, Anthropic to put like know-your-customer (KYC) restrictions? Doesn’t that make it harder then for users in China to get around that? how are they going to prove that they are not in China when they need to show that they have a non Chinese profile?
Zilan Qian (15:40)
Yeah, so I think one possibility is that then the whole transfer-station gray economy could get linked to the kind of traditional black market of facial recognition. There were jokes circulating around like the Chinese internet space as well as the Chinese-language Twitter space that you will just go to African or like Southeast Asian region and pay like a small fee, in exchange for foreigners scanning their faces and showing their passports. So it’s like doable and it’s like there’s previous case different from like AI and the whole access control. There’s the different place that prove Cases that prove that this black market exists. So you can just imagine how this kind of black market will connect with Transfer Station and put them together. So far we haven’t found solid evidence that this is exactly happening. I think there’s also some guidance on how not to trigger the kind of KYC verification. So it’s like a cat-and-mouse game that when you’re trying to put KYC in place or even when you’re trying to early days like request specifically foreign billing addresses and foreign phone numbers, people will find a way to get around it if they find your service really valuable. So it’s really a kind of a benefit-cost question is like is it worth it for you to spend so much effort? And if the model is really good then Of course you want to do whatever to get access to it.
Kyle Chan (17:21)
Right. Yeah. And I guess the sort of bigger picture here is that there is strong demand in China for access to American models, especially for Anthropic’s Claude models, it seems. And that’s despite the fact that you have strong Chinese models, there is still a thriving marketplace, it seems like, for access to American models.
Zilan Qian (17:43)
Yeah, I think it’s very interesting because I got asked a lot of why Chinese users want to use Anthropic models even though they have so many domestic alternatives. I think it’s like back to the kind of debate about whether good enough is good enough. Now Chinese models are good enough for a lot of tasks. But then with the several months capability gap, there are various things that you want to use For example, Fable for that for maybe Minimax or DeepSeek cannot fulfill for now. So that might be one reason that people want to use the frontier U.S. model. There’s also another reason that there’s a lot of individual developers in China trying to sell their products overseas. So sometimes they want to use the American models to integrate into their products. yeah, so these are the two main reasons. But also transfer station itself, the whole economy is now in a somewhat shaky moment in that like on one side you have all these like U.S. frontier companies mostly Anthropic trying to block the accounts. on the other hand you also have the Chinese government being increasingly aware of the vulnerability within this economy. And you have maybe Chinese models become more and more capable at a lot of tasks. And they are much cheaper than U.S. models. So whether this kind of two sources of external pressure And the shrinking demand can still sustain the economy is a really open question and let’s see. Yeah.
Kyle Chan (19:23)
Yeah, that is fascinating. Yeah. So yeah, we’ll see where this ends up going, especially pressure from both sides and the growing alternatives from China. Well, now I want to switch over to WAIC, the World AI Conference in Shanghai, where you recently attended as part of a longer trip in China and just wanted to get, some of your impressions from this. This was a very high-profile event. Xi Jinping gave a major keynote speech there about global AI governance. And there are many different industry participants and researchers and I believe a lot of foreigners were participating in talking about things like AI safety. So what were your impressions this year?
Zilan Qian (20:11)
Great. Now I’m going to give a ten-minute speech about I just returned from China from my seven-day trip and we are cooked.
Kyle Chan (20:20)
Ha.
Zilan Qian (20:22)
Joke aside, I think my overall impression, ‘cause it’s also my first World AI Conference, it’s chaotic. There’s just like so many things happen at the same time. I think my general sense is It seems not as optimistic as I expected. ‘Cause I expected, “AI is great, let’s go all-in on AI, AI+, everything”. well like AI plus everything does happen and then the AI is great also happens. But when people are actually discussing things you don’t feel like it’s super, super like energetic And one example is that at an AI for Science forum, The question of, “So now AI is getting more and more capable in scientific discovery, what’s the role of human scientists?” gets asked. And then at another high-profile AI research fireside chat, there’s also a conversation about how AI is now being used more and more by university students, and it seems it’s also eating into the peer review system. So how how will universities function when like AI is really having an impact in academic research? And back to the kind of will AI replace scientists, I think people seem to be like arrive in the framing of, AI can’t do A and B now. So we are kind of okay. And it might be able to do A and B in the next five or ten years. We are not sure where human scientists will be when that moment comes. So it seems that people are worried about all these job-loss and societal impact stuff, but they seem to have no answer. And also think like AI safety becomes a more salient topic. That people are more worried about for example, cyber risk from AI. That’s my general impression. Yeah.
Kyle Chan (22:24)
Yeah. Well, I want to tie this to a really amazing piece that you wrote about more generally Chinese attitudes towards AI and why this framing that we hear a lot about how Chinese people are more optimistic about AI and they’re more hopeful that it can be a positive force in the world and in their lives versus Americans who seem to be increasingly pessimistic and especially younger Gen Z, young college graduates who are, worried about finding a job. And then with the data-center backlash in the U.S., people seem to be mixed in their views of AI.
Zilan Qian (23:09)
Ha.
Kyle Chan (23:09)
So there’s that contrast, but your piece and your writing argues for a much more complicated view of how China sees AI. And I was wondering if you could talk more about, what’s really behind this AI optimism in China.
Zilan Qian (23:27)
Yeah, I think it’s like the motivation for writing that piece is back in February twenty-six. This year that we see a kind of a boom in the adoption of OpenClaw in China and there are thousands of people lining up at like Tencent’s Shenzhen headquarters and asking engineers to install OpenClaw on their phone. And when those incidents get picked up by foreign media, like a lot of the time people’s reaction is, look, China’s so optimistic about AI. All these people rushing to use AI agents. my gosh. And it seems to be a very convenient kind of framing to say that on one hand China’s optimistic, on the other hand U.S. is pessimistic. I think it misses a lot of the actual details behind what might happen. So the central argument of that piece is that fear and optimism can look the same on the surface. So I did a historical deep dive into the 1990s state-owned enterprise layoffs where because of reform and opening up and the whole economy switching toward a market economy They laid off a lot of the state-owned enterprise workers overnight. And then people not only lost their jobs, but also lost a lot of the social connection they built around the danwei (单位) they used to work for. However, on the other hand, because of this kind of switching to market economy, you also have other regions that really benefit from this kind of transformation. And people can become rich very shortly. You do business with foreigners, etc. So it kind of creates a moment where big social transformation comes and then a group of people benefits from it and a group of people become the losers. And when you’re really looking at the individual level, like anthropological research at that time like shows that People don’t really blame their whole society or the transformation. They blame themselves: w I didn’t really kind of seize the opportunity because there are various things you can do. For example, if you’re laid off, maybe you can go to the south and do business, or maybe you can go overseas and work and accumulate your capital and go back and start your business here. So there’s various like successful examples, nearby So when you are actually losing everything during this transformation, like the question you ask yourself is so seems to be like my own fault that I didn’t catch this kind of great moment and seize the opportunity. So I borrowed a phrase from the anthropologist Xiang Biao as the “last bus mentality” that One opportunity comes it’s like a bus and this might be the last bus. If you get on the bus then it drives you to a brighter future and if you don’t get on it then you’re kind of abandoned by everyone and just like be left there miserably. So I think this kind of mindset, this kind of anxiety is really present in every stage of China’s like development. For example, in the 2000s there’s the kind of English fever because of globalization and then people saying that English is extremely important and then you have Fengkuang Yingyu (疯狂英语, “Crazy English”), where crazy English where people will stand outside and yell out English words in order to memorize them. And then you have internet age and all the boom. And then for high school grads, you need to sign up for Computer science program in a university, and you have massive expansion of computer science program at the university level. And even in 2023, I believe, it’s like the early ChatGPT moment, there was cases where certain influencers trying to sell very low-quality AI lessons to the public, but then tens of thousands of people would just buy it because they really want to learn AI and learn what ChatGPT is about. And then fast forward earlier this year you have the OpenClaw moment. So across all these incidents, are people embracing AI? Yes, but what drives them seems to be the anxiety of fearing being abandoned by the transformation. So it’s really not, “Great, I love AI, AI is the best thing in the world. It’s really if I don’t learn AI, maybe I will lose my job. If I don’t learn AI, maybe I cannot even navigate a whole society. Just think about how China now becomes such a cashless society. Everything is so digital. So if you cannot use a phone, if you cannot just like navigate a whole app system, how are you able to get around especially in the kind of modern cities. So Yeah, that’s the main overview of what I’ve been writing.
Kyle Chan (28:49)
Yeah, I love that piece. I love the comparison with the crazy English movement and this just sort of open-arms embrace of English as not just a way to boost your career chances, but as something where if you don’t do this, you could fall behind. like diaodui (掉队) like this idea of kind of falling
Zilan Qian (29:11)
Yeah, definitely.
Kyle Chan (29:12)
Behind, falling out of line, and then this term the last bus. Like even just describing it makes me anxious because it’s the last chance to be part of this wave and if you’re not part of it, you’re going to be left behind structurally.
Zilan Qian (29:30)
I think the whole idea that if you don’t catch it, you will fall behind is also very present in the kind of whole education system or even the whole like discourse describing a normal Chinese person’s life. It’s if you don’t get to a good primary school you cannot get into a good middle school and then if you don’t get into a good middle school, you cannot get into a good high school. If you don’t get married at this age, you can’t have kids. And if you don’t have kids, you’ll be very miserable when you get old. It’s like at every stage of people’s life there’s always if you don’t do this, it will have such a bad consequence. So at almost every stage people have a very clear goal that like I need to rush to do this.
Kyle Chan (30:21)
Right, yeah. And the anthropologist who had coined this phrase the last bus mentality, Xiang Biao is the famous originator repurposer of the term involution, neijuan (内卷).
Zilan Qian (30:37)
Yeah.
Kyle Chan (30:38)
So this is very much connected to this intense competition, this feeling that there’s no space, you need to be kind of jostling with sharp elbows
Zilan Qian (30:47)
Mm.
Kyle Chan (30:47)
And trying to run faster and faster just to keep up, much less to get ahead. So that kind of anxiety I think is captured really well by your writing and his writing, and applies really well to the current AI moment. And this is also happening to some degree in the US, right, with the discussion of tokenmaxxing
Zilan Qian (31:11)
Mm-
Kyle Chan (31:11)
Which I guess now is already kind of getting passé where business leaders and, business school students would all be saying, “How many times can we say AI in one earnings call? Because if we don’t show that we’re AI-forward enough, then investors or our own competitors will think that we’re falling behind. And so there’s a bit of anxiety as well in the US, but maybe not quite as widespread and society wide as it is in China.
Zilan Qian (31:42)
Can definitely see like the same thing happen in the China side where some companies have this kind of AI KPI that you need to use AI in what percentage of your work and people even describe that there’s some sort of local AI arms race happening within a company where you’re racing against your colleagues trying to see like who adopts AI faster and more widely in your workflow. And even some tasks that can be done normally or just use some very basic AI integration, you need to come up with fancier ways of using AI in your workflow. You need to write new skills every day, etc. So I think a lot of this is job-loss anxiety and then the kind of tokenmaxxing, the kind of anxiety and uncertainty about AI is really shared across borders.
Kyle Chan (32:41)
Yeah. Well, when you were just recently in China, how did people talk about AI? away from the industry folks, how do normal Chinese people talk about AI, and how do they use AI in their daily lives? Was this something where you were seeing people constantly pulling out their phone to ask Doubao, the AI chatbot, a question? Or did you see, say, ads or billboards for AI classes and AI tutoring? How did you encounter AI when you were there?
Zilan Qian (33:21)
Yeah, so I think it really depends on cities. So I took a little detour to Shenyang before I went back to my home, which is Hangzhou. And I’ve also been to Shanghai and Beijing. I think Shenyang compared to these three cities, you just feel much, much less AI there. The only place I saw AI was like there’s a billboard about AI consulting service, integrating AI into your business, but it’s a very poorly designed one that I think AI can come up with something better. But then in Hangzhou it’s like once you land at the airport there are giant kind of Alibaba Cloud like advertisements of we are like helping Chinese companies to help your AI service go overseas etc. So I think these kind of cities are more AI-pilled in the sense of that. And I think It’s in general, like, especially f from like people who like my for example my parents and relatives like AI is more integrated into their everyday life in the sense that last time I went back, some of my older relatives are listening to AI-generated podcasts and I’m like, “It’s AI-generated.” They’re like, Yeah, we know, but it works. And this time everyone just like scrolling all this like AI-generated microdramas on their phones.
Kyle Chan (34:55)
Yeah.
Zilan Qian (34:56)
I’m like, “This is clearly AI-generated, do you enjoy it?. They’re like, “It just kills time,” and it’s like seems to be like more spicy than like human actors. So yeah, fine. And I think there are other, more subtle ways that like AI can enter your daily life. For example, AI is already sort of integrated into WeChat and Alipay. So if you’re already using all these services, then it seems more natural that like you will just maybe for some question you want to Google something or search on Baidu, you’ll ask the AI that’s integrated in Alipay or like WeChat. Yeah, that’s my general observation.
Kyle Chan (35:36)
Yeah. I find that the AI capabilities are embedded everywhere in the apps and platforms. And then even if you’re trying to find a coffee shop through Gaode Maps or something, it’s telling you all about the history of coffee or giving you too much
Zilan Qian (35:55)
Ha
Kyle Chan (35:56)
Like too much information. And I’m just like I just wanna find the nearest place to get a coffee ‘cause I need some caffeine to start my day. yeah.
Zilan Qian (36:03)
Yeah, absolutely.
Kyle Chan (36:05)
Ha.
Zilan Qian (36:06)
I somehow feel people’s habits seem to be like changing relatively easily in China. earlier this year when I went back, Alibaba, like Alipay was experimenting with its Tap! (碰一下) NFC payment, where you unlock your phone And enter Alipay and then touch your phone to their specific device and then you pay it. China leapfrogged the whole like credit card payment So they didn’t have that stage. And at that time they were kind of giving like red-packet money. So every payment you will have certain kind of discount and then people are at the stage of learning that kind of payment method. And then a few months later, when I went back, everyone very naturally pulled out their phone And it’s like, “Payment complete”. So I think there are various interesting ways to let people use AI. And it’s like similar to during the Spring Festival, there’s red pocket wars where several AI providers are giving people red pockets. to encourage them to use their AI services. So I think it’s just very interesting how like consumer behavior changes.
Kyle Chan (37:22)
Yeah, absolutely. I feel like sometimes from the U.S. point of view, there’s so much focus on what Beijing and the policymakers are doing top-down, and that’s driving a lot of this. But I’m always struck by how Chinese consumer attitudes are I don’t know how to describe it—much more open to trying things, especially when they’re not really they’re really beta versions, like they’re full of glitches or whatever, but I feel Chinese digital users are more willing to try things out and seem to be faster at that. And I’ll give a contrast. I was just recently in Germany and you
Zilan Qian (38:02)
Ha.
Kyle Chan (38:03)
I like grumble every time to anyone who will listen to me about how I have to wait in line at the grocery store as each person is paying with coins. counting out their euro coins and I’m just thinking, my God. I mean there is Apple Pay and, digital payment stuff in Germany as well. But, versus like China like ten, fifteen years or, at least ten years ago was already switched over to QR code payment. So any anyways, there’s variation obviously.
Zilan Qian (38:34)
I was talking to some of my friends about this phenomenon because we live in the UK and then it seems like something just like never changed compared to China. And it seems like we are talking about why people’s attitudes shift so much because sometimes when we mention things People in the UK sometimes say it’s like this the kind of current setup is fine, why do we need to change? Or, “We like to do things in the old way and we were reflecting that growing up, like all these like our neighborhoods changed drastically And the way that we go for transportation and payment changed drastically. It seems that we grew up amid massive change, so we’re used to the environment around us changing rapidly so that we are more comfortable with things changing and that we are just like more willing to adopt new things. But also related to the previous point that maybe we are also having the kind of anxiety of if we don’t use this then society will abandon us.
Kyle Chan (39:38)
Yeah. And also your point in that piece that the change is inevitable. So you either embrace it or you get left behind. Versus I do think that, part of the difference in the U.S. attitudes towards AI and especially data centers is that people feel they can push back and at least at the local community level, if you don’t want a data center, you can actually fight back and win. And block a data center from being built. Maybe there’s a sense in China that a lot of these bigger trends are sort of out of our control. And, whether it’s the economy or technology or something even bigger than that. And so might as well make the best of fluctuating time, I guess.
Zilan Qian (40:25)
Yeah, I do feel that. And I think the whole especially on technological determinism is very present even at the very high level discourse. Technology is the first driver of productivity. And I think people just draw a lot of lessons from like the Qing dynasty and how the government did back then refused to adapt to new technology and then lost the wars, ended up very miserably and it seems to serve as a constant lesson to the state and every Chinese person that you need to really embrace the technology, otherwise you will end up like the Qing dynasty, the Qing government.
Kyle Chan (41:09)
Yeah, definitely. So at the national level, at the individual level, at the company level, this phenomenon keeps playing out again and again. I want to get to another piece you recently wrote about AI labeling. And there’s a lot of discussion about AI-generated deepfakes, AI-generated content. Substack recently integrated, Pangram to help
Zilan Qian (41:35)
Yeah.
Kyle Chan (41:36)
Identify AI-generated content. There’s a strong stigma in the U.S. towards AI-generated content. maybe even if the text or the image is, fine, once we find out that it’s AI-generated, we’re kind of, ooh, I like I’d would prefer the human version instead. And then recently also the EU. has now implemented a set of rules around requiring watermarks that are visible to users, but also a digital signature or digital trace that is machine-readable as well. And there are already changes where Anthropic and OpenAI are starting to change some of their algorithms to deal with this. But what’s happening in China? Because it seems China was already concerned earlier about deepfakes or moving faster on some of the regulation. And some of this, as you’ve written about, some of the policies got started even before ChatGPT, made its debut in twenty two, I believe. What’s China trying to do with these AI label regulations?
Zilan Qian (42:42)
Yeah, I think the overarching concern is probably the same in China and the EU: making the AI content traceable, and then making everyone who handles it liable for keeping the trace intact. The earliest related regulation can be traced back to twenty nineteen where in There are network audio-video provisions that require prominent labeling of non-authentic audio and video generated by at that time it’s not AI, But deep learning and virtual reality technology. And then throughout like from twenty nineteen to twenty five, various regulations touch on this point. In twenty one there’s the algorithmic recommendation provisions, which asks platforms to add a label. before transmitting unlabeled content. And the twenty two Deep Synthesis Regulation expands it to you need to include text, image, audio, videos, and virtual scenes, I believe. It also introduced a two-tier labeling architecture where you need not only an explicit label but also an implicit label. And in twenty-five there were very detailed labeling measures. And coupled with national standards on AI, that really specifies what kind of labels are required, and how to format your metadata embedded in your content.
Kyle Chan (44:12)
Yeah, so there’s a whole raft of sort of evolving regulations or layered regulations around this space. Do you think a lot of this is about concerns—political-content concerns about deepfakes? Or do you think it’s sort of a more general sort of wanting to tie responsibility back to the creators? Or do you think it’s, more about users and, making sure that users at least have some awareness of what they are consuming? Whether it’s a short-form video or it’s some text. What do you think is kind of the underlying motivation behind all these measures?
Zilan Qian (44:49)
I think various motivations can drive this kind of regulation. There’s probably some kind of social stability consideration. There can also be like concern about AI misinformation, fake news affecting things. Can also be like AI-generated like scamming, like harming users’ interests or like people using AI-generated content to manipulate the public. So I think it’s like not that different, from a lot of the concerns outside China about AI-generated content.
Kyle Chan (45:24)
Yeah, and who’s responsible in terms of the labeling? is it the models, the companies behind the AI foundation models? Is it say, social media platforms where you might see AI-generated content? is it the creators who are publishing this stuff? Who’s responsible here?
Zilan Qian (45:46)
I think one interesting thing is that in the whole system everyone is responsible for something. For example, for the AI model like providers, you need to make sure that your output has the kind of implicit label which is the metadata embedded in the content. For example Anthropic’s invisible text watermark or C2PA metadata. And then you also need to have an explicit label, like a watermark that can be identified with the naked eye. if your output is easily confused with real humans. At the platform level, you’re responsible for labeling three different kinds of content. The first one is that the content where you detect an implicit label—the metadata embedded in it. And also the things that the user declares but the content might not have either explicit or implicit label, you need to label that. And for content the user didn’t declare and it doesn’t have metadata embedded. you also need to label it if your AI detection tool thinks is suspiciously AI-generated. So this is the platform’s responsibility as a distributor. And for users it’s more when you’re uploading AI-generated content to the platform, you need to declare that, and when you’re disseminating it, you need to make sure that if it’s AI-generated, you state it that it’s AI-generated.
Kyle Chan (47:24)
And how effective do you think these regulations are overall? Are they being enforced? It seems quite difficult to track and enforce all these different instances. And then on top of that, what are some of the challenges? you’ve written about, for example, the problem of false negatives and false positives, where you get content that is AI-generated, but failing to have a label, but then also non-AI content from humans, but then it gets mistakenly labeled as AI. So What’s happening there?
Zilan Qian (48:00)
Yeah, so how do they enforce it? I think the most visible enforcement we’re seeing now is from the regulators, specifically the Cyberspace Administration of China (CAC) and Ministry of Public Security (MPS). They’re trying to have multiple rounds of crackdown campaigns, in order to identify the unlabeled AI-generated content and also all the fake news that cause financial damage. So those regulators are very active. So every few months you have these campaigns not only to issue direct punishment but also to send signals saying that we are super serious about these things. And so it can be a signal for AI model providers, the platforms, all the users. And on the other hand, some platforms have more incentive to do more self-regulating stuff. For example RedNote, which brands itself around human connection and community, is doing a lot on AI content detection. They put out AI-generated-content governance proposals. And also they are developing a lot of technology for AI detection. They also have the kind of two-tier review system where their AI detector tools go in and do the first round and then they have human reviewers doing things on top of that. So there are various things you can try to enforce it. In general you can have a better environment where some AI-generated content can be identified but of course there’s false-positive and false-negative rates. False negatives are very hard. People are generating things with AI all the time. So sometimes you just can’t catch everything. It’s super easy to remove all the labels that the AI providers added on the content. For explicit watermark that you can see with the naked eye, you can just crop it and then remove the icon. I’m not sure—maybe people have done this with Gemini or GPT image generating content. And also it’s super funny because a lot of the AI model providers on one hand they’re watermarking everything coming out from their models. On the other hand they’re also having AI-powered watermark removal tools for the users.
Kyle Chan (50:39)
Ha.
Zilan Qian (50:40)
So it’s pulling both ways. And also for the implicit watermark—the metadata, it’s easy to remove. You just take a screenshot and then all the metadata will just go away for images, and there are various open-source tools on GitHub that help you to remove all these metadata. And our beloved Taobao also offers many services of this kind of watermark removal. So I think it’s somewhat hard to detect the AI-generated stuff if you want to achieve 100 percent—almost impossible. And on false-negative and false-positive rates, it’s a very tricky thing, because specifically on RedNote, there are many Chinese illustrators putting out digital art and sometimes their art will be flagged as suspected AI-generated. So it’s pretty bad for them. We also see for text on RedNote there’s a lot of content where people may follow certain templates to write it, but then the templates look AI-like or the templates being used to train AI. So it’s flagged as AI-generated. these kinds of things happen all the time. And I think it’s not that different from what’s happening with Substack and Pangram and many people are also complaining about their text being wrongly flagged as AI-generated. Coupled with, for example, the negative sentiment of AI-generated output, especially in the creator space, I think this phenomenon is shared inside and outside China people stigmatize AI-generated content, especially around creators, because creators’ work has been used to train AI in the first place. So if your artwork gets wrongly flagged as AI-generated, it can be really bad for you personally and there was one incident where an artist in China had her work flagged as AI-generated and there was a huge debate. And she needed to go on the platform to livestream herself drawing in order to prove that she was a human drawing the whole thing. So I think it’s a very hard thing to do content provenance in general. And China is taking its first stab and even before the EU, so there’s a lot of lessons we can draw from it.
Kyle Chan (53:11)
Yeah. It’s a problem that a lot of societies, a lot of countries are grappling with. And yeah, myself in my own writing, I find myself leaning into quirkier or more idiosyncratic sentence structures or words because to me, I’m thinking, I want to signal almost that this is not written by AI. This is me, this is my voice. And I worry about being mistakenly labeled AI-generated, and then I’m thinking then it will seem kind of cheap and mass-produced. There’s also this, it’s a longer-term phenomenon or a longer-term question about technology. And it goes back to Walter Benjamin and the art in the age of mechanical reproduction and if you can just produce things perfectly, mass-produce things perfectly then what’s the value of the human element? And it seems like there’s a premium placed on human, authentic, genuine handcrafted things physically and now digitally. But there are workarounds too, like distressed jeans is one example where you can mass-produce jeans that look customized, distressed with tears.
Zilan Qian (54:26)
Interesting.
Kyle Chan (54:27)
Right. So maybe this is part of a longer battle between humans and machines and identifying which is which.
Zilan Qian (54:35)
Yeah, but I think there’s also the point that maybe the distinction between humans and machines has been blurred even before AI. That’s why your text is being flagged as AI because it’s following certain templates. And your art may be flagged as AI because it has this kind of predictable trend. So when we are learning about writing and how to do art, I think specifically in China learning to do art is not like I’m a free-flow creator and I’m just so talented and I go into the art industry. Like writing. It has a process almost like taking gaokao (高考). Instead you take yikao (艺考) which is the art exam and you’re practicing how to draw these strokes endlessly every day and then maybe fourteen hours a day in the kind of cram school. So the system almost turns artists into machines. So the output these humans produce can look like a machine because the whole process is kind of mechanizing like human beings. So I think
Kyle Chan (55:55)
Yeah.
Zilan Qian (55:55)
It’s also a question of how we train people to write, to make art, and even to be human?
Kyle Chan (56:04)
Yeah, wow, this is getting very
Zilan Qian (56:06)
Mm.
Kyle Chan (56:07)
Deep and philosophical. I love it. Yeah. Well, your research, your writing has just been so fantastic. I learned so much from everything you’ve put out there. I’ll be sure to include a link to your Oxford China Policy Lab profile and to all the pieces we’ve discussed in the show notes. Are there other ways that people can follow you in your work?
Zilan Qian (56:30)
Yeah, I think people can follow my Substack for more serious reflection. I also sometimes post and translate memes from China on my Substack. And people can also follow my Twitter for more hot takes.
Kyle Chan (56:44)
What’s your Substack and what’s your Twitter handle?
Zilan Qian (56:47)
It’s my full name spelled out, Zilan Qian, so I think people can find it easily.
Kyle Chan (56:53)
Sounds great. Well, I will link to all of that. Thank you so much, Zilan, for an amazing conversation.
Zilan Qian (57:00)
Thank you for having me.
Kyle Chan (57:02)
If you like this episode, please rate and subscribe on YouTube, Spotify, or Apple Podcasts. You can find episode transcripts and more information on the High Capacity newsletter at highcapacity.org. I’m your host, Kyle Chan. Thanks for joining and see you next time.