By Kathy MeĂźmer, Martin Degeling, Anna Semenova, Alexander Hohlfeld, Greta Hess.

TikTok under investigation. Image generated by Dall-E

TikTok has been all over the news—and not just since the Digital Services Act (DSA) came into force. Rabbit holes, polarization, mental health issues, disinformation—the list of potential harms stemming from the use of TikTok is long and well-known from other platforms. But when we started to look into TikTok, there was one more observation that startled us: how much promotional content is on TikTok’s For You feed.

Like all online platforms that offer service free of charge, TikTok makes money from advertising. But TikTok is especially associated with the rise of shopping platforms like Temu and Shein as well as numerous other shops that sell low-quality products—including TikTok Shop, although it is not yet available in the European Union (EU).

This could threaten high consumer protection standards, a core value of the EU (Art. 38 of the EU Charter of Fundamental Rights) and something that is explicitly mentioned in the first article of the DSA. However, shops and the products sold in them tend to fall short of consumer protection standards in the EU. TikTok itself is also under scrutiny by consumer protection authorities for several non-transparent practices.

Consumer protection is a highly political and important topic, yet it is often neglected when we talk about systemic risks and platform regulation under the DSA. This neglect is, at the very least, concerning. After all, online shopping is a setting where young adults directly feel the effects of political developments and processes in their everyday lives, which might influence their trust in politics.

This is why we decided to focus on consumer protection in our TikTok audit. In the process, we learned a lot about TikTok’s inner workings, why the ad definition of the DSA (and other regulations) falls short, and how users actually use TikTok (findings that could also be helpful and relevant when it comes to other systemic risks). What we learned is interesting but also much more complex and ambiguous than we initially thought. But let’s start with a quick recap of what we did.

Risk-scenario-based audit approach

In February 2023, we published a framework for “Auditing Recommender Systems“ with reference to the DSA. But we didn’t just want to theorize; we also wanted to walk the talk and test our approach by auditing TikTok (which is designated as a Very Large Online Platform (VLOP) under the DSA). If you want to learn something about our methodological reflections, the decisions we made, and the insights we had, you can read about them in our blog posts.

To make a long story of complex methodological reflections short: In our research, we aimed to target the recommender system of TikTok’s For You feed. Therefore, we needed the perspectives of actual users and an understanding of the app’s behavior when running. That’s why we partnered with two organizations (pollytix and AIForensics) and ultimately opted for a mixed-method approach combining a representative survey with a semi-automated audit.

At the core of our methodology are scenarios that describe harms for individuals who use an element of a platform as well as the high-level risk that comes with it. For our study, we focused on the following scenario:

Young TikTok users in Germany between 18 and 25 years old end up overspending or with hidden costs after being exposed to (malicious and/or misleading) ads when swiping through the For You feed and this will lead to a low trust in consumer protection.

We’ve spent the last three months analyzing and discussing the data we’ve collected. Here are our main results:

Nearly all young adults use social media – TikTok is in third place when it comes to engagement several times a day

In order to achieve the most meaningful results possible, we had to make a number of decisions and narrow down the scope of our research. TikTok’s target group is young people, and we assumed that all TikTok users have experienced consumer protection-related issues on TikTok at some point. That is why we decided to focus on young adults between 18 and 25. We excluded minors from this sample, as they partially fall under different legal frameworks, have restricted accessibility (e.g., credit cards), and experience different harms. To be able to (language- and context-wise) understand the content these users might see, we further specified the affected user group as those located in Germany.

Here is what we learned about their usage behavior:

  • 99.5% of young adults between 18 and 25 in Germany use social media, 91.5 % of them intensively, which means they use at least one platform multiple times a day. On average, they use four different social media platforms, of which YouTube is the most used (92%), followed by Instagram (87%) and Snapchat (62%).
  • TikTok is ranked fourth with 59% usage overall, but it rises to third place in terms of engagement several times a day. 15% of young adults stated that they no longer use TikTok but have in the past. The most frequent reason for deleting the app was spending too much time on it.
  • Overall, young adults do not see TikTok as a trustworthy platform, with 70% considering it “rather untrustworthy” or “not trustworthy at all.”
  • TikTok is predominantly used for entertainment by 93% of young adults. 72% use it very or rather often to get inspired, and 63% use it to discover current trends. This high engagement, particularly for inspiration and trend-spotting, makes users especially susceptible to advertising on TikTok.
  • The most used feature on TikTok is the For You feed, with 89% of young TikTok users saying they use it often or very often.

If you want to learn more about the TikTok usage behaviour of young adults continue reading here.

Users estimate that every second post on TikTok is an ad

The majority of young adults on TikTok use the For You feed frequently, but just how much of that is actually advertising and might, therefore, pose a risk? When it comes to the platform involvement in our risk scenario, two findings from the survey are particularly interesting:

  • On average, users estimate that every second post on TikTok is an advertisement or a product recommendation.
  • Additionally, 62% of users agree that the line between advertising and entertainment is blurry.
  • This aligns with our scraping data:
    • Up to 20% of videos on TikTok are paid promotion, which are clearly discernible as an advertisement on TikTok.
    • In addition, there are numerous other promotional videos often classified as “organic marketing.” They reach an audience through the viral effects of the platform rather than relying on paid promotion. Brands like Temu and Shein heavily rely on this marketing through nano-influencers and discount codes, leveraging the authenticity of these influencers for the benefit of the brand. How many of these ads users see depends on what they are interested in.
    • To analyze this commercial and promotional content, outside of traditional ads, we looked at cues like the presence of shop links in creator bios and whether someone had previously posted a video labeled as “promotional.” The results showed that, on top of the 20% paid promotions, around 19% of the videos shown to our users are from accounts trying to sell or promote something - without paying advertising fees to TikTok. In total, we can therefore identify an average of 39% of the videos in users’ feeds as promotional.
  • These advertising strategies are effective:
    • 35% of TikTok users have purchased a product after seeing it on TikTok (42% of women and 29% of men).
    • Moreover, repetition plays a significant role: 50% say, that they bought something because they saw it in so many TikTok videos that they wanted it for themselves. This is driven by the nano-influencer strategy as well as by paid promotion: In our semi-automated tests, 43 of the 50 most-watched videos were paid promotion.

TikTok users often see ads that seem dubious, questionable, or even dangerous

Just because TikTok is an ad-driven platform, it does not mean that those ads are problematic, does it? That’s why we also asked about potentially “malicious” ads and learned that:

  • 58% of users say they often receive ads on TikTok that seem dubious or questionable.
  • 41% say they have seen ads or product recommendations on TikTok that they consider dangerous.
  • This is confirmed by our scraping data: 37% of the sites linked in ads we saw have poor or bad ratings on the review site Trustpilot.
  • With regard to paid promotion, we also used TikTok’s data to confirm that malicious ads exist: About 3% of ads on TikTok are deleted by the platform, but mostly only after they have reached the relevant target audience. The most common reasons are that the videos or accounts are banned due to suspicious behavior or that advertising for certain products and services (such as gambling) is not allowed in the respective country.

Two-thirds of young users who bought something via TikTok have already had bad experiences

But does this risk also lead to concrete harms in the target group? Yes, …but!

  • A majority of TikTok shoppers (71%) expressed satisfaction with their purchases. However, half of them (37%) also reported having at least one negative experience as well. Overall, a significant portion of TikTok shoppers (62%) had at least one negative experience.
    • Chief among these negative experiences was product or shop dissatisfaction, with over one-third (35%) receiving lower quality items than expected and a quarter struggling with returns or exchanges. Financial issues also plagued users, with over a quarter (28%) overspending due to unclear pricing and 16% falling into debt from their shopping activities.
    • Using the TikTok ad library, we found that nearly 7,500 ads were removed between September and April for “exaggerated financial claims,” totaling 50 million impressions before removal. One such ad, linked to a campaign with over 281,000 impressions in six months, was removed for this reason.
  • Men seem disproportionately affected by negative financial outcomes from TikTok shopping, with 44% reporting adverse effects compared to 29% of women, a statistically significant difference.
    • The precise cause of this gender gap is challenging to identify due to data limitations, such as variations in self-assessment of financial strain. However, an examination of TikTok’s ad library reveals only minimal overlap in advertisers targeting specific genders, with top advertisers and target audience interests differing significantly.
    • Ads for men predominantly focus on “games,” while those for women emphasize “beauty & personal care.” These findings suggest considerable divergence in ad content across genders, but further investigation is needed to establish causal links with gender disparities in financial impact.
  • Furthermore, our survey data indicate that users who self-report being susceptible to impulsive purchasing due to advertising or product recommendations also tend to report a higher incidence of negative financial consequences compared to less (self-reportedly) susceptible users (51% vs. 19%). This correlation suggests that initiatives aimed at empowering individuals to recognize and counteract the influence of advertising on their purchasing behavior could lead to a reduction in negative financial outcomes.

If you want to learn more about the harms, continue reading here.

YoungTikTok users perceive a lack of consumer protection

The trust in consumer protection on TikTok is quite low.

  • While most users are quite aware of the strategies that help them verify the trustworthiness of shops, interestingly, young users rely more on private providers like Trustpilot (82%) than on consumer organizations (76%) like the “Verbraucherzentralen” in Germany or BEUC on the European level.
  • Importantly, higher consumer literacy among our respondents correlates significantly with greater purchase satisfaction: Those with better literacy levels report fewer negative experiences. In the lower consumer literacy group, 33% had solely negative experiences, 40% both positive and negative, and 28% solely positive experiences. In contrast, among the higher literate, these numbers were 25%, 33%, and 42%, respectively.
  • Moreover, 45% state that Trustpilot and similar platforms are the most helpful measure, although Trustpilot is far from being independent (both frustrated users and companies that pay for fake reviews can easily influence the ratings there).
  • 65% believe that consumers are not adequately protected from negative shopping experiences on TikTok. This is underscored by the finding reported above, that many ads are only removed after they have already reached their audiences. This rate is even higher for those that have stopped using TikTok (n=242), here 76% think TikTok is not doing enough.
  • Young users have higher expectations of the platform and also see a responsibility to act.
    • 84% believe that TikTok, as a platform, should ensure that no harmful products are advertised and sold.
    • 83% think that consumers should be actively warned of dangers when buying via TikTok.
    • 73% think that TikTok should ensure that poor-quality products are not advertised.
  • Concerningly, two-thirds of the surveyed participants think it’s the individual’s own fault for having a bad experience. This perspective on users’ personal responsibility, paired with the recognition of harmful/low-quality products on the website, emphasizes the importance of consumer literacy programs.

What TikTok and enforcement agencies could do

As part of our research, we also want to outline a few ideas about how to improve the situation.

  1. 🪟 Transparency: TikTok is minimally DSA compliant with its advertisement library, but there is much more a multi-billion dollar operation could do to protect users and enforce rules:
    • Ensure clear and understandable labeling of promotional content, with user-friendly explanations about what users are seeing, why it is being shown, and who is paying for it.
    • Enforce existing rules more strictly. For instance, if a video is deemed inappropriate for advertising after a content moderation review, it should not remain available on the creator’s page without proper labeling.
    • Increase transparency regarding consumer protection by informing users when content they have interacted with is deleted.
  2. 🛡️ Protection and Enforcement:
    • TikTok already uses a system with various “warnings” shown to users when they leave the site (e.g., via a bio link). TikTok could be more aggressive in blocking certain links, such as those to shops if the user is not registered as a commercial entity, or by including third-party ratings of shops or other indicators of the trustworthiness of the linked site.
    • Implement more proactive labeling of users, videos, and comments where commercial interests are involved. This could be achieved through the automatic detection of products or promo codes from which creators and commentators earn money.
  3. đź“– Consumer Literacy
    • TikTok should invest in consumer literacy programs.
    • TikTok should actively involve users, for instance, by soliciting their feedback through ratings after they have made purchases.
  4. đź’Ľ Regulation: The proliferation of drop shipping and similar programs, enabling creators to establish businesses with minimal effort, increasingly blurs the distinction between content creation and advertising.
    • Despite the European Commission’s recent “influencer sweep,” which revealed that only a small fraction adhere to regulations, there has been no significant crackdown on these practices, allowing the unregulated market to persist.
    • Furthermore, the definitions of “advertising” in regulations like the DSA miss the reality of TikTok and should be adapted.

Summary

Our research aimed to understand if and to what extent TikTok’s For You feed poses systemic risks to the high standard of consumer protection that is enshrined in the EU Charter of Fundamental Rights. We focused on young adults in Germany, given TikTok’s popularity among this demographic. Our findings shed light on usage patterns:

  • Almost all young adults in Germany use social media, with TikTok ranking third in terms of engagement several times a day.
  • Despite its widespread usage, overall trust in TikTok is low, and it is predominantly used for entertainment and inspiration.

While the public discourse during European elections focuses on various actors using TikTok as a platform to spread political disinformation, our research indicates that most of the content is primarily concerned with consumerism, product placement, and the advertising of various goods and services to users. This leads to concrete harms in the target groups, showing that TikTok’s For You feed is a threat to consumer protection:

  • TikTok is being flooded with promotional content, which is turning the platform into an endless infomercial personalized to users’ interests.
  • Many users report receiving ads they perceive as dubious or even dangerous—a fact we can confirm with our scraping data.
  • Many users report negative experiences with TikTok-related shopping, including risks of financial harm.
  • While consumer literacy among users is notable, trust in consumer protection is low, with users relying more on private platforms like Trustpilot than official consumer protection services.
  • Despite this, young TikTok users express dissatisfaction with the lack of consumer protection measures on the platform and support for stricter safeguards against harmful products and advertisements. Ultimately, our findings left us quite ambivalent. On one hand, comparisons with data on online shopping in general show that TikTok as a platform does involve slightly higher risks for users, falling within the average range for problematic experiences. At the same time, the widespread use of fast fashion and drop shipping offerings on TikTok presents new consumer protection risks whose long term impacts we might not be able to assess yet.

We therefore conclude that TikTok poses a “normal” risk to consumer protection, but without mitigation the foreseeable negative impact is high.

Finally, we would like to highlight three interesting aspects of our results:

  1. It is intriguing how critically young adults view TikTok. Strikingly, despite their critical stance, users extensively use TikTok and also fall for problematic advertising content. While good consumer literacy has a positive influence, it cannot entirely prevent fraud. In addition, users—as we learned in the focus groups—act against their better judgment. This aspect can likely be applied to other areas, such as the spread of dis- and misinformation. Education and media literacy training are not the solution to all social media-related problems.
  2. The current regulation of advertising content on social media completely misses the reality of TikTok. This is a problem related to how “advertisement” is defined, for example, in the DSA. While the definitions of advertising have evolved continuously with the rise of social media, marketing strategies on TikTok, like those of Temu and Co, have introduced entirely new forms of advertising (e.g., through nano-influencers), which are currently not well defined.
  3. This form of advertising and consumption likely leads users to feel that consumer protection is inadequate. There seems to be extremely high dissatisfaction with TikTok’s business practices as well as a lack of trust in consumer protection institutions. The fact that young adults rely more on private providers like Trustpilot for their consumer literacy strategies rather than on official offerings from independent (and also financially independent) organizations highlights a significant gap.

Especially given the loss of trust among young people in politics, offerings that meet the real needs of young people, particularly in consumer protection, could be a good way to make politics tangible, that is, something that is oriented toward the interests of young people and from which they directly benefit.