Late last year, the public got its first glimpse into one of the EU Digital Services Act’s (DSA) key governance tools. Nineteen designated Very Large Online Platforms and Search Engines (VLOPs and VLOSEs) published systemic risk assessment and audit reports focusing on platform risks and mitigation measures.
The DSA’s required risk assessments and audits create the possibility to advance groundbreaking platform transparency and accountability, but this first round falls far short of realizing that potential.
Late last year, the public got its first glimpse into one of the EU Digital Services Act’s (DSA) key governance tools. Nineteen designated Very Large Online Platforms and Search Engines (VLOPs and VLOSEs) published systemic risk assessment and audit reports focusing on platform risks and mitigation measures.
The DSA’s required risk assessments and audits create the possibility to advance groundbreaking platform transparency and accountability, but this first round falls far short of realizing that potential. To deliver on the DSA’s promise, risk assessments and audits must address two foundational gaps:
In November, platforms and independent auditing firms released thousands of pages of highly technical documentation assessing risks and compliance. Euractiv has a good overview (behind a paywall), and most platforms were assessed to be “partly compliant” with DSA obligations (only Wikipedia was fully compliant).
The assessments represent an important milestone in the democratic regulation of digital platforms. For the first time, platform companies are required to systematically and publicly describe how their products and services connect to a range of risks, and the steps platforms are taking to manage these risks effectively.
However, in the absence of formal guidance or requirements from the European Commission, there is little uniformity or standards for rigor in the assessments. A Delegated Regulation provides guidance for auditors but does not include standard definitions, methodologies, or datasets. Each platform and auditor has largely taken its own approach, and there is a noticeable lack of reporting on specific metrics and data.
A preliminary review of several of the assessments suggests foundational gaps in assessing platform design and the data, metrics, and methods used to evaluate risks and mitigations.
Article 34 of the DSA requires in-scope platforms to identify, analyze, and assess systemic risks to the Union stemming from the design or functioning of their service and systems. Systemic risks are defined as:
This includes the design of recommender and algorithmic systems, content moderation systems, privacy, and data-related practices, among others.
The choices platforms make in designing their products can significantly impact the risk of societal and individual harm. Numerous comprehensive studies draw a link between design and risk, including the US National Academies of Sciences, Engineering, and Medicine’s Consensus Report on Social Media and Adolescent Health, the US Surgeon General’s Advisory, the Australia eSafety Commissioner’s Safety by Design efforts, the OECD’s Digital Safety by Design for Children, and many others. Design risk is increasingly a focus of social media regulation, including in the DSA and the UK’s Age Appropriate Design Code. Risks associated with design are also a central focus of product liability and addictive design litigation in the US.
Working with the USC Neely Center and the Tech Justice Law Project, the Knight-Georgetown Institute (KGI) recently developed a taxonomy to map consumer harms to specific platform design elements. This taxonomy connects design choices to consumer risks, including:
The following sections explore how risk assessments and audits address how design connects to consumer harm and systemic risk.
The first round of risk assessment fails to adequately assess how platform design contributes to problematic and harmful social media use in two key ways: (1) several risk assessments do not assess risks related to design, problematic social media use, and impacts on physical and mental well-being, particularly for minors; (2) even when platforms assess physical and mental well-being risks, assessments do not adequately describe the methods and data used to understand whether mitigation strategies are actually reducing risk.
Independent research links excessive or problematic social media use with systemic risks, including physical and mental well-being impacts. Problematic social media use can disrupt everyday activities, such as attending school or sleep, and can be associated with negative mental health outcomes, including depression. Indeed, research shows that social media use contributes to a drop in time associated with sleep and exercise.
Problematic social media use is closely tied to platform design. Multiple platform design features may extend time spent on social media, including infinite scroll, autoplay, engagement gamification, ephemeral content, and the timing and clustering of notifications.
Multiple risk assessments fail to meaningfully consider risks related to problematic and harmful use and the design or functioning of their service and systems. Facebook’s 2024 risk assessment assesses physical and mental wellbeing in a crosscutting way, but does not meaningfully consider risks related to excessive use or addiction.
Other assessments more centrally consider physical and mental well-being risks. Instagram’s report assesses physical and mental wellbeing risks and refers to “time management tools” as mitigations. Snap’s risk assessment devotes seven pages to physical and mental well-being risks, but the assessment fails to consider how platform design could contribute to physical and mental well-being risks by incentivizing problematic or harmful use. Snap’s assessment is broadly focused on risks related to harmful content. The assessment describes mitigations to reduce the prevalence of such content that could impact physical and mental well-being – including auto-moderating for abusive content or ensuring recommender systems do not recommend violative content. This, of course, is important.
However, the risk assessment and review of mitigations place almost no emphasis on risks of excessive use actually driven by Snap’s design. Snap’s focus on ephemeral content is presented as only a benefit – “conversations on Snapchat delete by default to reflect real-life conversations.” This ignores research on potential risks of problematic use associated with the urgency ephemeral content creates. Snap also uses gamified design features like streaks. Research into streaks suggests that this tool can increase daily use and is associated with problematic use. But these design choices are not meaningfully assessed.
YouTube, on the other hand, assesses physical and mental well-being risks and the design and functioning of the service and systems. This is what is actually required under Article 34. YouTube’s assessment expressly considers the “risk that the interface, design, or features of YouTube stimulate behavioral addictions in children using the service.” YouTube’s audit investigates measures the company takes to mitigate this risk. The auditor “inspected the platform interface for … features that protect minors from addictive behaviors while using the platform and determined that features such as ‘take a break’ reminders, daily screen time and downtime limits, disabling autoplay and notifications being silenced from 10PM to 8AM are enforced for minors.”
This is an important review of YouTube’s policies to protect minors from addictive behaviors and associated physical and mental well-being risks. However, the assessment and audit do not describe whether YouTube’s mitigations actually manage the risk of behavioral addiction. The assessment does not state whether the company or auditor relied on any internal data assessing the impacts of these features. A range of metrics could be useful. Companies track time spent on the platform, for example, which could inform whether the YouTube features actually help reduce problematic use and by how much.
Many risk assessments also fail to assess how the design and functioning of platform systems may enable and perpetuate unwanted and harmful content. Unwanted and harmful content contributes to a range of systemic risks including illegal content, impacts on fundamental rights, and a person’s physical and mental well-being.
Platforms may have design elements that encourage users to develop and disseminate harmful content. For example, some platforms enable lenses and filters that let users alter images and their appearance. These may include features like touching up an image of a sunset or adding funny animal features to a human face. But lenses may also apply so-called “beautification” to a user’s photos. Emerging research explores connections between filters and risks around body image, dissatisfaction, and depression. Much of this research focuses on specific risks to women and girls.
TikTok and Instagram appear to agree that filters have risks for their users. Indeed, Meta announced that it will shut down its Spark’s platform of third-party filters in January 2025. TikTok announced plans to block filters for users who report their age as under 18. Yet the risks and mitigation features related to platform design, filters, and body image do not appear in either Instagram’s 2024 or TikTok’s 2023 risk assessment. Pinterest’s 2024 assessment, on the other hand, does note that the platform prevents beauty filters because of the risk such filters “can change the way teens think about themselves.”
Assessments do not effectively consider how design connects to unwanted or harmful contact. Assessments and audits describe platform policies to manage unwanted and harmful contact risks, including privacy defaults for minors. But they do not appear to measure how policies and designs actually impact this risk. Without prevalence data or other measures, it is impossible to assess whether companies are, in fact, taking appropriate action to mitigate these risks.
Platform design plays an essential role in creating and mitigating risks of unwanted and harmful contact. User privacy defaults, which restrict user profile or content visibility to others outside the user’s network by default, are a key tool for managing harmful and unwanted contact. Some platforms enable user visibility by default and recommend user accounts to others outside their network, as well as accounts outside a user’s network to them. These designs pose particular risks to minors by enabling bad actors to target and/or mass contact their accounts. Research has found that expansive default account visibility and account recommendation are crucial design vulnerabilities for sextortion targeting minors.
As one would expect, risk assessments reference platform privacy protections. TikTok’s assessment describes steps to ensure accounts belonging to users under 18 are not recommended to people aged 18+ (and vice versa). Snap’s risk assessment states that the location-sharing feature of Snap Map is off by default to help minimize risks to minors. Instagram’s assessment describes private defaults for users under 18 in the EU/UK (and for users under 16 in the rest of the world). Pinterest has private defaults for users under the age of 16, and Pinterest’s risk assessment further describes connection rate limits designed to reduce the likelihood of mass connection requests.
Company audits confirm the presence of these policies. But do the defaults described across assessments effectively mitigate risks? The assessments and audits do not provide sufficient data to answer this question.
Snap, for example, suggests that it manages risk related to harmful contact because “by default users need to accept bi-directional friend requests or already have each other in their contact book to start communicating.” Ernst & Young’s audit confirmed that Snap had “proportionate measures to ensure the privacy, safety, and security of minors who use their services.”
But a lawsuit brought in the US by the New Mexico Department of Justice alleges very different facts. Referencing internal Snap figures, the New Mexico-focused lawsuit asserts that:
Snap’s own research … is reported to show that one-third of teen girls and 30% of teen boys were exposed to unwanted contact on its platform. One year later, 51% of Gen Z respondents to Snap surveys indicated that they or a friend experienced catfishing; of those, half said it happened to them in the last 3 months, 44% had actually shared images or information, and one-quarter were sextorted. This far exceeded the incidence on other platforms.
Independent research similarly alleges that criminals specifically target minors in the US, Canada, United Kingdom, Australia, and Europe for sextortion on Snap because “its design features provide a false sense of security to the victim that their photos will disappear and not be screenshotted.”
The scope of the problem is potentially quite significant. The New Mexico complaint describes internal employee communication indicating that the 10,000 user reports of sextortion Snap received each month “likely represent a small fraction of this abuse.” A separate US-based lawsuit focused on Meta exposed an internal Instagram user survey with stark findings: over a seven-day period, nearly twenty percent of 13-15-year-olds reported receiving unwanted sexually explicit content, and 13% experienced an unwanted sexual advance.
Snap and Instagram’s assessments, of course, note that their policies prohibit sextortion and sexual exploitation. But Snap’s risk assessment does not sufficiently assess how its design–the centrality of ephemeral content, potential recommendation of adults to minors through the Quick Add feature, and location-enabled Snap Map, among others–intersect with risk. Instagram’s assessment describes the presence of supervision tools for minors, educational resources, and automated systems to detect and remove accounts. But how do Snap and Instagram measure the efficacy of these tools in Europe? Did Snap or Instagram discuss internal research relating to unwanted contact and sextortion with the auditor? Did Snap’s audit investigate trends related to levels of user reported sextortion when coming to the conclusion that Snap was taking “proportionate measures” to manage the risk? We need far more detailed information to understand how risk and mitigations are concretely measured.
The DSA’s risk assessment and audits are a foundation to advance platform transparency and accountability. But much work is needed to ensure these assessments deliver on their promise. To begin, risk assessment and audits must work to address two critical gaps described at the outset of this commentary: (1) meaningfully assessing platform design and (2) improving the data, metrics, and methods used to evaluate risks.
Platform Design
The failure to consistently consider design undermines many of the current assessments. Too many assessments focus primarily on content risk and diminish or ignore how platform design intersects with risk. Moving forward, risk assessments and audits should incorporate evaluation of how design can create, exacerbate, or mitigate systemic risk. This focus will enable platforms, the European Commission, and broader civil society to more effectively understand, prevent, and mitigate systemic risks over time.
Methods, Metrics, and Data
The first round of reports demonstrates an insufficient focus on data and metrics for evaluating risk and mitigations. There is an urgent need to incorporate consistent and reliable definitions and measures into the risk assessment and audit process.
There is much work already underway that the risk assessments can build on. Platforms already use, for example, a range of metrics to track specific content, behavior, and user experiences related to categories of harm. Platforms evaluate how product design quantifiably impacts specific risks. User experience surveys, like the Snap and Instagram efforts, are an important way to understand prevalence from the perspective of users. There is already work underway to develop metrics for longitudinal, cross-platform studies of negative experiences. While hindered by existing platform data access restrictions, independent research by academics and civil society is also an essential forum for developing effective metrics for understanding risk, as are the Commission’s own investigations into platforms for non-compliance.
Systemic risk assessment can and must improve over time. Incorporating a greater focus on platform design as well as data, metrics, and methods can help the DSA’s risk assessments deliver on their promise.
/
KGI recently submitted comments to the European Commission on its draft Delegated Act on vetted researcher data access, highlighting the connection between vetted access and public access to data, offering recommendations for specific additional categories of data and research that could be enabled under the Act, and underlying the need to understand systemic risks both across platforms and across borders.
/
KGI highlights three areas where emerging and established research can inform strategies to guarantee the privacy, safety, and security of minors online: Access to Platform Data, Platform Design Features, and Engagement-based Recommender Systems.
/
A diverse community gathered to mourn the loss of CrowdTangle, Meta’s transparency tool that the company killed on August 14, 2024. The funeral highlighted why this tool and others like it are so important.
