Your guide to selecting the best Trust and Safety tools Download the Buyer's Guide to Trust and Safety 2.0

hero

Real-World Gaming Experiences with Responsible Safety

ActiveFence helps this augmented reality leader ensure safe interactions. We support immersive gameplay and real-world connections, enabling confident exploration both online and offline.

Female gamer immersed in a virtual reality environment with multiple screens.
At a Glance Encouraging people to explore the world is what Niantic does best, by building real-world AR games and apps that connect over 100 million users worldwide. But with location-based games and AR technology, such as in Pokémon GO, comes a range of Trust & Safety issues that need to be tackled to ensure their players have a safe and enriching experience - both online and off. To better understand user safety needs, their Trust & Safety team partnered with ActiveFence. By understanding the risks and tackling them with actionable insights as early as possible, ActiveFence enables their Trust and Safety team to inform their internal partners on ways to build and sustain appropriate, engaging, and safe spaces for the wide diversity of users who play their games.

The Challenge

For location-based games like Niantic’s, come many unique Trust & Safety challenges. “Our highest priority risk is the likelihood of physical or emotional harm, because that's unique to the experience we’re creating,” says Jen Weedon, Head of Adversarial Planning and Safety by Design at Niantic, “So, for Trust & Safety, we need to think about how to prevent or respond quickly to those types of harm.”

With more social features - both offline and online - Niantic faced multiple scenarios that could lead to safety and player experience risks. These include toxicity in content-based interactions, undesirable player behavior during offline meetups, and risks to the physical safety of players as they walk outside while playing.

“Because this technology is so new and constantly evolving, the potential for harm is enormous, and we don’t fully understand it yet, which makes it very hard to detect”

Company Info

PROFILE
Niantic is a technology company known for developing augmented reality (AR) mobile games like Pokémon GO, blending real-world exploration with digital experiences.
INDUSTRY
Gaming

The Solution

Niantic’s Trust & Safety team needed to ensure that new in-game features would be evaluated to ensure they were engaging and safe for players upon launching. They turned to ActiveFence for its elite team of researchers, policy analysts, and circumvention experts, requesting deep threat intelligence and red teaming to proactively detect and prevent unknown risks and policy loopholes early on during the product development cycle.

“We needed to understand all the scenarios that could go wrong, and ensure that we could either prevent them by creating new policies, or understand the different safety needs for different demographics.”

For example, in the case of the family-friendly game Pokémon GO, Niantic’s tolerance policy for profane behavior is much lower than, for example, that for a first-person shooter game aimed at a more adult audience.

“When you’re developing an experience for families to enjoy together, there are different risk profiles you have to think about as well. So the notion of customization in risk assessments is really important.”

For example, ActiveFence utilized a combination of abuse-scenario mapping and red teaming to stress test features before launch. This includes first mapping potential risks ahead of product development so the team can take into account potential risks and build into the product before development. Then, ActiveFence then red teams and tests out those abuse scenarios through various player personas, testing across multiple geo-locations, and mimicking threat activities to identify potential harms. This uncovered risks associated with physical gameplay, in-app abuse and abuse scenarios, security and privacy concerns, and other vulnerabilities in product or moderation capabilities. Other scenarios were also tested from a non-threat actor perspective, to identify gameplay issues and potential dangers caused by normal gameplay.

ActiveFence provided investigations into threat actor TTPs, reports, and actionable insights on ways players could exploit relevant features to carry out abuse, or, in some cases, increase risk to their own safety. Jen then shared the insights with their product and development teams to review t these safety challenges Specifically, they made several changes to ensure family safety - for example, adding user controls to customize who can see a player’s real-life locations, and updating their codes of conduct for specific games. ActiveFence also enables Niantic to understand the cross-platform threat actor ecosystem. This helps them identify and stop bad actors who are misusing the platform for fraudulent activity and cheating.

The Impact

By leveraging ActiveFence's cross-platform intelligence, Niantic is able to consider safety by design principles for every new feature development. They partner with their game teams early in the design and development process, to ensure they are considering privacy and physical safety, and are prepared for the safer launch of any new feature. “In talking with developers, they expect a certain level of Trust & Safety support. We’ve been able to give helpful tips to consider so they don’t have to do any mental heavy lifting in building those features,” says Jen. “They can use these findings to help them design features, make changes, or at least be as prepared as possible.” Normalizing this process has led Niantic employees to think more about responsible development in general, by getting in early to reduce risks and create safer player experiences.

“If you let toxic or sketchy behavior flourish, it's hard to walk back from that. So if you can get folks thinking about these things early on, ideally it leads to better long-term outcomes.”

Jen Weeden Headshot

“If you let toxic or sketchy behavior flourish, it's hard to walk back from that. So if you can get folks thinking about these things early on, ideally it leads to better long-term outcomes.”

Jen Weeden

Head of Adversarial Planning, Red Teaming& Safety by Design

Discover how threat actors exploit online games and how to protect your platform.

Read the Report

Related Case Studies

Group of diverse young people standing together in front of a rainbow flag, symbolizing LGBTQ+ support and inclusivity.
CASE STUDY

The Trevor Project

Explore how The Trevor Project leverages ActiveFence’s tools to create a safe, supportive space for LGBTQ+ youth.

Read More
cohere_banner
CASE STUDY

Cohere

Learn how Cohere partnered with ActiveFence to enhance trust & safety across their platform.

Read More
Smiling couple learning online together on a laptop at home.
CASE STUDY

Udemy

Discover how Udemy uses ActiveFence’s solutions to safeguard learners and educators worldwide.

Read More