Improve your detection and simplify moderation - in one AI-powered platform.
Stay ahead of novel risks and bad actors with proactive, on-demand insights.
Proactively stop safety gaps to produce safe, reliable, and compliant models.
Deploy generative AI in a safe and scalable way with active safety guardrails.
Online abuse has countless forms. Understand the types of risks Trust & Safety teams must keep users safe from on-platform.
Protect your most vulnerable users with a comprehensive set of child safety tools and services.
Our out-of-the-box solutions support platform transparency and compliance.
Keep up with T&S laws, from the Online Safety Bill to the Online Safety Act.
Over 70 elections will take place in 2024: don't let your platform be abused to harm election integrity.
Protect your brand integrity before the damage is done.
From privacy risks, to credential theft and malware, the cyber threats to users are continuously evolving.
Stay ahead of industry news in our exclusive T&S community.
Welcome to the ActiveFence Expert Exchange, where we interview Trust and Safety professionals to share insights and solutions for today’s top online safety challenges.
We recently spoke with Mike Pappas, CEO and Co-founder of Modulate. Known for its work in the gaming industry, Modulate helps online social platforms effectively detect disruptive voice chat behavior in online games. They provide voice toxicity detection for companies like Activision, Rockstar Games, Rec Room, Riot Games, and more, focusing on distinguishing acceptable banter from genuinely harmful behavior that drives players away.
In this post, we’ll share key takeaways from our discussion, and offer practical tips and strategies to help platforms create safer, more trustworthy online communities.
Let’s dive in.
Lately, my top focus has been helping platforms think critically and carefully about the kind of community they want to cultivate. To use our detection technology effectively, you first need to decide, “What am I trying to detect, and why?” This decision ties back to the kind of experience you want to create for your users.
In some cases, you might allow reclaimed language or flirty banter, while in others, maintaining a strictly family-friendly environment might be necessary. There’s room for reasonable disagreement between platforms, and part of the beauty of the internet is offering diverse options. That’s why it’s crucial for platforms to define their content moderation goals clearly from the outset.
We’re passionate about shifting from simply banning specific behaviors to focusing on preventing harm. Many current tech solutions rely on banning behaviors outright, but that may not work in all situations.
Take flirting, for example—what some consider sexual harassment may be perceived as flirting by others. Since Modulate works with voice, we can distinguish between flirting and harassment based on how all parties respond and determine if harm occurred. Instead of prohibiting flirting entirely, we emphasize the importance of ensuring interactions are comfortable and non-harmful. We advocate for tools that recognize harm, educate users on its causes, and promote better behavior. This approach rethinks how technology can prevent harm effectively.
It’s already very hard to moderate user-generated content because it’s not in a standardized format. I think the biggest impact that AI will have on Trust and Safety is putting the final nail in the coffin of the “Let’s just hire a bunch of people and have them review each individual case” method. That’s just not going to work, even for relatively small platforms—there’s too much content to review.
I don’t think any of this AI will replace human content moderation. Each platform has its own definition of what behaviors are acceptable. That’s not something AI can define on a platform by platform basis. It needs to get that answer from humans who deeply understand the platform’s mission.
We are an AI company, but I honestly hesitate to describe Modulate that way in Trust and Safety conversations because it often leads to thoughts of LLMs, with their unpredictability and lack of explainability. Our system is designed from the ground up to ensure transparency and give customers control over decisions. This focus means we use different AI systems, avoiding tools like LLMs, and ensuring our technology aligns with our principles.
I’ve talked a lot publicly about how some platforms avoid necessary moderation out of fear of enforcement actions from regulations like GDPR and COPPA. Their focus should be on enabling user protections while assessing if compliance supports or hinders this goal. If it hinders, they should strive to stay compliant while aligning with best practices and engaging regulators to promote more thoughtful, effective policies.
A look at Modulate’s ToxMod Executive Dashboard.
I think it depends on the industry they’re in. For example, in gaming, the decision makers are constantly thinking about things like, “Could this possibly slow down the game by a tenth of a frame?” That means that more decisions are deferred to engineers, and engineers will always have a little bit of a preference to build over buy. So the conversation with game studios tends to be about how much working with a vendor frees up your engineers to focus their attention elsewhere.
Think about how young kids often experiment with boundaries, testing reactions to different words or behaviors with their parents. If they do this in online spaces, they may unknowingly repeat harmful things, especially if they receive social rewards from others. They must have the meaning and impact of their words explained to them as they’re simply unaware. So instead of starting with a ban, platforms can first provide this explanation. Studies show this approach works.
For example, Apex Legends saw an 80% reduction in repeated offenses after providing this context. Assuming naivety and offering basic education can lead to significant behavior change and a safer platform.
We leverage ToxMod’s ability to monitor communities and detect emerging behaviors and harms. For example, the phrase “your body, my choice” has recently surfaced as a new manifestation of a pre-existing harm. ToxMod flags such issues early, allowing us to analyze their prevalence across platforms. This helps us determine whether it’s a rare occurrence amplified by media or a widespread issue actually affecting communities. With our broad deployment, we now have the statistical insights to assess and address these risks effectively.
Last summer, we released a case study with Call of Duty showcasing how a proactive approach uncovered significantly more harms than relying on user reports. This approach led to safer experiences, boosting engagement and retention as players enjoyed the game more.
We’ve also studied the impact of immediate action versus delayed feedback on harmful behavior. “Immediate” meaning acting within a few minutes—not necessarily a few seconds. We’ve found that quick action yields a 30% greater reduction in repeated harm compared to even a 30-minute delay. Fast feedback is crucial for learning. Think about how quickly we gain driving skills from immediate feedback during real-time practice as compared to classroom work. The same principle applies to fostering kindness and compassion in communities.
–
In the evolving landscape of Trust and Safety, emphasizing the importance of proactive measures, nuanced approaches, and adopting predictable AI can lead to healthier online communities. Platforms must define their unique goals, educate users, and act swiftly to address harmful behaviors. By leveraging innovative tools and embracing real-time feedback, businesses can effectively detect risks, reduce harm, and encourage prosocial behavior. These strategies not only create safer spaces but also enhance user engagement and retention, proving that prioritizing Trust and Safety is both a moral and strategic imperative.
Mike Pappas is the CEO/Co-founder of Modulate, which works with online platforms like Call of Duty and Rec Room to foster safer and more inclusive online voice chat. Mike’s work at Modulate ranges from developing new partnerships within the industry, monitoring trends and new opportunities for Modulate’s unique technology to have a positive impact, and reinforcing an internal and external culture of passion, respect, and personal growth. Outside of work, his passions include creating themed cocktail lists, long-distance running, and playing Nintendo games.
Discover how to help your teams respond faster to harmful activities.
Explore how gaming has evolved over 40 years, balancing safety and player experience in the face of new content moderation challenges.
The State of Online Gaming in 2025 report explores the safety challenges and trends shaping the future of player protection.
ActiveFence investigates how fraudsters target, evaluate, and access gamers' accounts in order to sell their digital assets.