Get the latest on global AI regulations, legal risk, and safety-by-design strategies. Read the Report
Protect your AI applications and agents from attacks, fakes, unauthorized access, and malicious data inputs.
Control your GenAI applications and agents and assure their alignment with their business purpose.
Proactively test GenAI models, agents, and applications before attackers or users do
The only real-time multi-language multimodality technology to ensure your brand safety and alignment with your GenAI applications.
Ensure your app is compliant with changing regulations around the world across industries.
Proactively identify vulnerabilities through red teaming to produce safe, secure, and reliable models.
Detect and prevent malicious prompts, misuse, and data leaks to ensure your conversational AI remains safe, compliant, and trustworthy.
Protect critical AI-powered applications from adversarial attacks, unauthorized access, and model exploitation across environments.
Provide enterprise-wide AI security and governance, enabling teams to innovate safely while meeting internal risk standards.
Safeguard user-facing AI products by blocking harmful content, preserving brand reputation, and maintaining policy compliance.
Secure autonomous agents against malicious instructions, data exfiltration, and regulatory violations across industries.
Ensure hosted AI services are protected from emerging threats, maintaining secure, reliable, and trusted deployments.
Get the State of Trust & Safety report 2024
New report explores the commercialization of disinformation, child predator networks,ย and violent terrorist groups
NEW YORK, February 15, 2023 โ ActiveFence, whose mission is to protect online platforms and their users from malicious behavior and content, today released its State of Trust & Safety 2023 report, which reveals new patterns in threat actor behaviors, tactics, and techniques. The report contains an analytically comprehensive evaluation of the top threats contributing to online harm โย from disinformation campaigns and child predator networks, to human exploitation operations and online terrorist organizations โ with the goal of reducing the prevalence of these dangerous activities to improve the digital safety of billions of global users.
As the internet continues to become an increasingly powerful and populated arena for business and society, so do the unique and sensitive issues that arise from unrestricted user generated content.ย
โEmpowering platforms to identify and remove harmful and misleading content is one of todayโs most complex, nuanced challenges,โ said Noam Schwartz, CEO and co-founder of ActiveFence. โI expect the 2023 threat landscape will be marked by increased professionalization, emboldened threat actors, and the blurring of lines between the digital and physical worlds. And with new regulations like the Digital Services Act (DSA), online platforms need to act swiftly to understand how to prevent and remediate these issues, and continue providing safe experiences for their users.โ
The commercialization of disinformation The Covid-19 pandemic and following years have ushered in a trend of โmass misinformation,โ whereby threat actors evolved their tactics in order to evade platform detection, and the volume of targets has increased. Political misinformation rose 150% between the fourth quarter of 2021 and the fourth quarter of 2022. In 2022, threat actors replaced bots with seemingly authentic accounts to spread disinformation, as we saw during the U.S. 2022 midterm elections. Disinformation is increasingly hosted on bespoke websites on mainstream platforms, making the content more difficult to detect and more easily shareable. We saw this play out during Russiaโs invasion of Ukraine.
ActiveFence observed an operation that distributed false and misleading anti-Muslim conspiracy theories ahead of the Uttar Pradesh-based state legislative assembly in India in March 2022. The operation exploited instant messaging and social media, spanning across four social platforms and impacting 600,000 followers with 100 posts per day. ActiveFence was able to issue a warning to its partners four and a half days before mainstream media reported on the activity, preventing dangerous narratives that could have caused social unrest.
More recently, ActiveFence spent more than seven months monitoring online activity in Brazil leading up to and following the presidential election. They alerted partners about the potential for political and societal disruptions over the contested outcome, including the protests leading to the violent storming of federal buildings that led to more than 1,200 arrests and the false narratives disseminated by Bolsonaro supporters that the riot was carried out by agents from the Workerโs Party.ย ย
Child exploitation online Online predators became bolder in 2022, especially as children are joining social media at a younger age. This has resulted in a rise in sextortion crimes, of which the FBI received 7,000 reported cases, resulting in over a dozen teen suicides. Additionally, this past year has seen an increase in minorsโ emulation of the adult performer creator economy, wherein minors are producing their own child sexual abuse material (CSAM).
ActiveFence discovered a CSAM network that shared links and ran advertisements to monetize child predator traffic, with just one of the entities sharing 120 videos to 197,000 followers, garnering 13 million views. ActiveFence notified the abused platforms of the new account activity, which forced the nefarious network to shift promotional strategies until it became too difficult to abuse the platform, ultimately closing their monetized CSAM offering.
Online terror promotion ActiveFence detected an extensive ISIS-supportive terrorist promotional network of 465 audio and video files reaching 418 publications in 11 languages. The network published videos showing the brutal executions of captured prisoners and audio of violence-inciting sermons. The threat actors evaded on-platform moderation by operating backup entities, but ActiveFence was able to locate these backup accounts and warned the abused platforms so they could remove the network, ultimately deterring network regeneration.
In observing ISIS supporters, ActiveFence noted their deployment of guerilla social media tactics in 2022 to exploit global events, such as Moroccoโs World Cup matches in Qatar. Supporters tagged snippets of ISIS propaganda using trending hashtags to reach diverse audiences quickly. ISIS also continued to focus on establishing a strong presence in Africa last year, as ActiveFence detected a 700% increase in ISIS propaganda productions in the first quarter of 2022, specifically recruiting in West Africa.
To read the full report, click here.
To learn more about how ActiveFence safeguards online platforms and users against online harm, please visit our website at www.activefence.com.
About ActiveFence: ActiveFence is the leading Trust and Safety provider for online platforms, protecting over three billion users daily from malicious behavior and content. Trust and Safety teams of all sizes rely on ActiveFence to keep their users safe from the widest spectrum of online harms, including child abuse, disinformation, hate speech, terror, fraud, and more. We offer a full stack of capabilities with our deep intelligence research, AI-driven harmful content detection and moderation platform.ย ActiveFence protects platforms globally, in over 100 languages, letting people interact and thrive safely online.
AI red teaming is the new discipline every product team needs. Learn how to uncover vulnerabilities, embed safety into workflows, and build resilient AI systems.
Discover how emotional support chatbots enable eating disorders and overdose risks, and what AI teams can do to safeguard users.
Align AI safety policies with the OWASP Top Ten to prevent misuse, secure data, and protect your systems from emerging LLM threats.