About header image

The Trust & Safety Glossary

A

Account Hijacking

The process by which a threat actor takes over a person’s account(s) in order to conduct unauthorized or illegal activities.

Accuracy

Refers to the degree to which automated or manual moderation tools make the correct decisions. Sometimes measured as an inverse of the false-positive rate. Read more

Adversarial Behavior

General term for unwanted on-platform behavior

Adware

Software that is maliciously installed to a user’s computer or mobile device, taking advantage of vulnerabilities to display unwanted advertisements. This type of software may also open the door for more malicious activity on the user’s device.

Alternative Social Networks

Social networks that were created as an alternative to mainstream social media.

Generally created to allow individuals to partake in activities that are banned on conventional platforms. As a result, these platforms are often used by threat actors to disperse malicious content and coordinate activities on mainstream networks.

Read more: The Online Battlefield

Anti-Money Laundering

The prevention, detection, and punishing of individuals who direct illegal funds into the financial system to fund terrorist or criminal networks with money that is then considered untraceable or legitimate.

Application Programming Interface (API)

A set of software codes that facilitate the interaction between software services, without transmitting information about how the other service works.

In Trust & Safety, APIs allow the transmission of data between platforms and vendors, allowing the incorporation of multiple sources of data to make moderation decisions and take actions.

At Scale

A reference to the volume of violative content and the flexibility of the system with the capacity to properly moderate the content.

Automated Detection

The use of automated technology, including AI, machine learning, NLP, and other mechanisms to detect violative content at scale.

While it allows for broad coverage of content, AI is limited by the learning sets it is provided, and so is often mostly English-language oriented, and lacks the ability to identify nuanced, or new violation types.

Read more: Read More

The use of automated technology, including AI, machine learning, NLP, and other mechanisms to detect violative content at scale.

While it allows for broad coverage of content, AI is limited by the learning sets it is provided, and so is often mostly English-language oriented, and lacks the ability to identify nuanced, or new violation types.

Read more: Content Detection Tools

Show less

B

Bad Actor

Motivated individuals, groups, or state actors that abuse technology platforms for harmful, violative activities that threaten the safety of users and the integrity of the platform.

Ban evasion

The act of bypassing platform moderation actions or circumventing a platform ban, often involves the creation of at least one additional account.

By creating new accounts, threat actors can return to a platform whose policies they have violated, in an effort to continue to do harm.

Banning

Permanently removing or blocking a user from the platform.

Read more: Policy Enforcement

Blocking

Preventing a user from posting on a platform.

Read more: Policy Enforcement

Blocklist

A list of unacceptable items, terms, or accounts not approved to exist on a platform.

Read more: Content Detection Tools

C

CP

Popular term among predator communities to denote “child pornography”

Child Safety

In Trust & Safety, refers to online risks to children, including exposure to harmful content, abuse, harassment, CSAM, and exploitation.

Read more: Policy Series: Child Safety

Child Sexual Abuse Material

Widely referred to as CSAM.

Images, text, or videos depicting the sexual abuse of minors (under 18 years old). For many platforms, this includes individuals who appear to be minors.

Christchurch Voluntary Principles

Established following the Christchurch, NZ mosque massacres, the Christchurch Voluntary Principles is a voluntary pledge by governments and companies, that are committed to eliminating terrorist and violent extremist content online. Read more

Clear Web

Websites that are publicly accessible to all audiences, through standard browsers and search engines. Estimated to only host 5% of online content.

Combating the Financing of Terrorism

A set of laws and regulations that require financial institutions and financial technology firms to assist law enforcement in their attempt to block terrorist entities from accessing funds.

Read more: Read More

A set of laws and regulations that require financial institutions and financial technology firms to assist law enforcement in their attempt to block terrorist entities from accessing funds.

Read more: Funding White Supremacy 

Show less

Comment Moderation

The process of detecting, flagging, removing, or denying abusive users the ability to post harmful comments.

Read more: Content Detection Tools

Community Guidelines

A set of policies and rules that allow an online platform to establish what is and is not acceptable content or behavior.

While not legally binding, these policies enable moderation teams to take action. Some pending legislation may make this mandatory in parts of the world.

Read more: ActiveFence Policy Guide

Community Moderation

Moderation activities that are conducted by community members, and not professional moderation teams.

Content Deletion

The most common moderation action: involves the deletion or removal of a piece of content.

Read more: Policy Enforcement

Content Moderation

The internal process of screening user-generated content posted to online platforms, in order to determine whether or not it violates policy, and take appropriate action against violative content.

Content Policy Manager

Team member involved in the dynamic process of creating and maintaining the community guidelines of an online platform.

Content Takedown Request

A legal request to take down content hosted and shared on an online platform, search engine, or internet service provider. Often used in the case of illegal content or copyright violations.

Cross-Platform Harassment

Harassment that spans across multiple platforms. Is enabled by the fact that most platforms only moderate based on on-platform activity.

Cyberbullying

The use of online platforms to bully individuals, generally refers to the abuse of children.

Cybercrime

Illegal online crimes or content that includes communications and actions regarding illegal online activities.

Cybersecurity

The protection of networks, devices and data as well as ensuring the confidentiality of information by denying unauthorized access.

Differs from online safety in that cybersecurity refers to the protection of devices, while online safety protects users and platforms.

Cyberstalking

The use of online platforms to abuse individuals generally refers to the abuse of adults (whereas cyberbullying refers to the abuse of children online).

D

Dark Web

The part of the online world that is only accessible via dedicated dark web browsers which use “onion routing” technology to allow completely anonymous browsing. While the dark web isn’t necessarily illegal or illicit, the anonymity it provides allows for these activities to take place.

Data Analyst

In a Trust & Safety team, the data analyst collects and analyzes data that contribute to the improvement of team processes, assist in measuring the effectiveness of decisions, moderators and policies, identify moderation and violation trend. This role is critical for teams working at scale.

Decision Tree

A tree-shaped model (or flow chart) of questions or decisions, where each outcome determines the next question or decision to be made, in order to come to an eventual outcome.

In Trust & Safety, a decision tree is often created to streamline and standardize the moderation decision process. Policy teams will create a decision tree for moderators to use when making a decision about a specific item o... Read More

A tree-shaped model (or flow chart) of questions or decisions, where each outcome determines the next question or decision to be made, in order to come to an eventual outcome.

In Trust & Safety, a decision tree is often created to streamline and standardize the moderation decision process. Policy teams will create a decision tree for moderators to use when making a decision about a specific item or account.

Show less

Deep Web

Online content that’s not indexed by standard search engines, and is therefore not easily accessible by most internet users. This includes personal mailboxes, content behind a paywall, or any other personal information.

The deep web differs from the dark web in that it does not require a specialized browser, nor does it imply anonymity.

Demonetization

Enforcement action, specifically on platforms that allow content monetization (through ads or otherwise), that blocks a user from earning income from their content.

Read more: Policy Enforcement

Digital Services Act

New legislative framework for digital services, proposed by the European Union in December 2020. This new law is aimed at making the online world safer, by establishing that activities that are illegal offline, should also be illegal online. The new law will require online platforms to take action against harmful and illegal content, specifically targeting hate speech, disinform... Read More

New legislative framework for digital services, proposed by the European Union in December 2020. This new law is aimed at making the online world safer, by establishing that activities that are illegal offline, should also be illegal online. The new law will require online platforms to take action against harmful and illegal content, specifically targeting hate speech, disinformation, and counterfeit products.

Show less

Disinformation

Intentionally misleading information that is shared and broadly distributed (disseminated) with the purpose of misleading or deceiving an audience. Often used as propaganda, disinformation has been widely used to sow public mistrust, influence elections, and legitimize wars. This is generally an organized, orchestra... Read More

Intentionally misleading information that is shared and broadly distributed (disseminated) with the purpose of misleading or deceiving an audience. Often used as propaganda, disinformation has been widely used to sow public mistrust, influence elections, and legitimize wars. This is generally an organized, orchestrated effort.

A distinction should be made between disinformation – the intentional dissemination of misleading content, and misinformation – non-intentional distribution of misleading content.

Read more on disinformation related to elections, health, warfare, and social radicalization.

Show less

Distributed Moderation

A type of content moderation where no individual person makes the moderation decision, rather community members (individual users) vote to determine if an item should or should not be allowed on the platform or forum.

Downranking

An enforcement method that allows a platform to reduce visibility of a user or their content. The content is ranked lower in search results or feeds, effectively reducing its visibility to other users.

Read more: Policy Enforcement

Doxxing

The online publication of private information such as an address or phone number, with the intent of harassing or inciting harm to the exposed individual(s). Read more

Duty of Care

In the UK’s drafted Online Safety Bill, online platforms have a Duty of Care to assess risks to their users, put policies and procedures to minimize that risk, and take actions to keep users safe.

Read more:  Read More

In the UK’s drafted Online Safety Bill, online platforms have a Duty of Care to assess risks to their users, put policies and procedures to minimize that risk, and take actions to keep users safe.

Read more:  The UK Online Safety Bill 

Show less

E

Enforcement

The broad range of actions taken by Trust & Safety teams when content violates policy.

Read more: Policy Enforcement

Error Rate

The rate at which items or moderation events are incorrectly identified. This is the inverse of accuracy.

Read more: Measuring Trust & Safety

F

False Negative

Violative content that was not flagged as being violative.

Read more: Measuring Trust & Safety

False Positive

Content that is incorrectly flagged as violative.

Read more: Measuring Trust & Safety

Feature Blocking

Enforcement action that involves blocking a user from accessing specific platform features – either those that have been misused in the past, or those with a high potential for misuse.

Read more: Policy Enforcement

Flagging

The act of notifying a moderation team of a potential policy violation. Can be conducted by either users or vendors, through manual or automated (AI-based) detection.

Read more: Content Detection Tools

Foreign Terrorist Organization

Foreign terrorist organizations are designated by the United States Department of State. United States law requires platforms to take action against content by foreign terrorist organizations.

Fraud

Defined as criminal deception intended to result in financial or personal gain.

In Trust & Safety, this may also refer to deceiving a user into providing their personally identifiable information (PII) or unknowingly providing access to their devices or accounts.

Fraudster

An individual who conducts fraud.

G

General Counsel

The full-time chief attorney for a company. This individual is often involved in the early stages of the establishment of a Trust & Safety team and is generally a key stakeholder in the creation of community guidelines and policies.

Grooming

The act of preparing or manipulating a minor into sexual victimization. Generally involves prolonged online communication, which may begin as non-sexual, and gradually escalates into sexually suggestive communication, before eventually leading to sexually offensive activities which may include physical contact.

Read more: Supporting Child Safety Read More

The act of preparing or manipulating a minor into sexual victimization. Generally involves prolonged online communication, which may begin as non-sexual, and gradually escalates into sexually suggestive communication, before eventually leading to sexually offensive activities which may include physical contact.

Read more: Supporting Child Safety

Show less

H

Harmful Content

Any text, image, audio, video, or other content posted online that is considered violative, malicious, deceptive, illegal, offensive, or slanderous.

Hash Database

In Trust & Safety, organizations such as NCMEC aggregate databases of image hashes that are related to various offenses (in this case, child safety). Platforms can then compare image hashes from their content to hashes of known malicious content (such as CSAM). This way, moderators do not have to view and analyze a potentially harmful piece of content and can compare its hash to that of recognized images.

Read more: Content Detection Tools

Hashing

Technology that creates a unique, fixed-length string of letters and numbers to represent a piece of data (often an image). The hash is non-reversible, meaning an image can’t be recreated from its hash.

Read more: Content Detection Tools

Hate Speech

Any expression or online content that incites, discriminates, justifies hatred, or promotes violence against an individual or group. Given distinct cultural and linguistic nuances, the detection of hate speech is often a complex task requiring regional and linguistic expertise.

Hiding

Platforms can temporarily hide content for a set amount of time, rather than immediately remove it.

Read more: Policy Enforcement

Human Intelligence

The collection of intelligence by means of interpersonal contact.

In Trust & Safety, specialized teams use human intelligence to infiltrate threat actor communities and proactively identify their means and methods.

Human Trafficking

Acts that involve the use of force, fraud or coercion to obtain forced labor or sexual acts. This may or may not involve cross-border transport.

I

Impersonation

Apps or websites that are intentionally created to resemble existing apps or services or appear to be a part of the user interface in order to gain access to personal data, passwords, bank accounts, etc.

Impersonation Of Individuals

The creation of fake accounts, often using a target’s name or photo, in order to cause harm to that individual.

Incident Response Analyst

Responsible for enforcing policy in time-sensitive, complex, or high severity violations. This individual may refer to an existing policy, or exercise discretion when conducting their investigations. If the policy doesn’t cover specific instances of their work, the incident response analyst may work with policy teams to amend the policy.

Intelligence

In trust & safety, intelligence is used to proactively alert platforms about impending risks, and to inform better moderation decisions.

Intelligence Desk

Internal or vendor teams that are responsible for on- and off-platform intelligence collecting in support of the Trust & Safety team’s efforts. Utilizing a broad range of tactics, including OSINT, WEBINT, and HUMINT, the team detects new threats and trends, identifies tactics, techniques, and procedures (TTPs), conducts investigations into suspicious account behaviors, “red teams” platform policies, and more.

Read more: Trust & Safety Intelligence

Investigations Specialist

Team member responsible for enforcement of policies created by policy teams. This individual investigates potential policy violations and determines whether or not a violation has been made, and then takes action against that user or content.

Read more: Building a Trust & Safety Team

K

Keyword Moderation

A form of content moderation that flags the instance of specific, potentially violative keywords used in text, audio, images, or videos posted on the platform. This type of moderation is limited in that it often lacks a contextual understanding of the keyword’s use, and requires constant updating of new violative keywords.

Read more: ... Read More

A form of content moderation that flags the instance of specific, potentially violative keywords used in text, audio, images, or videos posted on the platform. This type of moderation is limited in that it often lacks a contextual understanding of the keyword’s use, and requires constant updating of new violative keywords.

Read more: Content Detection Tools

Show less

Know Your Customer

A component of a financial institution’s anti-money laundering policy, Know Your Customer (KYC) is a requirement for financial institutions and certain financial technology firms to verify the identity of a client to prevent illegal access to funds (for example – funding of terrorist activities).

L

Labeling

Labeling involves applying a label to a particular piece of content. This is generally used when the content is not in violation of policy, but may negatively affect users. Frequently used in the case of potential mis/disinformation, or in violent/sexual content. Labels can be placed next to or below the content, to provide additional information, ... Read More

Labeling involves applying a label to a particular piece of content. This is generally used when the content is not in violation of policy, but may negatively affect users. Frequently used in the case of potential mis/disinformation, or in violent/sexual content. Labels can be placed next to or below the content, to provide additional information, or hide the content, requiring the user to consent to view the content.

Read more: Policy Enforcement

Show less

Law Enforcement Response Analyst

Member of the Trust & Safety or legal team, responsible for working with inbound and outbound communications with law enforcement, regarding content or users on the platform. Facilitates legal requests and takedowns.

Linguistic Expert

Member of the Trust & Safety team (or vendor) who is an expert in a specific language and can support nuanced moderation decisions in said language.

Loli

Child predator slang for an underage female with a childlike appearance, or a woman who is of age but physically looks, or dresses like a minor.

M

Malicious Content

Content that is created or shared with malicious intent. Includes but is not limited to child sexual abuse material, nudity, profanity, sexual content, bullying, terrorist content, violence, and disinformation.

Malware

Malicious software that is intentionally created to be intrusive and exploit other software or hardware. Usually aimed at stealing a user’s information, or causing damage to computers, servers, and networks.

Metadata

Information about a specific piece of content, user account, or shared item. Can include: the item’s name, time and date it was posted, the location where it was posted from, the user who posted it, all users who engaged with it, its view count, and more.

By analyzing an item’s metadata, Trust & Safety teams can make contextually informed moderation decisions.

Read more:... Read More

Information about a specific piece of content, user account, or shared item. Can include: the item’s name, time and date it was posted, the location where it was posted from, the user who posted it, all users who engaged with it, its view count, and more.

By analyzing an item’s metadata, Trust & Safety teams can make contextually informed moderation decisions.

Read more: Content Detection Tools

Show less

Misinformation

The unintentional creation or sharing of inaccurate or misleading information. This differs from disinformation in that misinformation is unintentional, while disinformation is the intentional distribution of misleading information.

Muting

Platforms can allow users to mute other users so that their activities do not appear on their feeds.

Read more: Policy Enforcement

N

Non Consensual Intimate Imagery

Sexually-explicit images that were either acquired unknowingly or unlawfully or were taken consensually but shared or posted online without consent.
NCII also include the sharing of intimate imagery beyond the scope of its intended use (ie. leaking intimate images shared on one platform, across other platforms)

 

O

Online Safety Bill (UK)

The UK’s Online Safety Bill is upcoming legislation that will require online platforms to take proactive action to keep users safe. The Bill outlines illegal, and some legal but harmful content that platforms will have to act against. The Bill’s current draft is in Parliament and is expected to pass by the end of 2022.

Read more: Read More

The UK’s Online Safety Bill is upcoming legislation that will require online platforms to take proactive action to keep users safe. The Bill outlines illegal, and some legal but harmful content that platforms will have to act against. The Bill’s current draft is in Parliament and is expected to pass by the end of 2022.

Read more: The UK Online Safety Bill

Show less

Online Sexual Harassment

Sexual harassment or misconduct that is conducted online. This is disproportionately aimed at women and/or members of the LGBTQIA+ community.

Online integrity

The perceived reputation, trustworthiness, and reliability of an online platform. The degree to which users feel that they and their information are safe within that platform.

Open Source Intelligence

Also knowns as web intelligence (WEBINT). Intelligence that is collected via publicly available information online.
In Trust & Safety, open-source intelligence is used to gain a contextual understanding of harmful activities, enabling proactive content moderation.

Overkill Rate

Refers to the rate at which a platform’s Trust & Safety team or automated detection mechanisms take unwarranted action against items.

P

Personally Identifiable Information (PII)

Information that potentially identifies an individual either directly or indirectly in conjunction with other data.

Phishing

A tactic of committing fraud, which involves the impersonation of legitimate websites, apps, or email addresses to deceive individuals into providing a fraudster with their personally identifiable information or account details.

PhotoDNA Technology

Microsoft software that is used to identify images involving child exploitation. This software creates image hashes, which are then compared to hashes of known instances of child exploitative imagery. Using this technology, platforms are able to detect and stop the distribution of such images.

Read more: Content Detection Tools

Policy

The content policy of a user-generated content website defines what can and can’t be posted to that specific website. Also known as Content Policy or Community Guidelines.

Policy Analyst

Member of the policy team, responsible for analyzing and examining the effectiveness of a company’s content policies.

Policy Enforcement

The team responsible for enforcing the platform’s content policies and taking action against violative content.

Policy Specialist/Policy Manager

Individuals in the Trust & Safety team responsible for establishing the platform’s community guidelines and defining what is and isn’t allowed to be shared on the platform.

Involves collaboration with internal teams as well as external agencies such as law enforcement, regulators, and industry partners.

Precision

The measure of true positives in automated moderation. Calculated by the number of correctly identified violations, divided by the total number of identified violations.

For example, if ten items were detected as hate speech by AI, but only eight of them were found to actually be hate speech, that AI had an 80% precision rate.

See Measuring Trust & Safety

Prevalence

A key metric for Trust & Safety teams. Measures the likelihood that platform users will be exposed to harmful content while using the platform.

Calculated by the number of views of malicious content, divided by the total number of content views in a sample set.

See Measuring Trust & Safety

Pro-Ana/Pro-Mia

Associated with eating disorder communities, these are abbreviated terms for pro-anorexia and pro-bulimia. Serve to self-identify as supportive of these eating disorders. 

See Eating Disorder Communities

Proactive Moderation

A form of content moderation that aims to detect malicious or violative content before it is seen or reported by others. Utilizes various techniques, including automated detection and intelligence collecting to identify the violative content before it has a chance to harm user safety or platform integrity.

Previously used only to detect illegal content like CSAM and terror, pending legislation may require platforms to proactively moderate harmful, not just illegal content.

See Harmful Content Detection

Proactive Removal Rate

A metric that indicates the rate at which action was taken on content or accounts prior to being posted or reported by other users.

Calculated by the number of proactively moderated items divided by the total number of moderated items.

R

Reach

Refers to the total number of views on a specific item of violative content. When reporting or conducting performance analysis, can be used as a measure of “total reach” of violative content, across the platform or across specific threat verticals.

Read more: Measuring Trust & Safety

Reactive Moderation

A moderation process that relies on a platform’s community or other individuals to identify and flag content that may be in breach of a platform’s policies. Due to its reliance on the community, violative content is often seen by multiple users, before action is taken.

Recall

The measure of how much of a platform’s malicious content is picked up by its moderation systems.

Calculated by the number of correctly identified malicious items, divided by the total number of malicious items on the platform. For example, if a platform has ten malicious pieces of content, and AI identified seven of them, that AI had a 70% recall rate. For most automated detection mechanisms, recall and precision are inversely correlated.

Read More: Measuring Trust & Safety

Red Team

In cyber security, the red team tests a software or platform’s defenses by proactively attempting to bypass or circumvent them. The team then reports on vulnerabilities found, in order to improve the platform’s defenses. Also known as white hat hackers.

Similarly, in Trust & Safety, a red team may proactively test detection methodologies by creating and sharing content with nuanced violations, in an attempt to bypass detection.

Read more: Content Detection Tools

Removal Rate

A user’s removal rate is the rate at which that user’s content is removed, and is used as a measure of that user’s risk to the platform. Calculated by the number of entries or posts removed, divided by the total number of entries or posts created by that user.

Request for Information (RFI)

In Trust & Safety, RFIs involve requesting intelligence or insights from an intelligence provider or internal intelligence desk. This research is then used to establish policies, take action, and understand on-platform trends.

Resilience

Refers to a moderator’s emotional resilience to exposure to harmful content.

To improve resilience, many Trust & Safety teams use tools that filter, blur, or otherwise obstruct the moderator’s view of the harmful imagery, such as ActiveFence’s CleanView. Additionally, many platforms employ automated detection mechanisms to automatically detect and remove extremely graphic content, without moderators having to view it.

Reverse Engineering

A method or process where attempts are made to replicate a system, process, machine, device, or software. Used in cybersecurity and red teams to find malicious code in software, websites, and apps.

Risk Score

An automated, AI-calculated score, denoting the level of threat that an item poses to user safety and the type of policy violation. Trust & Safety teams can use this metric to prioritize moderation queues.

Read more: Content Detection Tools

S

Safety by Design

The design and development of online products and services in a process that anticipates, detects, or eliminates online harms prior to their occurrence.

Read more: Safety by Design

Section 230

Frequently referred to as the 26 words that created the internet, Section 230 of the Communications Decency Act of 1996 limits the liability of technology companies for the user-generated content hosted on their platforms.

Suspension

An enforcement mechanism that involves the temporary, time-limited banning of an account.

Read more: Policy Enforcement

T

Tactics, Techniques, and Procedures (TTPs)

Generally provided by intelligence providers or internal intelligence desks, TTPs are a description of the techniques used by bad actors to conduct harm. In Trust & Safety, an understanding of bad actor TTPs can provide teams with the insights needed to proactively detect and stop on-platform harm.

Learn more: Trust & Safety Intelligence

 

Thinspo

Prevalent in eating disorder communities, the term references images or other content that encourages one to engage in extreme dieting behaviors. Also known as “thinspiration”, “bonespo”, “fitspo”.

Learn more: Eating Disorder Communities

Time to Action

A measurement of the time between when a violative piece of content is created, to when it is identified and actioned on. Can also refer to the time it takes to review and make a decision about a piece of content.

Toxicity

Often a subjective measurement of the level and type of harm that exists on a platform.

Transparency Report

Currently voluntary reports that provide transparency into the ways that data is handled and moderation decisions are made. This report communicates key metrics that can include: the volume, type and region of detected content, how that content was detected, what actions were taken, the volume of and reactions to appeals, and changes over time – among other metrics.

While currently voluntary, upcoming legislation in Germany, UK, EU and US will all require some level of transparency reports.

Read More: Read More

Currently voluntary reports that provide transparency into the ways that data is handled and moderation decisions are made. This report communicates key metrics that can include: the volume, type and region of detected content, how that content was detected, what actions were taken, the volume of and reactions to appeals, and changes over time – among other metrics.

While currently voluntary, upcoming legislation in Germany, UK, EU and US will all require some level of transparency reports.

Read More: The German Network Enforcement Act, the U.K. Online Harms Bill, the EU Digital Services Act and the U.S. Social Media Disclosure and Transparency of Advertisements Act

Show less

True Positive

Content that is correctly flagged as violative

Trust & Safety

Teams that are focused on the development, management, and enforcement of a platform’s policies to ensure that only acceptable content is posted online and that users are safe.

Trust & Safety Platform

A digital dashboard system for Trust & Safety teams, where moderators can access a prioritized queue of potentially violative content, conduct an analysis of that content, make a decision, and take action – all within the platform.

The platform can also assist in the creation of moderation workflows, allow managerial oversight of moderator decisions, and create transparency reports.

Read more: Trust & Safety Management

Trusted Flagger

An individual or vendor who is considered by the platform to be an expert in their field. Content flagged by this entity is therefore given special notice by moderation teams.

Read more: Content Detection Tools

U

Unsolicited Pornography

A type of abuse, involving the sending of unwanted or unrequested pornographic images. Prevalent in online dating and direct messaging platforms.

Unwanted Sexualization

The sending of unwanted requests for sexual activity.

User Flagging

A mechanism where individual users “flag” to moderation teams, content that is harmful or violative. Flagging is a common reactive reporting mechanism and is generally considered to have a high false-positive rate.

Read more: Content Detection Tools

User Generated Content

Items that are posted online by a user including social posts, profile photos, comments, audio, images, videos, forums, product/service listings, avatars, contest entries, or any content that a user may create, copy or share and other platform users are able to access it. Also known as UGC, or user-to-user content

V

Violative Content

Content that is in violation of a platform’s content policy or community guidelines. May include illegal, but also harmful and unwanted content.

Violent Extremism

In Trust & Safety, violent extremism involves the spread of content or actions online that are taken by extremist groups and their sympathizers to support extremist ideologies. This often includes images, videos, and text, as well as action taken against victim groups in live chats.

Oops!

Sorry, there were no results for that search term.