Gen AI Researcher

As a Red Team Specialist focused on Generative AI Models, you will play a critical role in enhancing the security and integrity of our cutting-edge AI technologies.

Your primary responsibility will be to conduct analysis and testing of our generative AI systems, including but not limited to language models, image generation models, and any related infrastructure.

The goal is to identify vulnerabilities, assess risks, and provide actionable insights to fortify our AI models and guardrails against potential threats.

Key Responsibilities:

• Simulated Cyber Attacks: Conduct sophisticated and comprehensive simulated attacks on generative AI models and their operating environments to uncover vulnerabilities.
• Vulnerability Assessment: Evaluate the security posture of AI models and infrastructure, identifying weaknesses and potential threats.
• Risk Analysis: Perform thorough risk analysis to determine the impact of identified vulnerabilities and prioritize mitigation efforts.
• Mitigation Strategies: Collaborate with development and security teams to develop effective strategies to mitigate identified risks and enhance model resilience.
• Research and Innovation: Stay abreast of the latest trends and developments in AI security, ethical hacking, and cyber threats. Apply innovative testing methodologies to ensure cutting-edge security practices.
• Documentation and Reporting: Maintain detailed documentation of all red team activities, findings, and recommendations. Prepare and present reports to senior management and relevant stakeholders.

Must-Have

• Proven record of AI vulnerabilities analysis
• Strong understanding of AI technologies and their underlying architectures, especially generative models and frameworks.
• At Least 5 years of experience in offensive cyber security, particularly in Cloud and API security.
• Familiarity with agentic frameworks and agentic development experience
• Proficiency in python.
• Excellent analytical, problem-solving, and communication skills.
• Ability to work in a fast-paced, ever-changing environment.

Nice-to-Have:

• Bachelor’s or Master’s degree in Computer Science, Information Security, or a related field.
• Proving record of building production quality pipelines and automations
• Experience with machine learning development frameworks and environments.
• Advanced Certifications in offensive cybersecurity (e.g. OSWE, OSCE3, SEC542, SEC522) are highly desirable.
• Certifications/background in DevOps/ML fields are highly desirable

ActiveFence is the leading tool stack for Trust & Safety teams, worldwide. By relying on ActiveFence’s end-to-end solution, Trust & Safety teams – of all sizes – can keep users safe from the widest spectrum of online harms, unwanted content, and malicious behavior, including child safety, disinformation, fraud, hate speech, terror, nudity, and more. 

Using cutting-edge AI and a team of world-class subject-matter experts to continuously collect, analyze, and contextualize data, ActiveFence ensures that in an ever-changing world, customers are always two steps ahead of bad actors. As a result, Trust & Safety teams can be proactive and provide maximum protection to users across a multitude of abuse areas, in 70+ languages. 

Backed by leading Silicon Valley investors such as CRV and Norwest, ActiveFence has raised $100M to date; employs 300 people worldwide, and has contributed to the online safety of billions of users across the globe.