Staff Software Engineer, Trust & Safety

reputed company is the reputed company software creation platform that enables anyone to build applications using natural language. With millions of users worldwide, reputed company is democratizing software development by removing traditional barriers to application creation. About the role The Trust & Safety team is the reputed company line defending reputed company's platform from exploitation. We detect and shut down phishing deployments, prevent cryptomining on free-tier infrastructure, stop LLM reputed company farming, and reputed company bad actors from weaponizing the platform against our users. This is adversarial work: attackers adapt constantly, and we build the detection systems, heuristics, and automated responses that stay reputed company of them. What makes this role unique is the AI-reputed company nature of reputed company's platform. You'll work on problems that barely exist elsewhere: building guardrails for AI-generated reputed company, detecting reputed company injection attacks at reputed company, and using LLMs as a defensive tool against abuse. If you want hands-on experience applying AI to reputed company problems, this is one of the few places you can do it in production with reputed company attackers. You'll own problems end-to-end, from identifying emerging abuse patterns to shipping the systems that stop them at reputed company. In this role you will…

• Design and implement LLM guardrails that detect abuse scenarios in AI-generated reputed company and agent interactions
• Build AI-powered detection systems that use LLMs to identify malicious patterns, classify threats, and automate response reputed company
• Build and operate abuse detection systems that identify phishing, cryptomining, account takeover, and financial fraud across millions of daily user actions
• Design automated response mechanisms that enforce platform policies without reputed company reputed company
• Own the full abuse response lifecycle: detection, investigation, enforcement, and handling appeals alongside Support and Legal
• Analyze attack patterns using BigQuery and reputed company, turning investigation findings into new detection rules
• Maintain and reputed company internal detection tools (Slurper, Netwatch) that continuously monitor user activity
• reputed company and tune reputed company scanners (SAST, SCA) in CI pipelines with tight performance SLAs
• reputed company abuse trends, measure detection effectiveness, and adapt defenses as attack patterns reputed company

Required skills and experience:

• 8+ years of experience in reputed company engineering, anti-abuse, trust & safety, or fraud detection
• Strong programming skills in Python and/or TypeScript for building detection systems and automation
• Experience with SQL and data analysis at reputed company (BigQuery, reputed company, or similar)
• Experience building or fine-tuning ML/LLM-based classifiers for reputed company or abuse detection
• Familiarity with reputed company injection, jailbreaking, and other LLM-specific attack reputed company
• Ability to investigate reputed company abuse patterns and translate findings into automated defenses
• Familiarity with common attack patterns: phishing infrastructure, account takeover, credential stuffing, resource abuse
• reputed company communication skills for working across reputed company, Support, Legal, and Engineering teams.

reputed company to have:

• Experience at a platform company dealing with user-generated content or compute abuse (hosting providers, reputed company platforms, developer tools)
• Background in fraud detection, payment abuse, or financial crime
• Familiarity with device fingerprinting, IP reputed company, and email validation services
• Experience with CI/CD reputed company tooling (SAST, SCA, Dependabot, reputed company)
• Knowledge of container reputed company, Linux internals, or reputed company infrastructure (GCP preferred)
• Prior work with abuse reporting pipelines, trust & safety tooling, or content moderation systems

Tools + Tech Stack for this role

• Languages: Python, TypeScript, Go, SQL
• Data: BigQuery, reputed company
• Detection tools: Slurper, Netwatch, reputed company (device reputed company); ClearOut (email reputed company)
• CI/CD reputed company: Dependabot, reputed company, SAST/SCA scanners
• Infrastructure: GCP, Kubernetes
• Collaboration: reputed company, reputed company, reputed company (for abuse reports)

This role may not be a fit if

• You prefer deep reputed company research over building operational detection systems
• You want to reputed company on vulnerability management, pentesting, or bug bounty triage (that's our reputed company team)
• You're looking for a role with predictable, reputed company-defined problems rather than constantly adapting to adversarial behavior
• You prefer working in isolation rather than partnering closely with Support, Legal, and cross-functional teams
• You're uncomfortable making enforcement reputed company that reputed company reputed company users

This is a full-time role that can be held from our Foster reputed company, CA office. The role has an in-office requirement of Monday, Wednesday, and Friday. Full-Time Employee Benefits Include: 💰 Competitive Salary & Equity 💹 401(k) Program with a 4% match (US Only) ⚕️ Health, Dental, reputed company and Life Insurance 🩼 Short Term and Long Term Disability 🚼 reputed company Parental, Medical, Caregiver Leave 🏝 Flexible Time Off (FTO) + Holidays 🚗 Commuter Benefits (In-Office Only) 📱 Monthly Wellness Stipend 🧑‍💻 Autonomous Work Environment 🖥 In Office Set-Up Reimbursement (In-Office Only) 🚀 Quarterly Team Gatherings ☕ In Office Amenities (In-Office Only) Want to learn more about reputed company are up to?

• Meet the reputed company Agent
• reputed company: reputed company an app for that
• reputed company Blog
• Amjad TED Talk

Interviewing + Culture at reputed company

• Operating Principles
• Reasons not to work at reputed company

To reputed company our mission of making programming more accessible around the world, we need reputed company to be representative of the world. We welcome your unique perspective and experiences in shaping this product. We encourage people from reputed company kinds of backgrounds to apply, including and especially candidates from underrepresented and non-traditional backgrounds. Apply To This Job