ML/applied engineer

What we're building: an open, reproducible benchmark measuring how open-weight LLMs (Kimi, DeepSeek, Qwen) hold up against prompt injection and jailbreak attacks compared to a frontier model, static versus adaptive. Public teaser. You'll be extending an existing attacker/defender/judge pipeline that already runs on AWS Bedrock, not building from scratch. You'll need: Strong Python, comfortable reading and extending an existing codebase (not writing throwaway scripts) Experience with LLM APIs (OpenAI/Anthropic/Bedrock SDKs), prompt engineering, and ideally some exposure to red-teaming or jailbreak research Comfortable with data pipelines: JSON schemas, batch jobs, versioned datasets Enough PyTorch/ML familiarity to run (not necessarily design) a training job on a single GPU instance, following a documented recipe Written English good enough to document findings clearly; this feeds a public paper Scope (mapped to the audit's backlog, executed under our review): Extend the data contract with two new scoring dimensions (injection, refusal_failure) per a spec we provide, without breaking existing validation Build attack tactic templates: indirect-document injection, tool-result injection, system-prompt extraction, encoding/obfuscation wrappers, multi-turn escalation (we provide examples and the pattern to follow) Build a scenario sampler that varies target model, injection vector, and policy context Build a provider adapter so the harness can call models outside AWS Bedrock (specifically Kimi K3 and GPT-5.x APIs), matching the existing Bedrock adapter's interface Wire the adaptive attack loop: feed guard/model misses back into the next round of attack generation (there's a documented stub to complete) Build a minimal scoring/serving entrypoint that takes a request and response and returns a verdict, off an existing model checkpoint Run small, budget-capped pilot batches to validate the pipeline before any full-size data generation (we approve each budget before it fires) Measure two-hop latency (request-side and response-side scoring) and report p95 Document everything in the repo as you go: this becomes the methodology section of a published paper, so clarity matters as much as code Explicitly not in scope, stays with us: what the numbers are allowed to claim, whether a training run proceeds, interpreting whether a result is good or bad, anything involving the full unlabeled corpus. To apply, answer: Link a project where you extended someone else's ML/data pipeline rather than built one from scratch. What did you have to understand first? Have you worked with LLM jailbreak/red-teaming techniques (indirect prompt injection, multi-turn escalation) before? Describe one, briefly. What's your experience with AWS Bedrock specifically, versus just the OpenAI/Anthropic APIs? Your available hours per week and time zone overlap with US Pacific.