Senior Data Pipeline Engineer — Python / BigQuery / Webhook Infrastructure

# Senior Data Pipeline Engineer — Python / BigQuery / Webhook Infrastructure We are building a scalable attribution and analytics infrastructure for a high-volume iGaming affiliate operation. This role is focused strictly on backend data infrastructure and warehouse engineering. NOT: * dashboards/BI * frontend * AI/ML * media buying --- # Planned Architecture SoftSwiss Webhooks ↓ Python FastAPI ingestion service ↓ Raw event DB/table (Postgres) ↓ Retry/replay logic ↓ Transformation jobs ↓ BigQuery raw warehouse ↓ Downstream analytics & attribution layer (handled separately) --- # Preferred Stack * Python (preferred over Node.js) * FastAPI or Flask * PostgreSQL * BigQuery * Docker * Cron/scheduled jobs * Webhook-first architecture * API reconciliation fallback --- # Scope ## Ingestion Layer Build a secure ingestion service that: * receives SoftSwiss webhook events * validates signatures * stores raw payloads safely * guarantees durability * prevents data loss --- ## Processing Layer Build normalization pipelines for: * signup * deposit * withdrawal * bet * win * bonus events Requirements: * click_id → user attribution persistence * FTD detection * replay-safe processing * idempotent event handling * retry/dead-letter handling --- ## BigQuery Warehouse Create: * raw warehouse tables * ingestion jobs * schema handling * scalable loading strategy Expected tables: * users * transactions * gameplay * bonuses * attribution mappings --- ## Reliability & Operations Requirements: * Dockerized deployment * logging * monitoring hooks * failure handling * documentation/runbook --- # Important Technical Preferences * Webhooks should be primary ingestion * API polling only used for reconciliation/fallback * Strong preference toward reliability and durability * Postgres-based replay/retry logic preferred for MVP simplicity --- # Deliverables * production-ready ingestion service * raw event persistence layer * normalization pipelines * BigQuery raw warehouse setup * replay/retry architecture * Dockerized deployment * documentation/runbook --- # Nice To Have * iGaming experience * attribution systems * click tracking experience * BigQuery optimization experience * Meta CAPI familiarity --- # To Apply Please include: 1. Relevant webhook/data pipeline experience 2. Experience with BigQuery 3. Preferred replay/retry architecture 4. Example of similar backend data infrastructure work 5. Python vs Node.js preference and why