Senior Data Engineer Needed – Pipeline Architecture, Data Modeling & Warehouse Optimization
Worldwide
We're hiring a Senior Data Engineer to design and scale the data infrastructure behind our analytics and product decisions. You'll own our pipelines end-to-end ingestion, transformation, and serving and be the go-to for data reliability and performance. Scope of Work: - Design and maintain ETL/ELT pipelines for structured and semi-structured data - Build dimensional data models (star/snowflake schemas) for analytics use cases - Set up data quality monitoring and alerting - Optimize warehouse performance (partitioning, clustering) and manage compute costs - Build orchestration workflows (Airflow/Dagster) with retry logic and SLAs - Work with our team to translate business needs into data models Required Skills: - 5+ years data engineering experience at production scale - Expert SQL (window functions, query optimization, execution plans) - Strong Python for pipeline development - Hands-on with Snowflake, BigQuery, or Redshift - Experience with Airflow, Dagster, or Prefect - Streaming experience (Kafka/Kinesis) a plus - Cloud experience (AWS/GCP/Azure) + Terraform - dbt experience a plus How to Apply: Skills Test (please complete to be considered): To keep this efficient for both sides, please include short written answers (a paragraph or two each) to these with your proposal: 1. Query fix: Describe how you'd diagnose and fix a slow SQL query on a large table what would you check first? 2. Pipeline design: How would you design a pipeline ingesting JSON events that handles duplicates and late-arriving data, with a daily SLA? Walk through your approach and trade-offs. 3. Failure story: Tell us about a time a pipeline silently broke or loaded bad data how did you catch it, and what did you build afterward to prevent a repeat? 4. Batch vs. streaming: When would you choose batch over streaming for a new pipeline, and what would you need to know first? Candidates who skip the written answers or give generic/copy-paste responses will not be considered. We're looking for people who can show their actual reasoning.
- More than 30 hrs/weekHourly
- 6+ monthsDuration
- ExpertExperience Level
$50.00
-
$95.00
Hourly- Remote Job
- Ongoing projectProject Type
Skills and Expertise
Activity on this job
- Proposals:50+
- Last viewed by client:1 hour ago
- Interviewing:0
- Invites sent:0
- Unanswered invites:0
About the client
- United StatesRichmond10:21 AM
- 2 hires, 0 active
Explore similar jobs on Upwork
How it works
Create your free profileHighlight your skills and experience, show your portfolio, and set your ideal pay rate.
Work the way you wantApply for jobs, create easy-to-by projects, or access exclusive opportunities that come to you.
Get paid securelyFrom contract to payment, we help you work safely and get paid securely.
About Upwork
- 4.9/5(Average rating of clients by professionals)
- G2 2021#1 freelance platform
- 49,000+Signed contract every week
- $2.3BFreelancers earned on Upwork in 2020
Find the best freelance jobs
Growing your career is as easy as creating a free profile and finding work like this that fits your skills.
Trusted by