Lead Data Platform Engineer
Worldwide
About the Role: Build the enterprise’s AWS Databricks data lake, including the lakehouse foundation, ingestion patterns, pipeline frameworks, and target data model. This role owns the technical buildout and design leadership for migrating key pipelines from Microsoft Fabric into AWS Databricks and implementing them into a new, scalable data model. Key Responsibilities: • Build the AWS Databricks data lake and lakehouse foundation, including workspace, storage, compute, catalog, security, and environment patterns. • Architect and build ingestion into Databricks from SAP S/4HANA, Salesforce, IoT, complaints, Microsoft Fabric, and other enterprise systems. • Design the target enterprise data model and implement migrated Microsoft Fabric pipelines into that model in AWS Databricks. • Implement Git, CI/CD, and engineering best practices. • Define platform standards for optimization, reliability, security, cost management, and handoff to DataOps for ongoing operations. • Mentor company’s engineering resources. Required Qualifications: • 10+ years of experience • Experience on AWS Databricks data platform architecture. • Lakehouse, data lake, medallion-style patterns, and Databricks integration experience. • Enterprise integration and pipeline orchestration for S/4HANA, Salesforce, IoT, and complaints data. • Data modeling and platform engineering. • CI/CD, infrastructure-as-code, and DevOps practices. • Technical leadership and mentoring.
- More than 30 hrs/weekHourly
- 6+ monthsDuration
- ExpertExperience Level
- Remote Job
- Ongoing projectProject Type
Skills and Expertise
Activity on this job
- Proposals:50+
- Last viewed by client:2 days ago
- Interviewing:13
- Invites sent:0
- Unanswered invites:0
About the client
- United StatesAtlanta9:00 PM
- $16K total spent5 hires, 2 active
- 142 hours
Explore similar jobs on Upwork
How it works
Create your free profileHighlight your skills and experience, show your portfolio, and set your ideal pay rate.
Work the way you wantApply for jobs, create easy-to-by projects, or access exclusive opportunities that come to you.
Get paid securelyFrom contract to payment, we help you work safely and get paid securely.
About Upwork
- 4.9/5(Average rating of clients by professionals)
- G2 2021#1 freelance platform
- 49,000+Signed contract every week
- $2.3BFreelancers earned on Upwork in 2020
Find the best freelance jobs
Growing your career is as easy as creating a free profile and finding work like this that fits your skills.
Trusted by