Lead Data Platform Engineer

Posted 3 days ago

Worldwide

Needs to hire 2 Freelancers
Summary

About the Role: Build the enterprise’s AWS Databricks data lake, including the lakehouse foundation, ingestion patterns, pipeline frameworks, and target data model. This role owns the technical buildout and design leadership for migrating key pipelines from Microsoft Fabric into AWS Databricks and implementing them into a new, scalable data model. Key Responsibilities: • Build the AWS Databricks data lake and lakehouse foundation, including workspace, storage, compute, catalog, security, and environment patterns. • Architect and build ingestion into Databricks from SAP S/4HANA, Salesforce, IoT, complaints, Microsoft Fabric, and other enterprise systems. • Design the target enterprise data model and implement migrated Microsoft Fabric pipelines into that model in AWS Databricks. • Implement Git, CI/CD, and engineering best practices. • Define platform standards for optimization, reliability, security, cost management, and handoff to DataOps for ongoing operations. • Mentor company’s engineering resources. Required Qualifications: • 10+ years of experience • Experience on AWS Databricks data platform architecture. • Lakehouse, data lake, medallion-style patterns, and Databricks integration experience. • Enterprise integration and pipeline orchestration for S/4HANA, Salesforce, IoT, and complaints data. • Data modeling and platform engineering. • CI/CD, infrastructure-as-code, and DevOps practices. • Technical leadership and mentoring.

  • More than 30 hrs/week
    Hourly
  • 6+ months
    Duration
  • Expert
    Experience Level
  • Remote Job
  • Ongoing project
    Project Type

Contract-to-hire opportunity

This lets talent know that this job could become full time.
Learn more
Skills and Expertise
Mandatory skills
Data Engineering
Databricks Platform
Activity on this job
  • Proposals:50+
  • Last viewed by client:2 days ago
  • Interviewing:
    13
  • Invites sent:
    0
  • Unanswered invites:
    0
About the client
Member since Feb 28, 2025
  • United States
    Atlanta9:00 PM
  • $16K total spent
    5 hires, 2 active
  • 142 hours

Explore similar jobs on Upwork

AI and Data Engineering CallerFixed-price‐ Posted 1 month ago
Data Engineering
Machine Learning
Data Science
Artificial Intelligence
Varicent/SPM Consultant for GuidanceHourly‐ Posted 3 weeks ago
SAP
ETL

How it works

  • Post a job icon
    Create your free profile
    Highlight your skills and experience, show your portfolio, and set your ideal pay rate.
  • Talent comes to you icon
    Work the way you want
    Apply for jobs, create easy-to-by projects, or access exclusive opportunities that come to you.
  • Payment simplified icon
    Get paid securely
    From contract to payment, we help you work safely and get paid securely.
Want to get started? Create a profile

About Upwork

  • Rating is 4.9 out of 5.
    4.9/5
    (Average rating of clients by professionals)
  • G2 2021
    #1 freelance platform
  • 49,000+
    Signed contract every week
  • $2.3B
    Freelancers earned on Upwork in 2020

Find the best freelance jobs

Growing your career is as easy as creating a free profile and finding work like this that fits your skills.

Trusted by

  • Microsoft Logo
  • Airbnb Logo
  • Bissell Logo
  • GoDaddy Logo