Senior Databricks Data Engineer – Hadoop Migration (1 Year contract)
Worldwide
*** YOU MUST UPLOAD YOUR RESUME IN MS WORD FORMAT TO BE CONSIDERED*** We are seeking a Senior Databricks Data Engineer for a 12-month project with a financial services client, focused on migrating an on-prem Hadoop environment to Databricks. Start: Mid-September Duration: 12 months Location: Remote Hours: Must work U.S. Eastern Time business hours (4pm - 1am IST) Responsibilities * Migrate data, pipelines, and workloads from Hadoop/Hive/HDFS to Databricks. * Build and migrate pipelines using Spark/PySpark, Python, SQL, and Delta Lake. * Refactor Hadoop workloads as needed for efficient execution on Databricks. * Perform data validation, performance tuning, and troubleshooting. * Implement Databricks governance and data engineering best practices. Required Skills * 6+ years of Data Engineering experience with strong hands-on Databricks experience. * Strong Spark/PySpark, Python, SQL, Delta Lake, Hadoop, Hive, and HDFS skills. * Experience with Hadoop-to-Databricks or large-scale cloud data migrations. * Experience with newer Databricks capabilities including Unity Catalog, Lakeflow Declarative Pipelines, Lakeflow Jobs/Workflows, Auto Loader, Liquid Clustering, Serverless Compute, and Databricks SQL. * Strong communication skills and ability to work U.S. Eastern Time business hours. * Databricks certifications. Nice to Have * Financial services/banking experience. This is a hands-on Data Engineering role focused on Hadoop-to-Databricks migration. *** YOU MUST UPLOAD YOUR RESUME IN MS WORD FORMAT TO BE CONSIDERED***
- More than 30 hrs/weekHourly
- 6+ monthsDuration
- ExpertExperience Level
$15.00
-
$18.00
Hourly- Remote Job
- Complex projectProject Type
Skills and Expertise
Activity on this job
- Proposals:5 to 10
- Last viewed by client:2 hours ago
- Interviewing:1
- Invites sent:0
- Unanswered invites:0
About the client
- United StatesSugar Land5:09 AM
- $1.6K total spent5 hires, 0 active
- 88 hours
Explore similar jobs on Upwork
How it works
Create your free profileHighlight your skills and experience, show your portfolio, and set your ideal pay rate.
Work the way you wantApply for jobs, create easy-to-by projects, or access exclusive opportunities that come to you.
Get paid securelyFrom contract to payment, we help you work safely and get paid securely.
About Upwork
- 4.9/5(Average rating of clients by professionals)
- G2 2021#1 freelance platform
- 49,000+Signed contract every week
- $2.3BFreelancers earned on Upwork in 2020
Find the best freelance jobs
Growing your career is as easy as creating a free profile and finding work like this that fits your skills.
Trusted by