Senior AI / Data Engineer: Medicare member eligibility module (Databricks)

Posted yesterday

Only freelancers located in the U.S. may apply.U.S. located freelancers only

Summary

We run member and prospect outreach for Medicare Advantage clients. The full system already works: ingest, identity, scoring, audience build, vendor file, matchback. We are not rebuilding it. One module is the problem. Eligibility, suppression, and release checks still depend on a large Excel workbook. The Databricks job that consumes those rules is in production. After a recent Snowflake permission change it has been failing, and even when it succeeds we do not fully trust the file. A wrong include or a missed suppression is a compliance issue, not a metric miss. PHI leaves the building. We need a senior engineer who can treat this as a system repair, not a notebook exercise. Read the live PySpark / SQL job. Trace the workbook into executable rules. Use an LLM only to accelerate extraction and to draft QC, then verify every rule against source tables before anything is trusted. Write the checks that catch duplicates, bad exclusions, test/control leakage, and count breaks before release. This is data engineering with model-assisted rule work. If your background is only model fitting, this is not the right job. If you have shipped person-level pipelines and can keep an LLM from inventing a business rule, it is. Stack on this module: Databricks, PySpark, Snowflake, SQL, Python. Git, AWS, SFTP, and Linux show up around the job. Excel is the current spec. US-based only. Background check, OIG screen, HIPAA training, and a BAA are required before data access.

  • More than 30 hrs/week
    Hourly
  • 3-6 months
    Duration
  • Expert
    Experience Level
  • $70.00

    -

    $120.00

    Hourly
  • Remote Job
  • Ongoing project
    Project Type

Contract-to-hire opportunity

This lets talent know that this job could become full time.
Learn more
Skills and Expertise
Mandatory skills
Data Engineering
PySpark
Activity on this job
  • Proposals:50+
  • Interviewing:
    0
  • Invites sent:
    0
  • Unanswered invites:
    0
About the client
Member since May 4, 2026
  • United States
    Giddings2:16 PM
  • $2.1K total spent
    7 hires, 0 active
  • 18 hours

Explore similar jobs on Upwork

Environment & Build Pipeline LockFixed-price‐ Posted 1 month ago
Java
DevOps
Deployment Automation
AI and Data Engineering CallerFixed-price‐ Posted 1 month ago
Data Engineering
Machine Learning
Data Science
Artificial Intelligence

How it works

  • Post a job icon
    Create your free profile
    Highlight your skills and experience, show your portfolio, and set your ideal pay rate.
  • Talent comes to you icon
    Work the way you want
    Apply for jobs, create easy-to-by projects, or access exclusive opportunities that come to you.
  • Payment simplified icon
    Get paid securely
    From contract to payment, we help you work safely and get paid securely.
Want to get started? Create a profile

About Upwork

  • Rating is 4.9 out of 5.
    4.9/5
    (Average rating of clients by professionals)
  • G2 2021
    #1 freelance platform
  • 49,000+
    Signed contract every week
  • $2.3B
    Freelancers earned on Upwork in 2020

Find the best freelance jobs

Growing your career is as easy as creating a free profile and finding work like this that fits your skills.

Trusted by

  • Microsoft Logo
  • Airbnb Logo
  • Bissell Logo
  • GoDaddy Logo