Senior Python Engineer for Data Pipeline

Posted 4 hours ago

Worldwide

Summary

## Senior Python Engineer – Production Data Pipeline & Infrastructure We are looking for an experienced **Python engineer** to diagnose, fix and improve an existing production data-processing pipeline running on a Linux VPS. The system is already built and operational. This is **not a rebuild project**. We need someone who can understand the existing architecture, identify reliability and performance issues, and implement practical fixes. ### Current Infrastructure * Python services and background workers * Linux VPS * MySQL / MariaDB * Large production database * Data ingestion and processing pipelines * Multiple processing queues * Browser-based automation * Image processing * Scheduled jobs * Process monitoring and automatic recovery * Logging and error handling ### The Work You will be responsible for investigating issues such as: * Processes unexpectedly stopping * Processing queues becoming delayed or stuck * Memory and resource usage * Worker stability * Database and processing bottlenecks * Inconsistent processing between pipeline stages * Monitoring and automatic recovery * Overall production reliability We want someone who is comfortable connecting to an existing Linux server via SSH, examining logs, processes, database activity and Python code, identifying the actual cause of problems, and implementing reliable fixes. ### Required Experience Strong practical experience with: **Python • Linux • MySQL/MariaDB • Background Workers • Data Pipelines • Process Management • Production Debugging • Performance Optimization** Experience maintaining an existing production system is more important to us than building something new from scratch. When applying, please briefly describe a similar Python production system or data pipeline you have diagnosed and improved. There is potential for **ongoing work** maintaining and improving the infrastructure after the initial project.

  • $30.00

    Fixed-price
  • Expert
    Experience Level
  • Remote Job
  • Complex project
    Project Type
Skills and Expertise
Mandatory skills
Python
Machine Learning
Nice-to-have skills
Django
Selenium
Activity on this job
  • Proposals:10 to 15
  • Interviewing:
    0
  • Invites sent:
    0
  • Unanswered invites:
    0
About the client
Member since Aug 26, 2024
  • GBR
    Leatherhead2:20 PM
  • $914 total spent
    71 hires, 12 active
  • 6 hours

Explore similar jobs on Upwork

Zero Trust Access Blog Content WritingFixed-price‐ Posted 1 month ago
Java
Golang
Zero Trust Architecture
Content Writing
Article Writing
Python
Firebase Cloud Firestore
Firebase
FastAPI
Asynchronous I/O
Python Asyncio

How it works

  • Post a job icon
    Create your free profile
    Highlight your skills and experience, show your portfolio, and set your ideal pay rate.
  • Talent comes to you icon
    Work the way you want
    Apply for jobs, create easy-to-by projects, or access exclusive opportunities that come to you.
  • Payment simplified icon
    Get paid securely
    From contract to payment, we help you work safely and get paid securely.
Want to get started? Create a profile

About Upwork

  • Rating is 4.9 out of 5.
    4.9/5
    (Average rating of clients by professionals)
  • G2 2021
    #1 freelance platform
  • 49,000+
    Signed contract every week
  • $2.3B
    Freelancers earned on Upwork in 2020

Find the best freelance jobs

Growing your career is as easy as creating a free profile and finding work like this that fits your skills.

Trusted by

  • Microsoft Logo
  • Airbnb Logo
  • Bissell Logo
  • GoDaddy Logo