Senior Python Engineer for Data Pipeline
Worldwide
## Senior Python Engineer – Production Data Pipeline & Infrastructure We are looking for an experienced **Python engineer** to diagnose, fix and improve an existing production data-processing pipeline running on a Linux VPS. The system is already built and operational. This is **not a rebuild project**. We need someone who can understand the existing architecture, identify reliability and performance issues, and implement practical fixes. ### Current Infrastructure * Python services and background workers * Linux VPS * MySQL / MariaDB * Large production database * Data ingestion and processing pipelines * Multiple processing queues * Browser-based automation * Image processing * Scheduled jobs * Process monitoring and automatic recovery * Logging and error handling ### The Work You will be responsible for investigating issues such as: * Processes unexpectedly stopping * Processing queues becoming delayed or stuck * Memory and resource usage * Worker stability * Database and processing bottlenecks * Inconsistent processing between pipeline stages * Monitoring and automatic recovery * Overall production reliability We want someone who is comfortable connecting to an existing Linux server via SSH, examining logs, processes, database activity and Python code, identifying the actual cause of problems, and implementing reliable fixes. ### Required Experience Strong practical experience with: **Python • Linux • MySQL/MariaDB • Background Workers • Data Pipelines • Process Management • Production Debugging • Performance Optimization** Experience maintaining an existing production system is more important to us than building something new from scratch. When applying, please briefly describe a similar Python production system or data pipeline you have diagnosed and improved. There is potential for **ongoing work** maintaining and improving the infrastructure after the initial project.
$30.00
Fixed-price- ExpertExperience Level
- Remote Job
- Complex projectProject Type
Skills and Expertise
Activity on this job
- Proposals:10 to 15
- Interviewing:0
- Invites sent:0
- Unanswered invites:0
About the client
- GBRLeatherhead2:20 PM
- $914 total spent71 hires, 12 active
- 6 hours
Explore similar jobs on Upwork
How it works
Create your free profileHighlight your skills and experience, show your portfolio, and set your ideal pay rate.
Work the way you wantApply for jobs, create easy-to-by projects, or access exclusive opportunities that come to you.
Get paid securelyFrom contract to payment, we help you work safely and get paid securely.
About Upwork
- 4.9/5(Average rating of clients by professionals)
- G2 2021#1 freelance platform
- 49,000+Signed contract every week
- $2.3BFreelancers earned on Upwork in 2020
Find the best freelance jobs
Growing your career is as easy as creating a free profile and finding work like this that fits your skills.
Trusted by