Backend Developer Needed to Optimize Data Integration Pipeline and Database Caching (Existing App)
Only freelancers located in the U.S. may apply.U.S. located freelancers only
Project Overview: I am building Reseller Bro, a mobile utility application designed for on-the-go users to analyze product images and instantly retrieve live, accurate marketplace valuation metrics. The front-end user interface and layout are largely complete. The application uses an AI workflow for visual recognition and keyword generation, but the backend data retrieval and sync pipeline needs optimization for speed and absolute accuracy. I am looking for a highly skilled backend developer to conduct a code audit, repair a broken data aggregation pipeline, and optimize our database caching architecture. The Core Tasks: Database Cache Failure: The app currently fails to properly save or recall data on repeat queries. We need a high-speed caching layer so that if an item has already been searched once, the second lookup pulls from the internal database instantly (under 1 second). Data Sync & Engine Accuracy: The live data gathering process is currently unstable and occasionally returns empty results even for common consumer items with deep market history. Once the AI identifies the item keywords, the data pipeline must reliably sync and aggregate live pricing data from major public e-commerce and secondary marketplaces. Data Array / Tier Mapping Mix-up: The app currently handles item condition mapping poorly, sometimes displaying irrational pricing tiers (e.g., valuing low-tier condition higher than top-tier condition). We need a clean pre-fetch matrix that normalizes marketplace conditions into three clean categories on the first search and automatically filters out pricing anomalies or extreme statistical outliers. Required Technical Skills: Strong backend development experience using Node.js or Python. Advanced expertise in data extraction, handling complex HTML/JSON payloads, managing dynamic connection protocols, and optimizing marketplace data synchronization. Deep experience with database optimization and high-speed caching layers (such as PostgreSQL, Redis, or similar). Ability to step into an existing codebase, audit backend logic, and refactor code without disrupting the current frontend UI layout. 💰 Budget & Performance-Based Milestones ($700 Total) This contract is fixed-price and strictly milestone-driven based on functional performance metrics. No milestones will be released without successful live-testing verification. Milestone 1 ($150) - Code Review & Cache Repair: Audit the inherited repository. Fix the database caching system so that previously queried items bypass the live search entirely and load data instantly (under 1 second). Milestone 2 ($250) - Data Pipeline Optimization & Matrix Mapping: Fix the live data aggregation process to consistently pull reliable pricing from target market channels using the AI-generated keywords. Implement data-cleaning and normalization rules to ensure price tiers always reflect a logical tier structure while removing extreme outliers. Milestone 3 ($300) - Final Integration & Testing: Fully connect the optimized database cache, clean data sorting logic, and application backend into a finalized, stable build ready for deployment testing. To Apply: Please respond with your relevant experience in optimizing high-speed database caches and managing high-volume marketplace data aggregation. Please briefly explain your typical approach to ensuring stable, reliable data connections with public e-commerce platforms. Full project source files and existing code repositories will be shared with top candidates during the interview phase.
$700.00
Fixed-price- ExpertExperience Level
- Remote Job
- Complex projectProject Type
Skills and Expertise
Activity on this job
- Proposals:15 to 20
- Last viewed by client:3 hours ago
- Interviewing:5
- Invites sent:0
- Unanswered invites:0
About the client
- USALas Vegas11:36 PM
- $1.1K total spent1 hire, 0 active
Explore similar jobs on Upwork
How it works
Create your free profileHighlight your skills and experience, show your portfolio, and set your ideal pay rate.
Work the way you wantApply for jobs, create easy-to-by projects, or access exclusive opportunities that come to you.
Get paid securelyFrom contract to payment, we help you work safely and get paid securely.
About Upwork
- 4.9/5(Average rating of clients by professionals)
- G2 2021#1 freelance platform
- 49,000+Signed contract every week
- $2.3BFreelancers earned on Upwork in 2020
Find the best freelance jobs
Growing your career is as easy as creating a free profile and finding work like this that fits your skills.
Trusted by