Big Data Developer job description template

An effective description can help you hire the best fit for your job. Check out our tips to provide details that skilled professionals are looking for.

Trusted by


Tips for Writing a Big Data Engineer Job Description

A big data engineer is a professional who is responsible for the management of data sets that are too big for traditional database systems to handle. They create, design, and implement data processing jobs in order to transform the data into a more usable format. They also ensure that the data is secure and complies with industry standards to protect the company’s information. 

Below, we will cover a sample job description, exploring the daily responsibilities and necessary qualifications for a big data engineer. 

The Job Overview

We are seeking a big data engineer to join our data analytics team. The successful candidate will be responsible for overseeing the creation and maintenance of our database infrastructure, including collecting and maintaining data, ensuring the integrity of our data, and creating and training data models.

Responsibilities

Below are some of the responsibilities of a big data engineer:

  • Design the architecture of our big data platform
  • Perform and oversee tasks such as writing scripts, calling APIs, web scraping, and writing SQL queries
  • Design and implement data stores that support the scalable processing and storage of our high-frequency data
  • Maintain our data pipeline
  • Customize and oversee integration tools, warehouses, databases, and analytical systems
  • Configure and provide availability for data-access tools used by all data scientists
Job Qualifications and Skill Sets

Below are the qualifications expected of a big data engineer:

  • 3 to 5 years of relevant data engineering experience
  • Bachelor’s degree or higher in computer science, data science, or a related field
  • Hands-on experience with data cleaning, visualization, and reporting
  • At least 2 years of relevant experience with real-time data stream platforms such as Kafka and Spark Streaming
  • Experience working in an agile environment
  • Familiarity with the Hadoop ecosystem
  • Experience with platforms such as MapReduce, Apache Cassandra, Hive, Presto, and HBase
  • Excellent analytical and problem-solving skills
  • Excellent communication and interpersonal skills

Big Data Developer Hiring Resources

Explore talent to hire
Learn about cost factors
ar_FreelancerAvatar_altText_292
ar_FreelancerAvatar_altText_292
ar_FreelancerAvatar_altText_292

4.8/5

Rating is 4.8 out of 5.

clients rate Big Data Developers based on 1K+ reviews

Hire Big Data Developers

Big Data Developers you can meet on Upwork

  • $35 hourly
    Gaurav S.
    • 4.8
    • (3 jobs)
    Delhi, NCT
    Featured Skill Big Data
    Data Warehousing
    Serverless Computing
    Deployment Automation
    AWS Systems Manager
    ETL
    Service Cloud Administration
    Golang
    JavaScript
    Python
    Node.js
    React Native
    Expert in High level and low level system design. DevOps and distributed systems are my passion. Other than software development, I also provide corporate training on new and upcoming technologies. When it comes to programming, I am a polyglot programmer with expertise in python and golang and a love hate relationship with Node and PHP
  • $50 hourly
    Mohamed S.
    • 5.0
    • (2 jobs)
    London, ENGLAND
    Featured Skill Big Data
    Data Mining
    Data Science
    Fraud Detection
    Data Analysis
    PySpark
    SAS
    Credit Scoring
    Apache Hadoop
    SQL
    Python
    As a seasoned Data Scientist and Technical Product Manager, I bring extensive experience in Financial Crime Risk and Credit Risk management, coupled with deep proficiency in Python, Spark, SAS (Base, EG, and DI Studio), Hadoop, and SQL. Transitioning into freelancing, I am eager to leverage my skills to contribute to diverse projects. While Upwork's guidelines restrict sharing direct links to external profiles, I am happy to provide a detailed portfolio from my LinkedIn upon request.
  • $40 hourly
    Feras A.
    • 5.0
    • (21 jobs)
    Oakville, ON
    Featured Skill Big Data
    Financial Statement
    Financial Analysis
    Tidyverse
    Data Analysis
    Microsoft Excel PowerPivot
    Data Modeling
    Automation
    R
    Power Query
    Microsoft Power BI
    Data Visualization
    SQL
    Intuit QuickBooks
    With 7+ years of experience in Power BI development, I help businesses transform raw data into actionable insights through scalable dashboards, automated workflows, and financial reporting solutions. I specialize in turning messy row data into interactive, drillable dashboards that save teams hours of manual work while delivering clarity, accuracy, and efficiency. My approach blends technical expertise with a strong focus on finance and process automation—so your data doesn’t just look good, it drives decisions. Core Expertise: 🔹 Power BI Development - Data wrangling (Excel/CSV/TXT/PDF → clean, structured datasets) - Data extraction , automation and integration using R programming , Power Automate and Zapier - Optimized star-schema data models for performance - Advanced DAX (time intelligence, dynamic aggregations, allocations) - Publishing & tenant administration (gateways, security, governance) 🔹 Projects Completed - Direct to customer sales dashboard - Amazon store analysis - RFM Analysis - Cohort Analysis - Profit & Loss (P&L) statements (monthly/quarterly/yearly) - Budget vs. Actuals, YoY, QoQ, MoM comparisons - Multi-entity/cost center breakdowns - Supplier/customer profitability analysis - Multi-currency and fiscal/calendar year support 🔹 Power Platform Solutions -Automating manual tasks (PDF/Excel conversions, file consolidation) -Scheduled data refreshes & seamless integrations Ongoing support & maintenance for reports and datasets Why Work With Me? ✅ Data Clarity: I turn scattered files into clear, interactive dashboards. ✅ Scalable Processes: From inconsistent CSVs to automated reporting pipelines. ✅ Ongoing Partnership: Training, documentation, and long-term support included. 📊 Let’s build reports that save time, improve decisions, and eliminate headaches.
Want to browse more talent? Sign up

Join the world’s work marketplace

Find Talent

Post a job to interview and hire great talent.

Hire Talent
Find Work

Find work you love with like-minded clients.

Find Work