Big Data Developer job description template

An effective description can help you hire the best fit for your job. Check out our tips to provide details that skilled professionals are looking for.

Trusted by


Tips for Writing a Big Data Engineer Job Description

A big data engineer is a professional who is responsible for the management of data sets that are too big for traditional database systems to handle. They create, design, and implement data processing jobs in order to transform the data into a more usable format. They also ensure that the data is secure and complies with industry standards to protect the companyโ€™s information. 

Below, we will cover a sample job description, exploring the daily responsibilities and necessary qualifications for a big data engineer. 

The Job Overview

We are seeking a big data engineer to join our data analytics team. The successful candidate will be responsible for overseeing the creation and maintenance of our database infrastructure, including collecting and maintaining data, ensuring the integrity of our data, and creating and training data models.

Responsibilities

Below are some of the responsibilities of a big data engineer:

  • Design the architecture of our big data platform
  • Perform and oversee tasks such as writing scripts, calling APIs, web scraping, and writing SQL queries
  • Design and implement data stores that support the scalable processing and storage of our high-frequency data
  • Maintain our data pipeline
  • Customize and oversee integration tools, warehouses, databases, and analytical systems
  • Configure and provide availability for data-access tools used by all data scientists
Job Qualifications and Skill Sets

Below are the qualifications expected of a big data engineer:

  • 3 to 5 years of relevant data engineering experience
  • Bachelorโ€™s degree or higher in computer science, data science, or a related field
  • Hands-on experience with data cleaning, visualization, and reporting
  • At least 2 years of relevant experience with real-time data stream platforms such as Kafka and Spark Streaming
  • Experience working in an agile environment
  • Familiarity with the Hadoop ecosystem
  • Experience with platforms such as MapReduce, Apache Cassandra, Hive, Presto, and HBase
  • Excellent analytical and problem-solving skills
  • Excellent communication and interpersonal skills

Big Data Developer Hiring Resources

Explore talent to hire
Learn about cost factors
ar_FreelancerAvatar_altText_292
ar_FreelancerAvatar_altText_292
ar_FreelancerAvatar_altText_292

4.8/5

Rating is 4.8 out of 5.

clients rate Big Data Developers based on 1K+ reviews

Hire Big Data Developers

Big Data Developers you can meet on Upwork

  • $100 hourly
    Amar K.
    • 5.0
    • (32 jobs)
    Bengaluru, KA
    Featured Skill Big Data
    API Development
    Flask
    Google App Engine
    Software Development
    Google Cloud Platform
    Amazon Web Services
    BigQuery
    PySpark
    Apache Airflow
    Apache Spark
    Data Engineering
    SQL
    Python
    Java
    I'm a software architect and data engineering expert with over 10 years of proven mastery in building scalable, secure, and high-performance backend and data-driven systems. My deep specialization includes robust data pipelines, sophisticated backend architectures, and strategic cloud solutions. I've successfully led and delivered critical technology initiatives for global industry leaders such as Goldman Sachs, Morgan Stanley, KPMG, and Oracle. Key Competencies: - Data Engineering & Databases โ€“ SQL (PySpark, Meltano, Oracle, Postgres, MySQL, Redshift, SQL Server, Snowflake, BigQuery), Data Warehousing, query optimization, Kafka, Spark, Airflow, DBT. - Cloud Platforms โ€“ AWS, GCP, Azure, OCI, Kubernetes, Docker, Terraform, CI/CD automation. - Backend Development โ€“ Java, Python (Django, Flask), modern web frameworks, microservices, API integrations. - Frontend Development โ€“ React.js, JSP/Servlets, HTML5, JavaScript (ES6+). Strategic & Technical Partnership: I go beyond simply delivering codeโ€”I partner with you to deeply understand your business challenges, advise on industry best practices, and implement tailored, future-proof solutions that optimize operations, reduce costs, and accelerate growth. Client Feedback: โ€œAmar consistently exceeds expectationsโ€”he doesn't just deliver technical solutions, he delivers strategic insights that drive real business results.โ€ โ€œHighly intelligent and experienced individual with deep knowledge across data engineering.โ€ โ€œTop performerโ€”will be working with Amar long time I hope!โ€ If you're looking for a seasoned expert who combines technical mastery and strategic vision, let's connect to discuss how I can help you build impactful, business-driven solutions.
  • $35 hourly
    Gaurav S.
    • 4.8
    • (3 jobs)
    Delhi, NCT
    Featured Skill Big Data
    Data Warehousing
    Serverless Computing
    Deployment Automation
    AWS Systems Manager
    ETL
    Service Cloud Administration
    Golang
    JavaScript
    Python
    Node.js
    React Native
    Expert in High level and low level system design. DevOps and distributed systems are my passion. Other than software development, I also provide corporate training on new and upcoming technologies. When it comes to programming, I am a polyglot programmer with expertise in python and golang and a love hate relationship with Node and PHP
  • $120 hourly
    Ioannis P.
    • 5.0
    • (8 jobs)
    Athens, I
    Featured Skill Big Data
    Cloud Computing
    SQL
    Machine Learning
    Data Science
    Deep Learning
    PyTorch
    Python
    I am an experienced Machine Learning Engineer, skillful in Python, Big Data, Deep Learning and Earth Observation. As a researcher, I am very familiar with the state-of-the-art and can provide in-depth analyses. I have completed several projects including time-series forecasting, anomaly detection, computer vision, remote sensing & satellite data analysis. I have implemented and deployed several Machine Learning models that are currently used in production: - An NLP model that identifies cardiology-related scientific abstracts and categorizes them into more specific categories. - A wildfire forecasting model that uses weather and satellite data. - An anomaly detection model that produces risk scores for vulnerable servers. I am excited to solve diverse problems as a freelance Data Scientist and waiting to tackle the next challenge.
Want to browse more talent? Sign up

Join the worldโ€™s work marketplace

Find Talent

Post a job to interview and hire great talent.

Hire Talent
Find Work

Find work you love with like-minded clients.

Find Work