Hire the best Apache Spark Engineers in Brazil

Check out Apache Spark Engineers in Brazil with the skills you need for your next job.
  • $55 hourly
    I focus on data engineering, software engineering, ETL/ELT, SQL reporting, high-volume data flows, and development of robust APIs using Java and Scala. I prioritize three key elements: reliability, efficiency, and simplicity. I hold a Bachelor's degree in Information Systems from Pontifícia Universidade Católica do Rio Grande do Sul as well as graduate degrees in Software Engineering from Infnet/FGV and Data Science (Big Data) from IGTI. In addition to my academic qualifications I have acquired a set of certifications: - Databricks Certified Data Engineer Professional - AWS Certified Solutions Architect – Associate - Databricks Certified Associate Developer for Apache Spark 3.0 - AWS Certified Cloud Practitioner - Databricks Certified Data Engineer Associate - Academy Accreditation - Databricks Lakehouse Fundamentals - Microsoft Certified: Azure Data Engineer Associate - Microsoft Certified: DP-200 Implementing an Azure Data Solution - Microsoft Certified: DP-201 Designing an Azure Data Solution - Microsoft Certified: Azure Data Fundamentals - Microsoft Certified: Azure Fundamentals - Cloudera CCA Spark and Hadoop Developer - Oracle Certified Professional, Java SE 6 Programmer My professional journey has been marked by a deep involvement in the world of Big Data solutions. I've fine-tuned my skills with Apache Spark, Apache Flink, Hadoop, and a range of associated technologies such as HBase, Cassandra, MongoDB, Ignite, MapReduce, Apache Pig, Apache Crunch and RHadoop. Initially, I worked extensively with on-premise environments but over the past five years my focus has shifted predominantly to cloud based platforms. I've dedicated over two years to mastering Azure and I’m currently immersed in AWS. I have a great experience with Linux environments as well as strong knowledge in programming languages like Scala (8+ years) and Java (15+ years). In my earlier career phases, I had experience working with Java web applications and Java EE applications, primarily leveraging the WebLogic application server and databases like SQL Server, MySQL, and Oracle.
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Scala
    Apache Solr
    Apache Kafka
    Bash Programming
    Elasticsearch
    Java
    Progress Chef
    Apache Flink
    Apache HBase
    Apache Hadoop
    MapReduce
    MongoDB
    Docker
  • $60 hourly
    Senior data engineer with product analytics and data science background, having worked with Fortune 500 companies (Procter & Gamble, Merck, Anheuser-Busch), as well as top-notch data-driven startups. Skilled in translating complex business problems into data solutions, designing data pipelines, providing high-quality data for data-driven insights and decision-making, as well as building KPIs, conducting statistical analyses, and creating impactful visualizations. Problems I'm good at solving: • Data Warehousing and Analytics • ETL / ELT data pipelines • SQL query tuning • Data Modeling and Database Design • Reporting • Data Analysis • Data Cleaning, Pre-Processing • Data Visualization • NLP problems I have a bachelor's in engineering from the top LATAM university (Universidade de São Paulo) with a track record of supporting organizations across various industries, including remote hiring, real estate, and consumer goods. Skills and Expertise ✅ SQL ✅ Python Databases ✅ Snowflake ✅ Redshift ✅ BigQuery ✅ Athena ✅ Trino ✅ Postgres ✅ MySQL Big Data Cloud Technologies ✅ Amazon Web Services – AWS Certified (Redshift, Athena, S3, Lambda, Glue ...) ✅ Google Cloud Platform Other Data Engineering Tools ✅ dbt ✅ Airflow ✅ Fivetran ✅ Git, Gitlab, and Github ✅ Rundeck ✅ Docker Data Visualization ✅ Looker (LookML Expert) ✅ PowerBI ✅ Metabase ✅ Looker Studio (Data Studio) Data Science and Machine Learning ✅ Sci-kit learn, pandas, etc ✅ NLP analysis ✅ Spark ✅ Databricks ✅ Hex ✅ Jupyter Notebooks User Behavioral Analytics ✅ Snowplow ✅ Indicative ✅ Heap ✅ Amplitude ✅ Google Analytics
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Snowflake
    dbt
    Amazon Redshift
    BigQuery
    ETL Pipeline
    Looker
    Data Analysis
    Data Modeling
    Data Visualization
    Business Intelligence
    Data Warehousing
    Machine Learning
    Python
    SQL
  • $65 hourly
    With over 15 years of experience as an IT professional, I bring a wealth of expertise as a Data Architect and Engineer. My solid Cloud Infrastructure and Database Administration skills enable me to deliver exceptional results. I thrive in leadership roles, excel when working independently, and actively contribute as a team player. My analytical, design, and problem-solving abilities set me apart, and I am committed to upholding the highest quality standards. My strong communication skills and belief in the transformative power of Scrum and Agile methodologies drive my effectiveness in project execution and team collaboration.
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Cloud Computing
    Scala
    Apache Kafka
    PostgreSQL
    Data Lake
    Docker
    ELK Stack
    Amazon EC2 Spot
    Oracle Database
    Terraform
    AWS Glue
    Microsoft SQL Server
    Amazon Athena
    PySpark
    Apache Superset
    Apache Airflow
  • $65 hourly
    Hello There! I'm a data engineer & data architect that focus on building scalable, reliable, and world class's Data solutions. I have a huge experience in Data Driven Development, Data Integrations, Data Modeling, Data Engineering, Data Architecture and AWS Data Services. My main focus is Data Engineering, Data Architecture ( specially Data Mesh ) and Cloud Computing. Talk Data with me, and let's change the world using DATA! I'm 3x AWS certified: # AWS Data Analytics Specialty # AWS Database Specialty # AWS Solutions Architect Associate
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Google Cloud Platform
    Amazon ECS
    Amazon DynamoDB
    Amazon Athena
    Amazon API Gateway
    Amazon RDS
    Amazon Redshift
    AWS Fargate
    AWS CloudFormation
    Python
    Data Engineering
    Apache Kafka
    AWS Glue
    SQL
  • $40 hourly
    Developing data-driven solutions for business applications and impacts. Particularly enjoying the process of identifying problems, discovering insights, validation through data, and implement solutions. Especially interested when they have positive influence on business and users
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Apache Airflow
    Google Cloud Platform
    AWS Lambda
    AWS Glue
    PySpark
    Plotly
    Terraform
    Amazon Redshift
    SQL
    Microsoft Power BI
    pandas
    Python
  • $38 hourly
    I’m a developer with experience in building websites for small and medium-sized businesses. Whether you’re trying to win work, list your services or even create a whole online store – I can help! I’m experienced in HTML and CSS3, PHP, jQuery, WordPress, and SEO I’ll fully project manage your brief from start to finish Regular communication is really important to me, so let’s keep in touch!
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Functional Programming
    Flutter
    Front-End Development
    Mobile App Development
    React
    API Development
    Apache Kafka
    Kotlin
    TypeScript
    Scala
    Golang
    Spring Boot
    Kubernetes
    Play Framework
  • $50 hourly
    Senior Backend Developer & Data Scientist: Turning Data into Decisions! 🤖 Hey there! I'm a seasoned Data Scientist and Senior Backend Developer with a knack for transforming complex data into actionable insights—and I promise not to bore you with too many numbers (unless you're into that)! With experience at top companies like Dell Technologies and Axur, I've mastered the art of using Large Language Models (LLMs) to boost data security and create predictive magic. Here's what I bring to the table: - AI and Machine Learning Wizardry: Crafting models that classify, cluster, and forecast like a pro using Python, SQL, and Apache Spark. - Senior Backend Development: Building robust, scalable backend systems with Django and Flask, ensuring your data flows smoothly and securely. - Data Analysis & NLP: Extracting insights from data is my jam, and I love making sense of the chaos with natural language processing. - Leadership & Mentoring: Leading teams to victory and mentoring future data wizards—no capes required. Whether you're looking to enhance your data capabilities, need a strategic partner for your next project, or just want to chat about the latest in AI and backend development, I'm here to help! Let's turn your data dreams into reality—one byte at a time! 🚀
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Amazon EC2
    Amazon S3
    AWS Amplify
    Amazon Web Services
    AWS Lambda
    Django
    Flask
    Data Analysis
    Deep Learning
    Large Language Model
    Data Mining
    Machine Learning
    SQL
    Python
  • $20 hourly
    Experienced in developing data science and analysis using mainly the python programming language and a little bit of scala. Knowledge of main technologies and frameworks used for that purpose, such as pandas, keras, tensorflow, spark and sklearn. Also experienced in building back end applications with async frameworks and both relation and non-relational databases.
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    PySpark
    Analytics
    Mathematics
    Microsoft Power BI
    Database
    ETL Pipeline
    Microsoft Azure
    Deep Neural Network
    SQL
    Machine Learning
    Data Science
    Python
    Deep Learning
    Machine Learning Model
  • $50 hourly
    I am a Data Engineer who loves the challanges of both the Operational and Analytical areas. Previously, I worked as a Data Science Analyst, focused on the Data Governance Journey in the biggest bank of Latin America. Furthermore, I am also a passionate DBA and SRE enthusiast. I graduated in Materials Engineering at the Federal University of São Carlos (UFSCar) and I have a MBA degree in Data Engineering at FIAP.
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Data Lake
    Terraform
    Big Data
    Microsoft Azure SQL Database
    Microsoft Azure
  • $25 hourly
    I am Christian Basilio, currently completing my degree in Public Management for Economic and Social Development at the Federal University of Rio de Janeiro (UFRJ). Throughout my journey, I have had the opportunity to work in various social movements, as well as in public bodies within the executive scope. My focus of study is in the field of data and geoprocessing, and I have worked in both the public and private sectors in this area. I have experience in startups and private technology companies, using tools such as Power BI, Tableau, Looker, Python, R, SQL, among others, for data analysis. In addition, I have experience in analytical work focused on investigative aspects, both in data journalism and for market and social media analysis. My experience ranges from data analysis for the private sector to social investigative journalism.
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Python Script
    MongoDB
    PySpark
    Apache Spark MLlib
    Databricks Platform
    SQL Programming
    Business Intelligence
    Microsoft Power BI
    Python
    R
    SQL
    Tableau
  • $35 hourly
    Back-end software engineer, specialized in Java. Bachelor's in Computer Science, with experience with mentoring and researching. Two years of experience developing and enhancing scalable and performative applications. Passionate about Back-End development, open source, as well as Developer Experience and Developer Productivity. Technical Habilities: * Java; * Scala | Apache Spark; * Kotlin | Ktorm; * C#; * Python; * Go | Golang | GORM; * Bash | Scripting; * Algorithms and Data Structures | Object Oriented Programming; * Javascript | Express | React | Node.js; * Git | Github; * Microsoft Azure; * Databricks; * Docker; * MySQL; * PostgreSQL; * gRPC; * REST APIs; Tools: * Github; * Visual Studio Code; * Intellij; * Linux; Personal Qualities: * Fast learner; * Time management; * Strong Problem Solving; * Good communication; * Good team player; * Mentoring | Teaching;
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    React
    Linux
    Microsoft Azure
    JavaScript
    Spring Boot
    C#
    SQL
    Git
    Docker
    Kotlin
    Golang
    Scala
    Python
    Java
  • $50 hourly
    I'm a Senior Data Engineer, Data Architect, and AWS Solutions Architect Associate with over 10 years of experience helping businesses unlock the power of their data. If you're looking to build scalable data solutions, streamline your data processes, or develop custom software, let's connect! I look forward to helping your business grow with data-driven solutions and innovative software. I'm passionate about creating scalable, efficient data solutions and custom software to solve complex challenges. Whether you need to build a robust cloud data architecture, streamline your ETL pipelines, or develop tailored software applications, I can deliver high-quality results that drive growth. What I Do Best: - Data Engineering & ETL Pipelines: I design and optimize data pipelines that automate data collection, transformation, and analysis, enabling faster and more accurate decision-making. - Cloud Data Solutions (AWS, Azure, GCP): I have extensive experience migrating and managing data on cloud platforms, ensuring secure, scalable, and cost-effective solutions. - Data Architecture: I create data architectures that provide a solid foundation for your business, ensuring data consistency, quality, and availability. - Custom Software Development: I develop tailored software solutions that integrate seamlessly with your business processes, whether you need microservices, APIs, or web applications. - Big Data & Analytics: I help businesses utilize big data tools like Hadoop and Spark to derive actionable insights and stay ahead of the competition. - Infrastructure as Code (IaC): I automate infrastructure management using tools like Terraform, ensuring efficient and consistent deployment and scaling of your cloud environments. Why Work With Me? - Proven Expertise: I've successfully delivered impactful solutions across industries, working with major clients like Podchaser and Accenture. - Problem-Solver: I thrive on tackling complex challenges and delivering innovative solutions that align with your business goals. - Collaborative Approach: I work closely with my clients to ensure the solution fits their needs. Communication and transparency are key in every project I take on. - Full-Service Solutions: With a background in software development and data engineering, I provide end-to-end services, from architecture design to deployment.
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Big Data
    Database Administration
    Data Modeling
    Python
    CI/CD
    DevOps
    Data Lake
    Data Engineering
    Terraform
    Apache Hadoop
    Cloud Security
    Cloud Architecture
    SQL
    ETL Pipeline
  • $45 hourly
    Tenho mais de 6 anos de experiência em engenharia de dados e analytics. Atualmente, sou Engenheiro de Dados onde coordeno uma equipe multidisciplinar focada em engenharia de dados, governança, arquitetura de soluções e visualização de dados. Programo em Python e PySpark e sou especialista em Databricks, Data Factory e Azure Functions. Minha atuação vai desde a definição de estratégias até a implementação de soluções técnicas, incluindo o desenvolvimento de PoCs em novas tecnologias para resolver problemas de negocio. Também atuo na definição de arquiteturas de MLOps, sempre buscando gerar valor e resultados significativos para a empresa. Algumas entregas relevantes: Transformação da Área de Analytics: Liderei a reestruturação completa da área, e construindo a infraestrutura, padronizando desenvolvimentos e dados no Data Lake e implementando processos de implantação com CI/CD. Essa transformação resultou em uma operação mais eficiente e organizada.
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Data Engineering
    Data Analytics & Visualization Software
    ETL
    AWS CodePipeline
    Data Analysis
    Artificial Intelligence
    Data Extraction
    Azure OpenAI Service
    Microsoft Azure
    Azure DevOps
    ETL Pipeline
    Python
    PySpark
    Databricks Platform
  • $40 hourly
    I am an analytic engineer, I work with scala, spark, python, sql. I have a lit of experience in ETL. I am good with dashboards by using Power BI, looker and tableau. I used to be a DBA (Database Administrator) as well, I used to work with Oracle, Microsoft Sql Server, and Sybase. I have many skills in microsoft office (word, excel, access), sharepoint, jquery, html, css, java, java script. I am very good with license, billing and budget as well.
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Microsoft Excel PowerPivot
    Oracle Database Administration
    Microsoft SharePoint Administration
    Oracle Data Guard
    Oracle Upgrade
    Oracle PLSQL
    Microsoft Azure SQL Database
    Microsoft Excel
    Tableau
    Looker
    MySQL
    Oracle
    Microsoft Power BI
    Snowflake
    SQL Server Integration Services
    Microsoft SQL Server
    SQL
    Python
    Scala
  • $12 hourly
    Software Developer holding a bachelor's degree in Computer Science along with graduate studies in Big Data and Data Science. With over 12 years of extensive experience in software development, I've specialized in processing large volumes of data for the last 8 years, utilizing technologies such as Hadoop, Spark, and Scala.
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Progress Chef
    Python
    Snowflake
    Databricks Platform
    Scala
    DevOps
    Big Data
    Apache Hadoop
    Java
    SQL
  • $100 hourly
    In the last 4 years, I have developed data and cloud solutions for top companies in the finance, marketing and education industries. I'm experienced in data engineering, cloud engineering and data analytics. The tech I have worked with the most is Python, AWS, SQL, Spark and Terraform, but I always love to check out cool new stuff! I have designed, architected and developed large scale data lakes, data pipelines for analytics, ETL, ELT and backend services, following DevOps and Software Engineering best practices. I will help your business with freelance and consulting services such as: • design and development of data lakes • adoption and deployment of Databricks • architecture of cloud solutions on AWS • infrastructure as code (IaC) • CI/CD pipelines • DevOps best practices • code review • data analytics and statistical modeling • team building • general consulting on data, cloud and software engineering I'm always eager to hear about interesting and challenging projects!
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Data Analytics
    Apache Airflow
    Big Data
    Data Engineering
    Databricks Platform
    Data Science
    Amazon Redshift
    Software Consultation
    Software Architecture & Design
    AWS Glue
    AWS Lambda
    Python
    SQL
  • $27 hourly
    Software Developer with 10 years experience. Worked in several areas such as web development, payment systems, industrial automation and mobile development. Played several roles on software development process (design, programming and planning). Lately working as backend and big data developer for e- commerce applications. Strong background on both R&D projects and real life projects and products. Highly motivated professional with experience working with multicultural and distributed teams around the world, familiar to work under every kind of circumstances and always commited to deliver the best results.
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    PostgreSQL Programming
    Amazon S3
    MySQL Programming
    AWS Lambda
    Node.js
    PHP
    Scala
    Golang
  • $20 hourly
    I'm a data engineer and software developer with 10 years of experience. I've been working with data engineering (extracting, loading, wrangling), and developing data pipelines using Python, Spark, Azure Data Factory, Oracle Data Integrator and SQL.
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Git
    Azure DevOps
    Databricks Platform
    Android
    Java
    Android App Development
    Oracle Database
    App Development
    Python
    ETL
    ETL Pipeline
  • $40 hourly
    I have more than 5 years of experience in Full Stack Engineering encompassing projects in cloud systems, web applications, mobile applications, data mining, and automation. I love to write data scrapers for online data mining applications and build a solution from ingestion to data visualization. My main languages are Python and JavaScript but tec is a way to achieve a solution so let's buid it together.
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Apache NiFi
    Apache Hadoop
    Apache Airflow
    Big Data File Format
    Python Script
    Big Data
    Business Intelligence
    Python
    Data Visualization
    Microsoft Power BI
  • $25 hourly
    Profile Graduated in Computer Engineering, experienced in programming with Python, extensive know-how in Apache Spark, Data Science, SQL, ETL, Business Intelligence tools. Also qualified video editing and for translations, with more than 10 years of english fluency.
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Video Editing
    Translation
    PySpark
    Python Script
    Business Intelligence
    SQL
    Microsoft Power BI
    pandas
    Python
  • $5 hourly
    Data professional with 4.5 years of experience delivering data-driven solutions and leading data engineering projects. Skilled in building pipelines, reducing costs, and fostering data-driven cultures using tools like Snowflake, Databricks, Apache Spark, and Tableau. Proficient in Power BI, SQL, Python, and Big Data processing, with expertise across AWS, Azure, and GCP cloud platforms. I specialize in scalable analytics and optimizing operations for multinational projects. Ready to unlock the full potential of your data and drive growth? Let’s transform your data into actionable insights that lead to success.Let’s make your next project a success!
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Cloud Computing
    Apache Airflow
    Snowflake
    Amazon Web Services
    ETL Pipeline
    Database
    ETL
    SQL Programming
    Power Query
    Data Analysis
    Python
    Microsoft Power BI
    Microsoft Excel
    Data Visualization
  • $15 hourly
    Hello! My name is Wylliams, I hold a degree in IT Management, and I'm currently preparing to pursue a post-graduate degree in Big Data. I work as a Data Engineer and also have experience as a Data Analyst. I specialize in creating ETL/ELT orchestration projects and continuously seek opportunities to optimize automations. I thrive on hands-on work! I have a strong passion for learning and regularly explore new tools and technologies to broaden my expertise. Alongside my Azure certifications, I also have hands-on experience with other cloud platforms like AWS, Huawei, and Tencent. Certifications: DP-203 Azure Data Engineer DP-900 Azure Data Fundamentals AZ-900 Azure Fundamentals PL-900 Power Platform Skills: Programming Languages: Python, Spark (Pyspark), PHP Databases: MySQL, PostgreSQL, SQL Server, MongoDB, Snowflake Big Data Tools: Databricks, Azure Cloud, Power BI, Power Platform, AWS, GCP I'm enthusiastic about leveraging my skills and knowledge to drive impactful data solutions. Let's connect and explore opportunities together!
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    MySQL
    MongoDB
    Big Data
    PostgreSQL
    MySQL Programming
    Database
    Azure Cosmos DB
    Apache Spark MLlib
    ETL
    Cosmos OS
    Microsoft Azure
    Amazon Web Services
    Databricks Platform
    ETL Pipeline
  • $15 hourly
    Passionate Data Scientist and AI professional with expertise in Python, Machine Learning, Deep Learning, NLP, Databases, and BigQuery. Skilled in promoting fairness and inclusiveness in language models. Fluent in English and Portuguese. Skills * Python, Matlab, C/C++, Linux * SQL * Machine Learning, Deep Learning, Data Analysis * Keras, PyTorch, TensorFlow, Scikit-Learn, Numpy * Natural Language Processing (NLP) * BigQuery, Spark
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Geospatial Data
    Microsoft Power BI
    Seismic
    Artificial Intelligence
    PySpark
    BigQuery
    Python
    Apache Spark MLlib
    Machine Learning
    Machine Learning Model
  • $25 hourly
    Certified as a Google Professional Data Engineer, I possess advanced expertise in designing, implementing, and managing secure, scalable, and reliable data solutions on Google Cloud Platform (GCP). My credentials extend to cloud certifications in Azure and ongoing pursuit of AWS certification, underscoring my commitment to staying at the forefront of cloud technologies. I am proficient in key technologies and methodologies, including Python, Django, Flask, Spark, GIT, ETL pipelines, Databricks, and a wide array of relational and non-relational databases such as Amazon RDS, Microsoft SQL Server, MongoDB, Cosmos DB, Azure Synapse, and BigQuery. My experience includes structuring data lakes and warehouses, managing complex data pipelines, implementing data governance frameworks, creating insightful dashboards, and leading data squad teams. Equipped with a solid foundation in data modeling, Agile methodologies, API development, and data infrastructure management, I excel in transforming business needs into scalable, efficient, and innovative data solutions.
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    GitHub
    FastAPI
    Data Warehousing & ETL Software
    ETL Pipeline
    Data Analytics
    Apache Airflow
    PySpark
    Python
    BigQuery
    Databricks Platform
    Microsoft Azure
    Amazon Web Services
    Google Cloud Platform
    Big Data
  • $10 hourly
    I consider myself a very interested, proactive, helpful, enthusiastic, organized, and, above all, curious person. I enjoy situations that make me "think outside the box" because I know they will make me more experienced in the future and I will have the opportunity to grow professionally and intellectually. In addition to this, I value good rapport and teamwork.
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Apache Hadoop
    Microsoft Azure SQL Database
    Databricks MLflow
    PySpark
    Apache Airflow
    AWS Glue
    Python Script
    Data Analytics & Visualization Software
    Data Engineering
    Data Analysis
    Databricks Platform
    Big Data
    Microsoft Azure
  • $15 hourly
    I'm a Data Engineer with experience in Big Data and BI solutions. Proven ability to migrate data warehouses to cloud-based data lakes using Azure and Databricks. Skilled in ETL/API integrations, data warehousing, and developing data pipelines (Python, Spark, Java) for cloud environments. Experience with pipeline orchestration tools (Rundeck, Airflow) and streaming data ingestion (Apache NiFi). Certified in Azure Fundamentals, Azure Data Fundamentals, and Databricks Data Engineer Associate.
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Data Modeling
    Python
    SQL
    Data Engineering
    ETL Pipeline
    Microsoft Azure
    Apache NiFi
    PySpark
    Databricks Platform
  • $27 hourly
    I’m a data engineer specializing in building efficient, scalable solutions with Azure and Python. Whether you need to streamline your data pipelines, harness the power of big data with Apache Spark, or drive insights through Power BI, I can help transform your data into actionable business outcomes. Let’s take your data operations to the next level. Azure Data Engineering: I design and implement scalable data pipelines using Azure services like Data Factory, Databricks, and Azure SQL, ensuring seamless data flow and integration. Python Expertise: With strong skills in Python, I automate data processes, build efficient scripts, and leverage libraries for data analysis and transformation. Big Data & Analytics: I specialize in Apache Spark, Hive, and MongoDB, working with large datasets to drive insightful business decisions through Power BI and Azure Data Lakes. Cloud Solutions: From AWS to Microsoft Azure, I have experience in cloud environments, optimizing infrastructure for data engineering tasks. Leadership & Collaboration: As a team leader, I guide projects from conception to execution, ensuring high performance, collaboration, and data-driven results.
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Java
    SQL
    Python
    Databricks Platform
    AWS CloudFormation
    Microsoft SQL Server
    Microsoft Power BI
    Microsoft Azure
    Azure DevOps
    Microsoft Azure SQL Database
    Data Lake
    MongoDB
    Data Analysis
    ETL
  • Want to browse more freelancers?
    Sign up

How hiring on Upwork works

1. Post a job

Tell us what you need. Provide as many details as possible, but don’t worry about getting it perfect.

2. Talent comes to you

Get qualified proposals within 24 hours, and meet the candidates you’re excited about. Hire as soon as you’re ready.

3. Collaborate easily

Use Upwork to chat or video call, share files, and track project progress right from the app.

4. Payment simplified

Receive invoices and make payments through Upwork. Only pay for work you authorize.