We are looking for a consultant who is expert in building ETL pipelines by extracting data from multiple source systems and write data into a relational database.
We are looking for consultant who has expertise in the following areas:
1. Expertise in Building ETL flows using Apache Spark and Python, Should have minimum 3 to 5 years of experience in Spark , Spark SQL and python.
2. Expertise in setting up Apache spark cluster with multiple nodes.
3. Expertise in performance tuning of ETL jobs on Spark clusters.
4. Expertise in writing Spark SQL and Python scripts for extracting data from relational databases and transforming the source data and loading the data into redshift.
5. Expertise in writing SQL queries