Upwork home
Sign up
Secondary

Hire the best Pyspark Developers in the United States

Check out Pyspark Developers in the United States with the skills you need for your next job.
Get started
  • $80 hourly
    • 5.0/5
    • (13 jobs)
    As a data architect, I have accumulated over 20 years of experience collaborating with various consulting companies. My expertise spans across multiple cloud platforms, including AWS (Glue, Lambda, Redshift, Athena), GCP (BigQuery, BigTable, GCS), Azure (ADF, Synapse Analytics, data lake, Analysis service), as well as Snowflake and DBT.. Additionally, I am skilled in Python programming and Airflow and have extensive knowledge of Kafka and Kinesis. Throughout my career, I have designed numerous dashboards utilizing Power BI, Tableau, and QlikSense. In addition, I have successfully developed CI/CD processes utilizing GitHub and Jenkins. I have gained valuable experience working with challenging clients across diverse industries, including banking, healthcare, energy, and marketing.
    Featured Skill Pyspark
    AWS Lambda
    Jenkins
    dbt
    Kafa
    Snowflake
    BigQuery
    Amazon Redshift
    PySpark
    Microsoft Azure
    AWS Glue
    Microsoft Azure SQL Database
    Microsoft Power BI
    Business Intelligence
    Qlik Sense
    Python
  • $60 hourly
    • 5.0/5
    • (4 jobs)
    I am a Data Analytics Engineer with a passion for turning raw data into actionable insights that drive business success. My expertise lies in designing and implementing scalable data pipelines, ensuring clean, reliable, and accessible datasets for analytics and decision-making. With a strong background in SQL, Python, and ETL processes, I specialize in building efficient data models that support real-time reporting and analytics. My proficiency in data visualization tools such as Power BI, Looker, and Tableau allows me to create compelling, interactive dashboards that help businesses uncover trends, monitor KPIs, and make data-driven decisions with confidence. Key Skills & Expertise: ✅ Data Pipeline Development – Extract, transform, and load (ETL) data efficiently from multiple sources. ✅ Data Modeling & Warehousing – Optimize relational and dimensional data models for performance. ✅ Visualization & Reporting – Build dynamic dashboards and reports in Power BI, Looker, and Tableau. ✅ SQL & Python Expertise – Write complex queries, automate processes, and analyze large datasets. ✅ Data Quality & Governance – Ensure accuracy, consistency, and security in data handling. ✅ Collaboration & Communication – Work closely with stakeholders to translate business needs into analytical solutions. My goal is to empower businesses with high-quality, meaningful data insights that enhance operations, optimize decision-making, and boost efficiency. Whether it’s streamlining data workflows, creating real-time dashboards, or implementing advanced analytics solutions, I’m here to turn your data into a strategic asset. Let’s connect and make your data work for you! 🚀
    Featured Skill Pyspark
    PySpark
    Apache Hadoop
    Informatica
    Keras
    Snowflake
    BigQuery
    MySQL
    Machine Learning
    Plotly
    Looker Studio
    Tableau
    Microsoft Power BI
    TensorFlow
    R
    Python
  • $50 hourly
    • 5.0/5
    • (4 jobs)
    I'm a engineer experienced in build data and software products. Whether you're trying to build data pipelines, CI/CD workflows, or create new websites, i can help. * Experienced in databricks and spark - building end to end data products including data pipeline, CI/CD workflow, operational tools. * Experienced in build websites or applications - especially in Java, Python
    Featured Skill Pyspark
    Ecommerce Website
    Script
    Scripting
    Web Application
    React
    Scala
    CI/CD
    Java
    Python
    Git
    Apache Airflow
    Databricks Platform
    Big Data
    Web Development
    PySpark
  • $60 hourly
    • 4.8/5
    • (74 jobs)
    Are you in search of a highly skilled professional to elevate your business operations and enhance customer interactions through intelligent chatbot solutions? Look no further! I am a dedicated LLM Engineer and AI enthusiast specializing in Python API integration, GPT (OpenAI Assistant, ChatGPT, GPT 4, Whisper), and various OpenAI solutions and LLMs in general! With a robust background in Artificial Intelligence and NLP.I have cultivated expertise in constructing chatbots using Langchain, OpenAI API, OpenAI Assistants, and open-source models on HuggingFace. My services cover a broad spectrum of chatbot development needs: 🌟 Are you encountering challenges in developing chatbots using Langchain, OpenAI API, OpenAI Assistants, or open-source models on HuggingFace? 🌟 Do you require expert assistance in seamlessly integrating chatbot functionalities into your website or application? 🌟 Are you aiming to create a personalized chatbot that effortlessly comprehends and responds to user queries? 🌟 Do you wish to optimize the performance and accuracy of your existing chatbot solution? 🌟 Have you come across any errors or issues in your chatbot's functionality that demand immediate troubleshooting? 🌟 Are you interested in creating an agent with various features (web, database search, vector store retrieval, ...)? Rest assured, I am here to provide comprehensive solutions and steadfast support for all your chatbot development needs. Why choose me? As a Creative Python API Integration and Python solution expert, I bring a wealth of experience in working with OpenAI LLMs (GPT 4, OpenAI Assistants, ChatGPT), Langchain, Pinecone, Weaviate, FAISS, and other LLM / VectorStores. Here are the technologies I specialize in: ✔️ LLM Agents (Access to Web, APIs, Databases, ...) ✔️ GPT Apps development using Python, Langchain, LLM (OpenAI Assistants / ChatGPT / GPT 4 / HuggingFace models), VectorStores (Pinecone, Weaviate, FAISS, ...) ✔️ Automation and integration using ChatGPT API. ✔️ Python solutions using ChatGPT API, Whisper API, GPT 4 model, and other OpenAI API services. ✔️ Integrating OpenAI's ChatGPT and GPT 4 API to handle user prompts. ✔️ Prompt Engineer with extensive experience in designing various problems for a variety of AI use cases. My commitment lies in crafting seamless and intuitive conversational experiences that consistently exceed the expectations of my esteemed clients. Together, we can create an intelligent chatbot solution tailored to your unique requirements, driving unprecedented success. If you are ready to unlock the transformative potential of chatbots and revolutionize your business, please feel free to reach out to me. Let's collaborate and turn your vision into a reality. Best regards, Chintan Soni
    Featured Skill Pyspark
    Data Analysis
    Databricks Platform
    CI/CD
    Generative AI
    PySpark
    Whisper AI
    Vector Database
    AI Chatbot
    Large Language Model
    Artificial Intelligence
    Django
    MySQL
    Python
    ChatGPT
    GPT-4
  • $60 hourly
    • 5.0/5
    • (3 jobs)
    ABOUT ME: I am Lead Data Engineer with strong software development background. I have over 10 years of professional experience in IT, 7 years of which in Data Engineering. I have MS in Software Engineering from DePaul University (Chicago, IL USA) WHAT I CAN DO FOR YOU: Having worked as a Lead Data Engineer in Fortune 500 big enterprises, I can help startups with with *developing comprehensive data governance and security strategies, *designing and implementing cloud data platforms (Azure, AWS, Databricks) * data warehouse modelling * data lake/data lakehouse modelling *cost optimization of data and ML pipelines *performance optimization of data and ML pipelines TECHNICAL SKILLS Python| Java| Scala| PySpark| Apache Spark| Apache Airflow| Databricks| AWS| Azure| AWS EMR| AWS GLUE | Azure Datafactory | Azure Synapse
    Featured Skill Pyspark
    Jakarta EE
    Android SDK
    Android App Development
    Data Lake
    Data Modeling
    Amazon Web Services
    Microsoft Azure
    AWS Lambda
    AWS Glue
    PySpark
    ETL
    Data Engineering
    Machine Learning
    Apache Spark
    Databricks Platform
    SQL
    Java
    Python
  • $100 hourly
    • 5.0/5
    • (9 jobs)
    2019-20. Researched, analyzed, designed, coded and implemented an automated car collision detection system for US car insurance company based on a convolutional neural network using Python, Tensorflow 2.0 and keras 2.0, as well as digital processing algorithms and GoogleMaps APIs. This system provides all the information needed for an operator to contact the nearest police and hospital with the accident location within a minute of the accident occurring, as well as providing a second-by-second animation useful for accident reconstruction. 2019 Researched, analyzed, designed, coded and implemented an Asset Management Risk Management system for a major asset investment firm based on a deep learning neural network, hidden markov chain, time series and NLP algorithms to reduce the expected risk associated with asset management an average of 10%. Used Python, Tensorflow, scikit-learn, BERT, spaCy and keras. The deployment platform was AWS Cloud with EC2, Sagemaker and AWS Deep Learning Containers. 2018. Recently received, in partnership with Oracle, the 2018 Innovation Challenge Award from The Guardian Life Insurance Company for architecting a prediction analytics system to increase sales of insurance products to new customer prospects using machine learning and neural networks. 2017-18. Researched, analyzed, designed, coded, implemented and deployed complete predictive/prescriptive analytics platform and dynamic pricing/ yield management on Azure Cloud for major international parking systems/services corporation to allow parking owners to predict potential demand for parking and maximize their profits by 15-20%. 2014-17. Led digital transformation of sales and marketing groups of a couple very large international corporations using predictive/prescriptive analytics and machine learning, from generating new sales leads to creating strategies for international sales campaigns, increasing selling revenue 3- to 6-fold, and cross-sales/up-sales by $100M+. 2008. As Chief Architect, lead a team of 30 architects in the US and 120 developers/dbas in India, to design and implement Ally Bank, the first US online bank, in a record six months, working 70-90 hours per week. Was responsible for securing a TARP grant of $6.5 billion by meeting extremely tight deadline.
    Featured Skill Pyspark
    Microsoft Azure
    PySpark
    AWS Glue
    Artificial Intelligence
    Big Data
    Analytics
    Cloud Computing
    Business Intelligence
    Machine Learning
    Deep Learning
    Natural Language Processing
  • $50 hourly
    • 5.0/5
    • (37 jobs)
    I am an experienced Data Engineer and R Shiny Developer with a strong background working with Fortune 500 companies and leading consulting firms. I specialize in building scalable data pipelines, developing data models, and creating interactive data applications using technologies like Python (PySpark), R, R Shiny, SQL, SAS, and Power BI. I have hands-on expertise with ETL processes, CI/CD pipelines, and cloud-based platforms such as Databricks and Teradata. I’ve also worked extensively in data integration, analytics, and business intelligence, helping organizations unlock actionable insights and optimize their data-driven decision-making processes. I am passionate about delivering high-quality, efficient solutions that meet both technical and business needs. Let’s collaborate to bring your data projects to life.
    Featured Skill Pyspark
    Visual Basic for Applications
    PostgreSQL Programming
    Databricks Platform
    PySpark
    Python
    Data Analysis
    Data Mining
    R Shiny
    Data Science
    Data Visualization
    SAS
    ggplot2
    Microsoft Excel
    R
    SQL
  • $70 hourly
    • 4.7/5
    • (12 jobs)
    Hello, I am Matthew (you can call me Matt). I truly love data and revealing to people what it can show. I'm a data scientist by trade (MS in Data Science from Columbia University's Fu Foundation School of Engineering and Applied Science) with specialties in Data Visualization, Machine Learning, Natural Language Processing, and Data Mining. I previously worked in financial compliance and healthcare technology, but I am here to work with anything data-related, particularly to its presentation. I am always focused first on providing the most comprehensive, polished products possible in a timely and transparent manner, because clients always deserve true honesty and quality from whomever they hire. I continue to ask "How can data help solve this problem?", whatever the problem might be. I look for clear trends and patterns to try to find the most accurate resolution possible. For me, it always comes down to numbers, and how they tell the larger story. Past Data Specialist Experience: - Transaction Modeling for Anti-Crime Modeling - Health Registries and Claims Data Analysis - Political partisanship and voter demographic dashboards - Machine Learning in stock market price data - Financial similarity matrices - LSTM NLP Summarization Model Programming Experience: - Python (numPy, Pandas, matplotlib, plotly, seaborn, Dash, scikit-learn (personally taught by creator of said package), Scipy, NLTK, Tensorflow) - R (dplyr, ggplot2, Rmarkdown, shiny, lubridate, zoo, knitr) - SQL (Oracle, MS SQL, Hive) - NoSQL (MondoDB) - LaTeX
    Featured Skill Pyspark
    ggplot2
    Data Visualization
    PySpark
    Microsoft Power BI
    Apache Hive
    R Shiny
    Apache Hadoop
    SQL
    Tableau
    Machine Learning
    Python
    Deep Learning
    Apache Spark
    R
  • $60 hourly
    • 5.0/5
    • (10 jobs)
    My husband and I are working as a team using our diverse skills to work on projects related to Computer Vision, Image Processing, Drone Software Development, Data Analysis and Augmented Reality. We also have experience in deploying AI/Computer Vision algorithms into Mobile (Native Platforms iOS & Android) and Web (Using Web Assembly & WebGL). We have a strong background in C++ and OpenCV. We have done many projects related to face detection and filters and deploy it into web.
    Featured Skill Pyspark
    Retrieval Augmented Generation
    LLM Prompt Engineering
    Deep Learning
    Data Science
    Microsoft Azure
    Azure Machine Learning
    WebGL
    Image Processing
    PySpark
    Databricks Platform
    OpenCV
    Computer Vision
    Augmented Reality
    C++
    Python
  • $50 hourly
    • 5.0/5
    • (3 jobs)
    🔍 Expert Data Scientist and Analytics Engineer With extensive experience in data science, analytics, and engineering, I am your go-to expert for turning complex data into actionable insights. As the current technical lead for a boutique consulting company, I have successfully led projects for a diverse range of clients, from large organizations like KPMG and Hagerty Consulting to public sector entities such as New York City Emergency Management. Key Skills and Expertise: Programming: Python, SQL, R Data Visualization: Tableau, PowerBI, Looker, Matplotlib, ggplot Data Engineering: Building robust data pipelines, ETL processes (dbt, BigQuery, Apache Spark, Hadoop) Machine Learning: Predictive modeling, NLP, sentiment analysis, algorithm development Analytics: Statistical analysis, A/B testing, KPI development, causal inference Technical Tools: Git, HTML/CSS, Power Automate, Tableau Site Administration, Agile (Jira), Process Flow Mapping What I Offer: Comprehensive Data Solutions: From data collection and cleaning to analysis and visualization, I provide end-to-end data solutions tailored to your needs. Actionable Insights: My work focuses on delivering clear, actionable insights that drive decision-making and improve business outcomes. Custom Dashboards and Reports: Using Tableau, PowerBI, or Looker I create intuitive and interactive dashboards that help you monitor key metrics and trends. Technical Leadership: With a proven track record of leading technical teams and projects, I ensure high-quality results delivered on time and within budget. Notable Projects: Cloud Data Warehouse Development: Developed a cloud-based database to manage over 2 million entries, increasing daily processing by 10x for all users. Technologies used: Python, REST APIs, Microsoft Azure SQL Database, Power Automate, Salesforce. Sentiment Analysis for UX Improvement: Implemented an NLP model for sentiment analysis and topic modeling, analyzing over 10,000 posts to extract actionable insights, identifying 80+ feedback points for website improvement. Technologies used: Python, NLTK, Vertex AI. Facebook Ads Optimization: Led Facebook Ads A/B testing initiatives, resulting in a 30% improvement in click-through rates and a 10% increase in conversions for clients. Technologies used: Looker, Google Analytics. I am passionate about leveraging data to solve real-world problems and am eager to bring my expertise to your next project. Let's work together to unlock the full potential of your data! Contact me today to discuss how I can help your business thrive.
    Featured Skill Pyspark
    Data Scraping
    Mathematical Modeling
    Jupyter Notebook
    Data Science
    Natural Language Processing
    GitHub
    HTML
    Web Design
    CSS
    PySpark
    Data Visualization
    SQL
    Microsoft Excel
    ArcGIS
    Python
  • $169 hourly
    • 4.7/5
    • (5 jobs)
    Professional Summary With a deep specialization in natural language processing, I am an experienced AI & Machine Learning Engineer and Educator in the technology and education sectors. My commitment to lifelong learning is a testament to my dedication and passion for staying at the forefront of AI research and applications, constantly updating my skills and knowledge to adapt to the rapidly evolving field. Technical Skills • Programming Languages: Python, R, SQL • Machine Learning/Deep Learning Libraries: TensorFlow, PyTorch, Keras, Scikit-Learn, XGBoost, LightGBM • Data Manipulation/Analysis Tools: Pandas, NumPy, dplyr, data.table • Data Visualization Tools: Matplotlib, Seaborn, Tableau, PowerBI, Plotly • Big Data Technologies: Hadoop, Apache Spark, AWS Redshift, Google BigQuery • Web Development: Streamlit, Flask, Django, HTML, CSS, JavaScript • Cloud Platforms: AWS, Azure, Google Cloud Platform, Docker, Kubernetes • Development Tools: GitHub, Git, Jupyter Notebook, R Studio • GIS Tools: ArcGIS, QGIS Additional Projects • Volcano Monitoring Machine Learning Project: Developed a machine learning model to predict volcanic eruptions using sensor data. Preprocessed time-series sensor data, including normalization and handling of missing values. Extracted relevant features from the time-series data significantly impacting volcanic activity predictions. Experimented with various machine learning algorithms, including SVM and neural networks, to identify the best performer. Implemented a real-time data processing pipeline that ingests sensor data continuously. Validated the model using a split-test approach and adjusted parameters to improve accuracy. Built visualizations to depict the prediction accuracy and feature importance, aiding in interpretability. • AI for Affordability of Oncology Immunotherapy Treatment: Contributed to an AI solution that predicts the Time To Next Treatment (TTNT) curve for oncology patients, aiding in personalized treatment planning. Collaborated with Mango Sciences and a diverse team to develop a predictive model and an interactive dashboard. The model enhances decision-making processes in treatment affordability and clinical pathways for immunotherapy. • AI-Driven Pool Management System: Develop an intelligent system for monitoring and controlling swimming pool environments. Automate management of pH levels, chlorine content, temperature, and overall water quality using AutoML tools. Install necessary Python libraries such as sklearn, seaborn, pandas-profiling, and AutoML tools like pycaret, h2o, and tpot. Import libraries and set up data frames for loading historical data on chemical levels and temperature. Clean and prepare data by removing outliers and ensuring data quality. Perform feature engineering and exploratory data analysis. Use various AutoML tools to test different AI models. Evaluate models based on metrics like mean absolute error (MAE) and accuracy to identify the most effective one. Reduce manual labor and enhance maintenance efficiency. Improve water quality and ensure safety. Facilitate the development and deployment of AI models for sustainable and cost-effective pool management solutions. • Price Prediction for Airbnb LA Mayflower Village: Boost predictive analytics capabilities to forecast key business metrics. Compiled extensive datasets from internal sales systems, customer feedback, and market analysis reports into a unified data repository. Performed exploratory analysis to discover patterns, anomalies, and correlations among data variables. Used Python tools like Seaborn and Matplotlib to create visual representations for effective stakeholder communication. Developed and refined features focusing on customer demographics, purchase history, and seasonal trends. Applied PCA to reduce the number of variables, enhancing model efficiency. Tested machine learning algorithms, including Random Forest, SVM, and neural networks, to identify the best predictive model. Used grid search and cross-validation to fine-tune models for optimal accuracy and reduced overfitting. Measured model performance using RMSE and MAE on a validation dataset. Conducted A/B testing to assess model effectiveness in real business settings. Integrated the finalized models into the production environment, linking them with business intelligence tools. Established a real-time dashboard to monitor model performance and its impact on business decisions. Set up a feedback loop with business units for continuous model refinement. Planned periodic updates to modeling techniques considering new data science advancements and market condition shifts.
    Featured Skill Pyspark
    BigQuery
    PySpark
    Microsoft Power BI Data Visualization
    Streamlit
    ArcGIS
    ggplot2
    Tableau
    Azure Machine Learning
    Amazon SageMaker
    XGBoost
    TensorFlow
    PyTorch
    Python Scikit-Learn
    Python
    SQL
  • $40 hourly
    • 5.0/5
    • (10 jobs)
    Hi everyone! I'm a professional Software Engineer at a FAANG company since five years ago. Coding and Data Engineering have always been huge passions in my life. Some of my expertise are in: - Cloud Architecture - Software Development - Data Engineering, ETL, - API design and implementation - Mobile and Web development in React - Network Engineering and Architecture Contact me with a message and we can schedule a Zoom call.
    Featured Skill Pyspark
    React Native
    ETL
    PySpark
    Data Engineering
    Network Engineering
    Software Development
    AWS Glue
    SQLAlchemy
    Microsoft Azure SQL Database
    Python
    Amazon Web Services
    React
    SQL
    Java
    pandas
  • $100 hourly
    • 5.0/5
    • (5 jobs)
    As a highly skilled and accomplished freelance Data Scientist and Data Engineer, I possess a Master's degree in Data Science with a specialization in Artificial Intelligence. My expertise lies in converting intricate data into actionable insights that drive impactful decision-making. With proficiency in Python, PySpark, Google Cloud, Azure, and more, I am well-versed in leveraging these tools to craft and deploy scalable machine learning models while establishing robust data infrastructures. My impressive track record speaks for itself. I have successfully scaled machine learning infrastructure to cater to 100,000+ customers, implementing over 100 parallel cost prediction models. Furthermore, I have excelled in developing high-capacity solutions for data inferencing, resulting in substantial cost savings through infrastructure optimization and cloud computing efficiency. Navigating complex backend migrations seamlessly, I have significantly enhanced data management and system efficiency. By collaborating closely with Data Engineers and DevOps professionals, I have deployed production models and generated pivotal insights, fostering a collaborative environment where synergies thrive. Combining a strong academic foundation in Artificial Intelligence with extensive practical experience, I play a pivotal role in supporting decision-making processes and contributing to clients' strategic objectives. With a meticulous eye for detail and an unwavering commitment to excellence, I take pride in delivering error-free work. Recognizing the critical role of accurate and high-quality data in driving informed decisions, I ensure each project receives my utmost dedication and expertise. As a freelance professional, I am driven by a passion to provide exceptional results every time. With an unwavering focus on excellence, I bring a wealth of skills and experiences to the table, elevating each project to new heights.
    Featured Skill Pyspark
    Apache Hadoop
    Cloud Computing
    Git
    Google Cloud Platform
    DevOps
    PySpark
    Data Science
    Python
    Databricks Platform
    Azure Machine Learning
    Deep Learning
    Machine Learning
  • $85 hourly
    • 5.0/5
    • (25 jobs)
    𝗔𝘇𝘂𝗿𝗲 𝗦𝗼𝗹𝘂𝘁𝗶𝗼𝗻𝘀 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁 | 𝗗𝗮𝘁𝗮 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿 | 𝗧-𝗦𝗤𝗟 | 𝗗𝗔𝗫 | 𝗣𝗼𝘄𝗲𝗿 𝗣𝗹𝗮𝘁𝗳𝗼𝗿𝗺 | 𝗔𝗜 𝗘𝘅𝗽𝗲𝗿𝘁 Are you 𝘀𝘁𝗿𝘂𝗴𝗴𝗹𝗶𝗻𝗴 to 𝗶𝗻𝘁𝗲𝗴𝗿𝗮𝘁𝗲, 𝗺𝗮𝗻𝗮𝗴𝗲, 𝗼𝗿 𝘀𝗰𝗮𝗹𝗲 your data infrastructure across multiple 𝗰𝗹𝗼𝘂𝗱 𝗽𝗹𝗮𝘁𝗳𝗼𝗿𝗺𝘀? Is your organization looking for an 𝗲𝘅𝗽𝗲𝗿𝗶𝗲𝗻𝗰𝗲𝗱 𝗮𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁 to streamline 𝗱𝗮𝘁𝗮 𝗽𝗶𝗽𝗲𝗹𝗶𝗻𝗲𝘀, 𝗮𝘂𝘁𝗼𝗺𝗮𝘁𝗲 𝘄𝗼𝗿𝗸𝗳𝗹𝗼𝘄𝘀, and harness the power of 𝗔𝗜 for insightful 𝗱𝗲𝗰𝗶𝘀𝗶𝗼𝗻-𝗺𝗮𝗸𝗶𝗻𝗴? With over 10 years of experience as a Certified Solutions Architect and Data Engineer, I’ve successfully delivered high-impact solutions for major enterprises including Microsoft, Wells Fargo, Thomson Reuters Elite, Coca-Cola, CVS, Walgreens, Capgemini, EPAM, nThrive, Pharmavite, USDA and Google. Also, worked with start ups like LimeGear, SourceGroup etc. As a multi-certified expert in Azure, AWS, GCP, and Power Platform, I specialize in crafting robust, scalable solutions that improve efficiency, reduce costs, and enhance data security. 𝗦𝗲𝗿𝘃𝗶𝗰𝗲𝘀 𝗜 𝗢𝗳𝗳𝗲𝗿: ☑️ Cloud Architecture & Integration: Designing and implementing secure, scalable architectures across Azure, AWS, GCP, and Snowflake ☑️ Data Engineering & ETL Solutions: Building optimized data pipelines using Azure Synapse, Databricks, and AWS Glue to deliver real-time insights ☑️ Data Warehousing & Analytics Dashboards: Creating comprehensive, interactive dashboards using Power BI, Tableau, and QuickSight to enable data-driven decision-making ☑️ SQL & DAX Optimization: Solving complex problems with T-SQL, DAX, and Python for highly efficient data models ☑️ Automation & AI Integration: Leveraging Azure Open AI and Power Platform for automating workflows and boosting business productivity ☑️ Database & Storage Management: Expertise in managing Azure SQL, PostgreSQL, Amazon RDS, S3, and other cloud storage solutions 𝗤𝘂𝗮𝗻𝘁𝗶𝗳𝗶𝗲𝗱 𝗥𝗲𝘀𝘂𝗹𝘁𝘀: ✔️ Increased data processing speed by 40% for a major client through efficient ETL pipelines ✔️ Improved business reporting capabilities, driving 30% faster decision-making with optimized Power BI dashboards ✔️ Reduced cloud infrastructure costs by 25% through effective cloud architecture design and resource management ✔️ Delivered secure, high-performance data migration from on-premise to cloud solutions for several Fortune 500 companies 𝗖𝗼𝗿𝗲 𝗘𝘅𝗽𝗲𝗿𝘁𝗶𝘀𝗲: ⭐ Cloud Platforms: Azure, AWS, GCP ⭐ Database Management: Azure SQL, PostgreSQL, MySQL, SQL Server, Oracle, Amazon RDS ⭐ Data Analytics: Power BI, Tableau, QuickSight, SQL, T-SQL, DAX ⭐ Automation & AI: Power Automate, Azure AI, Python, Kubernetes ⭐ Data Pipelines & ETL: Azure Data Factory, SSIS, Databricks, AWS Glue, Lambda ⭐ Security & Networking: Azure AD, AWS IAM, Azure Key Vault, Elastic Load Balancing As a proud U.S. citizen by choice, I am also the Co-Founder of Data Integrity Services, Inc., a Microsoft partner and featured Fabric Partner, with over 60 years of combined experience driving digital transformation for businesses across various industries. I’m passionate about giving back and helping others. In addition to my work, I teach T-SQL, Python, and DAX, focusing on making complex coding challenges easy to solve. I also contribute by donating a portion of our revenue to help those in need globally. 𝗟𝗲𝘁’𝘀 𝗕𝘂𝗶𝗹𝗱 𝘁𝗵𝗲 𝗙𝘂𝘁𝘂𝗿𝗲 𝗧𝗼𝗴𝗲𝘁𝗵𝗲𝗿! I’m committed to establishing long-term relationships with clients by delivering measurable results and innovative solutions. Reach out today to discuss how I can assist you. 𝗞𝗲𝘆𝘄𝗼𝗿𝗱𝘀: Azure Solutions Architect, Data Engineer, T-SQL, DAX, Power Platform, AI, SQL, Data Pipelines, ETL, Power BI, Tableau, Python, Automation, Databricks, AWS, GCP, Cloud Architecture, Azure Synapse, Data Warehousing, Kubernetes, SQL Server, PostgreSQL, Cloud Storage, Data Migration, SSIS, AI Integration, Data Security, Microsoft Partner, Data Engineering Solutions
    Featured Skill Pyspark
    Data Analysis
    Azure DevOps
    ETL
    Microsoft PowerApps
    PySpark
    Azure DevOps Server
    Microsoft SQL Server
    DevOps
    Microsoft Azure SQL Database
    Microsoft Power BI Development
    Microsoft Power Automate
    Databricks Platform
    SQL Server Integration Services
    SQL
    Microsoft Power BI
  • $70 hourly
    • 5.0/5
    • (3 jobs)
    Are you in search of a proficient Data Engineer or Analyst who can navigate the complexities of data pipelines, from initial debugging to creating insightful visualizations? I’m Owais, here to turn your data challenges into actionable insights. Why Partner with Me? Bespoke Data Solutions: Tailored data engineering and analysis services that meet your unique business objectives. End-to-End Pipeline Expertise: From data acquisition and cleaning to sophisticated analysis and visualization, leveraging tools like Python, SQL, DBT, and more. E-Commerce Data Mastery: Extensive experience in handling complex e-commerce datasets, ensuring your data not only informs but drives growth. Collaborative Success: Proven track record of working seamlessly with both upstream and downstream teams, ensuring smooth project execution. Professional Snapshot: Since joining HP in January 2022 as a Data Engineer, I’ve spearheaded projects that processed millions of records daily, integrating robust data management practices with SQL, Python, and cloud technologies (AWS & Azure). This role has sharpened my skills in: Developing large-scale ETL processes and data pipelines. Performing deep dives into data analysis and visualization, primarily using Python, Pandas, and PySpark. Ensuring data integrity through comprehensive error analysis, debugging, and monitoring. Empowering teams with data-driven insights, thanks to advanced analytics and machine learning techniques. What I Offer: Free Consultation: Let’s discuss how I can support your project or long-term data strategy. Adaptable Expertise: Whether it’s enhancing data pipeline efficiency, conducting error analysis, or visualizing complex datasets, I offer the flexibility and expertise to support diverse data needs. Ready to Transform Your Data into Insights? I’m committed to delivering exceptional value and building enduring partnerships. For a detailed discussion on how I can assist your project or team, please reach out via Upwork or email for a free consultation. Thank you for considering my expertise for your data engineering and analysis needs.
    Featured Skill Pyspark
    Tableau
    ETL
    Integration Testing
    Machine Learning
    Amazon Web Services
    Microsoft Azure
    Data Analysis
    Data Visualization
    dbt
    Databricks Platform
    PySpark
    Apache Spark
    Python
    Data Engineering
    SQL
  • $38 hourly
    • 5.0/5
    • (1 job)
    🌟 Dynamic Full Stack Developer | Creating Innovative Solutions for Enhanced User Experiences 🚀 Welcome! I'm an experienced and highly skilled Full Stack Developer with a passion for crafting cutting-edge solutions that drive remarkable user experiences. Here's what sets me apart: 💻 Versatile Technology Stack: Proficient in an extensive range of technologies including PHP, Python, HTML, CSS, JavaScript, React, Angular, Node.js, and more. I can effortlessly handle both front-end and back-end development, ensuring seamless integration and exceptional performance. 🌈 Aesthetic and Intuitive Interfaces: With a keen eye for design, I create visually stunning and user-friendly interfaces. By prioritizing usability and accessibility, I deliver dynamic and responsive websites and applications that captivate users from their very first interaction. 🛡️ Security at the Core: Protecting user data is paramount. I possess expertise in implementing robust authentication and authorization mechanisms, ensuring the utmost security across all levels of application development. 🎯 Solution-driven Mindset: I thrive in dynamic environments and leverage my problem-solving skills to overcome complex challenges. From troubleshooting to debugging, I'm adept at finding effective solutions to ensure smooth project execution. ⏱️ Timely Delivery, Uncompromised Quality: I'm committed to delivering projects on time and within budget while upholding the highest standards of quality. By following industry best practices, utilizing version control systems, and embracing collaborative development methodologies, I consistently deliver exceptional results. ✨ Let's Collaborate: I stay up-to-date with the latest industry trends and technologies, continuously expanding my skill set. With a passion for innovation and a focus on enhancing the overall client experience, I'm eager to bring your ideas to life and drive breakthrough efficiency. If you're seeking a Full Stack Developer who combines technical expertise, creativity, and a user-centric approach, I'm here to make your vision a reality. Let's connect and embark on an exciting journey together!
    Featured Skill Pyspark
    PySpark
    Shopify
    Payment Gateway Integration
    RESTful API
    Docker
    jQuery
    Amazon Web Services
    Flask
    SQL
    MongoDB
    Amazon DynamoDB
    JavaScript
    Django
    PHP
    Python
  • $120 hourly
    • 5.0/5
    • (1 job)
    With over 8 years of data science experience and a decade of strong analytical modeling and programming background, I help businesses further their digital transformation agenda by leveraging data science. My expertise in advanced statistical algorithms, machine learning and forecasting have allowed me to successfully lead numerous data science projects from inception to deployment. My expertise extends beyond technical skills. I excel in project management, ensuring timely delivery. With a Ph.D. in Physics and certificates in business strategy, I possess a strong foundation in algorithmic thinking, programming, mathematics, and statistics. This unique blend of technical skills and business acumen allows me to effectively bridge the gap between data science and business strategy.
    Featured Skill Pyspark
    Large Language Model
    Microsoft Power BI
    Git
    PySpark
    Artificial Intelligence
    Transformer Model
    SQL
    Deep Learning
    Tableau
    PyTorch
    Azure Machine Learning
    Python Scikit-Learn
    Python
    Machine Learning
  • $75 hourly
    • 5.0/5
    • (3 jobs)
    𝗘𝘅𝗽𝗲𝗿𝗶𝗲𝗻𝗰𝗲𝗱 𝗗𝗮𝘁𝗮 𝗦𝗰𝗶𝗲𝗻𝘁𝗶𝘀𝘁 (𝗲𝘅-𝗔𝗽𝗽𝗹𝗲, 𝗲𝘅-𝗚𝗼𝗼𝗴𝗹𝗲) 𝗜 𝘂𝗻𝗰𝗼𝘃𝗲𝗿 𝗽𝗮𝘁𝘁𝗲𝗿𝗻𝘀 𝗮𝗻𝗱 𝘁𝗿𝗲𝗻𝗱𝘀 𝗯𝘂𝗿𝗶𝗲𝗱 𝘄𝗶𝘁𝗵𝗶𝗻 𝘆𝗼𝘂𝗿 𝗱𝗮𝘁𝗮. 𝗪𝗵𝗲𝘁𝗵𝗲𝗿 𝘆𝗼𝘂'𝗿𝗲 𝗮𝗶𝗺𝗶𝗻𝗴 𝘁𝗼 𝗼𝗽𝘁𝗶𝗺𝗶𝘇𝗲 𝗼𝗽𝗲𝗿𝗮𝘁𝗶𝗼𝗻𝘀, 𝗲𝗻𝗵𝗮𝗻𝗰𝗲 𝗰𝘂𝘀𝘁𝗼𝗺𝗲𝗿 𝗲𝘅𝗽𝗲𝗿𝗶𝗲𝗻𝗰𝗲𝘀, 𝗼𝗿 𝗺𝗮𝗸𝗲 𝘀𝘁𝗿𝗮𝘁𝗲𝗴𝗶𝗰 𝗱𝗲𝗰𝗶𝘀𝗶𝗼𝗻𝘀, 𝗜 𝗱𝗲𝗹𝗶𝘃𝗲𝗿 𝘁𝗮𝗶𝗹𝗼𝗿𝗲𝗱 𝘀𝗼𝗹𝘂𝘁𝗶𝗼𝗻𝘀 . 📞 Invite me to your job on Upwork to schedule a complimentary consultation call to discuss how we can solve or answer your questions. I excel at finding technical solutions (SQL, Python, ML) to business problems. I’m experienced at communicating my findings as actionable recommendations for technical and non-technical stakeholders, including C-level audiences. Over 15 years of client services experience working with companies ranging from Fortune 5 to incubator stage, I understand how to work with people and resources reflecting the scale of the enterprise. "Kineret is an excellent thought-partner. She has the advantageous, well-honed ability to be flexible and open to ideas from other perspectives -- including thoughtfully asking for them -- and being firm when she knows she is right. She challenges you to think and demands your best as she demands so of herself. She was a great partner!" I'm the ideal freelancer for you if you're looking for a thought partner who will deliver insights and tech solutions that leverage your data. I am comfortable working with ambiguity and delivering actionable insights. "Kineret was a phenomenal group member that provided a collaborative energy to the group that was otherwise missing. Her insights were extremely thought-provoking!" Here are some of my technical skills: HDFS | Hadoop | PySpark | PyTorch | TensorFlow | Google Cloud Platform (GCP) | AWS | Databricks | Natural Language Processing (NLP) | Computer Vision | Tableau | Large Language Models (LLM) Data Analytics & Insights Data Visualization & Storytelling Strategic Business Analysis Machine Learning (ML) Artificial Intelligence (AI) Statistical Modeling Cross-Functional Leadership Data Reporting Experiment Design
    Featured Skill Pyspark
    Statistics
    Customer Segmentation
    Survey Data Analysis
    Predictive Modeling
    Experiment Design
    PySpark
    Analytics
    Natural Language Processing
    Computer Vision
    Machine Learning
    Time Series Forecasting
    SQL
    Data Science
    R
    Python
  • $120 hourly
    • 5.0/5
    • (2 jobs)
    I'm an enthusiastic Data Engineer, who is deeply interested in architecting, building, scaling, and optimizing data models, data pipelines, data lakes, and data warehouses. I'm an expert in Apache Spark for batch processing to handle terabytes of data. I'm always looking toward automation, self-service, and improving productivity for both developers and products. I believe in transparency and over-communication rather than staying in silence. Hire me today, or simply sent me an invite, we can discuss your projects.
    Featured Skill Pyspark
    Web Scraping
    Automation
    Unix Shell
    ETL
    Data Processing
    Microsoft Azure
    Big Data
    Snowflake
    Databricks Platform
    PySpark
    Apache Airflow
    Data Engineering
    Python
    Apache Spark
  • $40 hourly
    • 5.0/5
    • (4 jobs)
    Lukas is a software engineer with 4 years of experience doing both frontend and backend development for various tech companies based in the U.S. Lukas has experience building frontend applications using React, Vue.js, JavaScript, CSS/SCSS, HTML5, Redux, Jest, jQuery, and D3.js. He also has experience working with Node.js, Flask, Django, Java, Spring, Flask, Rust, Python, SQL, NoSQL databases, etc. Lukas also has experience working with the Cloud, having hands on experience with many AWS technologies such as S3, DynamoDB, SQS, EC2, Lambda, Route53, EMR, ELB, Networking and more.
    Featured Skill Pyspark
    Amazon S3
    PySpark
    AWS Lambda
    Amazon EC2
    Vue.js
    jQuery
    SQL
    Java
    C++
    Django
    Node.js
    React
    Python
    JavaScript
  • $68 hourly
    • 5.0/5
    • (2 jobs)
    👋 Hello, and welcome to my profile! I'm a passionate and results-driven professional with extensive expertise in the fields of Artificial Intelligence, Machine Learning, MLOps, and Python. If you're looking for a seasoned expert to lead your AI initiatives or solve complex problems with cutting-edge technology, you're in the right place. 🤖 As a Lead Generative AI & Prompt Engineer, I specialize in building AI systems that can generate creative and context-aware content. I have a deep understanding of Natural Language Processing (NLP), and I've worked on projects ranging from chatbots to content generation tools. My experience allows me to craft AI models that understand and mimic human language effectively. 🧠 In my role as an AI & ML Engineer, I've designed and developed machine learning models for a variety of applications, such as predictive analytics, image recognition, recommendation systems, and more. I'm proficient in using libraries like TensorFlow and PyTorch, ensuring robust and scalable solutions. 🔧 Additionally, I have a strong background in MLOps, ensuring that AI and ML models are deployed efficiently and can be maintained effortlessly. I can set up continuous integration/continuous deployment (CI/CD) pipelines, containerization, and orchestration solutions to keep your AI systems running smoothly. 🐍 Python is my go-to language for implementing AI and ML solutions. I leverage its versatility and vast library ecosystem to create powerful and efficient code that drives your projects forward. 🌟 Here's what I can bring to the table: ================================================ ✨ Expertise in designing, developing, and deploying AI and ML models. ✨ Proficiency in NLP for creative content generation. ✨ MLOps skills for efficient model deployment and maintenance. ✨ Strong Python programming skills. ✨ A commitment to delivering high-quality, scalable solutions. ✨ Excellent communication and project management skills to ensure smooth collaboration. 💡 Whether you're looking to automate tasks, enhance your business with AI-driven insights, or explore the possibilities of generative AI, I'm here to help. Let's work together to turn your AI dreams into reality. 📈 Don't hesitate to reach out and discuss your project requirements. I'm eager to collaborate on exciting ventures and contribute my expertise to your success. Let's embark on this AI journey together!
    Featured Skill Pyspark
    Chatbot
    ChatGPT
    Data Integration
    Data Science
    Data Science Consultation
    Prompt Engineering
    AI Consulting
    PySpark
    Machine Learning
    Artificial Intelligence
    AI Security
    AI Development
    PostgreSQL
    Python
  • $200 hourly
    • 5.0/5
    • (6 jobs)
    I’m Will, a Data Scientist specializing in quantitative analytics for the financial and sports betting domains. When it comes to quantifying impacts and outcomes, there are a lot of options and considerations. I bring expertise to tailor solutions to your needs. Standard Services Include: - Build a model from scratch with your data - Validate your model or system - Obtain opening and closing lines for major betting markets - Identify statistically significant predictors that impact outcomes I work as a Quantitative Analyst for one of the world's leading banks. I hold a Master's degree in Applied and Computational Mathematics and Statistics: Data Science Specialization from the University of Notre Dame and I hold a Bachelor's degree in Business Administration: Finance Concentration from the University of Tennessee. Statistical explanations and predictive modeling are all projects that I enjoy taking on. Proficient in Python, SQL, Databricks, and more. I craft customized data-driven solutions to drive high performance. I offer expertise in leveraging data for extracting value in the complex industry of sports betting.
    Featured Skill Pyspark
    PySpark
    Statistical Analysis
    Data Analysis
    Data Engineering
    Artificial Intelligence
    Sports
    Business
    SQL
    Databricks Platform
    Python
    Machine Learning
  • $80 hourly
    • 5.0/5
    • (2 jobs)
    Data Scientist with a passion for enabling analytics and using data to drive insights. Experience with languages Python, R, SQL. Interests include data engineering, data visualization, statistical analysis, causal inference, data science, machine learning, exploring new algorithms and latest technologies. Experiences with AI automation, LLMs, predictive modeling, data warehouses, ETL, engineering architecture, BI (Power BI), and predictive analytics.
    Featured Skill Pyspark
    Google Analytics
    Artificial Intelligence
    Generative AI
    Databricks Platform
    Databricks MLflow
    PySpark
    SQL
    R Shiny
    R
    Python
    Machine Learning
  • $55 hourly
    • 5.0/5
    • (3 jobs)
    I’m Ernie Maldonado, and I’m a seasoned data scientist with vast experience in data analytics, data engineering and machine learning. I have implemented data driven analytics solutions to monitor and increase revenue as well as reduce costs. From the technical point of view, these goals can be achieved by implementing a cost effective data platform, along with developing useful metrics, increasing automation and finally creating actionable tasks for people. From the business side, these goals can be achieved by developing strong relationships with internal and external clients, always striving to meet their needs and delivering high quality products. Finally, from the talent point of view, it is essential to have a strong and cohesive team, that works well together pursuing shared goals. We share the same goals of increased revenue and cost reduction and believe that we can achieve those through technical excellence, practical experience and client satisfaction. These shared goals and values assure me that we will work well together and build a long lasting and successful partnership
    Featured Skill Pyspark
    Database
    Big Data
    PySpark
    Apache Hadoop
    R
    Econometrics
    Economics
    SQL
    Python
    SAS
    Information Technology
    Machine Learning
    Engineering & Architecture
    Statistics
    Data Engineering
  • $45 hourly
    • 4.9/5
    • (2 jobs)
    I’m a data engineer with extensive experience in managing and improving complex data systems for businesses and political campaigns. Whether you're looking to optimize your data architecture, enhance data quality, or integrate new data solutions, I can help. - Proficient in Python, SQL, R, HTML, CSS, JavaScript, and TypeScript - Expert in using tools like GCP, PostgreSQL, MongoDB, and Apache Spark for data processing and analytics - Full project management from initial data assessment to the final reporting stages - Regular communication is key to successful project outcomes, so I’m committed to maintaining clear and consistent contact throughout our collaboration. Formal education has included Bachelor's in Economics, Master's in Social Impact, and Certification of Completion for Code Fellows' Intensive Bootcamp for Advanced Software Development in Python.
    Featured Skill Pyspark
    Google Cloud Platform
    Apache Airflow
    Jupyter Notebook
    Git
    Tableau
    PySpark
    pandas
    NumPy
    JavaScript
    CSS 3
    HTML5
    Bash
    R
    SQL
    Python
  • $110 hourly
    • 5.0/5
    • (1 job)
    Companies hire me to make the impossible possible. I have over 30 years experience working the technology landscape. - Project Management - Technology Integration - Software Development Lifecycle & CI/CD - Data Management & Analytics & AI - On-site and Cloud (AWS) - Client / Server - Cybersecurity After completing my degree in Bioengineering at the University of California, San Diego (UCSD) I started my career in Biotech and Computational Chemistry writing software utilizing early AI to perform chemistry and design drugs. I have been published for this work. I have also worked in clinical healthcare (Epidemiology and Infection Control), Supply Chain and the Finance industry where my technology skills matured to the level of architecting and beyond into the world of business leadership. My strength lies in listening to a client's challenge and then drawing on my experience to craft solutions. My data-driven expertise allows companies to manage and understand their data better. I think outside the box. If you are a small or startup company I can help you lay out your technology roadmap. If you are a mid-size or large company I can work with existing infrastructure and technology and make it better and more efficient. In all cases I can help recruit and mentor talent for companies to maintain their technology strategy. I am confident I can bring great value to your company so that you can impact the world! I look forward to our first conversation. (I am available on LinkedIn.)
    Featured Skill Pyspark
    AWS Development
    SQL
    API Documentation
    Scripts & Utilities
    Windows Framework
    Linux
    ETL Pipeline
    Data Warehousing
    PySpark
    Python
    Java
    C++
    ASP.NET
    .NET Core
    .NET Framework
  • $60 hourly
    • 4.6/5
    • (2 jobs)
    As an accomplished AI/ML Scientist specializing in spatial analytics, I am dedicated to harnessing cutting-edge technologies to address complex location-based challenges across diverse industries. With extensive expertise in data engineering, machine learning models, and spatial analytics, I excel in crafting innovative solutions tailored to the unique needs of sectors such as healthcare, navigation mapping, and traffic safety. Services Offered: AI/ML Development: Proficient in developing advanced machine learning models customized for detection, analysis, and predictive insights across various domains. Spatial Analytics: Expertise in leveraging spatial data analysis techniques to extract actionable insights and optimize decision-making processes for enhanced efficiency and effectiveness. Heatmap Creation: Skilled in generating informative heatmaps to visualize spatial data patterns, identify hotspots, and drive strategic planning for improved outcomes. Location Optimization: Specialized in determining optimal locations for new facilities or infrastructure through comprehensive spatial analysis and predictive modeling, ensuring maximum impact and efficiency. Key Skills: Data Engineering, Machine Learning, Spatial Analytics, Geographic Information Systems (GIS), Predictive Modeling, Heatmap Visualization, Location Intelligence Industry Experience: Healthcare, Navigation Mapping, Traffic Safety, Retail Site Selection, Why Choose Me: With a profound understanding of AI/ML methodologies and spatial analytics techniques, I offer a unique blend of expertise to deliver tailored solutions that address specific industry challenges effectively. My commitment to innovation, precision, and client satisfaction ensures exceptional results that drive tangible value and competitive advantage. Let's Collaborate: If you're seeking expertise in AI/ML development, spatial analytics, heatmap creation, or location optimization, I'm here to collaborate and deliver bespoke solutions aligned with your business objectives. Let's discuss how we can leverage technology to overcome your location-based challenges and unlock new opportunities for growth and success.
    Featured Skill Pyspark
    BigQuery
    Machine Learning
    Google Cloud Platform
    Databricks Platform
    GIS
    K-Means Clustering
    Classification
    PySpark
    Python
    SQL
    Data Science Consultation
    Computer Vision
    Spatial Analysis
    Statistics
    Artificial Intelligence
  • Want to browse more freelancers?
    Sign up

How hiring on Upwork works

1. Post a job

Tell us what you need. Provide as many details as possible, but don’t worry about getting it perfect.

2. Talent comes to you

Get qualified proposals within 24 hours, and meet the candidates you’re excited about. Hire as soon as you’re ready.

3. Collaborate easily

Use Upwork to chat or video call, share files, and track project progress right from the app.

4. Payment simplified

Receive invoices and make payments through Upwork. Only pay for work you authorize.

Top cities for Pyspark Developers in the United States
Show more