Hire the Best Pyspark Developers
in the United States

🏆 𝐄𝐱𝐩𝐞𝐫𝐭-𝐕𝐞𝐭𝐭𝐞𝐝 - 𝗧𝗼𝗽 𝟭% 𝗼𝗻 𝗨𝗽𝘄𝗼𝗿𝗸 ✅ 𝟕+ 𝐘𝐞𝐚𝐫𝐬 𝐨𝐟 𝐄𝐱𝐩𝐞𝐫𝐢𝐞𝐧𝐜𝐞 ⏰ 𝟏𝟐𝟎𝟎𝟎 + 𝐔𝐩𝐰𝐨𝐫𝐤 𝐇𝐨𝐮𝐫𝐬 | 𝟏𝟎𝟎+ 𝐏𝐫𝐨𝐣𝐞𝐜𝐭𝐬 𝐃𝐞𝐥𝐢𝐯𝐞𝐫𝐞𝐝 ⭐️ $𝟕𝟎𝟎𝐤+ 𝐞𝐚𝐫𝐧𝐢𝐧𝐠 𝐨𝐧 𝐔𝐩𝐰𝐫𝐨𝐤 I’m Abdul, an Upwork Certified AI/LLM engineer with 7+ years of experience helping startups and growing teams build smart AI solutions. From autonomous agents, MVPs, AI-native products to intelligent analytics and cloud-ready deployments, I will help scale your operations, workflows, and revenue. —------------------------------------------------------------------- ⭐️𝐓𝐫𝐮𝐬𝐭𝐞𝐝 𝐛𝐲 𝐈𝐧𝐝𝐮𝐬𝐭𝐫𝐲 𝐋𝐞𝐚𝐝𝐞𝐫𝐬⭐️ “Abdul helped us rethink our lead gen with AI-powered decision-making. His work with LLMs and pipelines was key.” — HS, Director of Data & ML, EZ Pack “Game-changer. Abdul brought serious AI expertise that moved the needle fast.” — Thomas E., CEO, Mission BI —------------------------------------------------------------------- 𝐂𝐨𝐫𝐞 𝐒𝐞𝐫𝐯𝐢𝐜𝐞𝐬 & 𝐄𝐱𝐩𝐞𝐫𝐭𝐢𝐬𝐞 ➡️ 𝐆𝐞𝐧𝐞𝐫𝐚𝐭𝐢𝐯𝐞 𝐀𝐈 & 𝐂𝐨𝐧𝐯𝐞𝐫𝐬𝐚𝐭𝐢𝐨𝐧𝐚𝐥 𝐀𝐠𝐞𝐧𝐭𝐬 Autonomous LLM agents, domain-tuned business assistants, task automation, human-like conversational interfaces (OpenAI, Claude, open-source models) ➡️ 𝐑𝐀𝐆 (𝐑𝐞𝐭𝐫𝐢𝐞𝐯𝐚𝐥-𝐀𝐮𝐠𝐦𝐞𝐧𝐭𝐞𝐝 𝐆𝐞𝐧𝐞𝐫𝐚𝐭𝐢𝐨𝐧) Hybrid semantic search + generation, document & knowledge-base Q&A, contextual accuracy tuning, vector-store architecture ➡️ 𝐋𝐋𝐌 𝐅𝐢𝐧𝐞-𝐓𝐮𝐧𝐢𝐧𝐠 & 𝐄𝐯𝐚𝐥𝐮𝐚𝐭𝐢𝐨𝐧 Domain-specific fine-tuning (QLoRA), benchmark evaluation (perplexity, BLEU, custom QA metrics), reliability & guardrail testing ➡️ 𝐀𝐈 𝐀𝐮𝐭𝐨𝐦𝐚𝐭𝐢𝐨𝐧 Python-based workflow automation, process orchestration, manual-work reduction, API & systems integration, n8n, make. ➡️ 𝐀𝐈-𝐏𝐨𝐰𝐞𝐫𝐞𝐝 𝐀𝐧𝐚𝐥𝐲𝐭𝐢𝐜𝐬 Multi-agent data analysis, auto-generated dashboards & reports, structured + unstructured data pipelines, decision recommendations ➡️ 𝐂𝐥𝐨𝐮𝐝-𝐑𝐞𝐚𝐝𝐲 𝐃𝐞𝐩𝐥𝐨𝐲𝐦𝐞𝐧𝐭 Production-scale deployment on AWS (Bedrock, SageMaker), GCP (Vertex AI), Azure, and Databricks —------------------------------------------------------------------- 💰 𝐊𝐞𝐲 𝐀𝐜𝐡𝐢𝐞𝐯𝐞𝐦𝐞𝐧𝐭𝐬 -Helped clients raise $3M+ through AI-powered growth. -Built 50+ AI systems, from multi-agent tools to insight engines. -Drove a 17% revenue increase in 7 months for a US startup. -Impacted 300,000+ users through data-led product strategies. -Graduated in the Top 5% in my Master’s degree class (specialization in AI) —------------------------------------------------------------------- 🛠️ 𝐓𝐞𝐜𝐡 𝐒𝐭𝐚𝐜𝐤 LLMs & Agents: GPT-4, Claude, LLaMA, Mistral, LangChain, LlamaIndex, MemGPT RAG Stack: FAISS, ChromaDB, Pinecone, Weaviate LLM Ops: Fine-tuning, Prompt Engineering, Evaluation Metrics, Guardrails Dev & Infra: Python, FastAPI, LangGraph, Streamlit, Chainlit, Flask Data Pipelines: PySpark, SQL, Event-Driven Workflows, Cloud Functions Cloud Platforms: AWS, GCP, Azure, Bedrock, SageMaker, Vertex AI, Databricks —------------------------------------------------------------------- I am available to understand your project needs and deliver the best solution efficiently. Click on the “Message” button and let’s have a chat —------------------------------------------------------------------- Keywords associated with my skill set: AI Agent Developer, AI Consultant, Chatbot Developer, AI Automation Expert, AI Integration Specialist, Machine Learning Engineer, LLM Developer, Generative AI, Agentic AI, Multi-Agent Systems, Large Language Models (LLMs), GPT-4, Claude, LLaMA, Mistral, LangChain, LlamaIndex, LangGraph, Retrieval-Augmented Generation (RAG), Semantic Search, Knowledge Retrieval, Vector Databases, FAISS, ChromaDB, Pinecone, Weaviate, Custom Embeddings, Memory Layers, Prompt Engineering, LLM Fine-Tuning, LLM Evaluation, MLOps, Model Deployment, AI Data Analysis, NLP, Python, FastAPI, Chainlit, Data Pipelines, PySpark, SQL, AWS, GCP, Azure, Bedrock, Vertex AI, SageMaker, Databricks, Cloud Functions, API Integration, Open Source Models, Healthcare AI Engineer, Healthtech, Edutech, Fintech.

I'm a data professional with a Bachelor degree in computer science, with 8+ years of experience in the field and have been working in small and big data enterprise solutions. During my professional path, I worked in several roles like DBA, Data Architect, Data Engineer, Data Governance and Business Analytics team support. My experience comes from hybrid environment projects with cloud providers like AWS, Google and Azure. I can mention some of the most challenging projects in my professional career where I am proud to produce an impact in the organizations. The actual Chilean Public Education payment service, which managed almost 3,2% of the country's PIB can calculate and assign the Education money for every public school in the country using variable metrics managed by user, my expertise in Databases and integrations was key in the actual calculation workflow process. I was able to developed a service to recollect Chilean Public transportation data from real-time social network and combine with the buses GPS data for analytics, so the Managers could make decisions for quality service improve from the service user perspective taking actions rapidly. I was part of the team that create and deploy the first Africa online tax service in Kenya, where I took in charge of the modeling and support of the Oracle operational database, the service was a game changing, lowing in big manner, the cost of the tax recollection in Kenya. And lastly I design and had feed the Datawarehouse of a Chilean financial big company with the data required that allowed to the company made the correct decision about what king of products needed to focus. All of it, with a mix of database services like Azure SQL Server, GCP Bigquery, MongoDB and PostgreSQL. With the work on mixed environments and budgets, I can mix open-source technologies like low-code solutions like Pentaho, Airflow, db and Processmaker, with proprietary services in the cloud like lambda functions, Athena, Google Compose, Azure ADF or Glue, plus coding skills like SQL, python, Java and Regular Expressions (for text data query). I also have experience with Data visualization tools like Google Data Studio, Clickview, PowerBI, SSRS, Webfocus and Pentaho Reporting Services, knowledge of it because of the need to support BI Teams in some projects. Its seems likely that I missed a more specialized technical profile, but it was the result of the need to work with disparate clients and industries, however I feel very confident with the whole enterprise data journey path, from data modeling and processing to data visualization and it allows me to see the big picture in data solutions implementations. Regards.

I'm a Data Engineer with 3 years of experience, specializing in distributed databases and SQL optimization. In my recent role, I built a 60TB data warehouse using Greenplum that processes 2 million rides, generating 10-20 million records daily. I have extensive experience with MPP databases like Greenplum, ClickHouse, and Redshift. I'm proficient in SQL and Python, which I use for building ETL pipelines. In my previous role, I completely rebuilt a data processing system, improving performance from 17 hours to 2 hours, helping clients access reports much faster. I also automate workflows with Airflow and work with AWS services like Redshift and S3 for data storage and processing.

I am Data & AI Architect with 12+ years of experience across multiple sectors, currently focusing on modern data stacks and building efficient data layers for AI use cases My experience related to enterprise level and high profile projects in the past, but now I'm helping startups and small-mid sized companies to achieve their goals! My core competences are: Data Modeling, Data Architecture on Cloud platforms, Data Integration and AI Data layers, ETL and Business Intelligence, Database Administration Solution Architecture : Design of solution architectures for a data processing systems of various scale and purpose. Definition of modern stack technical solutions including data storage, network, data processing (ELT/ETL), BI and AI/ML components. Process and methods definitions and optimizations AI/ML : Amazon Quicksuite, LLM API Integration (OpenAI, Gemini, Antropic), AWS Sagemaker, pgvector, Databricks Mozaic AI, Snowflake Cortex/Agents Data Modelling : Modelling of OLTP and Datawarehouse systems. It could be design of new schema, normalization/denormalization of existing model, Enterprise datawarehouse design based on Kimball/Inmon, Data Lake and Data Vault architectures, Modernization of existing data landscape Data Lakes : Modern datalakes built on cloud storages as GCS/S3/Azure Datalake Gen2 with Databricks, AWS Glue, Trino, Apache Iceberg DBA Activities : DB migrations, Backup & Recovery, Upgrades, Instance configurations, DB Monitoring, Horizontal scaling, Streaming/BDR replications. Sharding with postgreSQL extensions Data Integration and ETL : Traditional batch ETL - Informatica, Talend, AWS Datapipeline, Matillion ETL Serverless ETL - AWS Lambda, AWS Glue, Batch, AWS DMS, Google Cloud Functions, Databricks Streaming ETL - Apache NiFi, Kafka, Kinesis streams SaaS ETL - Stitch, Alooma, Fivetran, Airbyte BI/data miodeling layers - dbt/Dataform/Prefect Direct loading with DBMS tools & scripting Data Governance and MDM: Design and implementation of solutions based on Informatica DG/MDM, Alation, Atlan, Azure Purview and custom dataquality solutions BI Systems : Design of BI systems and implementation. I had experience with major industry leading tools as Tableau, PBI, Looker and cloud alternatives. Additionally i had experience with old-style reporting solutions from SAP, Qlick, Jasper Cloud containerization and deployment : Docker, Mesos/Kubernetes Java development : EE/SE , Spring, Hibernate, RESTful APIs, Maven Clouds : - Cloud migrations (AWS, Azure, GCP) - Cloud infrastructures (VPCs, EC2, Loadbalancing, Autoscaling, Security in AWS/GCP) Please note that minimum engagement is part-time (20 hrs/week), this will ensure the quality of delivered solution and mutual benefit. Thank you for getting to the end of this extensive details and looking forward working on a project together :) Best Regards, Yegor.

Are you looking to unlock the full potential of Palantir Foundry and AIP? I am a Senior Palantir Foundry Developer and Platform Administrator with extensive experience architecting and deploying ontology-backed applications for a massive enterprise deployment of over 6,000 registered users. Beyond building scalable data pipelines and custom applications, I provide hands-on training and SME-level support to developers, ensuring your team adopts industry best practices in ontology design, application architecture, and AI integration. My technical expertise is grounded in over 16 years of experience as a rigorous Operations Research Analyst. I don't just write code; I bridge the gap between complex quantitative analysis and production-ready software. I have also completed the comprehensive Ontologize Foundry & AIP Foundations for Engineers curriculum. Core Expertise: Data Engineering: PySpark, SQL, Pipeline Builder, and robust data integration. Ontology & App Development: Ontology design, Workshop (no-code apps), and OSDK + React front-end development. AI Platform (AIP) & MLOps: AIP Logic, RAG-assisted AIP Agents, LLM-in-code patterns, semantic search, and the Model Catalog. I am available for part-time, project-based engagements on Fridays, as well as nights and weekends (US Central Time). Let's connect to discuss how we can accelerate your next Palantir initiative.

I am a certified AWS Developer with over 10 years of working experience with AWS Cloud, who specializes in server-less architectures such as ETL processes, serverless APIs, serverless websites. ETL processes: - various compute environments: from EMR, via Glue, Batch, to Lambda with focus to be config driven and as serverless as possible. - custom orchestration systems with Step Functions, Scheduled Events, On-demand, queueing and post processing - Python, PySpark, NodeJS (with Typescript) APIs: - as serverless as possible to reduce cost and development time, with ability to deploy whole API service as single Lambda function (NestJS, FastifyJS, ExpresJS, FastAPI, Flask) - Python, NodeJS (with Typescript) Serverless websites: - as serverless as possible for reports about system status, small intake forms, management tools, etc. - build whole websites with static generated content, server site rendering (NuxtJS), - Python, NodeJS (with Typescript) I prefer to use Serverless Framework as IaaC (infrastructure as a code) for deployment with a connection with GitHub Actions as a pipeline. Also, I have experience with and I am open to use other deployment tools such as: AWS CDK, SAM, Terraform, etc. I also have experience designing DynamoDB tables (one table design principle) and connecting them with Lambdas as data producers, and creating DynamoDB Streams with Lambdas as event processors (which could've been useful in your project). Let's talk about your project needs and use my 10 years of working experience with AWS. Tomasz