Hire the Best Pyspark Developers
in Poland

🚀 Build scalable data infrastructure 🛠 Automate data workflows 📊 Deliver actionable analytics I’m a certified data consultant specializing in Microsoft Fabric, Azure Databricks, SQL, Python, and Power BI. I help companies, from SaaS startups to global enterprises, turn complex, fragmented data into reliable, analytics-ready datasets that drive faster decisions and product growth. My work spans the full data lifecycle: designing architecture, building ETL pipelines, managing data lakes, and delivering secure reporting layers. I’ve delivered 20+ successful remote projects across logistics, maritime, energy, and real estate - from setting up 5+ Microsoft Fabric environments from scratch to managing infrastructure for Verizon AI’s 80M+ company dataset. Core Skills 📶 Data Engineering & Analytics: Microsoft Fabric, Azure Databricks, SQL Server, T-SQL, Python, PySpark, DAX, M ☁️ Cloud & Orchestration: Azure Data Factory, ADLS Gen2, Medallion Architecture, CI/CD, Git 🔌 Integrations: REST APIs, CRMs (Salesforce, HubSpot), ERPs (SAP, QuickBooks), IoT data 📊 Data Visualization: Power BI (Embedded), Apache Superset Example Use Cases End-to-end SaaS data infrastructure for lead-generation platforms ETL pipelines consolidating CRM, ERP, and IoT data into analytics-ready datasets Embedded analytics portals for client KPI tracking Role-based reporting for operations, finance, and product teams Microsoft Certified PL-300 – Power BI Data Analyst Associate DP-700 – Fabric Data Engineer Associate DP-500 – Azure Enterprise Data Analyst Associate Clients choose me because I: 🔸 Deliver on time and to spec 🔸 Take ownership of the whole solution, from ingestion to analytics 🔸 Communicate clearly with both technical and non-technical teams 🔸 Bring both enterprise-scale engineering and startup agility to projects If you’re looking for a technical data consultant who can architect, build, and optimize your data infrastructure - let’s connect.

I have over 9 years of experience in Data Engineering (especially using Spark and pySpark to gain value from massive amounts of data). I worked with analysts and data scientists by conducting workshops on working in Hadoop/Spark and resolving their issues with big data ecosystem. I also have experience on Hadoop maintenance and building ETL, especially between Hadoop and Kafka. You can find my profile on stackoverflow (link in Portfolio section) - I help mostly in spark and pyspark tagged questions.

Full-Stack AI Lead with 15+ years of experience in AI/ML/Data Engineering. Specialist in productionizing Document AI, Agentic LLM Workflows, and Scalable Big Data systems. Master’s in Applied Mathematics. 𝐀𝐠𝐞𝐧𝐭𝐢𝐜 𝐀𝐈 & 𝐋𝐋𝐌 - Architecting production-grade agents and multi-intent workflows using LangGraph and LangChain. - Implementing NeMo Guardrails, DeepEval, and Langfuse to ensure security and near-zero hallucination rates. - Developing high-performance FastAPI backends for real-time operational data integration. 𝐃𝐨𝐜𝐮𝐦𝐞𝐧𝐭 𝐈𝐧𝐭𝐞𝐥𝐥𝐢𝐠𝐞𝐧𝐜𝐞 & 𝐎𝐂𝐑 - Specialist in extracting structure from "dirty" or non-digital sources using Visual NLP and OCR. - Lead Developer of Spark OCR. - Author and main contributor of open-source projects: Spark PDF and ScaleDP. - Expertise in document layout analysis, data extraction, and semantic search. 𝐏𝐫𝐨𝐝𝐮𝐜𝐭𝐢𝐨𝐧 𝐌𝐋𝐎𝐩𝐬 & 𝐁𝐢𝐠 𝐃𝐚𝐭𝐚 - Architecting scalable infrastructures using Vector DBs (Milvus, Pinecone, Qdrant). - Optimizing Apache Spark (PySpark, Streaming) and Databricks pipelines; reduced processing time from days to hours. - Implementing robust CI/CD via Databricks Asset Bundles, Jenkins, and GitHub Actions. 𝐏𝐫𝐢𝐯𝐚𝐜𝐲 & 𝐂𝐨𝐦𝐩𝐥𝐢𝐚𝐧𝐜𝐞 - Automated de-identification and redaction for GDPR/HIPAA. - Specialist in masking sensitive information in DICOM, PDF, and image formats. 𝐓𝐞𝐜𝐡𝐧𝐨𝐥𝐨𝐠𝐢𝐞𝐬 AI & LLM: LangGraph, LangChain, RAG, Llama 3, Gemini, GPT-4, Vertex AI, Hugging Face, GliNER. Backend: Python, Scala, FastAPI, Node.js, PostgreSQL, MongoDB. Vector DBs: Pinecone, Milvus, Qdrant. Infrastructure: AWS, GCP, Azure, Docker, Kubernetes, Jenkins, AirFlow. Big Data: Apache Spark (PySpark, Streaming, MLlib), Kafka, Kinesis.

- 10+ years of professional experience in JVM-based software development. - Expertise in Scala’s modern FP stack (Typelevel, ZIO, Akka), data engineering (Hadoop, AWS, Spark), and Java (Spring) for high-performance solutions. - A product-oriented mindset, focused on understanding the business domain deeply to deliver solutions that align with and exceed business expectations.

🔥 Looking for a Data Engineer who knows how to scale? I help enterprises transform raw data into real-time insights using Databricks, Azure, and streaming architectures that deliver results. 👉 Press “𝐈𝐍𝐕𝐈𝐓𝐄 𝐓𝐎 𝐓𝐇𝐄 𝐉𝐎𝐁” or “𝐒𝐄𝐍𝐃 𝐀 𝐌𝐄𝐒𝐒𝐀𝐆𝐄” and let’s talk about how I can help you build a high-performance data platform. 🚀 Case Study: I partnered with Databricks Professional Services (via Azure) to deliver a production-grade, structured streaming pipeline for a client that ingests over 4 billion events per day. The result? A scalable, monitored system that powers real-time analytics and intelligent automation—proving that AI Chatbots and Agents can thrive even in high-volume, enterprise-grade environments. 📜 Certifications: 🧠 Generative AI – Databricks 🏗️ Azure Databricks Platform Architect – Databricks 🧱 Databricks Lakehouse – Databricks 🔧 Databricks Platform – Databricks 🛡️ Platform Administrator – Databricks I’m a certified Data Engineer with deep expertise in Databricks, Azure, and distributed data systems. I’ve worked with enterprise clients (1000+ employees) to deploy scalable, multi-region data platforms that power real-time analytics and business growth. ✅ As a Data Engineer, I deploy enterprise-grade data infrastructure by rolling out Databricks across 13 regions to support global operations with high availability and performance. ✅ As a Data Engineer, I build streaming pipelines that process billions of records daily with near real-time SLAs (<1 minute), enabling fast, actionable insights. ✅ As a Data Engineer, I drive impact-driven analytics that enabled a client to onboard 500+ new customers by unlocking insights from their data lake through Databricks-powered analysis. ✅ As a Data Engineer, I collaborate with Microsoft Professional Services to optimize Azure and Databricks performance, ensuring cost efficiency and speed at scale. ✅ As a Data Engineer, I design secure and reliable architectures—implementing Medallion Architecture, Delta Lake optimizations, and CI/CD workflows for robust data operations. 🤔 What’s stopping us from building something powerful together? Send me a message, and let’s kick off your next data engineering project!

💪Technical Expertise: ✅ Python -> FastAPI, PostMan,BeautifulSoup, Selenium, Requests ✅ SQL -> PostgreSQL, SQLite, MS SQL, MySQL ✅ No-SQL Databases (MongoDB) ✅ Orchestration -> Airflow, Talend, Pentaho, Databricks, Azure Data Factory) ✅ Containerization and CI/CD (Docker, Kubernetes) ✅ Cloud Provider Experience (Azure, AWS) ✅ Data Modelling 𝐌𝐲 𝐒𝐞𝐫𝐯𝐢𝐜𝐞𝐬 💻: 🛠️ 𝗘𝗧𝗟 𝗣𝗶𝗽𝗲𝗹𝗶𝗻𝗲: Need that pipeline to move your data from source to destination? JSON, CSV, EXCEL, SQL sources to any database of your choice. On-prem or cloud platform? You want to use Databricks or another service? In real-time or batches? You need consultations on choosing the best approach/service/design for your business usecase? I have got you covered - just a booking away to providing you the optimal solution. 📊 𝐖𝐞𝐛 𝐒𝐜𝐫𝐚𝐩𝐢𝐧𝐠: Data collation can be a tedious task asmost of the data are not readily available in a needed format and/or location, JavaScript complexities, dynamic pages, login - I have got you covered. Want your data collated in an organized and formatted manner - JSON, CSV, XLSX, or loaded to a database via an API - just name it. 🌐 𝐀𝐏𝐈 𝐃𝐞𝐯𝐞𝐥𝐨𝐩𝐦𝐞𝐧𝐭: Looking for a reliable REST API for your business? Our solution delivers robust performance, secure authentication, and clear Pydantic validations. With intuitive documentation, we make integration effortless. Let's streamline your business processes! 📈 𝐃𝐚𝐭𝐚 𝐀𝐧𝐚𝐥𝐲𝐬𝐢𝐬: Looking to uncover insights from your business data to drive revenue growth, optimize processes, or reduce costs? Our data analysis offers detailed insights and clear visualizations to help you make informed decisions. Let's transform your data into actionable strategies! ⚙️ 𝐀𝐮𝐭𝐨𝐦𝐚𝐭𝐢𝐨𝐧𝐬: Need a UI to automate your tasks - tkinter, Gradio, Streamlit? I have got you covered. Just with a button click, all your tasks will be automated and save you the stress of doing them manually. 🛠 𝐂𝐮𝐬𝐭𝐨𝐦 𝐒𝐜𝐫𝐢𝐩𝐭𝐢𝐧𝐠: Developing custom scripts and applications for specific use case which includes data cleaning, automated form filling and task scheduling. Just one click away. 🔍 𝐖𝐡𝐲 𝐥𝐞𝐭 𝐦𝐞 𝐡𝐚𝐧𝐝𝐥𝐞 𝐲𝐨𝐮𝐫 𝐭𝐚𝐬𝐤𝐬? ✔️ Diverse Experience ✔️ Proven track record 📋 of succesful deliveries. ✔️ Attention to detail - Watchful eyes 👀 to ensure nothing is missed . ✔️ On-time delivery - You'll get your deliverable as at when due ⏰, even earlier 😁 ✔️ Excellent communication skills - All details of the project at your finger tips 📞