Hire the Best Pyspark Developers
in Poland

I am a computer science student specializing in systems engineering, passionate about machine learning and data analysis. I have hands-on experience with Python, SQL, and PySpark, gained through academic projects and self-learning. I aim to develop innovative solutions in AI and Data Science.

Over 22 years of experience working on enterprise class systems. Variety of projects and customers. Always focused on performance and reliability. Always on cutting edge of technology. Keeping in mind business objectives and customers interests. Competencies: • Solution Selection, Architecture Design, Implementation, Customization, Migration, Internal Adoption - Freelance Consulting & Audit BigData & Machine Learning Expertise: • Machine Learning Models adoption from business use-case until production maintenance • Data Collection, Data pipelines, Data Workflows • Big Data ETL & Mining • System Integration I have a solid portfolio of enterprise level, successfully implemented solutions as well as ad-hoc optimizations of existing systems in large, global organizations.

Full-Stack AI Lead with 15+ years of experience in AI/ML/Data Engineering. Specialist in productionizing Document AI, Agentic LLM Workflows, and Scalable Big Data systems. Master’s in Applied Mathematics. 𝐀𝐠𝐞𝐧𝐭𝐢𝐜 𝐀𝐈 & 𝐋𝐋𝐌 - Architecting production-grade agents and multi-intent workflows using LangGraph and LangChain. - Implementing NeMo Guardrails, DeepEval, and Langfuse to ensure security and near-zero hallucination rates. - Developing high-performance FastAPI backends for real-time operational data integration. 𝐃𝐨𝐜𝐮𝐦𝐞𝐧𝐭 𝐈𝐧𝐭𝐞𝐥𝐥𝐢𝐠𝐞𝐧𝐜𝐞 & 𝐎𝐂𝐑 - Specialist in extracting structure from "dirty" or non-digital sources using Visual NLP and OCR. - Lead Developer of Spark OCR. - Author and main contributor of open-source projects: Spark PDF and ScaleDP. - Expertise in document layout analysis, data extraction, and semantic search. 𝐏𝐫𝐨𝐝𝐮𝐜𝐭𝐢𝐨𝐧 𝐌𝐋𝐎𝐩𝐬 & 𝐁𝐢𝐠 𝐃𝐚𝐭𝐚 - Architecting scalable infrastructures using Vector DBs (Milvus, Pinecone, Qdrant). - Optimizing Apache Spark (PySpark, Streaming) and Databricks pipelines; reduced processing time from days to hours. - Implementing robust CI/CD via Databricks Asset Bundles, Jenkins, and GitHub Actions. 𝐏𝐫𝐢𝐯𝐚𝐜𝐲 & 𝐂𝐨𝐦𝐩𝐥𝐢𝐚𝐧𝐜𝐞 - Automated de-identification and redaction for GDPR/HIPAA. - Specialist in masking sensitive information in DICOM, PDF, and image formats. 𝐓𝐞𝐜𝐡𝐧𝐨𝐥𝐨𝐠𝐢𝐞𝐬 AI & LLM: LangGraph, LangChain, RAG, Llama 3, Gemini, GPT-4, Vertex AI, Hugging Face, GliNER. Backend: Python, Scala, FastAPI, Node.js, PostgreSQL, MongoDB. Vector DBs: Pinecone, Milvus, Qdrant. Infrastructure: AWS, GCP, Azure, Docker, Kubernetes, Jenkins, AirFlow. Big Data: Apache Spark (PySpark, Streaming, MLlib), Kafka, Kinesis.

- 10+ years of professional experience in JVM-based software development. - Expertise in Scala’s modern FP stack (Typelevel, ZIO, Akka), data engineering (Hadoop, AWS, Spark), and Java (Spring) for high-performance solutions. - A product-oriented mindset, focused on understanding the business domain deeply to deliver solutions that align with and exceed business expectations.

🚀 Build scalable data infrastructure 🛠 Automate data workflows 📊 Deliver actionable analytics I’m a certified data consultant specializing in Microsoft Fabric, Azure Databricks, SQL, Python, and Power BI. I help companies, from SaaS startups to global enterprises, turn complex, fragmented data into reliable, analytics-ready datasets that drive faster decisions and product growth. My work spans the full data lifecycle: designing architecture, building ETL pipelines, managing data lakes, and delivering secure reporting layers. I’ve delivered 20+ successful remote projects across logistics, maritime, energy, and real estate - from setting up 5+ Microsoft Fabric environments from scratch to managing infrastructure for Verizon AI’s 80M+ company dataset. Core Skills 📶 Data Engineering & Analytics: Microsoft Fabric, Azure Databricks, SQL Server, T-SQL, Python, PySpark, DAX, M ☁️ Cloud & Orchestration: Azure Data Factory, ADLS Gen2, Medallion Architecture, CI/CD, Git 🔌 Integrations: REST APIs, CRMs (Salesforce, HubSpot), ERPs (SAP, QuickBooks), IoT data 📊 Data Visualization: Power BI (Embedded), Apache Superset Example Use Cases End-to-end SaaS data infrastructure for lead-generation platforms ETL pipelines consolidating CRM, ERP, and IoT data into analytics-ready datasets Embedded analytics portals for client KPI tracking Role-based reporting for operations, finance, and product teams Microsoft Certified PL-300 – Power BI Data Analyst Associate DP-700 – Fabric Data Engineer Associate DP-500 – Azure Enterprise Data Analyst Associate Clients choose me because I: 🔸 Deliver on time and to spec 🔸 Take ownership of the whole solution, from ingestion to analytics 🔸 Communicate clearly with both technical and non-technical teams 🔸 Bring both enterprise-scale engineering and startup agility to projects If you’re looking for a technical data consultant who can architect, build, and optimize your data infrastructure - let’s connect.

I am a Senior Data Engineer with 9+ years of experience architecting high‑performance data pipelines, cloud data warehouses, and enterprise BI solutions. I specialise in DBT‑driven transformation layers, BigQuery optimisation, and large‑scale ETL/ELT orchestration – turning raw, fragmented data into trusted, analytics‑ready assets that power strategic decisions. My approach combines deep data modelling expertise with modern engineering practices: version‑controlled transformations, automated testing, CI/CD for data, and cost‑aware cloud design. I deliver systems that are not only scalable and reliable but also maintainable and well‑documented for your entire team. ✔️𝐒𝐩𝐞𝐜𝐢𝐚𝐥𝐢𝐳𝐞𝐝 𝐄𝐱𝐩𝐞𝐫𝐭𝐢𝐬𝐞. - DBT (Data Build Tool): Advanced modelling, incremental strategies, macro development, and rigorous data testing for analytical warehouses. - BigQuery Optimisation: Partitioning, clustering, slot management, and query cost control – delivering sub‑second responses on petabyte‑scale data. - Big Data Processing: Spark (PySpark) and Kafka for stream/batch processing, with efficient handling of high‑volume, high‑velocity datasets. ✔️𝐒𝐭𝐫𝐨𝐧𝐠 𝐄𝐱𝐩𝐞𝐫𝐢𝐞𝐧𝐜𝐞 𝐢𝐧 - E‑commerce & retail analytics: building 360° customer views, inventory forecasting, and real‑time sales dashboards. - Financial data platforms: regulatory reporting, fraud detection pipelines, and GDPR‑compliant data masking. - Real‑time streaming architectures: Kafka + BigQuery + dbt for near‑real‑time operational intelligence. ✔️𝐏𝐫𝐢𝐦𝐚𝐫𝐲 𝐒𝐭𝐚𝐜𝐤. - Cloud Platforms: GCP (BigQuery, Composer, Pub/Sub), Azure (Data Factory, Synapse, Databricks), AWS (Glue, Redshift, S3). - Warehousing & Lakes: BigQuery, Snowflake, Azure Synapse, MSSQL, PostgreSQL, MySQL. - Transformation & Orchestration: dbt (core/cloud), Apache Airflow, Prefect, Azure Data Factory. - Languages: Advanced SQL, Python (Pandas, PySpark, Polars), DAX, M (Power Query). - BI & Visualisation: Power BI (Service, Premium, Paginated Reports), Tableau, Looker Studio. - Big Data Tech: Apache Spark, Kafka, Flink, Databricks (Delta Lake). - DevOps for Data: Git, CI/CD (GitHub Actions, Azure DevOps), Docker, Terraform (basic). - Data Governance: RLS, column‑level security, data lineage, and quality monitoring (Great Expectations). ✔️𝐖𝐡𝐞𝐭𝐡𝐞𝐫 𝐲𝐨𝐮 𝐧𝐞𝐞𝐝 𝐭𝐨 - Migrate legacy ETL to a modern dbt + BigQuery stack with full CI/CD, - Build a high‑throughput streaming pipeline that ingests millions of events per second, - Design a self‑service semantic layer that empowers business users with reliable KPIs, - Or optimise your existing warehouse for cost and performance without sacrificing freshness, I deliver a clear roadmap, measurable improvements, and a system your team can confidently own. ✔️𝐖𝐡𝐚𝐭 𝐲𝐨𝐮 𝐠𝐞𝐭 ✅ A data platform built for scale – not just today, but for 10x growth ✅ Cloud cost optimisation that reduces your monthly bill by 20‑40% ✅ Daily updates (Slack/email) with progress, blockers, and next steps ✅ Clean, modular dbt code with full documentation and automated tests ✅ Production‑ready pipelines with monitoring, alerting, and rollback strategies Message me or invite me to a quick 15‑min call – I will walk you through a tailored architecture sketch for your project. Keywords: Data Engineer, DBT, BigQuery, Big Data, ETL, ELT, Cloud Analytics, Power BI, MSSQL, Python, SQL, GCP, Azure, AWS, Data Warehousing, Snowflake, Databricks, Apache Airflow, Kafka, Spark, PySpark, Data Pipeline, Business Intelligence, Dashboard, Data Modelling, DAX, M, Data Lake, Data Integration, E‑commerce Analytics, Financial Data, Real‑time Streaming, CI/CD, DevOps, Data Governance, Performance Optimisation, Cloud Migration, Senior Data Engineer, Remote, Full‑time.