Hire the best Apache Kafka developers

Check out Apache Kafka developers with the skills you need for your next job.
Clients rate Apache Kafka developers
Rating is 4.7 out of 5.
4.7/5
based on 384 client reviews
  • $35 hourly
    I am creative, curious, analytical, and often considered a "Problem-Solver" at work. Having ease at learning new applications and programming languages by myself, I love to dive deeper into new concepts and tools of Software Development. I have a bachelor's degree in Computer Science. I have been working on the solution of scalable microservices on Java Spring Framework, in order to deliver new features used on multiple Client applications. Check below my main experiences: • Design and development of scalable microservices integrated with RESTful HTTP APIs and Messaging Brokers (Kafka and IBM MQ), based on Hexagonal/Clean Architecture. • Creating and deploying CI/CD solutions on OpenShift with Jenkins and AWS. • Projects implemented with Java Spring Framework, using Spring Boot with dependency management through Maven and code versioning by Git. • Automated testing practices using Testing Pyramid (Unit, Integration, and Component tests automation) to have a maintainable test suite. • Problem prospecting and solving using Observability tools such as Amazon CloudWatch, Splunk, and Grafana. • Experience at DevOps culture and Scrum Methodology working with synergic squads to design and build high-end applications and use of the project management platforms JIRA and Confluence. As I have a great interest and love for video games, I have been studying Game Programming in my spare time, mainly C++ programming and Unreal Engine development.
    vsuc_fltilesrefresh_TrophyIcon Apache Kafka
    CI/CD
    Linux
    Jenkins
    Apache Tomcat
    Microservice
    AWS Lambda
    .NET Framework
    Computer Science
    SQL Programming
    Spring Framework
    Entity Framework
    API
    AWS CloudFormation
    AWS Application
    Python
    Java
    C++
  • $45 hourly
    HI I have 10 years of experience in programming (backend). My main programming language is Golang, and I have experience creating microservice applications using Docker, Kubernetes (ECS and GKE), setup CI/CD using GitLab-ci or Jenkins, and most of my projects using Echo framework for APIs and for connecting internal services I use Grpc or Kafka.
    vsuc_fltilesrefresh_TrophyIcon Apache Kafka
    Kubernetes
    PostgreSQL
    Back-End Development
    RESTful API
    RabbitMQ
    Swagger
    Git
    Stripe
    Microservice
    Google Cloud Platform
    Amazon ECS for Kubernetes
    MySQL
    Docker
    Golang
  • $95 hourly
    => Let's Connect Hello, I'm Dima, a seasoned CyberSecurity Specialist and Turnkey Infrastructure Expert specializing in BigData solutions and data analysis, utilizing a DevOps approach. => Expertise Overview With a robust passion for constructing SOC, SOAR, and SIEM solutions, my primary focus lies in developing data ingestion, enrichment, and analysis pipelines, ensuring they are highly available and fault-tolerant. My expertise extends to building central logging and real-time processing platforms from the ground up, optimizing them for performance, security, and reliability across multiple environments, whether in the cloud or on-premise. => Value Proposition My commitment is to deliver solutions that not only centralize security and threat intelligence but also facilitate enhanced control over data, ultimately contributing to infrastructure cost savings. => Technological Summary CyberSecurity:------- > Wazuh, Suricata, pfSense BigData:--------------- > Kafka, ElasticSearch, OpenSearch Data Processing:----- > FluentD, Vector.dev, Apache NiFi Infra as Code:--------- > Terraform, cdktf, cdk8s Virtualization:--------- > Proxmox, VMware Containerization:----- > Kubernetes Clouds:---------------- > AWS, Hetzner, DigitalOcean, Linode Automation:----------- > Jenkins, GitHub Actions Monitoring:----------- > Zabbix, Grafana, Kibana, Prometheus, Thanos Mail:--------------------> MailCow SMTP/IMAP, Postfix VPN:------------------- > OpenVPN Server Programming:-------- > Bash, Python, TypeScript Operating Systems:- > CentOS, RHEL, Rocky Linux, Ubuntu, Debian => Personal Attributes • Leadership: Leading by example with a team-first approach • End-to-End Execution: Proficient from POC to Enterprise-level implementation • Resilience: Demonstrating high thoroughness and endurance • Adaptability: A quick, can-do architect and experienced troubleshooter • Optimization: Adept in process and performance optimization • Documentation: Skilled technical documentation writer • Vision: A visionary in technological implementation and solution provision
    vsuc_fltilesrefresh_TrophyIcon Apache Kafka
    Elasticsearch
    Linux System Administration
    Apache Hadoop
    Email Security
    Machine Learning
    ELK Stack
    Cloudera
    Zabbix
    MySQL
    Big Data
    Apache NiFi
    PfSense
    Red Hat Administration
    Proxmox VE
    Amazon Web Services
  • $50 hourly
    10 yrs of experience, will develop a production-ready SaaS application for you from A to Z: ✅ login and registration flow ✅ business flow/rules using OOP, SOLID, KISS, GRASP principles ✅ booking/appointment functionality if needed ✅ integrate with payment systems such as Stripe and Paypal ✅ fetch data from 3rd party services, REST api, if needed ✅ integrate your SaaS with ChatGpt or any other AI system ✅ an admin panel to manage users, payments, etc ✅ test the code using unit tests, integration tests and e2e tests to avoid bugs ✅ deploy the app to AWS, Azure or on-prem Message me to get a FREE consultation about your product idea! p.s. I did SaaS for such domains as: Ed Tech, AI, Health Care Other domains I worked with: Airline, Transportation, Real Estate, Smart Home (IoT), IT Security, Banking ----------------------------------------------------------------------------------------------- Below are technologies are for Upwork search engine Architectures: - Microservice - Monolith - SOA (Service Oriented Architecture) - Event Driven - Serverless - Event Sourcing and CQRS Programming languages: - Java 6, 7, 8, 11, 15, 17, JavaScript, TypeScript Frontend: - ReactJs, Angular, VueJs, Jquery Protocols/connectivity: - HTTP, HTTPs, FTP, SSH - REST, SOAP - RabbitMQ, Kafka, ActiveMQ - AWS SNS, AWS SQS, AWS Kinesis, AWS MSK - WebSockets Authentication/Security: - JWT, OAuth2 - Spring Security - AWS Cognito - Keycloak 21.0 - auth0 - social logins (Facebook, Google) Web Frameworks: - NestJS, NextJS, Node JS - Spring Boot (REST API, MVC, Data Jpa, Security, etc) - Quarkus - Google Guice - Java EE (EJB, JPA, etc) ORMs: - Hibernate - iBatis - Spring Data Jpa - Spring Data Mongo - Spring Data Neo4j - JDBC template Databases: - MySQL 5.7, 8.0 - MSSQL - Postgresql 14 - Oracle - Sybase - Mongodb - Neo4j - Redis Clouds: - AWS: EC2, ECS, S3, Lambda, Kinesis, MSK, EventBridge, SQS, SNS, CloudWatch, CloudFormation, RDS, Aurora, Cognito - Azure - GCP - Heroku CI/CD: - Jenkins - TeamCity - Github Actions - AWS Codebuild - SonarQube Devops: - Docker - Docker Compose - Kubernetes (k8s) - Linux Others: - Chatwoot
    vsuc_fltilesrefresh_TrophyIcon Apache Kafka
    ChatGPT API Integration
    JavaScript
    NodeJS Framework
    MySQL
    SOAP
    PostgreSQL
    Web Development
    RabbitMQ
    RESTful API
    Spring Boot
    SQL
    Node.js
    Java
    Amazon Web Services
  • $120 hourly
    Freelancer/consultant with a focus on quality and speed (but quality first). I began Canusi GmbH in 2019 and have made many of the mistakes early entrepeneurs and freelancers make. Today I work on a part time basis freelancing with a focus on solving business problems, by drawing on SOLID, TDD, and DDD. I've experience mentoring engineers/project managers and also scaling up teams quickly by improving engineering processes, and reducing internal overhead on decision making. I specialize particularly in Java microservices with Spring and Kafka, with near a decade in data driven systems and around half of that in event driven system design. What I expect of partners: - willingness to be direct and tackle difficult decisions - focus on long term relationships and investment - ownership of the vision and process What you can expect from me: - a "whatever it takes" mentality, and the resilience to see through difficult moments in a project - laser focus on your priorities, not the politics - high confidence that the technology delivered will be effective, well maintained, and scalable - impecceable communication skills Experience - 10 years Java - 4 years Scala - 5 years in ETL, Oracle, Postgresql, and relational database management - 2 years technical lead/project manager Certification - Scrum Master - Kubernetes
    vsuc_fltilesrefresh_TrophyIcon Apache Kafka
    Kubernetes
    PostgreSQL
    Amazon Web Services
    RESTful Architecture
    Docker
    Akka
    Java
    Python
    Scala
  • $30 hourly
    Summary: Software Engineer, Golang, Java, ML. Kafka,Spark and Flink. London Kafka Summit Speaker, Confluent Kafka Community Advocate.
    vsuc_fltilesrefresh_TrophyIcon Apache Kafka
    Database
    Containerization
    Docker
    Video Stream
    Distributed Computing
  • $50 hourly
    A Backend Software Engineering with more than 6 years of experience. Have worked with large-scale backend/distributed systems and big data systems. A DevOps engineer with 4 years of experience - both on-premises and AWS, experienced with K8s, Terraform, Ansible, CI/CD. Currently working as Principal Engineer/ Solution Architect role.
    vsuc_fltilesrefresh_TrophyIcon Apache Kafka
    DevOps
    Docker
    Elasticsearch
    GraphQL
    Scala
    Serverless Computing
    Architectural Design
    Kubernetes
    Apache Spark
    Apache Hadoop
    Amazon Web Services
    API Development
  • $100 hourly
    I help companies to design and build software. My interests are Big Data and Machine Learning. During recent years is particularly focused on scalable computing and data grids. Skills: Apache Spark, Apache Kafka, Apache Hadoop, GigaSpaces, Machine Learning, Scala, Java and Spring. Can be found blogging at dyagilev.org
    vsuc_fltilesrefresh_TrophyIcon Apache Kafka
    Spring Framework
    Java
    Scala
    Apache Hadoop
    Apache Spark
  • $99 hourly
    - PyTorch, Machine Learning, LangChain, Elasticsearch, Cassandra, Neo4j, Kafka - AWS, GCP, Azure, Docker, K8s and Distributed Systems - ReactJS, NodeJS, Python, Spring Boot, Java, Postgres, Mysql - Top Rated Plus and 100% Success Rate since May 2022 I have contributed to projects in 20+ organizations in adapting Distributed Applications for different use cases such as creating search solutions for big data, identifying fake & duplicate businesses, recommendation model, finding anomalies and log monitoring in domains like finance, directory services, e-commerce and registrar services. I provide complete distributed application solution from setting up cluster, tuning, optimizations, queries, data modeling and client implementation in Spring Boot, Micro Services, Python, AWS, ElasticSearch, Cassandra, Hive, Apache Kafka. I am a B.Tech in computer science with 15+ years of IT experience. I contributed to enterprise level projects for Yelp, Bank of America, Capitol One and YP.com.
    vsuc_fltilesrefresh_TrophyIcon Apache Kafka
    React
    Spring Boot
    Elasticsearch
    RESTful API
    Apache Cassandra
    Selenium WebDriver
    Big Data
    ELK Stack
    Hibernate
    Django
    Java
    Node.js
    Python
  • $50 hourly
    "⭐️⭐️⭐️⭐️⭐️ Recommended. Always fast, efficient and always open to discussion. The assigned tasks were performed at a high level. It was easy to find common ground..." I am Top-Rated Full Stack Developer with 8 years of experience in Web/Software development. I do development, designing, debugging, covering with tests, analytics of architecture, finding the best solutions for your needs, increasing performance, finding solutions, team-leading, and managing teams of up to 10 people. I was working on various projects and different roles. I am great as a standalone developer and also perfect as a team player. I worked with 90+ Clients Across Various Platforms 🏆🔥 I am proficient in: Font-end ✅ HTML/HTML5 ✅ CSS/CSS3 ✅ JavaScript ✅ Jquery/Ajax ✅ JSP ✅ AngularJS/Angular2+ ✅ React ✅ Bootstrap ✅ FontAwesome Databases ✅ MySQL ✅ PostgreSQL ✅ MS Access ✅ MSSQL ✅ OracleDB ✅ MongoDB Application/Web Servers ✅ Apache Tomcat ✅ NGINX Programming Languages/ Technologies ✅ Java ✅ TypeScript ✅ JavaScript ✅ Software Architecture Design ✅ UML ✅ Reverse Engineering ✅ Design Patterns ✅ JDBC ✅ RMI ✅ Maven ✅ JSP ✅ Servlets ✅ Web Services ✅ JPA ✅ DataSource ✅ DBCP ✅ Rest ✅ SQL ✅ SpEl Frameworks ✅ Spring ✅ Spring Security ✅ Spring MVC ✅ Hibernate ✅ JPA ✅ Spring Data ✅ Spring Oauth2.0 ✅ EJB ✅ JUnit Development Tools ✅ Eclipse ✅ NetBeans ✅ IntelliJ idea ✅ Git Bash ✅ SmartGit ✅ Visual Studio Code ✅ Sublime Text Editor ✅ JUnit/ Log4J If you are looking for a developer who can deliver the highest quality code, facilitate easy and comprehensive communication, engage in detail-oriented programming, and ultimately satisfy the customer, you have found the right person. Looking forward to our cooperation Best, Oleg.
    vsuc_fltilesrefresh_TrophyIcon Apache Kafka
    Apache Tomcat
    JUnit
    MongoDB
    CSS 3
    Jenkins
    Web Design
    Java
    PostgreSQL
    Web Application
    Spring Boot
    Spring Framework
    SaaS
    RESTful Architecture
    Angular
  • $40 hourly
    I'm a motivated and reliable full-stack software developer with a strong skill set across multiple technologies. My experience covers both front-end and back-end development, and I maintain a well-organized and documented coding approach to facilitate project scalability. I'm a proactive and dedicated learner with a growth-oriented mindset, consistently seeking personal growth and expanding my technical expertise. With several years of experience collaborating with clients and leading diverse teams, I prioritize responsibility, effective communication, and efficiency. I offer a free consultation service where I'm more than willing to share my extensive experience in assisting clients with building web and mobile applications from the ground up. This includes tasks like setting up MVPs, selecting the appropriate technology stack, outlining a practical plan and timeline, estimating the budget, and more. Please feel free to reach out to me for this complimentary service, and together, we can turn your concept into a reality.
    vsuc_fltilesrefresh_TrophyIcon Apache Kafka
    Apache Spark
    ClickHouse
    Apache Druid
    Kotlin
    Apache Flink
    Aerospike
    Spring Boot
    Amazon Athena
    Amazon Redshift
    MongoDB
    Kubernetes
    Python
    Java
    Docker
  • $30 hourly
    Optimistic, forward-looking software developer with 1.5+ years' background in creating and executing innovative software solutions. I have worked on Big Data project and have experience to handle a huge amount of data. - I'm experienced in Spark, Scala, Hive, HDFS. - I'm experienced in Docker and Kubernetes. - I have a good data handling skill. - Design before Code, Always!
    vsuc_fltilesrefresh_TrophyIcon Apache Kafka
    Docker Compose
    PySpark
    Kubernetes
    Apache Hive
    Apache Flume
    Docker
    Apache Hadoop
    MySQL
    Apache Spark
    Spring Boot
    RESTful API
    Java
    Scala
    Python
  • $85 hourly
    I pride myself on achieving a 𝗽𝗲𝗿𝗳𝗲𝗰𝘁 𝗿𝗲𝗰𝗼𝗿𝗱 𝗼𝗳 𝟱-𝘀𝘁𝗮𝗿 𝗿𝗮𝘁𝗶𝗻𝗴𝘀 𝗮𝗰𝗿𝗼𝘀𝘀 𝗮𝗹𝗹 𝗽𝗿𝗼𝗷𝗲𝗰𝘁𝘀. My expertise in 𝗰𝗹𝗼𝘂𝗱 𝗱𝗮𝘁𝗮 𝗲𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴 and 𝗳𝘂𝗹𝗹-𝘀𝘁𝗮𝗰𝗸 development has been honed through experience with premier institutions like 𝗚𝗼𝗹𝗱𝗺𝗮𝗻 𝗦𝗮𝗰𝗵𝘀, 𝗠𝗼𝗿𝗴𝗮𝗻 𝗦𝘁𝗮𝗻𝗹𝗲𝘆, a member of the 𝗕𝗶𝗴 𝗙𝗼𝘂𝗿 and a 𝗙𝗼𝗿𝘁𝘂𝗻𝗲 𝟱𝟬𝟬 company. With over 9 years of experience in Data Engineering and Programming, I bring a commitment to excellence and a passion for perfection in every project I undertake. My approach is centered around delivering not just functional, but 𝗵𝗶𝗴𝗵𝗹𝘆 𝗲𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝘁 𝗮𝗻𝗱 𝗼𝗽𝘁𝗶𝗺𝗶𝘇𝗲𝗱 code, ensuring top-quality outputs that consistently impress my clients. My expertise combined with extensive experience on both GCP and AWS Cloud platforms, allows me to provide solutions that are not only effective but also innovative and forward-thinking. I believe in going beyond the basics, striving for excellence in every aspect of my work, and delivering results that speak for themselves. 𝗖𝗵𝗼𝗼𝘀𝗲 𝗺𝗲 𝗶𝗳 𝘆𝗼𝘂 𝗽𝗿𝗶𝗼𝗿𝗶𝘁𝗶𝘇𝗲 𝘁𝗼𝗽-𝗻𝗼𝘁𝗰𝗵 𝗾𝘂𝗮𝗹𝗶𝘁𝘆 𝗶𝗻 𝘆𝗼𝘂𝗿 𝗽𝗿𝗼𝗷𝗲𝗰𝘁𝘀 𝗮𝗻𝗱 𝗮𝗽𝗽𝗿𝗲𝗰𝗶𝗮𝘁𝗲 𝗮 𝗳𝗿𝗲𝗲𝗹𝗮𝗻𝗰𝗲𝗿 𝘄𝗵𝗼 𝗮𝘂𝘁𝗼𝗻𝗼𝗺𝗼𝘂𝘀𝗹𝘆 𝗺𝗮𝗸𝗲𝘀 𝗼𝗽𝘁𝗶𝗺𝗮𝗹 𝗱𝗲𝗰𝗶𝘀𝗶𝗼𝗻𝘀, 𝘀𝗲𝗲𝗸𝗶𝗻𝗴 𝗰𝗹𝗮𝗿𝗶𝗳𝗶𝗰𝗮𝘁𝗶𝗼𝗻𝘀 𝗼𝗻𝗹𝘆 𝘄𝗵𝗲𝗻 𝗮𝗯𝘀𝗼𝗹𝘂𝘁𝗲𝗹𝘆 𝗻𝗲𝗰𝗲𝘀𝘀𝗮𝗿𝘆. 𝗔𝗿𝗲𝗮𝘀 𝗼𝗳 𝗘𝘅𝗽𝗲𝗿𝘁𝗶𝘀𝗲: - 𝗖𝗹𝗼𝘂𝗱: GCP (Google Cloud Platform), AWS (Amazon Web Services) - 𝗣𝗿𝗼𝗴𝗿𝗮𝗺𝗺𝗶𝗻𝗴 𝗟𝗮𝗻𝗴𝘂𝗮𝗴𝗲: Java, Scala, Python, Ruby, HTML, Javascript - 𝗗𝗮𝘁𝗮 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴: Spark, Kafka, Crunch, MapReduce, Hive, HBase, AWS Glue, PySpark, BiqQuery, Snowflake, ETL, Datawarehouse, Databricks, Data Lake, Airflow, Cloudwatch 𝗖𝗹𝗼𝘂𝗱 𝗧𝗼𝗼𝗹𝘀: AWS Lambda, Cloud Functions, App Engine, Cloud Run, Datastore, EC2, S3, - 𝗗𝗲𝘃𝗢𝗽𝘀: GitHub, GitLab. BitBucket, CHEF, Docker, Kubernetes, Jenkins, Cloud Deploy, Cloud Build, - 𝗪𝗲𝗯 & 𝗔𝗣𝗜: SpringBoot, Jersey, Flask, HTML & JSP, ReactJS, Django 𝗥𝗲𝘃𝗶𝗲𝘄𝘀: "Amar is a highly intelligent and experienced individual who is exceeding expectations with his service. He has very deep knowledge across the entire field of data engineering and is a very passionate individual, so I am extremely happy to have finished my data engineering project with such a responsible fantastic guy. I was able to complete my project faster than anticipated. Many thanks...." "Amar is an exceptional programmer that is hard to find on Upwork. He combines top-notch technical skills in Python & Big Data, excellent work ethic, communication skills, and strong dedication to his projects. Amar systematically works to break down complex problems, plan an approach, and implement thought-out high-quality solutions. I would highly recommend Amar!" "Amar is a fabulous developer. He is fully committed. Is not a clock watcher. Technically very very strong. His Java and Python skills are top-notch. What I really like about him is his attitude of taking a technical challenge personally and putting in a lot of hours to solve that problem. Best yet, he does not charge the client for all those hours, He still sticks to the agreement. Very professional. It was a delight working with him. and Will reach out to him if I have a Java or Python task."
    vsuc_fltilesrefresh_TrophyIcon Apache Kafka
    Google App Engine
    BigQuery
    Software Development
    Web Development
    Machine Learning
    Big Data
    PySpark
    Google Cloud Platform
    Amazon Web Services
    Apache Airflow
    Apache Spark
    Data Engineering
    SQL
    Python
    Java
  • $70 hourly
    "Yonadav B. proved to be a valuable freelancer for us who quickly resolved our KPI aggregation problems and transformed our system into a scalable one. Despite minimal onboarding, Yonadav swiftly fixed the issues and proceeded to rewrite the aggregation system in Python. His ability to work under pressure, technical expertise, and dedication to delivering high-quality work were truly commendable. We highly recommend Yonadav to any company in need of urgent data science or pipeline solutions, as his contributions to our project were invaluable." - Filipe Silvestrim of Oxolo "I highly recommend Yonadav on every level of a project. He was our lead developer working on a reporting / media buying platform which has helped us scale our spend by over 1000%. He's been truly wonderful to work with, understands all instructions and needs given, and works around the clock to get it done. I am looking forward to continuing building this product with Yonadav and other projects." - Raymond Attipa of Duke Digital Group With over 10 years of experience, I have engaged in a huge amount of software products development. They cover micro services, backend daemons, ETL pipelines, automation scripts, web portals, APIs, and more. I have completed a master’s degree in Computer Science along with equivalent training, fellowship, and work experience. High quality, in-time delivery, active communication, and optimal products are my focal points. I enjoy sharing my thoughts and ideas with clients to improve their business. TECHNICAL SKILLS Programming Language: - Python | SQL | R | Javascript Backend - Django | DRF | FastAPI | Flask | Pyramid - Unit testing Frontend: - React - Ant Design | Material Design | Bootstrap Database - MySQL | PostgreSQL | MS SQL | MongoDB Devops - Kubernetes | Docker - CI / CD Cloud Services - Amazon Web Services | Google Cloud Platform | Azure
    vsuc_fltilesrefresh_TrophyIcon Apache Kafka
    Full-Stack Development
    Machine Learning
    API
    Data Engineering
    Apache Airflow
    ETL
    Solution Architecture
    PostgreSQL
    SQL
    Web Application
    JavaScript
    Django
    React
    Docker
    Python
  • $60 hourly
    10+ years as Senior Java Engineer. Accenture, Credit Suisse & Unibet ex-employee. Skilled in Java 11, Spring Boot, REST APIs, Web Applications, Relational & NoSQL Databases, Scrum, AWS, Angular, Docker. I also worked as a consultant on complex projects for investment banks - Barclays Capital, Deutsche Bank and Citi. Great experience with: ⇨ Java stack: Java 11, Spring Boot, Spring Security, Spring Data, Spring MVC, Spring Cloud, JPA/Hibernate, Spring Hateoas ⇨ REST API: design and development, documentation (swagger) ⇨ Databases/ORMs: Oracle, MSSQL, MySQL, PostgreSQL ⇨ Messaging: JMS, ActiveMQ, Tibco, Kafka ⇨ NoSQL: MongoDB ⇨ AWS: EC2, S3, SQS, Route 53, IAM, Lambda ⇨ DevOps and Infrastructure: Bamboo, TeamCity, Linux, Docker, Docker Swarm, DockerHub, Ant, Apache Maven/Gradle, Jenkins, GitLab, GitLab CI, GitHub, BitBucket, Sentry.io, ELK, SonarQube, Git, SVN ⇨ Frontend: HTML/CSS, JavaScript, JQuery, Angular, npm, gulp ⇨ Other Java libraries: RestAssured, JUnit/Mockito/PowerMock, Lombok, FreeMarker/Thymeleaf/Velocity, DropWizard, Jackson/Gson
    vsuc_fltilesrefresh_TrophyIcon Apache Kafka
    CI/CD
    Amazon Web Services
    Spring MVC
    Java Persistence API
    Hibernate
    API
    Spring Boot
    RESTful API
    SQL
    Docker
    Web Application
    AngularJS
    Java
    MySQL
    Spring Framework
  • $50 hourly
    Since 2008, I have dedicated myself to the mastery of software development, offering extensive experience across various projects. This includes web front-end and back-end development, mobile, desktop, and cloud computing solutions. My commitment to excellence is evident in everything I do, and my portfolio speaks to my capability, versatility, and innovation. Skills and Technologies: • Languages: Proficient in Java, Node.js, Angular, Groovy, and Kotlin. Always updated with the latest trends and best practices. • Clean Architecture: A strong advocate of clean and efficient design, ensuring each project's robustness and scalability. • Front-End Development: Expertise in Angular, Vue.js, Nuxt, Tailwind, Bootstrap, crafting responsive and intuitive user interfaces. • Back-End Development: Skilled in Java, Node.js, and Groovy to develop server-side applications that are as resilient as they are expandable. • Cloud Computing: A solid track record with Amazon Web Services (AWS), designing and implementing scalable cloud solutions. Project Experience: I've successfully navigated projects from conception to full-scale implementation, building solid Minimum Viable Products (MVPs) and following through with maintenance and enhancements. My hands-on approach guarantees that your projects continue to thrive and adapt to the ever-changing technological landscape. Thought Leadership: Outside of coding, I contribute to the community through Java-focused blogs and online tutorials, offering insights into AWS best practices and efficient cloud solutions. My passion extends beyond mere development; I am here to foster growth and inspire innovation within the industry. Why Work With Me? My goal is simple: to provide top-tier software development and cloud computing services. I bring a unique blend of creativity, technical expertise, and forward-thinking strategies, allowing me to align with your vision and create solutions tailored to your needs. If you seek a seasoned developer who can adapt in a heartbeat, look no further. Let's work together to make your next project a resounding success. Feel free to reach out, and let's discuss how I can contribute to your success story.
    vsuc_fltilesrefresh_TrophyIcon Apache Kafka
    Java
    Node.js
    JHipster
    Angular
    AWS Server Migration
    TypeScript
    Kotlin
    AWS Lambda
    Unix
    Docker
    SQL
    MongoDB
    Android App Development
    Spring Framework
    Amazon Web Services
  • $50 hourly
    Are you looking for a DevOps or SysAdmin? You’re on the right way!👇🏼 My name is Sofiia and I'm a DevOps and SysAdmin at one of the top dev firms in Ukraine in Kyiv as named by Clutch in 2020 🏆 About me: - 3 years of experience as system administrator - Programming languages: Java, JavaScript - Technologies & frameworks: Spring, Hibernate, Kafka, OOP, jQuery - Configuration/Automation tools: Kubernetes, Kiali, Istio, Calico, Consul, Terraform, Helm, Ansible - CI/CD: GitHub actions, Jenkins, Spinnaker - Cloud: AWS, Azure - Testing: jUnit, Postman - Databases: PostgreSQL, SQLite, MySQL, JDBC, FireBase - Deployment & VCS: Git, GitHub, Docker - Segments: Magento Commerce, WordPress Marketing websites, WordPress Woocommerce, ERP/CRM systems, corporate security access domain Skynix is a team of professionals focused on LAMP and JAM stacks. We deliver complex web and mobile applications development using PHP (Laravel, Yii2, Zend, or CMS systems like Magento or WordPress) and JavaScript (React, Vue, TypeScript, Ember). Why Skynix: - Providing development services starting from 2015 - 20+ team members (Senior 25%, Middle 45%, Junior 30%) and keep on growing - Only professional and enterprise-level of attitude - the project will be delivered, you can just enjoy the process - Great at remote work with various time zones - Our developers are good in English and learn skills extremely fast Need to empower your business? Let's work together
    vsuc_fltilesrefresh_TrophyIcon Apache Kafka
    CI/CD
    Docker
    Hibernate
    Azure DevOps
    AWS CloudFormation
    Kubernetes
    Azure DevOps Server
    AWS CodeDeploy
    Terraform
    DevOps
    GitHub
    AWS CodePipeline
    Java
    System Administration
  • $120 hourly
    Hello! As a seasoned Software Engineer, my expertise lies in developing cutting-edge streaming systems that drive business intelligence and growth. I'm passionate about leveraging data to its full potential through innovative cloud and analytics solutions. Select Accomplishments: • Led the architecture of Unity Technologies' streaming platform, reducing costs by 35%. • Modernized LCBO's data systems, significantly enhancing performance and user experience. • Improved Askuity's data processing speed by 60%, optimizing system efficiency • Contributed to open-source projects related to Kafka, Flink, Druid, Stream processing and Data Observability * Trained 100+ students, preparing them for data-centric industry roles. Core Skills: • Programming: Scala, Java, Python, Go, SQL. • Data Processing: Kafka, Flink, Spark, Beam, Scio, Dataflow. • Data Storage: BigQuery, ElasticSearch, Druid, Postgres, Cassandra, Athena, Redis. • Cloud Platforms: Kubernetes, Docker, Terraform, GCP, Azure, AWS. • System Monitoring: Prometheus, Grafana, OpenTelemetry, Datadog, Argo. Credentials: • B.A. in IT with Honors, specializing in Big Data for IoT from York University. Get in Touch: Are you interested in transforming your data infrastructure? I'm here to help your business excel with real-time data insights. Let's talk!
    vsuc_fltilesrefresh_TrophyIcon Apache Kafka
    Apache Beam
    Apache Flink
    Data Processing
    Data Visualization
    Database
    Data Analysis
    Data Management
    API Development
    Data Ingestion
    Data Mining
    Software Development
    Java
    Scala
    ETL Pipeline
  • $110 hourly
    With 90+ 5 stars projects on UPWORK, I have been helping SaaS businesses to build scalable, secure, high available and high performing environments on AWS following DevOps best practices. Here is what I can help you with: ✅ . Review of your AWS current architecture performance, security and costs ✅ . Architecture of new AWS infrastructure and Data Pipelines ✅ . Migrations to AWS, AWS Cost Reduction, Database Optimisations, ✅ . Managed Services SLA Problems that I solve: ⚡️. Performance At Scale: I build and deploy industry-tested and proven infrastructure code that ensures your application runs and performs at a high standard and scalability without compromising security. ⚡️. Cost Optimisation: My infrastructure architecture enables the efficient use of resources and can help you save up to 90% in your AWS costs ⚡️. Security & Compliance: I can help you Scale and innovate quicker while building a solid foundation that will help you maintain a secure, compliant environment. ⚡️. Training & DevOps. I can help you increase your team's technology adoption while automating your DevOps processes. Technologies that I adopt and I'm an expert at: ➡️. AWS services such as s EC2, ECS, EKS (Kubernetes), S3, RDS, Step Functions, Lambda, AWS Gateway API, DynamoDB, Beanstalk, Elasticache, SNS, SES, IAM, ControlTower, WAF, Shield, CodePipeline, Code Deploy, Cloudformation, Cloudfront, CloudWatch, VPC, Route 53, Certificate Manager. ➡️. Supporting services such as Terraform, Rancher, ISTIO, Prometheus, Grafana, Teleport, Papertrail, Graylog, New Relic, Dynatrace, DataDog, Kafka, NEO4J and Spot.io. ➡️. Strong CI/CD DevOps engineering using GitHub/BitBucket Pipelines, CircleCI, ARGOCD. Industries: — FinTech — Media — Ecomm — and more! ----- Here are some of the case studies of the clients I have helped with: 1. MONETRACK - Deploying a client stack in minutes on AWS instead of weeks Industry: Finance Problem: An unreliable process that caused constant frustration and delays. Process: Each deployment would take days to weeks to be set up, requiring between 5-6 people. Results: They now deploy a client stack in minutes instead of weeks using AWS EKS and a customised CI/CD process. In addition, I have: · Improved deployment time from weeks to minutes. · 80% reduction in costs per deployment. · Much greater scalability and reliability than its Legacy infrastructure. · Secure alignment with their client's compliance requirements. 2. Tourism Media - Great flexibility, deploying applications faster and secure on AWS Industry: Media Problem: With no CI/CD integration across all deploys, the job to integrate each stack was inconsistent and pruned to errors. Process: Deployment was not consistent across the board; they were also not resource-friendly Result: They can now push out a consistent deployment process across the company while being quickly adopted by their teams. In addition, I also have: · Improved deployment time by 60% · 70% costs reduction in EC2 instances · Greater scalability and reliability than their Legacy infrastructure. · Consistent DevOps process across the organisation 3. Value-Ad - Scale 10X more on AWS with great stability and without the extra cost. Industry: Finance Problem: An infrastructure that could not scale securely and effectively was harming their client base. Solution: Deploy a new AWS EKS (Kubernetes) environment that is scalable, highly available, secure (PCI-DSS compliant) and cost optimised using AWS Well Architect Framework best practices. Result: They can now handle 10X more requests with excellent stability and without extra cost than their legacy infrastructure. In addition, I have added: · Greater scalability and reliability · Deployments are simplified and done in minutes instead of hours · Increased security and reduced risk within the delivery cycle. · Greater degree of standardisation · 32% cheaper than its previous infrastructure ----- Now a bit of myself: ** 85+ projects 5 stars on UPWORK **. ** EXPERT-VETTED, top 1% experts on UPWORK ** I'm a passionate professional with 17 years of experience in enabling business strategies through the practical application of technology architecture principles, practices, and tools. User-centric and solution-oriented, I have managed projects and teams across Australia, New Zealand, the USA, Brazil, the UK, Ukraine, Singapore, Vietnam and India over the past 14 years. A problem-solver in a broad range of technology stacks covering Architecture, Development, Integration, Data, Infrastructure, and Support. Seamless configuration and migration include fast deployment of critical and complex workloads on the cloud with hands-on expertise on AWS Services. I take pride in my relentless passion and drive in delivering effective and sustainable results. I always wanted to create a company that empower our clients with everything they need to drive innovation. Send me a message. I would love to discuss your project.
    vsuc_fltilesrefresh_TrophyIcon Apache Kafka
    Amazon Virtual Private Cloud
    AWS Lambda
    Amazon EC2
    AWS CloudFront
    Amazon DynamoDB
    Serverless Computing
    AWS Server Migration
    Linux System Administration
    Amazon RDS
    Amazon S3
    Rancher
    Kubernetes
    Amazon Web Services
    Docker
    Terraform
  • $55 hourly
    🚀 Elevating Data for a Brighter Tomorrow 🌐 📊 Data & AI Enthusiast | Architect | Innovator 🔗 Let's Connect and Transform the Data Landscape Together 🌟 About Me: I'm not your typical Data Architect. I'm a relentless seeker of elegant solutions in the labyrinth of data. With over a decade of software development experience, my journey has been a relentless pursuit of harnessing the power of data to shape the future. 💡 My Path: I've always been drawn to the intersection of technology, innovation, and data. Over the years, I've sculpted a career marked by: 🔹 Deep Domain Knowledge: A versatile architect with an unquenchable thirst for understanding diverse domains. Experienced in most domains, I've dived deep into the essence of industries. 🔹 Innovation Incarnate: I thrive on redefining the limits of what's possible. In R&D, I've crafted innovative software designs and solutions that change the game. 🔹 Technology Savvy: My toolkit boasts cutting-edge technologies - Scala, Python, Akka, Spark, Kafka, AWS, GCP, and more. It's not about the tech; it's about what we can achieve with it. 🔹 Architectural Vision: Actively shaping high-level and low-level design decisions, I'm the maestro behind the symphony of architecture. 🔹 End-to-End Expertise: My journey spans from network infrastructure to the cloud, from code to deployment. Versatility is my middle name. 🌌 My Vision: My career is a quest to conquer the data cosmos. I'm not just solving problems; I'm crafting solutions to real-world challenges. The thrill of making technology serve humanity fuels my passion. 🌎 The Future Beckons: In a world awash with data, I'm your guide to unlock its potential. If you're ready to embark on a journey where data isn't just a tool but a game-changer. Let's Connect.
    vsuc_fltilesrefresh_TrophyIcon Apache Kafka
    Amazon Web Services
    Data Integration
    Google Cloud Platform
    ETL Pipeline
    Database Architecture
    Big Data
    Snowflake
    BigQuery
    Data Science
    Machine Learning
    Python
    Apache Cassandra
    Apache Spark
    Scala
  • $56 hourly
    • Java Developer with 10 years of IT experience (Java SE/EE, JMX/RabbitMQ, Spring Cloud Netfix/Hibernate/Elastic Search Frameworks) • Groovy Developer with 2 years of IT experience (Grails/Micronaut/Hibernate/Spring/Spring Boot/Quarkus, JavaScript/ HTML/CSS) • C# Developer with 2 year of IT experience (ASP.NET/MVC Framework) • Kotlin Basics (Learning in progress) • Successful practical experience in business application support (Enterprise-level of high availability systems). • AWS (EBS, EC2, S3, SES, SQS, Route 53), Azure DevOps, GCP (Cloud Run, SQL, GCS, PUBSUB) • Key business domains: Software Development; E-government, EdTech. • Worked successfully in the multicultural environment and remotely. • Hands on experience in design, development, testing. • Technical Sciences, Ph.D.
    vsuc_fltilesrefresh_TrophyIcon Apache Kafka
    Core Java
    Elasticsearch
    RabbitMQ
    Stripe
    Agile Software Development
    Apache Groovy
    Spring Framework
    Git
    Grails
    Spring Boot
    C#
    Apache Maven
    Docker
    JavaScript
    MySQL
  • $65 hourly
    Skills : Kafka, Kafka streams, Confluent, Spring Kafka , Spring boot , Spring security, Java 11 , REST, AWS, Docker Experience : 20 yrs Companies/clients worked with : ING, G-Star, equensWorldLine, ABN AMRO, Deutsche Bank, Vodafone, Accenture, TCS, UTI Project domains : Banking, Telecom, Retail Role : Dev & Ops Responsibilities : - Design & Development - Support Education : Bachelors of Engineering Accomplishments : Several innovation awards Please look at my LinkedIn Profile for more info.
    vsuc_fltilesrefresh_TrophyIcon Apache Kafka
    Microservice
    AWS Application
    Java
  • $30 hourly
    Key Skills: • Total 18 Years of experience in Software Development using Java J2EE(Spring, Spring Boot, Spring Security, Spring Cloud, Micronaut, Hibernate, JPA, Struts, EJB, JMS, JSF, GWT, ActiveMQ, Apache CAMEL, Apache Kafka, Apache Pulsar, Microservices ) • J2EE(Spring, Spring Boot, Spring Security,Spring Cloud, Micronaut, Hibernate, JPA, Struts, EJB, JMS, JSF, GWT, ActiveMQ, Apache CAMEL,Apache Kafka, Apache Pulsar, Microservices ) • Serverless frameworks (Azure Functions, Durable Functions, AWS Lambda, IBM Openwhisk) • Extensive working knowlegde of Cloud Infrastructure and Marketplace applications(Azure -Function Apps, Durable Fucntions, APIM, Azure Service Bus, Key Vault, AWS - Lambda Functions, AWS API Gateway, S3, AWS SQS, IBM Cloud). • Distributed Edge Computing and have good experience in working with systems of high scale. • Worked on Auth0, Okta and integrating many systems using Auth0 • Worked on Siddi and Complex Event Processing (CEP Systems) • Containers (Docker, Kubernetes, Azure AKS) • Extensively worked on Performance Testing using various tools like Apache Pulsar Perf, Jmeter, YCSB and getting Benchmark numbers. • Search Engine (Solr, Lucene, Elastic Search) • Expertise in development of Plug-ins, Widgets, Jive Apps, Web Services, Themes on Jive Platform and well versed with Jive SBS 4.5, 5.0, 6.0, 7.0, 8.0, jive-n, jive-x platforms • Expertise in using build tools like Maven, Ant, Jenkins, Cruise Control and setting up CI/CD pipelines • Expertise in writing/consuming Web Services (REST and SOAP) • ETL Tools (Talend Open Studio and ETL Works). Good working knowledge of Talend Open Studio and cloud based ETLWorks and writing and integrating jobs with other systems like Salesforce, Mulesoft, Sharepoint, MercuryGate, Ring Central, Twilio, SendinBlue, DAT, BlueYonder, 123Loadboard, DAT etc and other Enterprise Web applications. • Expertise in deploying web applications on Windows/Linux/Unix servers • Good Hands on experience in managing and configure Servers like Tomcat,Apache, SUN ONE AS – 8.1, WEBLOGIC 8.1, Glassfish v3 and 4 • Certified as SCJP(Sun Certified Java Professional) • Certified as SCWCD(Sun Certified Web Component Developer) Professional • Certified MongoDB developer from MongoDB University Over the last few years I have worked with global clients like Albertsons, UST-Global, British American Tobacco(BAT), Grail Research, Sony, Hitachi, Morgan Stanley, Bank of New York and have gained considerable experience in architecting and developing highly scalable Enteprise web and data solutions.
    vsuc_fltilesrefresh_TrophyIcon Apache Kafka
    SonarQube
    J2EE
    ETL Pipeline
    Performance Testing
    Microsoft Azure
    Web Service
    JSON
    Java
    JUnit
    API Integration
    Spring Boot
    Azure DevOps
    Docker
    Microservice
    Spring Batch
  • $30 hourly
    ✅ My expertise in Python/ Cloud/ Machine Learning/ Dashboard/ Analytics/ NLP/ Data Model/ API/ App & Software Development ✅ Over 150 happy clients in my profile. ✅ Python Full Stack Expert ✅ Machine Learning Expert ✅ Data Analytics Expert ✅ Data Model Expert ✅ Dashboard Developer ✅ Google Cloud Expert ✅ Google Data Studio Expert ✅ Frontend Development ✅ Backend Development ✅ SEO, GA. GTM, PPC, & Digital Marketing Expert ✅ API Expert I have been working for more than 15 years in Data Mining, Data Model, Data Management, Machine Learning, Artificial Intelligent, Automation, Statistical Analysis, Data Analytics, Visualization, Dashboard, Mobile App, Web and Full Stack Development. My technical skills are - 1. Machine Learning Frameworks and Libraries:- Tensorflow, PyTorch, Beautiful Soup, CNTK, Scikit-learn, Spark MLib, Keras, OpenCV, Pandas, NumPy, SciPy, Matplotlib, MATLAB, Firebase etc 2. Data Science and Analytics:- Analytics- Power BI, Kibana, R, SAS, SPSS, STATA Cloud- AWS, Azure, Google Cloud Database- SQL, MySQL, MongoDB, Postgresql, Neo4j, GraphDB, SQLite BigData Processing- Hadoop, Spark, Scala Automation Testing- Selenium, Zephyr, Cucumber, LambdaTest 3. Dashboard Development:- Python, D3, Chart, Kibana, Grafana, KPI, Google Data Studio, Power BI, Tableau, DAX, Excel, Elastic Search, Klipfolio, etc. 4. Full Stack Development:- Python, JavaScript, Typescript, React.js, Node.js, Next.js, Angular.js, D3.js, PHP, Laravel, Docker, Django, Bootstrap, jQuery, Vue.js, UI/UX, Git, etc.
    vsuc_fltilesrefresh_TrophyIcon Apache Kafka
    Airtable
    Marketing Data Analytics
    Zapier
    Data Modeling
    Kibana
    Elasticsearch
    Microsoft Power BI
    Dashboard
    Neo4j
    Node.js
    React
    JavaScript
    Google Ads
    Machine Learning
    Python
  • $30 hourly
    6+ years of experience in architecting, designing and developing software across large scalable distributed systems and web applications. In my past experiences, I have been responsible for end-to-end development of features for Paytm Mall (Ecommerce), Paytm Smart Retail (B2B) and Paytm For Business(Merchant Platform). I am currently working on development of inhouse analytics platform for flipkart as Abobe Analytics is not scaling anymore at Flipkart's scale. Languages: Java, Scala, Python, JS Technologies: Spring, Spring Boot, Apache Flink, Spark,Django , Node.js, Express, Flask Data: Hibernate, Hadoop, Hive, Hbase, Druid, MySQL, SQLite, PostgreSQL, Elastic Search, Redis, SQLAlchemy Others: Kafka, RabbitMQ, Jenkins, Kibana, Nginx, Gunicorn, Celery, Supervisor, Datadog, JIRA, Git, CI/CD, TDD
    vsuc_fltilesrefresh_TrophyIcon Apache Kafka
    Amazon Web Services
    Google Cloud Platform
    JavaScript
    Java
    Python
    Big Data
    Apache Hive
    Apache HBase
    Apache Hadoop
    Apache Spark
    Apache Flink
    SQL
    Elasticsearch
    Django
  • $100 hourly
    Senior Software Engineer/Tech Lead who delivered dozen of projects as a tech lead with MILLIONS of users, this profile speaks for itself! I am a well-rounded Software Engineer and Tech Lead, ranking in the top 3% of Stack Overflow contributors with a reputation of over 14K. With 17+ years of experience, I have worn many hats, ranging from developer and DevOps roles to lead engineer positions. My leadership in highly successful projects has led to multiple promotions to C-level executive roles, such as CTO and CIO. A Client Review: "Simply amazing! Not often you find a freelancer that has a passion for what he does but also has the heart of a teacher. Good attention to detail goes above and beyond what you ask for; a great communicator ... there is just too much to mention here. I will continue to hire for future projects!" I contributed as the architect and lead developer to multiple banking and business critical product developments from inception to launch and operations. One of them are now being consumed by millions of people and hundreds of apps daily; I have all the tools needed to do a successful project. Here is the list of services I can provide: ♦ Solution Architecture and Design ♦ Lead Development Teams ♦ Review Code and Architecture, Spot Wrong Design Decisions, and Code Smells. ♦ Domain Model Analysis and Design ♦ Data Model Design, SQL Coding, and Optimization ♦ NoSQL Solution Design ♦ Mentoring ♦ Develop Java, Kotlin, C#, Groovy, Grails, Ruby, Ruby on Rails, Angular, WCF/ASP.NET Core, WPF/WinForms, XML,Jenkins, Gradle and Groovy applications, I also developed small projects in Python. ♦ Troubleshooting and Bug Fixing Critical Issues ♦ Pair Programming ♦ Testing and QA
    vsuc_fltilesrefresh_TrophyIcon Apache Kafka
    Web Development
    Software Architecture & Design
    Kotlin
    RESTful API
    Spring Framework
    Amazon Web Services
    Kubernetes
    JavaScript
    SQL
    C#
    DevOps
    Ruby
    Microservice
    Spring Boot
    Java
  • $50 hourly
    I am experienced in Database development, Performance Tuning, Database and System Administration, Webserver Administration, Database Warehousing and Business Intelligence. I have more then 8500 hours of Database Development, Webserver and System administration work online. Managed FileMaker, Oracle, MS SQL Server, MySQL, PostgreSQL databases on Windows and Linux platforms. My area of specialization is Performance Tuning and High Availability. I am Oracle Certified Professional OCP in both Development and Administration. Provided training on FileMaker, Oracle, MS SQL Server, MySQL and PostgreSQL development and administration. I am also experienced in software analysis, design, coding and integration of web based systems. My Goal is to meet business requirements with respect to quality, functionality, performance, availability and on-time delivery. I have good knowledge of Software Engineering and SDLC and achieved delivery of quality software within timelines.
    vsuc_fltilesrefresh_TrophyIcon Apache Kafka
    System Administration
    Apache HTTP Server
    AWS Server Migration
    Database Optimization
    PHP
    DevOps
    SQL Server Integration Services
    Linux System Administration
    Database Programming
    Amazon RDS
    Amazon Aurora
    Performance Optimization
    Microsoft SQL Server
    Claris FileMaker
    MySQL
  • Want to browse more freelancers?
    Sign up

How it works

1. Post a job (it’s free)

Tell us what you need. Provide as many details as possible, but don’t worry about getting it perfect.

2. Talent comes to you

Get qualified proposals within 24 hours, and meet the candidates you’re excited about. Hire as soon as you’re ready.

3. Collaborate easily

Use Upwork to chat or video call, share files, and track project progress right from the app.

4. Payment simplified

Receive invoices and make payments through Upwork. Only pay for work you authorize.

Trusted by

How to Hire Top Apache Kafka Developers

How to hire Apache Kafka developers

Looking for a high-throughput, fault-tolerant, data streaming solution for processing large volumes of messages? An Apache Kafka developer can help.

So how do you hire Apache Kafka developers? What follows are some tips for finding top Apache Kafka consultants on Upwork.

How to shortlist Apache Kafka professionals

As you’re browsing available Apache Kafka consultants, it can be helpful to develop a shortlist of the professionals you may want to interview. You can screen profiles on criteria such as:

  • Technology fit. You want a developer who understands how to integrate Apache Kafka with the rest of your technology stack.
  • Project experience. Screen candidate profiles for specific skills and experience (e.g., building a website activity tracking pipeline).
  • Feedback. Check reviews from past clients for glowing testimonials or red flags that can tell you what it’s like to work with a particular Apache Kafka developer.

How to write an effective Apache Kafka job post

With a clear picture of your ideal Apache Kafka developer in mind, it’s time to write that job post. Although you don’t need a full job description as you would when hiring an employee, aim to provide enough detail for a contractor to know if they’re the right fit for the project.

Job post title

Create a simple title that describes exactly what you’re looking for. The idea is to target the keywords that your ideal candidate is likely to type into a job search bar to find your project. Here are some sample Apache Kafka job post titles:

  • Need help building low-latency log aggregation solution with Apache Kafka
  • Seeking Java developer with Kafka Pepper-Box and JMeter expertise
  • Developing a Change Data Capture (CDC) agent with Kafka

Apache Kafka project description

An effective Apache Kafka job post should include:

  • Scope of work: From message brokers to real-time analytics feeds, list all the deliverables you’ll need.
  • Project length: Your job post should indicate whether this is a smaller or larger project.
  • Background: If you prefer experience with certain industries, software, or developer tools, mention this here.
  • Budget: Set a budget and note your preference for hourly rates vs. fixed-price contracts.

Apache Kafka developer responsibilities

Here are some examples of Apache Kafka job responsibilities:

  • Design and develop data pipelines
  • Manage data quality
  • Implement data integration solutions
  • Troubleshoot and debug data streaming processes

Apache Kafka developer job requirements and qualifications

Be sure to include any requirements and qualifications you’re looking for in an Apache Kafka developer. Here are some examples:

  • Proficiency in Java and/or Scala
  • Data streaming
  • CDC
  • Data engineering

Apache Kafka Developers FAQ

What is Apache Kafka?

Apache Kafka is an open-source stream-processing solution developed by LinkedIn and later donated to the Apache Software Foundation. The software platform aims to provide a low-latency, high-throughput solution for processing real-time data feeds.

Apache Kafka uses the publish/subscribe messaging pattern common in distributed systems. Kafka instances typically exist as clusters of nodes called brokers that can receive messages from multiple producers (any apps sending data to the cluster) and deliver them to multiple consumers (any apps receiving data from the cluster). Producers publish messages to Kafka topics (i.e., categories of messages), while consumers subscribe to Kafka topics. It is through this topic categorization that the brokers are able to determine where messages need to be delivered.

Apache Kafka is a popular choice among developers looking to build message brokers, website activity trackers, and analytics pipelines that must deal with large volumes of real-time data from disparate sources.

How much does it cost to hire an Apache Kafka developer?

The first step to determining the cost to hire an Apache Kafka developer will be to define your needs. Rates can vary due to many factors, including expertise and experience, location, and market conditions.

Cost factor #1: project scope

The first variable to consider when determining scope is the nature of the work that needs to be completed. Not all Apache Kafka development projects are created equal. Creating a simple log aggregator to collect log files off different servers into a central place for processing will typically take less time than building out a multistage data streaming pipeline for your SaaS (software-as-a-service) product.

Tip: The more accurately your job description describes the scope of your project, the easier it will be for talent to give you accurate cost estimates and proposals.

Cost factor #2: Apache Kafka developer experience

Choosing the right level of expertise for the job is closely tied to how well you determined the scope of your project. You wouldn’t need an advanced Apache Kafka developer to create your own custom site analytics dashboard using Kafka. On the other hand, building a large-scale enterprise messaging system will require the skills of a seasoned Apache Kafka developer.

Beyond experience level, you need to consider the type of experience the talent possesses. The following table breaks down the rates of the typical types of Apache Kafka developers you can find on Upwork.

Rates charged by Apache Kafka developers on Upwork

Level of Experience Description Hourly Rate
Beginner Familiarity across the technology stack. Data engineering fundamentals (e.g., data streaming, data quality, data integration). Can use Kafka for basic website tracking, messaging, and data streaming. $40-70+
Intermediate Professional full-stack developers or data engineers. Experience working with high-throughput data needs, microservices architectures, and multistage data streaming pipelines. $70-100+
Expert Advanced full-stack developers or data engineers with years of experience in big data. Capable of managing teams of developers and engineers. Advanced knowledge of application architectures, data streaming technologies, and data processing solutions. $100-130+

Cost factor #3: location

Location is another variable that can impact an Apache Kafka developer’s cost. It’s no secret that you can leverage differences in purchasing power between countries to gain savings on talent. But it’s also important to factor in hidden costs such as language barriers, time zones, and the logistics of managing a remote team. The real advantage to sourcing talent remotely on Upwork is the ability to scan a global talent pool for the best possible person for the job. Location is no longer an obstacle.

Cost factor #4: independent contractor vs. agency

The final variable regarding talent cost is hiring an independent contractor vs. an agency. An agency is often a “one size fits all” model, so you’ll often have access to a designer, a project manager, an engineer, and more. When hiring individuals you have total autonomy regarding who is responsible for which part of the project, but you’ll need to source each of those skills separately.

The trade-off between hiring individuals vs. hiring an agency is the level of administrative overhead you incur personally in coordinating tasks among all members of the team. Project scope and personal preference will determine which style is a better fit for your needs.

Apache Kafka developer tips and best practices

Understand your partition data rate limitations

In Kafka, messages are organized into topics that can be divided into a number of smaller partitions. Partitions allow your Kafka cluster to process the data in a particular topic in parallel across multiple brokers. This capacity for parallel processing is what enables Kafka to deliver high-throughput messaging.

Of course, even high-throughput systems are going to have their limitations. Messages sent to a partition exist in a log for a configurable period of time or until a configurable size limit is reached. Exceed that retention limit prematurely, and it’s possible you can start losing messages before consumers can pull them from the topic partition.

That’s why it’s important to understand the data rate of your topic partitions. Simply multiply the average message size times the number of messages per second to calculate your average retention rate. This will enable you to figure out how much retention space is required to guarantee data is retained for the desired period of time.

Widen those consumer socket buffers for high-speed ingestion

The default settings for consumer socket buffers tend to be around 100 KB (Kafka 2.4.x), which is too small for high-throughput environments. For low-latency, high-bandwidth networks (10 Gbps or higher), it might be necessary to bump those values up to 8 or 16 MB. You can tune the socket buffer setting for consumers with the “socket.receive.buffer.bytes” parameter.

Tune your memory buffer and batch sizes for high-throughput producers

On the producer side of the equation, high-throughput environments will likely require a change to the default memory sizes for your “buffer.memory” and “batch.size” parameters. These values are trickier to set than your consumer socket buffers as they depend on a number of factors, including producer data rate, number of partitions, and the total memory you have available. Larger buffers aren’t necessarily always better, because having too much data buffered on-heap can lead to increased garbage collection—a process that will compete for resources and affect your importance. Best practices should be established based on the unique configuration and settings of your Kafka data streaming system.

View less
Schedule a call