— Among the Top 1% of Upwork Freelancers // Top Rated Plus Freelancer & Expert Vetted Freelancer
— Ranked Top 5% in Python Across LinkedIn
I have more than 10 years of experience in Full Stack Engineering and DevOps Engineering, encompassing projects in cloud systems, web applications, mobile applications, data mining, and automation. My current main area of interest is within large data-driven applications in a distributed, scalable, cloud environment. I have many years of experience in writing screen and data scrapers for online data mining applications (with experience automating scraping data from thousands of sites).
DATA ENGINEERING / DATA LAKES
I have been designing, scaling, and implementing Data Lakes sense before we called them "data lakes". I have worked extensively with AWS Partners to train their teams and to deliver Data Lake solutions to the customers. I design and develop the ETL processes and am familiar with integrating a wide range of APIs.
Tools I have used include: Python, Node.js, AWS Lambda, AWS S3, Tableau, Quicksight, PowerBI, Redshift, DynamoDB, MySQL, PostgreSQL, Aurora, AWS RDS, MongoDB, Elasticsearch, etc.
I have integrated many third party tools to such Data Lakes including Google Analytics, Salesforce, Google Ads, Bing Ads, Facebook Ads, ServiceNow, and many other similar tools.
I have been writing data scrapers and other data intensive scripts/ utilities for many years. My language of choice for many of these is Python or R, and I regularly work with Scrapy, BeautifulSoup, lxml, etc. However, I can successfully develop a utility in many scripting languages and packages, including: Bash, PHP, Ruby, Perl, etc.
I have written screen scrapers for over 500 websites, including Google Search and Craigslist. I have a very high working knowledge of HTML, which helps me quickly write any screen scraper you find yourself needing.
DEVOPS / AUTOMATION
With most of my projects, I am responsible for everything from ground up. This, of course, includes the code of the application, but also the automated pipelines, automated deployments, version control, and Infrastructure-as-Code. I have vast experience with designing and developing system architectures in Google Cloud and Amazon Web Services.
I have used many available DevOps tools in these projects, with my favorites consisting of Terraform, CloudFormation, Gitlab, Github, Jenkins, Ansible, Docker, Packer, AWS Code Services, TravisCI, and CircleCI.
Whether you need to migrate data, design infrastructure, or automate pipelines and deployments, I have the experience and expertise to help.
I prefer developing complex and unique web services and/or user interfaces rather than creating simple websites which are similar to each other.