Cloud Engineer
What is “The Cloud”?
The cloud is synonymous with modern computing. A majority of businesses—from startups to massive corporations—use it in some capacity to augment or streamline their existing operations, data storage, hosting, and app deployment. So what is the cloud and how is it changing traditional server setups across the globe?
The U.S. National Institute of Standards and Technology (NIST) defines the cloud as “a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.”
In short, it’s allowing companies to reengineer their back-end architectures (servers, databases, application software, and more) and put them in virtual environments where they can be accessed remotely, without requiring physical server hardware of their own. Cloud services (also known as web services) are a blanket way to describe these computing capabilities.
The benefits of using cloud and web service providers
Adopting an infrastructure-as-a-service (IaaS) platform like Amazon Web Services (AWS) or a platform-as-a-service model (PaaS) from a provider like Heroku can take some of the responsibilities of hosting, maintaining, updating, and scaling up server operations off the hands of developers and IT managers. An IaaS will provide you with all the components you need to build a backend architecture, offering flexibility and plenty of scalability. Note that deploying a site or app’s back end to the cloud with an IaaS requires more configuration, while deploying with a PaaS doesn’t require as much configuration—just enable the services you need within the platform’s environment, push the code to it, and it handles the rest.
While their clients benefit from a virtual environment, cloud service providers themselves have massive data centers that are as big as multiple football fields. These servers are usually set up in a way that some can fail (or, even a majority of them), but not in a way that all of the services will be taken down. It’s important to know that it’s still possible for cloud computing to have a single point of failure, or to have certain services fail.
This has brought about the need for more engineers who know how to integrate, work with, and fix cloud-based operations. You may not need a full-time network engineer if you’re operating in the cloud, but you’ll definitely need a skilled development operations professional to ensure things run smoothly.
Here’s a look at a few of the top providers in the cloud services market:
- Amazon Web Services (AWS): Popular with startups looking to get things going in the cloud for cheap, this platform is also powerful enough for large, enterprise operations.
- Google Cloud Platform: This popular PaaS offers cloud computing, storage, big data and API services so you can build and launch sites or complex applications in the cloud. It offers things like SQL and NoSQL database services, analytics, virtual machines, all of which can be mixed and matched to suit your needs.
- Microsoft Azure: An enterprise-level Paas and IaaS cloud provider, it offers mobile and web app deployment and scaling, database services, virtual machines, mobile back ends, machine learning, and more.
- Heroku: A popular PaaS where applications can be completely built, deployed and run in the cloud.
- Rackspace: This PaaS offers cloud computing through its infrastructure (either dedicated servers, public cloud, or private cloud, all of which can be mixed and matched for a hybrid environment), or their partnerships with Azure and AWS.
- Cloud Foundry: This open-source PaaS written in Ruby and Go offers cloud computing services with an enterprise-grade option, Pivotal.
- Xen Cloud Platform (XCP): This open source virtualization solution that provides cloud computing and back-end virtualization. XCP includes an enterprise-ready set of tools with the Xen Hypervisor, with the Xen API for cloud, storage and networking operations.
- Oracle: This enterprise giant has made recent updates to its cloud capabilities, helping big businesses leverage the cloud, but also offering small- to medium-sized businesses cloud computing services.
- Apache Cloudstack: a free, open-source cloud software for creating and deploying cloud services that has excellent support for virtualization and the AWS API.
Beyond data centers: making the move to the cloud
With the rise of the cloud, the server landscape is rapidly changing, with more server-side operations being pushed off-site. Small, medium, and enterprise-level companies can easily expand the size, storage, and processing power of their servers in a way that takes less time, and less money.
For startups in particular, these cloud-based platforms take server upkeep and support off their plates so they can focus on growing their businesses. They also offer flexibility and speed, with the ability to scale up quickly when needed. Scaling up can be hard to do in a physical data center: ordering new hardware, provisioning, and racking, and stacking can take anywhere from three to six weeks. In the cloud, you can provision capacity on the fly.
The key to moving to the cloud is striking the right balance for your organization—whether that’s a hybrid approach or all-in.
Adopting a hybrid cloud approach
A majority of businesses are finding that a hybrid approach works best, leaving some things on traditional, local servers and moving more resource-heavy applications to the cloud. This can be a permanent strategic solution, or a stop along the way to going 100% cloud-based. There are benefits to leaving certain portions of your back-end infrastructure on virtualized local servers or co-located data centers, while moving more resource-heavy applications to the cloud.
For example, migrating a back-end architecture to Amazon Web Services (AWS) allows for automatic scaling, the ability to fail over multiple availability zones, its accommodations for peaky traffic and intensive operations (e.g., some of our machine learning models), and the ability to free up a team to focus on application-specific work rather than solving problems Amazon can address.
Public cloud vs. private cloud
A common approach is to divvy up a server’s workload with a mix of private (onsite) and public (cloud services) clouds. Larger, enterprise companies often opt for a private cloud/onsite server architecture. One reason? Protecting sensitive data.
For some organizations in regulated industries, like finance, there are restrictions on what information they can store in the cloud. Because of this, they have to strike a balance between storing sensitive information on-site while still making it available in the cloud, so they can take advantage of the agility and scalability the cloud offers.
Critical apps are often better suited for a private cloud for security and reliability reasons. The main concerns with critical apps are performance (speed, reliability, and no downtime) and security of your information. A privately hosted cloud is a good bet for these, giving you more control along with the flexibility you want.
Less critical apps like web servers, backup services, and infrastructures are safer in the public cloud. There, you’ll get servers that allow you to free up space on-site, plus the benefit of temporary scalability if you need it, with increased capacity just a click away. Also, a public cloud configuration can even be treated like software code and placed in a repository where developers can edit, adjust, and run tests against your current configuration, which is helpful for ensuring a successful deployment, and for getting the most out of the cloud.
Should you move to the cloud? What to consider before deploying to the cloud
For small businesses, there are some clear advantages to using the cloud—whether it’s going entirely to the cloud, or using a hybrid approach that breaks up your server workload between an on-site infrastructure and the cloud.
The benefits? It’s an invisible, offsite server that you can scale up when and how you want. If you have an application that requires a lot of space, data, or resources, you can shift that over to the cloud while freeing up space in your current setup.
Small businesses can strike a strategic balance between traditional on-site servers and cloud servers, so it’s important to ask yourself a few questions about your setup before choosing the one that’s right for you. And if you're not sure you can always seek the advice from cloud consulting specialists.
- Take a look at your existing back-end infrastructure. What are your requirements, and what are your end-user’s requirements? What will diversifying to the cloud do to help with these? Consider compatibility of your server-side software. While some businesses don’t move all of their server-side architecture to the cloud, it’s helpful to ensure cloud compatibility in the components they keep on-site.
- Decide what should go where. Plan how you’ll virtualize your back end. If you’re a small business, basic server functions like an email server or an app server could probably stay on-site. Or, apps that don’t require as much data storage could stay local, too. Be sure to prioritize your needs. You may opt to divvy up the workload with a hybrid cloud environment, keeping mini servers on-site to handle smaller workloads, like file sharing servers. These can even be designed to sync up with cloud drives.
- What’s your budget? This will help narrow down which cloud service is right for you. The great thing about the cloud? Flexibility. If you start out small and find you need more, you can easily upgrade subscriptions or buy more data—no need to switch out hard drives.
- Who should you hire or have on your team to help? Do you have a server professional available to help with maintenance or fixes? Because you don’t have to worry about hardware with cloud servers, it can be a more seamless integration for IT professionals, but it’s not without its quirks. Make sure you have a dev ops engineer who has plenty of experience integrating with the cloud and handling issues with network reliability that can arise.
- How scalable does it need to be? How much growth you anticipate in terms of traffic and data for your application or site’s server play a very important role in how you decide to set up your server. You’ll want to be able to expand your server space without having to totally replace it, whether that means starting with a setup that allows you to switch out hard drives for hard drives with more memory, or virtualizing your setup across numerous smaller servers.
- Security. Security is always a big concern—if you have very sensitive information being stored on your server, you may opt to keep that on-site while moving less sensitive information to the cloud. Or, go for a private cloud/hybrid cloud environment that allows you to maintain a more secure environment, and lets your IT professionals keep tighter control on what data is stored/shared where.
The need for cloud engineers
From a talent standpoint, running your server operations in the cloud means you won’t need the same network and storage engineers on hand to take care of day-to-day server issues—but maintenance and support won’t be totally off the table.
Look for a cloud server architect with plenty of experience deploying operations to the cloud. Some key skills and expertise to look for in a dev ops engineer are:
- Configuration management skills: Chef, Puppet, Ansible, etc.
- Virtualization experience: VMware, Kernel-based Virtual Machine (KVM), Xen, etc.
- Public cloud experience: Amazon Web Services, Google, Rackspace