Lets start with the obvious – what are the generic categories of service offered by all service providers?

The Cloud service providers are numerous. The top two are, not surprisingly, Amazon Web Services and Microsoft Azure. If we look a little more closely at these two (sorry Google and IBM), we will see that each have their own set of acronyms for the products and services they offer. Very broadly speaking (I did mention complexity as one of the disadvantages of Public Cloud!), these can be separated in to the physical platforms that we may already be familiar with (compute, storage, and network), the software that runs on top (database, additional software to support functionality), and the features that improve overall efficiency (auto scaling for example). To understand these products and services, let’s say storage, it is necessary to understand what each service description actually means. It now becomes even more complex, as storage can be broadly categorised in to ‘tiers’ - block, file, object, backup and recovery, and archive. Block is the higher cost and typically higher performing tier of storage (I am using my words very carefully as this is a minefield, with higher network speeds on Ethernet using IP being compared with ‘lossless’ fibre channel protocol block storage making this discussion a little contentious) and suitable for certain types of data requiring high performance. Block storage uses LUNs (disks) to store data, and typically requires a filesystem to manage the data being written to blocks on the disk. The LUN is presented to a host across a fibre channel connection using SCSI to communicate with the disk, or an Ethernet connection using iSCSI (which encapsulates the SCSI commands on a TCP/IP protocol connection). Block storage is the highest performing tier, and the data written to block storage is referred to as ‘structured’.

File storage is fairly self-explanatory, and can be seen in all workplaces when a user connects to their desktop and opens a file. Data held on file storage is referred to ‘unstructured’, and uses NFS, SMB and CIFS protocols to access file ‘shares’ (folders). File storage is typically a medium performing tier of storage, and can be easily shared between end users and servers, unlike block storage.

Object storage on the other hand is more complex in its use cases, and is typically a lower cost option. Object storage is again used for specific types of data, and is lower performing, and can be treated essentially as a ‘cheap-and-deep’ tier of storage. Object storage stores data in objects, and each object contains meta data describing that object. Because high performance is not a characteristic of Object storage, objects will consist of mainly static data, and will be replicated among other object stores in other datacentre locations to provide very high resilience against data loss. The complexity continues when considering the type of storage that is being used at the physical layer. This can be either a hard drive, or a solid state drive (SDD). The key premise behind Object storage is availability through replication of objects between datacenters and regions.

For compute resources, to fully understand what is on offer from a service provider (Amazon refer to their Cloud compute as Elastic Cloud Compute, or EC2 for short, and Microsoft have Virtual Machines), you need to have an appreciation of the physical aspects of the virtual machine that is being provisioned. Remember, this is all virtualised, so a virtual machine will be provisioned from a physical server running a specific rating of CPU, with a finite number of physical cores. The virtual machine will then be configured with a count of virtual CPUs (vCPU), which are directly related to the physical and logical cores on the host server. The physical platform will be shared with other virtual machines running on the same physical server. The pricing for compute will be based on the performance characteristics of the physical server, and there will be varying levels of performance available, with the lower performing platforms being at a lower price point. It is therefore essential that the performance profile of the application moving to a Cloud compute platform is understood.

We have already acknowledged that public Cloud resources are expensive. And don't let the service provider tell you otherwise. Having said that, it is then up to you to ensure that you design your Cloud environments using the additional tools and mechanisms that improve efficiencies and lower cost dramatically. Remember that your resource does not necessarily need to be running 100% of the time. When it is shutdown it should not be costing you money. That is just for starters. Whether you need to provision a compute resource at all is a consideration that can reduce the cost further, taking advantage of the ephemeral compute instances that only exist while running a specific task. Designing your public Cloud platform to incorporate these innovative features is essential.

With the highest utilisation being driven by virtualisation, and the consolidation of resource on to fewer physical units, the cost of running a private Cloud will inevitably be far below than that of a traditional infrastructure. This is compelling when considering adopting a Cloud strategy, public and private. The reduced cost associated with a private Cloud includes a reduction in the support and management run rate through the introduction of automation and orchestration. The private Cloud infrastructure design introduces a far more flexible approach to support. It is possible through this design to move roles to regions that are remote to the infrastructure, which allows for consolidation of teams and regional functions. All of this counts towards reduced overall cost. As we have demonstrated in the previous sections, a virtualised infrastructure results in considerable efficiencies, and no longer are there large pools of unused resource; no longer are there lengthy delays in standing up new infrastructure. And the tools available to run a Cloud service will reduce the support overheads needed to monitor, alert, escalate and report.

The impact of infrastructure outages and faults on the business functions is also reduced, and in some cases eliminated entirely through the inherent nature of Cloud, which more or less incorporates DR and HA by design. This allows, in many cases, for the business units to continue functioning without an interruption to the service delivered by IT. This is all achieved as a consequence of virtualisation, making it very easy to move a process from one platform or physical location to another, in many cases seamlessly, or simply distributing the process across multiple locations. This is also the case for storage and network, and so there is no reason why a business process cannot be 'live' in two locations at the same time, making failover in the event of an outage or catastrophe seamless to the end users. The business continuity model described here allows for a public Cloud to exist at multiple sites, and so utilise the resource that was once sitting idle waiting for disaster recovery to be invoked. Again, this all has a positive impact on cost for private Clouds.

And finally, you have a choice. By adopting a public Cloud infrastructure approach to support business functions, you shift your costs from CapEx to OpEx. And although a private Cloud remains a CapEx commitment, it offers you an overall reduction through its efficiencies.

Amazon Web Services began the Cloud revolution about 13 years ago, and have comfortably led the race to deliver Cloud services since its inception. AWS has evolved to such a degree that their selection of service offerings is vast. They have a multitude of mechanisms for everything. Each Cloud service provider will likely provide services for serverless computing, containers, Big Data analytics, you name it, but AWS will provide additional levels of service across all of these, and more. Just take a look at the product list and you’ll see what I mean. It is in some ways quite beneficial, to have a one stop provider for all your needs, but it does also risk introducing an element of confusion. For some, it will be somewhat overwhelming to have such an array of services, most of which will be irrelevant. The culture that AWS want to foster among their customers is that they should 'own' your IT for you. This will be appealing in some ways by taking much of the burden of infrastructure design, build and support away. But it is not necessarily what everyone will want, and one must be wary of their ultimate agenda. To understand the AWS agenda, you must take a look at their leadership principles, which drive high service quality and customer satisfaction to extremes, providing a route to ultimate domination in their field, be it Cloud computing or retail. The customer comes first. This may be considered a very good quality, and it is, but remember that one day you may want to prise yourself away from this relationship. That will not be so easy. Some existing AWS users may also suggest that their charging structure is not easy to understand. Get that wrong and you are in trouble.

AWS is now under growing pressure from its main rival, Microsoft Azure, who have gained considerable ground on AWS over a relatively short period. This is certainly in some part down to the ability of Microsoft to deliver services that are more compatible and a better fit for most end user's current requirements. Some key words and phrases below may go some way to help in understanding AWS a little more.

Elastic Cloud Compute (EC2), provides a very large range of compute platforms for all workload profiles. Starting with T2 instances, that are burstable to support high workloads when needed, and are typically used for lighter web type workloads, through to the M5 and M4 range of instances that scale up to 94 vCPU in a single instance. The burstable instances, such as the T2, will support higher workloads than the baseline typically support, and this will be chargeable when used. Additionally, auto scaling can add EC2 resource based on certain conditions dynamically when needed, and can even be formed from spare compute resource at far lower cost to the consumer. There really is a lot to choose from in the AWS compute product set.

AWS S3 is object storage, typically used to store very large volumes of data, with multiple copies of the data stored across a number of object stores in different regions. The data can then be used for analytics (for example Amazon Elasticsearch, or Redshift), or simply an object store for large volumes of data. Object storage differs from the other storage types (file and block for example), and would not normally support database queries. Objects are stored in buckets and the retrieval of objects requires the exact bucket name and key which makes SQL queries a little more challenging. AWS S3 storage is designed to be fast in region when connected to EC2, and supports large volumes of data that is not likely to change frequently, such as website static files, video streaming and the like. As mentioned already, the whole premise behind object storage is to distribute multiple copies of data (objects) across multiple regions, providing, according to AWS, ‘eleven nines’ availability.

Amazon EBS storage is built for performance, supporting high IOPS, and built traditionally on a fibre channel (FC) SAN and presented as block storage to the host. The SAN does not necessarily need to be FC, and in IaaS terms is irrelevant, which is one of the factors that makes public Cloud a compelling choice. The infrastructure decisions are for the service provider to decide. EBS storage, as you would expect, is ideal for database applications and any application the depends on the infrastructure service supporting high sustained IOPS. AWS will replicate EBS storage within the availability zone to provide high availability (note: replication within the availability zone is not Disaster Recovery, that is a separate tier).

Amazon EFS, as the name suggests, is file storage using NFS as the access protocol. File storage stores data as files, not surprisingly, and has a multitude of uses, too many to capture effectively here. EFS has the capability of being exported and mounted on your on-premise servers, which makes the movement of data in to public Cloud simple (for realistic data volumes that is). When you sit at your desktop at work, it is File Storage that makes your desktop folders available.

AWS Elastic Beanstalk is a service that provides versatility to application deployments. Very briefly, AWS EB will manage an applications deployment in the AWS Cloud without the application owner needing to be concerned about the infrastructure that it runs on. If an application repository is accessible through GIT or Visual Studio, Elastic Beanstalk will automatically build the environment that that application will run from, with no intervention needed. There are many other AWS tools and services that provide similar levels of versatility, using mechanisms such as containers and container framework tools like Kubernetes.

Amazon Redshift provides a data warehouse facility that scales up from 00’s of GB to multi-petabyte storage, allowing for large scale analytics on the high volume of stored data. Redshift consists of a set EC2 nodes that are organised in to a cluster. The Redshift cluster consists of a leader node that accepts queries from applications, and a number of compute nodes that execute the queries in parallel. When launching a cluster, the application owner will specify a node type, either DS (dense storage) or DC (dense compute). DS nodes are optimised for large data workloads and use HDD storage, and DC optimise for compute, using SSD storage. Again, this is used as an example of an AWS service that provides ‘Big Data’ style analytic capability.

AWS Lambda is another versatility tool, similar to Elastic Beanstalk in its purpose, by allowing the end user to run code without the need to provision or manage servers. For example, as described on the AWS website, Lambda can be used to execute code in response to a trigger. If data is uploaded, the trigger may execute code that validates the data, or performs another function needed in response to the uploaded data completing. With Lambda, the end user will only pay for the compute time used, and Lambda will scale automatically depending on resource requirements.

An Amazon VPC is a virtual network environment dedicated to an end users Amazon account. This provides a virtual network environment in which to run AWS resources (EC2, EBS etc). A VPC virtual network will support multiple subnets. Security between subnets in a VPC is achieved using security groups and network ACLs. A public subnet is defined as one that has an Internet connection through either the default gateway, or NAT gateway. A private subnet is defined as one without an Internet connection. Early versions of EC2 would have been launched in to what is called EC2-Classic, which is a single flat network shared by AWS customers. All new accounts will be EC2-VPC only. An EC2 instance launched in a default VPC will have both a private and public IPv4 address, with an internet gateway allowing instances to communicate with the internet. A non-default subnet will however have only a private IP address, and no access to the internet. A typical AWS environment (account) will likely consist of multiple VPCs with firewalls providing secure separation between the different VPC functions.

Amazon CloudFront is a content delivery network (CDN), which represents the edge of the Cloud. In other words, your content is cached closer to your end users, and therefore more quickly available. The content can be anything from static files that make up your website, or media files that will benefit from being 'hosted' closer to the user. Using CloudFront also provides you with greater visibility of where your content is being requested globally, and the type of access, through its usage reports.

As you would expect from Microsoft, Azure relies very much on its ability to offer Microsoft core products as part of its Azure Cloud service offerings, such as SQL Server, Office and .NET. Microsoft is also a significant and compelling Cloud service provider through its Office365 service. This has all helped Azure gain considerable ground on AWS, having arrived late to the Cloud ‘party’. However, there is no avoiding the fact that Microsoft Azure is still a comparatively new service provider (I use this term very loosly as Azure has been in service since 2010!), despite having a significant global coverage for its Cloud services. In terms of the actual services offered by Azure, you will not be wanting. AWS may have a vast eye watering catalogue of services, but that does not mean that similar services, albeit perhaps fewer overall, are also available on Azure. For example, all the expected container services supporting Docker and Kubernetes are a staple of the underlying service offerings. There are analytics and IoT services. So it may seem just a little pedantic to suggest that Azure has fewer service offerings when compared with AWS.

The reality is that Microsoft is itself evolving as a corporation, and embracing open source and openness in general. Their Azure portal is more intuitive than the AWS console, and over time this familiarity will play a large part in their future Cloud success. If I were to bet on one of these two horses, it would most likely be Microsoft, despite the interminable drive of the Amazon train.

Some (only a few) common names and terms used for Microsoft Azure for common services are show below to get you started.

Regions are where Microsoft runs its Cloud services. These regions are similar to Availability Zones in AWS, except that where Amazon will have multiple Availability Zones in one region, Microsoft will offer a distinct region, which may contain multiple datacenters where services run from. Until quite recently (I am writing this in February 2018), Microsoft Azure was limited in the flexibility of its disaster recovery options. Regions were paired, and the customer would be limited to that pairing for its Cloud based operation. That has changed, and now Microsoft is also adopting the concept of Availability Zones, which allow the customer to choose where it runs its production and disaster recovery environments. The reason for putting some emphasis on this recent change (rather than simply explaining the new feature) is to draw attention to the fact that Microsoft is still playing catch-up to a certain extent, despite having more regions globally than any other Cloud service provider.

These are the compute building blocks in Azure, and provide ‘on-demand and scalable computing resource’. The physical resource supporting Azure compute is of course Microsoft Hyper-V, delivering all the features that Hyper-V supports. The VM will run either Windows or Linux, in the same way as AWS.

Blob Storage is Microsoft Azures version of Object storage, used for large volumes of data, stored as objects and typically supporting very high data volumes. The Blob storage offering from Azure differs from AWS S3 storage in a few key areas. Azure Blob storage consists of three types: Block Blobs, Append Blobs and Page Blobs. Unlike AWS S3, the Page Blob is designed to support the reading and writing of arbitrary ranges of bytes, and will therefore support data structures like databases. The basic premise of object storage remains though, and will distribute multiple copies of data across multiple regions, sustaining concurrent loss of data across two sites.

High performance storage, comparable to AWS EBS storage. I will not talk in detail on this tier of storage as it is covered in some detail under the general technical description for block storage. If you need storage that will support sustained high IOPS, the Azure Disk Storage is what you will most likely need.

This tier of storage is totally self-explanatory, and one of the benefits of Microsoft’s naming convention. Zero ambiguity. As described in the AWS section, file storage stores data as files and can be shared among multiple machines. This is very different to Disk storage, which are attached to individual virtual machines.