April 04, 2016 - by Bill Fine

Share:

Private cloud is dead (long live the private data center).

A few weeks ago Wired magazine chronicled The Epic Story of Dropbox’s Exodus From the Amazon Cloud Empire. Dropbox left Amazon, citing high costs, and is now running its business in private data centers. Is this an anomaly, or the beginning of a trend? Are other companies making the same choice, to move away from Amazon, but doing so less publicly?

At Joyent, we build open source software to automate data centers. Like Amazon, we operate our own public cloud. Unlike Amazon, we also offer the software that powers our public cloud to enterprises who want to run their own data centers. Simply put, we make it easy for companies to switch from the public cloud to a private data center, or to use both in a hybrid model. And, as I look across our customer base and sales pipeline, I see a clear trend toward a hybrid cloud model emerging among what I call “cloud mature” companies.

I recently met with a large AWS customer that is planning to migrate a large percentage of its infrastructure into its own private data center. Their situation is typical. They are spending millions on public cloud services today, and pumping petabytes of data into a public cloud object store that they cannot take with them. As they look a couple of years down the road, they see an ever-growing public cloud bill and no way out. Shifting to a hybrid model that includes a private data center makes financial and strategic sense, IF their team can efficiently handle the operations of a private data center and keep pace with innovations in the public cloud.

To prove that running their own private data center could be simple and easy at the scale they require, we agreed to work with them to stand up an environment (on commodity hardware, in a colocation facility) that modeled their planned usage over the next five years. From an object storage perspective, that meant a private data center that could handle over 40 petabytes of data, concurrently processing 40 gigabits per second of reads and writes. A small team stood up and began operating their environment, based on Joyent’s Triton and Manta software, in one week. Good-bye S3. Hello to tens or perhaps hundreds of millions of dollars in savings.

Many still believe that operating a private compute and storage service at this scale is too complex for the average IT department. While that once may have been the case, in the same way it was once true that public clouds were less secure than private data centers, it isn’t anymore. Yes, VMware is expensive and OpenStack is hard, but new open source technologies, like Joyent’s Triton and Manta, make standing up and operating a private compute or storage service simple and easy.

But what about the challenge of keeping pace with innovations in the public cloud? It isn’t enough for a private data center to be both cheaper than public cloud options and simple to operate. AWS is more than just EC2 and S3. Higher level database services like RDS are wildly popular. For a private data center to be successful today it needs to be able to deliver the developer-friendly services that public cloud users have come to expect and depend upon.

Here, again, the march of technology comes to the rescue. While even just a year ago, the ability to deliver database as a service was beyond the means of the typical IT shop, that isn’t the case today. New tools and development techniques are emerging that deliver the benefits of services like RDS, without the lock-in (and lock-out) that comes with higher level, public cloud, proprietary services. Leveraging these new technologies, IT shops can now easily deliver developer-friendly services that are, in important ways, better than those available in the public cloud.

The public cloud isn’t going anywhere. Amazon has nearly $10 billion in annualized cloud sales and is growing at nearly 70% year-on-year. Enterprises are flocking to Amazon, Google, Microsoft, Joyent and other public clouds for good reason. Public clouds provide speed and agility to organizations, and cost savings for companies with unpredictable infrastructure needs. But, as technology advances and as companies born in the cloud grow up, what we are starting to see is the emergence of a new definition of hybrid cloud, that starts in the public cloud and then extends to the private data center as an enterprise matures. Look beyond the headline-grabbing public cloud growth numbers, and what you will find is a growing class of “cloud-mature” customers that are realizing three things:

1. For predictable workloads, running in a private data center is much cheaper than running in a public cloud,
2. If they keep pumping their data into a proprietary, public cloud object store, the gravity of that data is going to lock them into a single public cloud provider forever, and
3. Technology advances are making operating private data centers a viable option.

In an interesting twist, growing adoption of the public cloud and increasing levels of cloud maturity among the biggest public cloud users is what is making the private data center relevant again.