The Cloud Killed Infrastructure, Long Live Infrastructure!

Vertical clouds are on the rise, as traditional clouds give way to specialization.

The infrastructure narrative of the last 15 years has been dominated by the cloud. Its growth has driven a surge of innovation in everything from hardware, to software, to operations – and resulted in the largest migration of workloads in the history of the industry. A handful of hyperscalers have driven this growth, and their impact is hard to overstate. They dominate the talent pool. They dominate market cap growth. And they dominate global IT spend, with the top few spending more that $5B in 2019 (and at least one over $10B). As a result, they also have outsized influence on the entire supply chain from chips to software, which they now use to drive agendas confined to their (relatively) narrow view of how infrastructure should look. 

From this very well-earned position of dominance, they’ve convinced large swaths of the industry that they are the only companies who can build scalable and cost-effective cloud infrastructure upon which the rest of the industry should be content building applications. (Until, of course, they decide to launch competitive services as a matter of course.) 

However, like all architectural epochs, the large centralized clouds are just another waypoint on the long arc of IT infrastructure. And their extreme success is now resulting in yet another shift that is naturally pushing workloads back out to more specialized infrastructure. And that trend is the focus of this post. 

This shift, by the way, is in no way any failing on the part of the traditional clouds. In fact, they’re still relatively early in their journey of consuming traditional IT, and will continue to do that for decades – growingly significantly along the way. They are doing exactly what they should, building a very general offering to support the long tail of IT applications. 

However, for the most part, to offer a higher-level resource, they end up just bundling the usual low-level resources in new ways — and they optimize them for the mean workload. As a result, the trad clouds are often too general, too inflexible, and their services too shallow. And so a new crop of infrastructure companies are being built to fill the growing need around rich, vertically integrated services. Or vertical clouds, as we like to call them. 

Vertical clouds, which are entirely focused on a specific type of workload or cloud service, tend to be far more sophisticated, far more cost effective, and far more performant. And while they may be built on top of the trad, centralized clouds to start with, more and more we’re seeing them using special purpose physical infrastructure too. 

These companies are now able to do this because (a) cloud applications are increasingly loosely coupled, allowing their developers to pick and choose what cloud infrastructure services they use, and perhaps most significantly, because (b) the size of the cloud has reached a point where the markets for these individual cloud services are large enough to support large, viable independent companies.

And so, in many ways, we’re entering a new and incredibly exciting era of infrastructure, in which any infrastructure service (and really any common sub-component of an application) is fair game to build a company around as a verticalized cloud. The better you are at building the infrastructure, the better the service will be. And because the market is large enough to sustain this, the large central clouds are structurally disadvantaged to compete. The primary question to startups in infrastructure has ironically shifted from “what if AWS/GCP/Azure decides to compete with you?” to “why aren’t you competing more directly with AWS/GCP/Azure?”

To pull away from the abstract, let’s look at a few areas where this is concretely changing how infrastructure companies are being built as used. We’ve selected three, but there are many many others (and given time we’d love to write those up too):

Front end devs are consuming the back end, and compute is coming to them: Traditionally, the compute logic for building a web page sits server side and is implemented by a back end team. However, there is an order of magnitude more front end developers, and so we’re seeing the rise of many frameworks that enable front end developers to build sophisticated apps using cloud services directly as a back end. The most popular emergent stack for this is the JAMstack.

Interactive web apps clearly benefit from compute being closer to the user. And because the back end is now being abstracted as a set of services, the teams that implement them can push them closer to the user by taking advantage of the CDN tier. And that’s exactly what we’re seeing happen now. Many new services, from databases, to rendering engines, to collaboration tools, are being implemented as workloads and run on next-generation CDNs such as Fly.io and CloudFlare; this provides both power to the front end and performance to the users. Naturally, Fly and CloudFlare build their own infrastructure, because ultimately that’s the best way to provide low cost and good performance, if you have the ability to do so. 

AI is driving new hardware build-outs: Different workloads benefit from different configurations of hardware and software. So much so that running a workload on a platform not tuned for it can result in orders of magnitudes of loss in cost and performance. While there are many classes of workloads that both fall in this category and are popular across the cloud user base, perhaps the most notable of those are AI workloads. It’s well known that Facebook, Google, Microsoft, and many other companies have built bespoke clusters for AI training. 

AI workloads differ dramatically from the traditional cloud applications built around web servers and databases. A specialized cloud not only has different silicon built for AI/ML computation, but also a different scheduler, network interconnect, modes for managing failure, and many other design aspects tuned for this purpose. Given all this, it’s little wonder that over the last few years we’ve seen AI-focused clouds such as MosaicML grow in popularity. 

This isn’t to say that the trad clouds can’t provide AI-focused services. All of the popular ones do. But it’s yet another service, sitting alongside hundreds of others. And so for those users for whom AI workload cost and performance are paramount, it’s understandable that they’re increasingly exploring other options. 

App platforms going full stack: The previous two examples are horizontal infrastructure components that sit well below the app layer. However, we’re also starting to see the trend move to higher-level app platforms. One of our favorite examples of this is the work being done at Mighty. Mighty runs the browser as a service in the cloud to make web apps faster (loading, interactivity, stacking, etc.). The team chose specific hardware and integrated it with its own Chromium browser to provide an experience that’s orders of magnitudes better than traditional public cloud offerings for Remote Desktop software, while rivaling a native laptop experience.

—

Before we wrap it, it’s worth noting that a lot of both technical and business innovation over the last decade is helping enable this trend. Technically, it’s easier than ever to stand up a bare metal hardware offering. And the hardware vendors have adapted their business models to allow for full OpEx rather than CapEx. 

We also want to make the point that the focus of this piece is on infrastructure companies that are built by infrastructure teams whose expertise is, well, infrastructure. There are many such teams that are fully capable of building scalable systems from hardware on up. In the past, the markets simply weren’t large enough to sustain this level of specialization. But they are now. And a new era of infrastructure is following. And we’ll all be better for it. 

***

The views expressed here are those of the individual AH Capital Management, L.L.C. (“a16z”) personnel quoted and are not the views of a16z or its affiliates. Certain information contained in here has been obtained from third-party sources, including from portfolio companies of funds managed by a16z. While taken from sources believed to be reliable, a16z has not independently verified such information and makes no representations about the enduring accuracy of the information or its appropriateness for a given situation. In addition, this content may include third-party advertisements; a16z has not reviewed such advertisements and does not endorse any advertising content contained therein.

This content is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. You should consult your own advisers as to those matters. References to any securities or digital assets are for illustrative purposes only, and do not constitute an investment recommendation or offer to provide investment advisory services. Furthermore, this content is not directed at nor intended for use by any investors or prospective investors, and may not under any circumstances be relied upon when making a decision to invest in any fund managed by a16z. (An offering to invest in an a16z fund will be made only by the private placement memorandum, subscription agreement, and other relevant documentation of any such fund and should be read in their entirety.) Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z, and there can be no assurance that the investments will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by Andreessen Horowitz (excluding investments for which the issuer has not provided permission for a16z to disclose publicly as well as unannounced investments in publicly traded digital assets) is available at https://a16z.com/investments/.

Charts and graphs provided within are for informational purposes solely and should not be relied upon when making any investment decision. Past performance is not indicative of future results. The content speaks only as of the date indicated. Any projections, estimates, forecasts, targets, prospects, and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Please see https://a16z.com/disclosures for additional important information.