Microsoft to throttle Gen AI services for 'excessive' users

Microsoft to throttle Gen AI services for ‘excessive’ users

Time Stamp: November 2, 2023 2:45 AM
Microsoft to throttle Gen AI services for 'excessive' users PlatoBlockchain Data Intelligence. Vertical Search. Ai.

Microsoft has changed the terms and conditions for its online services to include warning that “excessive” users of its generative AI services will have their access restricted.

The new language appeared in a November 1 update to Microsoft’s legalese spotted by licensing-watchers Cloudy With A Chance Of Licensing.

The restrictions are described in a new clause of the document titled “Capacity Limitations”, is:

Excessive use of a Microsoft Generative AI Service may result in temporary throttling of Customer’s access to the Microsoft Generative AI Service.

The document does not, however, define “excessive use”, how long a “temporary” restriction might last, or exactly what happens during “throttling”.

The Register is also keen to know if it’s ethical to throttle an AI. They do have feelings, after all.

But we digress.

The need for throttles suggests bottlenecks in Microsoft’s AI architectures that could degrade performance for users, which could mean those who update Windows to start using its AI Copilots could enjoy a less-than-brilliant service. That could deter some from future use of Microsoft’s AI, a poor outcome for the software giant.

Or perhaps Microsoft just wants to keep its costs low. The Windows giant has already warned investors that scaling its cloudy AI will cost lots of money. On its October 24 earnings call the company revealed capital expenditures Q1 2024 reached $11.2 billion, with part of that sum consumed by “investments to scale our AI infrastructure.” The company also noted that its margins would be “offset by the impact of scaling our AI infrastructure to meet growing demand.”

That’s because all the servers and GPUs needed to power AI aren’t cheap to acquire. They’ll also cost plenty to operate, as they consume electricity and push data out over Microsoft’s networks.

Preventing overenthusiastic users from constant high usage could save Redmond serious coin.

Which may be needed as Microsoft reportedly makes a loss on GitHub’s AI Copilot.

And speaking of Copilot: Microsoft said it made its generative 365 Copilot generally available this week though as customers commenting on that announcement point out, you’ll need an enterprise plan, at least 300 seats signed up paying $30 a month for Copilot, and you need to call Redmond. Not quite so generally available as expected. That’s $9,000 a month right off the bat.

Other AI providers already impose some usage restrictions. OpenAI, for example, rate limits its API, and explains that’s needed to “ensure that everyone has fair access to the API.”

“If one person or organization makes an excessive number of requests, it could bog down the API for everyone else. By throttling the number of requests that a single user can make, OpenAI ensures that the most number of people have an opportunity to use the API without experiencing slowdowns,” the document adds.

OpenAi also limits access to its free tier. “If we believe you are not using the free tier in good faith, we may charge you standard fees or stop providing access to the Services,” states the outfit’s Terms Of Use.

Text-to-image outfit Midjourney’s Terms of Service states “If You purchase an unlimited plan, we will try to reasonably offer You unlimited access to the Services. However, we reserve the right to rate limit You to prevent quality decay or interruptions to other customers.”

The Register has asked Microsoft to explain the undefined terms in its new clause and will update this story if we receive a substantive response. ®

Laura Dobberstein contributed reporting on OpenAI’s legalese.

Time Stamp:

More from The Register