Slack leverer indbygget og sikker generativ AI drevet af Amazon SageMaker JumpStart | Amazon Web Services

Slack leverer indbygget og sikker generativ AI drevet af Amazon SageMaker JumpStart | Amazon Web Services

This post is co-authored by Jackie Rocca, VP of Product, AI at Slack

Slack is where work happens. It’s the AI-powered platform for work that connects people, conversations, apps, and systems together in one place. With the newly launched Slack AI—a trusted, native, generative artificial intelligence (AI) experience available directly in Slack—users can surface and prioritize information so they can find their focus and do their most productive work.

We are excited to announce that Slack, a Salesforce company, has collaborated with Amazon SageMaker JumpStart to power Slack AI’s initial search and summarization features and provide safeguards for Slack to use large language models (LLMs) more securely. Slack worked with SageMaker JumpStart to host industry-leading third-party LLMs so that data is not shared with the infrastructure owned by third party model providers.

This keeps customer data in Slack at all times and upholds the same security practices and compliance standards that customers expect from Slack itself. Slack is also using Amazon SageMaker inference capabilities for advanced routing strategies to scale the solution to customers with optimal performance, latency, and throughput.

“With Amazon SageMaker JumpStart, Slack can access state-of-the-art foundation models to power Slack AI, while prioritizing security and privacy. Slack customers can now search smarter, summarize conversations instantly, and be at their most productive.”

– Jackie Rocca, VP Product, AI at Slack

Foundation models in SageMaker JumpStart

SageMaker JumpStart is a machine learning (ML) hub that can help accelerate your ML journey. With SageMaker JumpStart, you can evaluate, compare, and select foundation models (FMs) quickly based on predefined quality and responsibility metrics to perform tasks like article summarization and image generation. Pretrained models are fully customizable for your use case with your data, and you can effortlessly deploy them into production with the user interface or SDK. In addition, you can access prebuilt solutions to solve common use cases and share ML artifacts, including ML models and notebooks, within your organization to accelerate ML model building and deployment. None of your data is used to train the underlying models. All the data is encrypted and is never shared with third-party vendors so you can trust that your data remains private and confidential.

Tjek den SageMaker JumpStart model page for available models.

Slack AI

Slack launched Slack AI to provide native generative AI capabilities so that customers can easily find and consume large volumes of information quickly, enabling them to get even more value out of their shared knowledge in Slack.  For example, users can ask a question in plain language and instantly get clear and concise answers with enhanced search. They can catch up on channels and threads in one click with conversation summaries. And they can access personalized, daily digests of what’s happening in select channels with the newly launched recaps.

Because trust is Slack’s most important value, Slack AI runs on an enterprise-grade infrastructure they built on AWS, upholding the same security practices and compliance standards that customers expect. Slack AI is built for security-conscious customers and is designed to be secure by design—customer data remains in-house, data is not used for LLM training purposes, and data remains siloed.

Slack delivers native and secure generative AI powered by Amazon SageMaker JumpStart | Amazon Web Services PlatoBlockchain Data Intelligence. Vertical Search. Ai.

Løsningsoversigt

SageMaker JumpStart provides access to many LLMs, and Slack selects the right FMs that fit their use cases. Because these models are hosted on Slack’s owned AWS infrastructure, data sent to models during invocation doesn’t leave Slack’s AWS infrastructure. In addition, to provide a secure solution, data sent for invoking SageMaker models is encrypted in transit. The data sent to SageMaker JumpStart endpoints for invoking models is not used to train base models. SageMaker JumpStart allows Slack to support high standards for security and data privacy, while also using state-of-the-art models that help Slack AI perform optimally for Slack customers.

SageMaker JumpStart endpoints serving Slack business applications are powered by AWS instances. SageMaker supports a wide range of instance types for model deployment, which allows Slack to pick the instance that is best suited to support latency and scalability requirements of Slack AI use cases. Slack AI has access to multi-GPU based instances to host their SageMaker JumpStart models. Multiple GPU instances allow each instance backing Slack AI’s endpoint to host multiple copies of a model. This helps improve resource utilization and reduce model deployment cost. For more information, refer to Amazon SageMaker tilføjer nye inferensfunktioner for at hjælpe med at reducere omkostningerne til implementering af fundamentmodeller og latens.

Følgende diagram illustrerer løsningsarkitekturen.

Slack delivers native and secure generative AI powered by Amazon SageMaker JumpStart | Amazon Web Services PlatoBlockchain Data Intelligence. Vertical Search. Ai.

To use the instances most effectively and support the concurrency and latency requirements, Slack used SageMaker-offered routing strategies with their SageMaker endpoints. By default, a SageMaker endpoint uniformly distributes incoming requests to ML instances using a round-robin algorithm routing strategy called RANDOM. However, with generative AI workloads, requests and responses can be extremely variable, and it’s desirable to load balance by considering the capacity and utilization of the instance rather than random load balancing. To effectively distribute requests across instances backing the endpoints, Slack uses the LEAST_OUTSTANDING_REQUESTS (LAR) routing strategy. This strategy routes requests to the specific instances that have more capacity to process requests instead of randomly picking any available instance. The LAR strategy provides more uniform load balancing and resource utilization. As a result, Slack AI noticed over a 39% latency decrease in their p95 latency numbers when enabling LEAST_OUTSTANDING_REQUESTS compared to RANDOM.

For more details on SageMaker routing strategies, see Minimize real-time inference latency by using Amazon SageMaker routing strategies.

Konklusion

Slack is delivering native generative AI capabilities that will help their customers be more productive and easily tap into the collective knowledge that’s embedded in their Slack conversations. With fast access to a large selection of FMs and advanced load balancing capabilities that are hosted in dedicated instances through SageMaker JumpStart, Slack AI is able to provide rich generative AI features in a more robust and quicker manner, while upholding Slack’s trust and security standards.

Lær mere om SageMaker JumpStart, Slack AI , how the Slack team built Slack AI to be secure and private. Leave your thoughts and questions in the comments section.


Om forfatterne

Slack delivers native and secure generative AI powered by Amazon SageMaker JumpStart | Amazon Web Services PlatoBlockchain Data Intelligence. Vertical Search. Ai.Jackie Rocca is VP of Product at Slack, where she oversees the vision and execution of Slack AI, which brings generative AI natively and securely into Slack’s user experience. Now she’s on a mission to help customers accelerate their productivity and get even more value out of their conversations, data, and collective knowledge with generative AI. Prior to her time at Slack, Jackie was a Product Manager at Google for more than six years, where she helped launch and grow Youtube TV. Jackie is based in the San Francisco Bay Area.

Slack delivers native and secure generative AI powered by Amazon SageMaker JumpStart | Amazon Web Services PlatoBlockchain Data Intelligence. Vertical Search. Ai.Rachna Chadha er Principal Solutions Architect AI/ML i Strategic Accounts hos AWS. Rachna er en optimist, der mener, at den etiske og ansvarlige brug af kunstig intelligens kan forbedre samfundet i fremtiden og bringe økonomisk og social velstand. I sin fritid kan Rachna godt lide at bruge tid med sin familie, vandreture og lytte til musik.

Slack delivers native and secure generative AI powered by Amazon SageMaker JumpStart | Amazon Web Services PlatoBlockchain Data Intelligence. Vertical Search. Ai.Marc Karp er en ML-arkitekt hos Amazon SageMaker Service-teamet. Han fokuserer på at hjælpe kunder med at designe, implementere og administrere ML-arbejdsbelastninger i stor skala. I sin fritid nyder han at rejse og udforske nye steder.

Slack delivers native and secure generative AI powered by Amazon SageMaker JumpStart | Amazon Web Services PlatoBlockchain Data Intelligence. Vertical Search. Ai.Maninder (Mani) Kaur er AI/ML Specialist lead for strategiske ISV'er hos AWS. Med sin kunde-først tilgang hjælper Mani strategiske kunder med at forme deres AI/ML-strategi, fremme innovation og accelerere deres AI/ML-rejse. Mani er en fast tilhænger af etisk og ansvarlig AI og stræber efter at sikre, at hendes kunders AI-løsninger stemmer overens med disse principper.

Slack delivers native and secure generative AI powered by Amazon SageMaker JumpStart | Amazon Web Services PlatoBlockchain Data Intelligence. Vertical Search. Ai.Gene Ting is a Principal Solutions Architect at AWS. He is focused on helping enterprise customers build and operate workloads securely on AWS. In his free time, Gene enjoys teaching kids technology and sports, as well as following the latest on cybersecurity.

Slack delivers native and secure generative AI powered by Amazon SageMaker JumpStart | Amazon Web Services PlatoBlockchain Data Intelligence. Vertical Search. Ai.Alan Tan er en Senior Product Manager hos SageMaker, der leder indsatsen inden for store modelslutninger. Han brænder for at anvende maskinlæring til analyseområdet. Uden for arbejdet nyder han udendørslivet.

Tidsstempel:

Mere fra AWS maskinindlæring