Activați CI/CD pentru punctele finale Amazon SageMaker cu mai multe regiuni

Amazon SageMaker și SageMaker inference endpoints provide a capability of training and deploying your AI and machine learning (ML) workloads. With inference endpoints, you can deploy your models for real-time or batch inference. The endpoints support various types of ML models hosted using Containere de învățare profundă AWS or your own containers with custom AI/ML algorithms. When you launch SageMaker inference endpoints with multiple instances, SageMaker distributes the instances across multiple Availability Zones (in a single Region) for high availability.

In some cases, however, to ensure lowest possible latency for customers in diverse geographical areas, you may require deploying inference endpoints in multiple Regions. Multi-Regional deployment of SageMaker endpoints and other related application and infrastructure components can also be part of a disaster recovery strategy for your mission-critical workloads aimed at mitigating the risk of a Regional failure.

Proiecte SageMaker implements a set of pre-built MLOps templates that can help manage endpoint deployments. In this post, we show how you can extend an MLOps SageMaker Projects pipeline to enable multi-Regional deployment of your AI/ML inference endpoints.

Prezentare generală a soluțiilor

SageMaker Projects deploys both training and deployment MLOPs pipelines; you can use these to train a model and deploy it using an inference endpoint. To reduce complexity and cost of a multi-Region solution, we assume that you train the model in a single Region and deploy inference endpoints in two or more Regions.

This post presents a solution that slightly modifies a SageMaker project template to support multi-Region deployment. To better illustrate the changes, the following figure displays both a standard MLOps pipeline created automatically by SageMaker (Steps 1-5) as well as changes required to extend it to a secondary Region (Steps 6-11).

The SageMaker Projects template automatically deploys a boilerplate MLOps solution, which includes the following components:

1. Amazon EventBridge monitoare AWS CodeCommit repositories for changes and starts a run of AWS CodePipeline if a code commit is detected.
2. If there is a code change, AWS CodeBuild orchestrates the model training using SageMaker training jobs.
3. After the training job is complete, the Registrul modelului SageMaker registers and catalogs the trained model.
4. To prepare for the deployment stage, CodeBuild extends the default Formarea AWS Cloud template configuration files with parameters of an approved model from the model registry.
5. Finally, CodePipeline runs the CloudFormation templates to deploy the approved model to the staging and production inference endpoints.

The following additional steps modify the MLOps Projects template to enable the AI/ML model deployment in the secondary Region:

6. O replică a lui Serviciul Amazon de stocare simplă (Amazon S3) bucket in the primary Region storing model artifacts is required in the secondary Region.
7. The CodePipeline template is extended with more stages to run a cross-Region deployment of the approved model.
8. As part of the cross-Region deployment process, the CodePipeline template uses a new CloudFormation template to deploy the inference endpoint in a secondary Region. The CloudFormation template deploys the model from the model artifacts from the S3 replica bucket created in Step 6.

9–11 optionally, create resources in Ruta Amazonului 53, Gateway API Amazon, și AWS Lambdas to route application traffic to inference endpoints in the secondary Region.

Cerințe preliminare

Create a SageMaker project in your primary Region (us-east-2 in this post). Complete the steps in Construirea, automatizarea, gestionarea și scalarea fluxurilor de lucru ML folosind Amazon SageMaker Pipelines until the section Modificarea exemplului de cod pentru un caz de utilizare personalizat.

Update your pipeline in CodePipeline

In this section, we discuss how to add manual CodePipeline approval and cross-Region model deployment stages to your existing pipeline created for you by SageMaker.

1. On the CodePipeline console in your primary Region, find and select the pipeline containing your project name and ending with deploy. This pipeline has already been created for you by SageMaker Projects. You modify this pipeline to add AI/ML endpoint deployment stages for the secondary Region.
2. Alege Editati.
3. Alege Adaugă etapa.
4. Pentru Nume de scena, introduce SecondaryRegionDeployment.
5. Alege Adaugă etapa.
6. În SecondaryRegionDeployment stage, choose Add action group.In this action group, you add a manual approval step for model deployment in the secondary Region.
7. Pentru Numele acțiunii, introduce ManualApprovaltoDeploytoSecondaryRegion.
8. Pentru Furnizor de acțiuni, alege Manual approval.
9. Leave all other settings at their defaults and choose Terminat .
10. În SecondaryRegionDeployment stage, choose Add action group (după ManualApprovaltoDeploytoSecondaryRegion).In this action group, you add a cross-Region AWS CloudFormation deployment step. You specify the names of build artifacts that you create later in this post.
11. Pentru Numele acțiunii, introduce DeploytoSecondaryRegion.
12. Pentru Furnizor de acțiuni, alege Formarea AWS Cloud.
13. Pentru Regiune, enter your secondary Region name (for example, us-west-2).
14. Pentru Input artifacts, introduce BuildArtifact.
15. Pentru ActionMode, introduce CreateorUpdateStack.
16. Pentru StackName, introduce DeploytoSecondaryRegion.
17. În Format, Pentru Numele artefactului, Selectați BuildArtifact.
18. În Format, Pentru Nume fișier, introduce template-export-secondary-region.yml.
19. Întoarce Use Configuration File pe.
20. În Format, Pentru Numele artefactului, Selectați BuildArtifact.
21. În Format, Pentru Nume fișier, introduce secondary-region-config-export.json.
22. În Capabilitățile, alege CAPABILITY_NAMED_IAM.
23. Pentru Rol, alege AmazonSageMakerServiceCatalogProductsUseRole created by SageMaker Projects.
24. Alege Terminat .
25. Alege Economisiți.
26. În cazul în care o Save pipeline changes dialog appears, choose Economisiți din nou.

Modificați rolul IAM

We need to add additional permissions to the Gestionarea identității și accesului AWS (IAM) rol AmazonSageMakerServiceCatalogProductsUseRole creat de Catalog de servicii AWS to enable CodePipeline and S3 bucket access for cross-Region deployment.

1. Pe consola IAM, alegeți Roluri în panoul de navigare.
2. Căutați și selectați AmazonSageMakerServiceCatalogProductsUseRole.
3. Choose the IAM policy under Numele politicii: AmazonSageMakerServiceCatalogProductsUseRole-XXXXXXXXX.
4. Alege Editați politica și apoi JSON.
5. Modify the AWS CloudFormation permissions to allow CodePipeline to sync the S3 bucket in the secondary Region. You can replace the existing IAM policy with the updated one from the following GitHub repo (see lines:16-18, 198, 213)
6. Alege Revizuiți politica.
7. Alege Salvează modificările.

Add the deployment template for the secondary Region

To spin up an inference endpoint in the secondary Region, the SecondaryRegionDeployment stage needs a CloudFormation template (for endpoint-config-template-secondary-region.yml) and a configuration file (secondary-region-config.json).

The CloudFormation template is configured entirely through parameters; you can further modify it to fit your needs. Similarly, you can use the config file to define the parameters for the endpoint launch configuration, such as the instance type and instance count:

{
  "Parameters": {
    "StageName": "secondary-prod",
    "EndpointInstanceCount": "1",
    "EndpointInstanceType": "ml.m5.large",
    "SamplingPercentage": "100",
    "EnableDataCapture": "true"
  }

To add these files to your project, download them from the provided links and upload them to Amazon SageMaker Studio in the primary Region. In Studio, choose Browser de fișiere and then the folder containing your project name and ending with modeldeploy.

Upload these files to the deployment repository’s root folder by choosing the upload icon. Make sure the files are located in the root folder as shown in the following screenshot.

Modify the build Python file

Next, we need to adjust the deployment build.py file to enable SageMaker endpoint deployment in the secondary Region to do the following:

• Retrieve the location of model artifacts and Registrul Amazon de containere elastice (Amazon ECR) URI for the model image in the secondary Region
• Prepare a parameter file that is used to pass the model-specific arguments to the CloudFormation template that deploys the model in the secondary Region

Puteți descărca actualizată build.py file and replace the existing one in your folder. In Studio, choose Browser de fișiere and then the folder containing your project name and ending with modeldeploy. Locate the build.py file and replace it with the one you downloaded.

The CloudFormation template uses the model artifacts stored in a S3 bucket and the Amazon ECR image path to deploy the inference endpoint in the secondary Region. This is different from the deployment from the model registry in the primary Region, because you don’t need to have a model registry in the secondary Region.

Modify the buildspec file

buildspec.yml contains instructions run by CodeBuild. We modify this file to do the following:

• Install the SageMaker Python library needed to support the code run
• Pass through the –secondary-region and model-specific parameters to build.py
• Add the S3 bucket content sync from the primary to secondary Regions
• Export the secondary Region CloudFormation template and associated parameter file as artifacts of the CodeBuild step

Deschideți buildspec.yml file from the model deploy folder and make the highlighted modifications as shown in the following screenshot.

Alternatively, you can download the following buildspec.yml file to replace the default file.

Add CodeBuild environment variables

In this step, you add configuration parameters required for CodeBuild to create the model deployment configuration files in the secondary Region.

1. On the CodeBuild console in the primary Region, find the project containing your project name and ending with deploy. This project has already been created for you by SageMaker Projects.

2. Choose the project and on the Editati meniu, alegeți Mediu inconjurator.

3. În Configurare avansată secțiune, deselectați Allow AWS CodeBuild to modify this service role so it can be used with this build project.
4. Add the following environment variables, defining the names of the additional CloudFormation templates, secondary Region, and model-specific parameters:
   1. EXPORT_TEMPLATE_NAME_SECONDARY_REGION - Pentru Valoare, introduce template-export-secondary-region.yml si pentru Tip, alege Text simplu.
   2. EXPORT_TEMPLATE_SECONDARY_REGION_CONFIG - Pentru Valoare, introduce secondary-region-config-export.json si pentru Tip, alege Text simplu.
   3. AWS_SECONDARY_REGION - Pentru Valoare, enter us-west-2 and for Tip, alege Text simplu.
   4. CADRU - Pentru Valoare, introduce xgboost (replace with your framework) and for Tip, alege Text simplu.
   5. MODEL_VERSION - Pentru Valoare, enter 1.0-1 (replace with your model version) and for Tip, alege Text simplu.
5. Copiați valoarea lui ARTIFACT_BUCKET into Notepad or another text editor. You need this value in the next step.
6. Alege Update environment.

You need the values you specified for model training for FRAMEWORK și MODEL_VERSION. For example, to find these values for the Abalone model used in MLOps boilerplate deployment, open Studio and on the Browser de fișiere menu, open the folder with your project name and ending with modelbuild. Navigate to pipelines/abalone și deschideți pipeline.py file. Search for sagemaker.image_uris.retrieve and copy the relevant values.

Create an S3 replica bucket in the secondary Region

We need to create an S3 bucket to hold the model artifacts in the secondary Region. SageMaker uses this bucket to get the latest version of model to spin up an inference endpoint. You only need to do this one time. CodeBuild automatically syncs the content of the bucket in the primary Region to the replication bucket with each pipeline run.

1. În consola Amazon S3, alegeți Creați găleată.
2. Pentru Numele cupei, introduceți valoarea lui ARTEFACT_BUCKET copied in the previous step and append -replica to the end (for example, sagemaker-project-X-XXXXXXXX-replica.
3. Pentru Regiunea AWS, enter your secondary Region (us-west-2).
4. Leave all other values at their default and choose Creați găleată.

Approve a model for deployment

The deployment stage of the pipeline requires an approved model to start. This is required for the deployment in the primary Region.

1. In Studio (primary Region), choose Resurse SageMaker în panoul de navigare.
2. Pentru Select the resource to view, alege Registrul modelului.
3. Choose model group name starting with your project name.
4. In the right pane, check the model version, stage and status.
5. If the status shows pending, choose the model version and then choose Starea de actualizare.
6. Change status to Aprobat, Apoi alegeți Starea de actualizare.

Deploy and verify the changes

All the changes required for multi-Region deployment of your SageMaker inference endpoint are now complete and you can start the deployment process.

1. In Studio, save all the files you edited, choose merge, and choose the repository containing your project name and ending with deploy.
2. Choose the plus sign to make changes.
3. În schimbată, adăuga build.py și buildspec.yml.
4. În Neurmată, adăuga endpoint-config-template-secondary-region.yml și secondary-region-config.json.
5. Enter a comment in the Rezumat câmp și alegeți Comite.
6. Push the changes to the repository by choosing Împinge.

Pushing these changes to the CodeCommit repository triggers a new pipeline run, because an EventBridge event monitors for pushed commits. After a few moments, you can monitor the run by navigating to the pipeline on the CodePipeline console.

Make sure to provide manual approval for deployment to production and the secondary Region.

You can verify that the secondary Region endpoint is created on the SageMaker console, by choosing Contul Meu in the navigation pane and confirming the endpoint status in Activitate recenta.

Add API Gateway and Route 53 (Optional)

You can optionally follow the instructions in Apelați un punct final al modelului Amazon SageMaker folosind Amazon API Gateway și AWS Lambda to expose the SageMaker inference endpoint in the secondary Region as an API using API Gateway and Lambda.

A curăța

To delete the SageMaker project, see Delete an MLOps Project using Amazon SageMaker Studio. To ensure the secondary inference endpoint is destroyed, go to the AWS CloudFormation console and delete the related stacks in your primary and secondary Regions; this destroys the SageMaker inference endpoints.

Concluzie

In this post, we showed how a MLOps specialist can modify a preconfigured MLOps template for their own multi-Region deployment use case, such as deploying workloads in multiple geographies or as part of implementing a multi-Regional disaster recovery strategy. With this deployment approach, you don’t need to configure services in the secondary Region and can reuse the CodePipeline and CloudBuild setups in the primary Region for cross-Regional deployment. Additionally, you can save on costs by continuing the training of your models in the primary Region while utilizing SageMaker inference in multiple Regions to scale your AI/ML deployment globally.

Please let us know your feedback in the comments section.

Despre Autori

Mehran Najafi, dr., este un arhitect senior de soluții pentru AWS, concentrat pe soluții AI/ML și SaaS la scară.

Steven Alyekhin is a Senior Solutions Architect for AWS focused on MLOps at Scale.