All 7 Amazon SageMaker & Machine Learning on AWS questions
More in Amazon SageMaker & Machine Learning on AWS
What is SageMaker hyperparameter tuning, and how does it help improve model accuracy? How do SageMaker endpoints support real time inference, and how do they differ from batch transform? What is SageMaker Studio, and how does it improve the machine learning development experience? How does SageMaker support MLOps practices such as model monitoring and automated retraining pipelines?
Advertisement
AWS › Amazon SageMaker & Machine Learning on AWS

How does SageMaker support multi model endpoints and model versioning for cost efficient deployment at scale?

Advanced

Answer

SageMaker multi model endpoints let you host many different models behind a single endpoint, dynamically loading each model into memory only when it is actually requested and unloading less frequently used models to save resources, which is significantly more cost efficient than deploying a separate dedicated endpoint for every single model, especially useful when you have hundreds of similar models, such as one per customer.
aws sagemaker create-model --model-name multi-model-example --primary-container Image=xgboost,ModelDataUrl=s3://my-bucket/models/,Mode=MultiModel
Real-world example A company offering personalized pricing models for thousands of individual retail clients uses a SageMaker multi model endpoint to serve all of those models cost effectively from shared infrastructure, rather than paying for thousands of individually running endpoints.

Common follow-ups: What are the tradeoffs of multi model endpoints in terms of latency for infrequently used models?;How does model versioning work when updating one of many models behind a multi model endpoint?

AWS Cost Management & Billing;Auto Scaling Groups
All 7 Amazon SageMaker & Machine Learning on AWS questions