Machine Learning Lens

AWS Well-Architected Framework

Abstract and introduction
Well-Architected Framework pillars
Well-Architected machine learning lifecycle
Well-Architected machine learning design principles
Well-Architected machine learning
Conclusion
Contributors
References
Document history
Best practices arranged by pillar
Best practices by ML lifecycle phase
Notices
AWS Glossary

Best practices by ML lifecycle phase - Machine Learning Lens

Documentation AWS Well-Architected AWS Well-Architected Framework

Business goal identification phase ML problem framing phase Data processing phase Model development phase Model deployment phase Model monitoring phase

Cette page n'a pas été traduite dans votre langue. Demande de traduction

Best practices by ML lifecycle phase

PDF RSS

There are six phases in the machine learning lifecycle and within each there are six pillars of the Well-Architected Framework. The following index lists the best practices by lifecycle phase.

Business goal identification phase

Operational excellence pillar

MLOE-01: Develop the right skills with accountability and empowerment
MLOE-02: Discuss and agree on the level of model explainability
MLOE-03: Monitor model compliance to business requirements

Security pillar

MLSEC-01: Validate ML data permissions, privacy, software, and license terms

Reliability pillar

There are no reliability pillar best practices for business goal identification.

Performance efficiency pillar

MLPER-01: Determine key performance indicators

Cost optimization pillar

MLCOST-01: Define overall return on investment (ROI) and opportunity cost
MLCOST-02: Use managed services to reduce total cost of ownership (TCO)

Sustainability pillar

MLSUS-01: Define the overall environmental impact or benefit

ML problem framing phase

Operational excellence pillar

MLOE-04: Establish ML roles and responsibilities
MLOE-05: Prepare an ML profile template
MLOE-06: Establish model improvement strategies
MLOE-07: Establish a lineage tracker system
MLOE-08: Establish feedback loops across ML lifecycle phases
MLOE-09: Review fairness and explainability

Security pillar

MLSEC-02: Design data encryption and obfuscation

Reliability pillar

MLREL-01: Use APIs to abstract change from model consuming applications
MLREL-02: Adopt a machine learning microservice strategy

Performance efficiency pillar

MLPER-02: Use purpose-built AI and ML services and resources
MLPER-03: Define relevant evaluation metrics

Cost optimization pillar

MLCOST-03: Identify if machine learning is the right solution
MLCOST-04: Tradeoff analysis on custom versus pre-trained models

Sustainability pillar

MLSUS-02: Consider AI services and pre-trained models
MLSUS-03: Select sustainable Regions

Data processing phase

Operational excellence pillar

MLOE-10: Profile data to improve quality
MLOE-11: Create tracking and version control mechanisms

Security pillar

MLSEC-03: Ensure least privilege access
MLSEC-04: Secure data and modeling environment
MLSEC-05: Protect sensitive data privacy
MLSEC-06: Enforce data lineage
MLSEC-07: Keep only relevant data

Reliability pillar

MLREL-03: Use a data catalog
MLREL-04: Use a data pipeline
MLREL-05: Automate managing data changes

Performance efficiency pillar

MLPER-04: Use a modern data architecture

Cost optimization pillar

MLCOST-05: Use managed data labeling
MLCOST-06: Use data wrangler tools for interactive analysis
MLCOST-07: Use managed data processing capabilities
MLCOST-08: Enable feature reusability

Sustainability pillar

MLSUS-04: Minimize idle resources
MLSUS-05: Implement data lifecycle policies aligned with your sustainability goals
MLSUS-06: Adopt sustainable storage options

Model development phase

Operational excellence pillar

MLOE-12: Automate operations through MLOps and CI/CD
MLOE-13: Establish reliable packaging patterns to access approved public libraries

Security pillar

MLSEC-08: Secure governed ML environment
MLSEC-09: Secure inter-node cluster communications
MLSEC-10: Protect against data poisoning threats

Reliability pillar

MLREL-06: Enable CI/CD/CT automation with traceability
MLREL-07: Ensure feature consistency across training and inference
MLREL-08: Ensure model validation with relevant data
MLREL-09: Establish data bias detection and mitigation

Performance efficiency pillar

MLPER-05: Optimize training and inference instance types
MLPER-06: Explore alternatives for performance improvement
MLPER-07: Establish a model performance evaluation pipeline
MLPER-08: Establish feature statistics
MLPER-09: Perform a performance trade-off analysis
MLPER-10: Detect performance issues when using transfer learning

Cost optimization pillar

MLCOST-09: Select optimal computing instance size
MLCOST-10: Use managed build environments
MLCOST-11: Select local training for small scale experiments
MLCOST-12: Select an optimal ML framework
MLCOST-13: Use automated machine learning
MLCOST-14: Use managed training capabilities
MLCOST-15: Use distributed training
MLCOST-16: Stop resources when not in use
MLCOST-17: Start training with small datasets
MLCOST-18: Use warm-start and checkpointing hyperparameter tuning
MLCOST-19: Use hyperparameter optimization technologies
MLCOST-20 - Setup budget and use resource tagging to track costs
MLCOST-21: Enable data and compute proximity
MLCOST-22: Select optimal algorithms
MLCOST-23: Enable debugging and logging

Sustainability pillar

MLSUS-07: Define sustainable performance criteria
MLSUS-08: Select energy-efficient algorithms
MLSUS-09: Archive or delete unnecessary training artifacts
MLSUS-10: Use efficient model tuning methods

Model deployment phase

Operational excellence pillar

MLOE-14: Establish deployment environment metrics

Security pillar

MLSEC-11: Protect against adversarial and malicious activities

Reliability pillar

MLREL-10: Automate endpoint changes through a pipeline
MLREL-11: Use an appropriate deployment and testing strategy

Performance efficiency pillar

MLPER-11: Evaluate cloud versus edge options for machine learning deployment
MLPER-12: Choose an optimal deployment option in the cloud

Cost optimization pillar

MLCOST-24: Use appropriate deployment option
MLCOST-25: Explore cost effective hardware options
MLCOST-26: Right-size the model hosting instance fleet

Sustainability pillar

MLSUS-11: Align SLAs with sustainability goals
MLSUS-12: Use efficient silicon
MLSUS-13: Optimize models for inference
MLSUS-14: Deploy multiple models behind a single endpoint

Model monitoring phase

Operational excellence pillar

MLOE-15: Enable model observability and tracking
MLOE-16: Synchronize architecture and configuration, and check for skew across environments

Security pillar

MLSEC-12: Restrict access to intended legitimate consumers
MLSEC-13: Monitor human interactions with data for anomalous activity

Reliability pillar

MLREL-12: Allow automatic scaling of the model endpoint
MLREL-13: Ensure a recoverable endpoint with a managed version control strategy

Performance efficiency pillar

MLPER-13: Evaluate model explainability
MLPER-14: Evaluate data drift
MLPER-15: Monitor, detect, and handle model performance degradation
MLPER-16: Establish an automated re-training framework
MLPER-17: Review for updated data/features for retraining
MLPER-18: Include human-in-the-loop monitoring

Cost optimization pillar

MLCOST-27: Monitor usage and cost by ML activity
MLCOST-28: Monitor Return on Investment for ML models
MLCOST-29: Monitor endpoint usage and right-size the instance fleet

Sustainability pillar

MLSUS-15: Measure material efficiency
MLSUS-16: Retrain only when necessary

Warning Javascript is disabled or is unavailable in your browser.

To use the Amazon Web Services Documentation, Javascript must be enabled. Please refer to your browser's Help pages for instructions.

Document Conventions

Best practices arranged by pillar

Notices

Did this page help you? - Yes

Thanks for letting us know we're doing a good job!

If you've got a moment, please tell us what we did right so we can do more of it.

Did this page help you? - No

Thanks for letting us know this page needs work. We're sorry we let you down.

If you've got a moment, please tell us how we can make the documentation better.

Avez-vous besoin d’aide ?

Essayez AWS re:Post
Contactez un expert AWS IQ

Confidentialité Conditions d'utilisation du site Préférences de cookies

Sur cette page

Business goal identification phase
ML problem framing phase
Data processing phase
Model development phase
Model deployment phase
Model monitoring phase

Sélectionner vos préférences de cookies

Best practices by ML lifecycle phase

Business goal identification phase

Operational excellence pillar

Security pillar

Reliability pillar

Performance efficiency pillar

Cost optimization pillar

Sustainability pillar

ML problem framing phase

Operational excellence pillar

Security pillar

Reliability pillar

Performance efficiency pillar

Cost optimization pillar

Sustainability pillar

Data processing phase

Operational excellence pillar

Security pillar

Reliability pillar

Performance efficiency pillar

Cost optimization pillar

Sustainability pillar

Model development phase

Operational excellence pillar

Security pillar

Reliability pillar

Performance efficiency pillar

Cost optimization pillar

Sustainability pillar

Model deployment phase

Operational excellence pillar

Security pillar

Reliability pillar

Performance efficiency pillar

Cost optimization pillar

Sustainability pillar

Model monitoring phase

Operational excellence pillar

Security pillar

Reliability pillar

Performance efficiency pillar

Cost optimization pillar

Sustainability pillar

Cette page vous a-t-elle été utile ?

Rubrique suivante :

Rubrique précédente :

Avez-vous besoin d’aide ?