Model Deployment and Operations
Overview
H2O Driverless AI offers model deployment, management and monitoring capabilities for the IT and DevOps teams. The platform makes it convenient for IT to deploy the winning model across a broad range of production environments. In addition, the data science and data engineering teams can monitor the performance of the model for any drifts in predictions and scores over time, as well as manage any re-training or tuning necessary at run-time.
Operationalizing Models
As enterprises “make their own AI”, new challenges emerge:
- Maintaining reproducibility, traceability, and verifiability of machine learning models
- Recording experiments, tracking insights, reproducibility of results
- Searchability of models (or querying models)
- Collaborating within the DS and ML teams
- Visualizing model performance (drift, degradation, A/B testing)
Operationalizing models crosses functional boundaries
- DevOps and IT teams are usually heavily involved
- Model operations should require minimal changes to existing application workflows
- Maintain data and model lineage in case of rollbacks, regulatory compliance
Model Deployment Options with Driverless AI
Driverless AI offers the following options for deploying machine learning (ML)
models, depending on where the AI application is running:
- 1. As a cloud service
- 2. As a local REST endpoint
- 3. As a Java object file
The model could be directly deployed in a cloud service. Driverless AI offers the ability to export the model directly in AWS Lambda or Sagemaker. Ideal for building and running your AI applications in AWS. Configuration details can be seen here.
The model could be configured to run on a local REST server. Driverless AI offers the ability to deploy the scoring pipeline on a local server. Ideal for AI workloads in on-premises environments. Configuration details can be seen here.
The model could be abstracted into a Java object as a standalone model scoring engine. Driverless AI allows downloading the model as a Plain Old Java Object (POJO) or Model Object Optimized (MOJO) file. POJO and MOJO files are standalone scoring engines. Ideal for running AI applications in low-latency environments such as edge devices or on-premises. Configuration details can be seen here.
Model Operations
As AI applications enter production:
- Data scientists and data engineers need to manage the transition of ML models
- Development → Staging → Production
- Model accuracy can drift over time. Data scientists need alerts if drift exceeds certain thresholds.
- DevOps/IT need a central store for models, model artifacts, related inference, etc.
- ML engineers may need to re-calibrate or re-tune production models
- Seamless collaboration between data science, DevOps and IT teams becomes important
Key MLOps Capabilities in Driverless AI Today
Model Management
(Data Science Teams)
- Projects Workspace
- Model store
- Collaboration
Model Deployment
(IT/DevOps)
- Deploy to different environments – in the cloud, on-premises
Model Monitoring
(Business/Data Science)
- Specify metrics or parameters to monitor
MLOps with H2O Driverless AI
Model Monitoring and Management Done Right
Driverless AI includes new capabilities for model administration, monitoring and management. These capabilities allow:
- DevOps teams to monitor the models for system health checks
- Data science teams to monitor metrics around drift detection, model degradation, A/B testing
- Provides alerts for recalibration and retraining
Telemetry and Alerts
Driverless AI can monitor models for drift, anomalies, model metrics and residuals, and provide alerts on a dashboard for potential re-tuning or re-training of models.
Model comparison
The MLOps offers capabilities to compare multiple models by looking at their confusion matrices. Moreover, our A/B testing functionality helps to run and compare multiple models in production before they are deployed in production.
Full model traceability
Data scientists can track back a prediction on a specific model and investigate the report to understand how it was created. This also includes the ability to frequently retrain and publish updated models to the runtime environment.
Model Governance
It is crucial for IT and DevOps to implement access rules and decision rights across the enterprises as AI goes into production. MLOps provides important capabilities such as role-based access controls for models as well as tracking who built the model and who deployed it.
Deploy Model as a REST Endpoint
