Description
Describe the bug
SageMaker Model Monitor provides one-time monitoring job schedule[Link].
However, ValueError
are raised when specifying CronExpressionGenerator.now()
.
To reproduce
Using SageMaker-Monitoring-Feature-Attribution-Drift-for-Endpoint.ipynb:
- Change
schedule_expression
toCronExpressionGenerator.now()
- Specify
data_analysis_start_time
anddata_analysis_end_time
in ModelExplainabilityMonitor'screate_monitoring_schedule
type(model_explainability_monitor) # sagemaker.model_monitor.clarify_model_monitoring.ModelExplainabilityMonitor
schedule_expression = sagemaker.model_monitor.CronExpressionGenerator.now()
model_explainability_monitor.create_monitoring_schedule(
analysis_config=model_explainability_analysis_config,
endpoint_input=sagemaker.model_monitor.EndpointInput(
endpoint_name=endpoint_name,
destination="/opt/ml/processing/input/endpoint",
features_attribute=features_jmespath,
inference_attribute=confidence_score_jmespath,
),
output_s3_uri=monitor_output_s3_uri,
schedule_cron_expression=schedule_expression,
data_analysis_start_time="-PT2H",
data_analysis_end_time="-PT1H"
)
Error Message:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In[63], line 3
1 schedule_expression = sagemaker.model_monitor.CronExpressionGenerator.now()
----> 3 model_explainability_monitor.create_monitoring_schedule(
4 analysis_config=model_explainability_analysis_config,
5 endpoint_input=sagemaker.model_monitor.EndpointInput(
6 endpoint_name=endpoint_name,
7 destination="/opt/ml/processing/input/endpoint",
8 features_attribute=features_jmespath,
9 inference_attribute=confidence_score_jmespath,
10 ),
11 output_s3_uri=monitor_output_s3_uri,
12 schedule_cron_expression=schedule_expression,
13 data_analysis_start_time="-PT2H",
14 data_analysis_end_time="-PT1H"
15 )
File ~/anaconda3/envs/python3/lib/python3.10/site-packages/sagemaker/model_monitor/clarify_model_monitoring.py:1104, in ModelExplainabilityMonitor.create_monitoring_schedule(self, endpoint_input, analysis_config, output_s3_uri, constraints, monitor_schedule_name, schedule_cron_expression, enable_cloudwatch_metrics, batch_transform_input, data_analysis_start_time, data_analysis_end_time)
1102 # create schedule
1103 try:
-> 1104 self._create_monitoring_schedule_from_job_definition(
1105 monitor_schedule_name=monitor_schedule_name,
1106 job_definition_name=new_job_definition_name,
1107 schedule_cron_expression=schedule_cron_expression,
1108 )
1109 self.job_definition_name = new_job_definition_name
1110 self.monitoring_schedule_name = monitor_schedule_name
File ~/anaconda3/envs/python3/lib/python3.10/site-packages/sagemaker/model_monitor/model_monitoring.py:1562, in ModelMonitor._create_monitoring_schedule_from_job_definition(self, monitor_schedule_name, job_definition_name, schedule_cron_expression, data_analysis_start_time, data_analysis_end_time)
1559 message = "Creating Monitoring Schedule with name: {}".format(monitor_schedule_name)
1560 logger.info(message)
-> 1562 self._check_monitoring_schedule_cron_validity(
1563 schedule_cron_expression=schedule_cron_expression,
1564 data_analysis_start_time=data_analysis_start_time,
1565 data_analysis_end_time=data_analysis_end_time,
1566 )
1568 monitoring_schedule_config = {
1569 "MonitoringJobDefinitionName": job_definition_name,
1570 "MonitoringType": self.monitoring_type(),
1571 }
1572 if schedule_cron_expression is not None:
File ~/anaconda3/envs/python3/lib/python3.10/site-packages/sagemaker/model_monitor/model_monitoring.py:1536, in ModelMonitor._check_monitoring_schedule_cron_validity(self, schedule_cron_expression, data_analysis_start_time, data_analysis_end_time)
1531 message = (
1532 "Both data_analysis_start_time and data_analysis_end_time are required "
1533 "for one time monitoring schedule "
1534 )
1535 _LOGGER.error(message)
-> 1536 raise ValueError(message)
ValueError: Both data_analysis_start_time and data_analysis_end_time are required for one time monitoring schedule
Expected behavior
Create one-time monitoring job schedule.
ModelBiasMonitor (Model Bias Monitoring Job) can create one-time monitoring job schedule, because data_analysis_start_time
and data_analysis_end_time
are correctly passed in ModelBiasMonitor's _create_monitoring_schedule_from_job_definition
function.
ModelBiasMonitor's _create_monitoring_schedule_from_job_definition
However, ModelExplainabilityMonitor's _create_monitoring_schedule_from_job_definition
function does not pass data_analysis_start_time
and data_analysis_end_time
params, consequently ValueError was raised in _check_monitoring_schedule_cron_validity
called in _create_monitoring_schedule_from_job_definition
.
data_analysis_start_time
and data_analysis_end_time
params are defined in ModelExplainabilityMonitor's create_monitoring_schedule
args.
ModelExplainabilityMonitor's _create_monitoring_schedule_from_job_definition
ModelMonitor's _check_monitoring_schedule_cron_validity
In this commit, codes to support for one time monitoring schedule were merged, however ModelExplainabilityMonitor's one time monitoring schedule looks missed (data_analysis_start_time
and data_analysis_end_time
as ModelExplainabilityMonitor's create_monitoring_schedule args are added).
System information
- SageMaker Python SDK version: v2.0.8 (latest)
- Framework name (eg. PyTorch) or algorithm (eg. KMeans): N/A
- Framework version: N/A
- Python version: 3.10.13 | packaged by conda-forge
- CPU or GPU: CPU
- Custom Docker image (Y/N): N
Additional context
N/A