-
Notifications
You must be signed in to change notification settings - Fork 154
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Lakehouse monitoring integration #156
Lakehouse monitoring integration #156
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some initial comments, but thanks for doing this @s-udhaya !
...t_root_dir}}/{{template `project_name_alphanumeric_underscore` .}}/monitoring/README.md.tmpl
Outdated
Show resolved
Hide resolved
...t_root_dir}}/{{template `project_name_alphanumeric_underscore` .}}/monitoring/README.md.tmpl
Outdated
Show resolved
Hide resolved
...{{template `project_name_alphanumeric_underscore` .}}/resources/monitoring-resource.yml.tmpl
Outdated
Show resolved
Hide resolved
...{{template `project_name_alphanumeric_underscore` .}}/resources/monitoring-resource.yml.tmpl
Outdated
Show resolved
Hide resolved
...e `project_name_alphanumeric_underscore` .}}/monitoring/metric_violation_check_query.py.tmpl
Outdated
Show resolved
Hide resolved
...e `project_name_alphanumeric_underscore` .}}/resources/monitoring-workflow-resource.yml.tmpl
Outdated
Show resolved
Hide resolved
@@ -143,6 +143,41 @@ Model validation contains three components: | |||
To set up and enable model validation, update [validation.py](../validation/validation.py) to return desired custom metrics and validation thresholds, then | |||
resolve the `TODOs` in the ModelValidation task of [model-workflow-resource.yml](./model-workflow-resource.yml). | |||
|
|||
|
|||
### Setting up model monitoring |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
### Setting up model monitoring | |
### Setting up Lakehouse monitoring |
Change elsewhere too please to match our branding
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I understand we are still discussing on this internally. I will update this when we finalize the unified naming convention
- Minor readme changes - Use dafault assets_dir path for monitoring
ff13cdd
to
37bf41b
Compare
- Accept inference table name from CLI - Merge monitoring related resources into a single file - Parametrize the metric and validation threshold
37bf41b
to
a7c882b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good Overall. Thank you for working on this. Left some comments. Can we also do an E2E test on this?
...e `project_name_alphanumeric_underscore` .}}/monitoring/metric_violation_check_query.py.tmpl
Outdated
Show resolved
Hide resolved
...e `project_name_alphanumeric_underscore` .}}/monitoring/metric_violation_check_query.py.tmpl
Outdated
Show resolved
Hide resolved
...name_alphanumeric_underscore` .}}/monitoring/notebooks/MonitoredMetricViolationCheck.py.tmpl
Show resolved
Hide resolved
...ut_root_dir}}/{{template `project_name_alphanumeric_underscore` .}}/resources/README.md.tmpl
Show resolved
Hide resolved
...ut_root_dir}}/{{template `project_name_alphanumeric_underscore` .}}/resources/README.md.tmpl
Show resolved
Hide resolved
...{{template `project_name_alphanumeric_underscore` .}}/resources/monitoring-resource.yml.tmpl
Outdated
Show resolved
Hide resolved
Where did you find documentation for creating monitors through asset bundles as demonstrated in this PR? I see no documentation in the databricks asset bundle configurations reference, nor the Lakehouse Monitoring page Is this due to the feature being in private preview? |
@RegaCaska the DAB docs might still be generating, you can ask in the CLI GitHub, but if you'd like you can use the Terraform or SDK docs to configure it in DABs since it uses the same structure. |
This PR integrates lakehouse monitoring, with automatic retraining component into mlops-stack.