Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update operation fails for AI Studio Project #564

Open
marvinbuss opened this issue Jul 30, 2024 · 12 comments
Open

Update operation fails for AI Studio Project #564

marvinbuss opened this issue Jul 30, 2024 · 12 comments

Comments

@marvinbuss
Copy link

marvinbuss commented Jul 30, 2024

With the latest v1.14.0 I am able to deploy Azure AI Studio resources (hub and project) just fine.

Update operations on the project fail though. Every update causes the following validation error:

{
│   "error": {
│     "code": "ValidationError",
│     "severity": null,
│     "message": "Managed network cannot be disabled once enabled.",
│     "messageFormat": null,
│     "messageParameters": null,
│     "referenceCode": null,
│     "detailsUri": null,
│     "target": null,
│     "details": [],
│     "innerError": null,
│     "debugInfo": null,
│     "additionalInfo": null
│   },
...
│ }

My property bag does not include any managed vnet references:

...
  body = {
    kind = "Project"
    properties = {
      description   = "AI Studio Project - ${var.ai_studio_project_name}"
      friendlyName  = title(replace(var.ai_studio_project_name, "-", " "))
      hubResourceId = var.ai_studio_hub_id
    }
  }
...

The same terraform resource definition is used for initial creation and for updates. Not sure what causes the issue.
Is anyone experiencing the same issues?

Two assumptions:

  1. Either the azapi provider does not send the kind property as part of the update operation.
  2. There is an issue on the product side.
@ms-henglu
Copy link
Member

Hi @marvinbuss ,

Thank you for taking time to report this issue.

Would you please also share the complete config to help investigate? Thanks.

@ms-henglu
Copy link
Member

Thanks @marvinbuss ,

I believe this is an upstream API issue, that the managed network is enabled by default, and in the following requests, the managed network must be specified.

Please check whether below config could solve the issue:

body = {
    kind = "Project"
    properties = {
      description   = "AI Studio Project - ${var.ai_studio_project_name}"
      friendlyName  = title(replace(var.ai_studio_project_name, "-", " "))
      hubResourceId = var.ai_studio_hub_id
      managedNetwork = {
        status = {
          status = "Active"
        }
      }
    }
  }

@marvinbuss
Copy link
Author

Let me give this a try and report back. Thanks for your prompt response.

@marvinbuss
Copy link
Author

I tried the following configs:

Option 1: (like you suggested)
Resulted in the same error as mentioned above.

resource "azapi_resource" "ai_studio_project" {
  type      = "Microsoft.MachineLearningServices/workspaces@2024-04-01"
  name      = var.ai_studio_project_name
  location  = var.location
  parent_id = "/subscriptions/${data.azurerm_client_config.current.subscription_id}/resourceGroups/${var.resource_group_name}"
  tags      = var.tags
  identity {
    type         = "SystemAssigned"
    identity_ids = []
  }

  body = {
    kind = "Project"
    properties = {
      description   = "AI Studio Project - ${var.ai_studio_project_name}"
      friendlyName  = title(replace(var.ai_studio_project_name, "-", " "))
      hubResourceId = var.ai_studio_hub_id
      managedNetwork = {
        status = {
          status     = "Active"
        }
      }
    }
  }

  response_export_values    = []
  schema_validation_enabled = false # Can be reverted once this is closed: https://github.com/Azure/terraform-provider-azapi/issues/524
  locks                     = []
  ignore_casing             = false
  ignore_missing_property   = true
}

Option 2:
Resulted in the same error as mentioned above.

resource "azapi_resource" "ai_studio_project" {
  type      = "Microsoft.MachineLearningServices/workspaces@2024-04-01"
  name      = var.ai_studio_project_name
  location  = var.location
  parent_id = "/subscriptions/${data.azurerm_client_config.current.subscription_id}/resourceGroups/${var.resource_group_name}"
  tags      = var.tags
  identity {
    type         = "SystemAssigned"
    identity_ids = []
  }

  body = {
    kind = "Project"
    properties = {
      description   = "AI Studio Project - ${var.ai_studio_project_name}"
      friendlyName  = title(replace(var.ai_studio_project_name, "-", " "))
      hubResourceId = var.ai_studio_hub_id
      managedNetwork = {
        status = {
          status     = "Active"
          sparkReady = true
        }
      }
    }
  }

  response_export_values    = []
  schema_validation_enabled = false # Can be reverted once this is closed: https://github.com/Azure/terraform-provider-azapi/issues/524
  locks                     = []
  ignore_casing             = false
  ignore_missing_property   = true
}

Option 3:
Resulted in a new error ("message": "Project workspace shouldn't define it's own managed network properties.",).

resource "azapi_resource" "ai_studio_project" {
  type      = "Microsoft.MachineLearningServices/workspaces@2024-04-01"
  name      = var.ai_studio_project_name
  location  = var.location
  parent_id = "/subscriptions/${data.azurerm_client_config.current.subscription_id}/resourceGroups/${var.resource_group_name}"
  tags      = var.tags
  identity {
    type         = "SystemAssigned"
    identity_ids = []
  }

  body = {
    kind = "Project"
    properties = {
      description   = "AI Studio Project - ${var.ai_studio_project_name}"
      friendlyName  = title(replace(var.ai_studio_project_name, "-", " "))
      hubResourceId = var.ai_studio_hub_id
      managedNetwork = {
        isolationMode = "AllowOnlyApprovedOutbound"
        status = {
          status     = "Active"
          sparkReady = true
        }
      }
    }
  }

  response_export_values    = []
  schema_validation_enabled = false # Can be reverted once this is closed: https://github.com/Azure/terraform-provider-azapi/issues/524
  locks                     = []
  ignore_casing             = false
  ignore_missing_property   = true
}

Looks like we are blocked.

@marvinbuss
Copy link
Author

@ms-henglu and other ideas what could be wrong?

@marvinbuss marvinbuss changed the title Update operation fails for Update operation fails for AI Studio Project Aug 1, 2024
@marvinbuss
Copy link
Author

Short update from my side:

  • I enabled Trace logs to review the API calls that are made to the resource provider.
  • I extracted some of the information and validated the via Postman.
  • I am able to replicate the issue via Postman right now.
  • Right now, I am assuming that this is a provider issue.

Will follow-up once I have any updates.

@aldodfm
Copy link

aldodfm commented Aug 1, 2024

just sharing, it's happening the same to me, but using Azure Bicep...

@marvinbuss
Copy link
Author

Thanks for sharing @aldodfm. Let me follow-up internally.

@ms-henglu
Copy link
Member

Hi @marvinbuss - About hubResourceId = var.ai_studio_hub_id, what should I set for the hubResourceId?

@marvinbuss
Copy link
Author

For hubResourceId you must specify the resource ID of another AML workspace of kind Hub.
Sample can be found here:

@marvinbuss
Copy link
Author

marvinbuss commented Aug 2, 2024

Today, another error message started showing up even though nothing has changed on the TF module:

│ {
│   "error": {
│     "code": "ValidationError",
│     "severity": null,
│     "message": "Project workspace shouldn't have it's own Key Vault",
│     "messageFormat": null,
│     "messageParameters": null,
│     "referenceCode": null,
│     "detailsUri": null,
│     "target": null,
│     "details": [],
│     "innerError": null,
│     "debugInfo": null,
│     "additionalInfo": null
│   },
│   "correlation": {
│     "operation": "",
│     "request": ""
│   },
│   "environment": "westeurope",
│   "location": "westeurope",
│   "time": "2024-08-02T07:42:06.4382655+00:00",
│   "componentName": "account-rp",
│   "statusCode": 400
│ }

Is anyone seeing the same error when updating a project?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants