GCP: Composer Cluster IP Allocation Policy invalid on plan and apply

[eahrend]

Expected Behavior

It should plan and apply without issue. This is the terraform resource Im using
https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/composer_environment#nested_ip_allocation_policy_c2

Actual Behavior

When I set the composer node config to this, there is an error

	composerNodeConfig := &cc.ComposerEnvironmentConfigNodeConfig{
		Subnetwork:     jsii.String(cloudComposerCluster.Subnetwork),
		Network:        jsii.String(cloudComposerCluster.Network),
		ServiceAccount: cloudComposerServiceAccount.Email(),
		Tags:           jsii.Strings(cloudComposerCluster.Tags...),
		IpAllocationPolicy: &cc.ComposerEnvironmentConfigNodeConfigIpAllocationPolicy{
			ClusterSecondaryRangeName:  jsii.String(cloudComposerCluster.PodRangeName),
			ServicesSecondaryRangeName: jsii.String(cloudComposerCluster.ServiceRangeName),
		},
	}

It returns:

                        ╰─ Key 'ipAllocationPolicy': Unable to deserialize value as cdktf.IResolvable | array<@cdktf/provider-google.composerEnvironment.ComposerEnvironmentConfigNodeConfigIpAllocationPolicy> | undefined
                            ├── 🛑 Failing value is an object
                            │      { '$jsii.struct': [Object] }
                            ╰── 🔍 Failure reason(s):
                                ├─ [as array<@cdktf/provider-google.composerEnvironment.ComposerEnvironmentConfigNodeConfigIpAllocationPolicy>] Value is not an array
                                ╰─ [as cdktf.IResolvable] Value does not have the "$jsii.byref" key

Converting it to an array will get past the synth, however a new error pops up

	composerNodeConfig := &cc.ComposerEnvironmentConfigNodeConfig{
		Subnetwork:     jsii.String(cloudComposerCluster.Subnetwork),
		Network:        jsii.String(cloudComposerCluster.Network),
		ServiceAccount: cloudComposerServiceAccount.Email(),
		Tags:           jsii.Strings(cloudComposerCluster.Tags...),
		IpAllocationPolicy: &[]cc.ComposerEnvironmentConfigNodeConfigIpAllocationPolicy{
			{
				ClusterSecondaryRangeName:  jsii.String(cloudComposerCluster.PodRangeName),
				ServicesSecondaryRangeName: jsii.String(cloudComposerCluster.ServiceRangeName),
			},
		},
	}

Will result in a successful synth, however during the plan phase it'll say that I am missing the values for other resources in that block. Which is weird because in the provider having all of those fields will result in a conflict (and manually adding them causes this conflict as well)
https://github.com/hashicorp/terraform-provider-google/blob/97596918928c1f6bccd6d53a10270f8465efe98a/google/services/composer/resource_composer_environment.go#L288-L344

I looked at the issue for this provider with vanilla TF, and they suggest to just use a single ip_allocation_policy, but there isn't a way to do this in cdktf, as just having a single ip allocation policy fails on the synth

Steps to Reproduce

Check actual behavior, contains relevant code

Versions

language: go
cdktf-cli: 0.20.8
node: v20.4.0
cdktf: 0.20.8
terraform: v1.7.0

This is what I can get since doing cdktf debug has another error

Providers

Running this command gives an error:

% cdktf provider list
Pre-built provider information not found
Error: Pre-built provider information not found

Gist

No response

Possible Solutions

N/A

Workarounds

Tried using an escape hatch, but that also gives an error

Anything Else?

No response

References

Related issue and suggested fix:
hashicorp/terraform-provider-google#15456

Created this PR to potentially fix

Help Wanted

• [ x] I'm interested in contributing a fix myself

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[eahrend]

This looks like an issue with the base provider and the lack of a 1:1 from HCL to TF complaint JSON. Created a PR for the provider, which when tested locally it works:
GoogleCloudPlatform/magic-modules#11401

[eahrend]

Confirmed this works with terraform compliant JSON in the 6.1.0 release of GCP's provider. @ansgarm any idea when that will be ported over to CDKTF?

[jsteinich]

You could try building the provider locally (https://developer.hashicorp.com/terraform/cdktf/concepts/providers#add-provider-to-cdktf-json) to verify that the provider is working as expected.

If it is, could make a PR against https://github.com/cdktf/cdktf-repository-manager/blob/main/provider.json to update the version that is being pre-built.

If it isn't, it may also be possible to fix by updating https://github.com/hashicorp/terraform-cdk/blob/main/packages/%40cdktf/provider-generator/lib/get/generator/custom-defaults.ts which exists because of a certain incompatibility in Terraform JSON.

[eahrend]

Oh nice, thank you @jsteinich , I'll give that a shot tomorrow.

Apologies, will need to test this week.