Local Process with Kubernetes (Preview) is now available in Visual Studio 2019 and Visual Studio Code

[greenie-msft]

Local Process with Kubernetes (Preview)

Local Process with Kubernetes puts your development computer in the context of your Kubernetes cluster. Write, test, and debug your microservice code on your development workstation, while consuming dependencies from your cluster. Redirect traffic from the cluster to your development computer to effectively remote debug. Local Process with Kubernetes gives you the confidence of full end-to-end testing during development with the speed and flexibility of running code directly on your development computer.

Local Process with Kubernetes (Preview) is now available in both Visual Studio 2019 and Visual Studio Code.

• How Local Process with Kubernetes Works

• How to use Local Process with Kubernetes in Visual Studio 2019

• How to use Local Process with Kubernetes in Visual Studio Code

[AnandThanumalayan]

This is provided the Kubernetes cluster that the development computer tries to connect is accessible from the public network?
We usually deploy the AKS cluster nodes in private subnets for security reasons and manage deployments to it via a bastion host, even for dev environments.

[greenie-msft]

Hi @anandthanu

In theory, if you are able to execute kubectl commands from the bastion host, Local Process with Kubernetes should work.

Local Process with Kubernetes is now publicly available and I'm interested in hearing how this goes for your scenario. Please keep me posted and/or feel free to reach out to me directly via email with any further questions.

Nicholas.Greenfield@microsoft.com

[kenneth-trivec]

Hello,

This seems to look very promising since we were often having issues with quickly getting development up and running with DevSpaces.

However I have a few questions regarding working with multiple persons and their respective child spaces. How is traffic routed when we have different people redirecting traffic for multiple different services, because with the current DevSpace implementation everyone has their own child space which inherits services from the parent-space. This made it so we aren't blocking each other.

I currently am unable to see parent-space services to redirect traffic from when selecting a child-space. Is there something I am missing or is this something that still needs to be considered?

Thanks in advance.

[Gordonby]

Hi @kenneth-trivec

It's a good question, and you're correct - in this preview Local Process with Kubernetes won't facilitate the creation of a child space and deploying your service to that new space.
Bringing routing to Local Process with Kubernetes is next on our list of priorities.

As a workaround; In your existing Dev Space enabled AKS cluster you can use Local Process with Kubernetes to work on an existing service deployed in a child namespace.

[Eneuman]

Just a heads-up

The link to "How to use Local Process with Kubernetes in Visual Studio 2019" is not working.

I think is should be redirected to
https://docs.microsoft.com/en-us/azure/dev-spaces/how-to/local-process-kubernetes-visual-studio
but right now it redirects to
https://docs.microsoft.com/en-us/azure/dev-spaces/how-to/local-process-kubernetes-visualstudio

[Gordonby]

Thanks for pointing it out. I've fixed our short link to the page now. https://aka.ms/localprocesswithk8s-vs

[ahaeber]

Is it possible to use local process in non-VS environments? In particular JetBrains Rider would be lovely, but I'm happy with running from terminal as well.

[Gordonby]

Hi @ahaeber - we're currently focussing on the two VS environments, but we have it on the backlog to start looking at other IDE's.
I've not used Rider before, so thanks for the suggestion.

[econforti]

Hi @ahaeber - we're currently focussing on the two VS environments, but we have it on the backlog to start looking at other IDE's.
I've not used Rider before, so thanks for the suggestion.

Hi @Gordonby , your answer include that is on the track to work on Linux/ VS Code?

[luckyluc74]

Is it possible to use Local Process with Kubernetes in Visual Studio 2019 in combination with kubernetes from Docker Desktop? It would be nice that it would support generic kubernetes cluster like k3d/k3s/k8s and not having the requirement of using an azure account ( read: vendor lockin)

[Gordonby]

Hi @luckyluc74

We are working through supporting non-AKS clusters in our new “Local Process with Kubernetes” experiences. We don't yet have dates for when this will be supported, but it’s at the top of our backlog. The emphasis of this work initially is to provide a seamless experience on local/on-prem clusters.

[luckyluc74]

Hi @Gordonby

Thanks for your update. Looking forward to this new “Local Process with Kubernetes” with support for non-AKS clusters. As a developer I mostly use my local (non-AKS) kubernetes cluster for developing and testing. And this feature would greatly enhance my productivity.

[kanonkanin]

Great stuff:-). Looking forward to the possibility to facilitate the creation of a child space as mentioned above.

[jmhill]

Yes, the improved launch configuration is very nice - we would also need 1) linux support and 2) the equivalent of the 'clone' flow from the prior version of the VS Code extension/azds cli, to get parity with what our workflow looked like in the azds connect approach.

Although since we're also making use of Review Apps, we could always just start work on a branch by opening a PR to trigger a deploy of the relevant service to the PR (child) namespace, which would give us something to work with right away as a workaround. The 'clone' option was pretty slick though.

Thanks again

[markphillips100]

I have pods that make use of aad-pod-identity and a managed identity to access a keyvault for sensitive app configuration data on startup using the AddAzureKeyVault extension. The non-sensitive config data are maintained in configmaps.

1. Am I right that LPK can only pass environment and file/volume data back to the local environment and so I'd have to supply override env data by using KubernetesLocalProcessConfig.yaml?
2. If, for example, I pass a SqlConnectionString via the override yaml and it still specifies it's using a managed identity rather than credentials, is the connection to SQL Database proxy'ed via the remote container and will it continue to use the same managed identity?

[Gordonby]

Hi @markphillips100

1. Yes, you would use the KubernetesLocalProcessConfig.yaml to override any existing environment variables that were pulled back to the local environment from the pod.
2. Managed Identity isn't a supported configuration at the moment. We've investigated it, and whilst it is possible to get to the IMDS, it does introduce a security concern. It's still in the backlog, and we will continue to investigate the best way to offer support for Pod Identity.

[markphillips100]

Thanks @Gordonby. I'll try the yaml override and see how I go. Firstly though, can you explain how I might use the file in my situation (let me know if separate issues should be created in regards to LPK please).

TL;DR
I'd like to tell LPK (via perhaps the KubernetesLocalProcessConfig.yaml) the relative path of a local file to use for environment variables. Can this be done using the config yaml file or in some other way? Also, where is the config yaml supposed to be located? At solution level, in the project, or is it configurable via launch.json perhaps?

My Inner-Loop...
I develop Pulumi code in my repos to facilitate the infrastructure requirements for the application code being developed in the same repo. Infrastructure and app code commits are kept together basically.

I supply a Pulumi config property to the Pulumi code to change the behaviour of the infrastructure deployment slightly. Instead of deploying HELM charts like in a CI/CD stage, the code creates Pulumi "chart" Outputs (variables accessible via Pulumi CLI tool) containing the data that would normally be passed to the HELM chart. The values.yaml essentially, albeit in json format so I have a ps1 script that can convert back to yaml.

I use devspace.sh to build and deploy my inner-loop images to my dev cluster in a developer-specific k8 namespace. The "deployment" configuration in devspace.yaml directly references the previous yaml values file with no developer changes required. So essentially I can execute devspace run stack-up to build my developer-specific infrastructure, and then execute devspace dev and I'll get developer-specific images built and deployed with their stdout piped to the terminal.

I imagine I'd still use this inner-loop process but debugging (via attaching in VSCode) is troublesome, especially as SourceLink support to private repos is lacking. I'd like to use VS2019 to debug if possible, and the F5 approach LPK offers is very compelling, considering it should also allow for sourcelink to work.

So my goal is to build upon the scripts that I already have and just produce LPK supporting environment yaml files (with development-specific modification to bypass the managed identity issue) as part of the devspace run stack-up step. If a specific project's LPK configuration can be set to point at this relative-path environment variables yaml then all should be good.

[Gordonby]

Hey @markphillips100,

It might be good to create this issue properly over to our new LPK repo to keep the discussion tight.
All this talk of devspace.sh (whatever that is 😕) , and LPK in the Azure Dev Spaces repo is going to get confusing.

I've dropped an example of the file in our sample app : https://github.com/microsoft/mindaro/blob/master/samples/BikeSharingApp/ReservationEngine/KubernetesLocalProcessConfig.yaml
It's to be located in the root of the project, hopefully the example above illustrates this.
We currently only look in the yaml file (from your local machine).

[markphillips100]

@Gordonby yep no problem. Sorry to confuse. I'll just generate your specific yaml files as required.

I did log a separate issue (#341) related to using the extension. Unfortunately it prevents me using it as it breaks my other kubectl workflows. I'll log it in the other repo too.

[kanonkanin]

Hi.

I am getting an error now. It worked OK before summer.
I am using exact same settings then and now.
Excerpts from log (lpk-library-2020-08-12-12-41-45-9672):

Logging handled exception: Microsoft.DevSpaces.Common.Exceptions.UserVisibleException: {"Request":null,"Response":null,"RequestId":null,"ClientRequestId":null,"CorrelationRequestId":null,"Format":"Failed to deploy remote restore job for pod deployment 'test/xxx' with error: Operation returned an invalid status code 'UnprocessableEntity'","Args":[],"StackTrace":"
at Microsoft.DevSpaces.Library.Connect.KubernetesRemoteEnvironmentManager.StartRemoteAgentAsync(RemoteContainerConnectionDetails remoteContainerConnectionDetails,
etc

[daniv-msft]

Hi @kanonkanin,

Thank you for reporting this issue. We are aware of it, and we'll release a fix in the coming weeks.

This issue is due to one of the jobs we're trying to create that has a name too long. This job's name is a concatenation of your isolateAs header (if you're using isolation), the name of the service you debug and random characters. This would look like:
kanonkan-my-looooong-service-name-restore-1234

If this name is longer than 63 characters, then we get this UnprocessableEntity error.

Are you working on VS Code and using the isolation feature? If yes, please consider updating your isolateAs field in tasks.json to have a shorter length while we're working on the fix.

On a side note, if you have other issues related to Local Process with Kubernetes, could you please open them in https://github.com/Microsoft/mindaro ? This is where we're tracking such issues now.

[rmcfarlane82]

I have tried to use local process but cant get it to list namespaces, i can list them in kubectl but in visual studio it gives me a permissions error.
not sure where to ask about this so if anyone can point me in the right direction.
i get the error below

{
  "timestamp": "2020-09-07 13-44-39.302",
  "message": "Refresh the list of namespaces displayed for the selected cluster",
  "subscriptionId": "[REDACTED]",
  "exception": {
    "ClassName": "Microsoft.VisualStudio.Kubernetes.Debugging.Core.Errors.ContainerToolsException",
    "Message": "An error occurred during the activation of a particular registration. See the inner exception for details. Registration: Activator = KubernetesClient (DelegateActivator), Services = [Microsoft.DevSpaces.Common.Kubernetes.IKubernetesClient], Lifetime = Autofac.Core.Lifetime.CurrentScopeLifetime, Sharing = None, Ownership = ExternallyOwned ---> An error occurred during the activation of a particular registration. See the inner exception for details. Registration: Activator = KubernetesClient (ReflectionActivator), Services = [Microsoft.DevSpaces.Common.Kubernetes.IKubernetesClient], Lifetime = Autofac.Core.Lifetime.CurrentScopeLifetime, Sharing = None, Ownership = OwnedByLifetimeScope ---> An exception was thrown while invoking the constructor 'Void .ctor(Microsoft.DevSpaces.Common.Kubernetes.IK8sClientFactory, k8s.KubeConfigModels.K8SConfiguration, Microsoft.DevSpaces.Common.IO.IFileSystem, Microsoft.DevSpaces.Common.IO.IPlatform, Microsoft.DevSpaces.Common.IEnvironmentVariables, Microsoft.DevSpaces.Common.Logging.ILog)' on type 'KubernetesClient'. ---> User: [REDACTED] does not have appropriate auth credentials in kubeconfig (See inner exception for details.) (See inner exception for details.) (See inner exception for details.)",
    "Data": null,
    "InnerException": {
      "ClassName": "StreamJsonRpc.RemoteInvocationException",
      "Message": "An error occurred during the activation of a particular registration. See the inner exception for details. Registration: Activator = KubernetesClient (DelegateActivator), Services = [Microsoft.DevSpaces.Common.Kubernetes.IKubernetesClient], Lifetime = Autofac.Core.Lifetime.CurrentScopeLifetime, Sharing = None, Ownership = ExternallyOwned ---> An error occurred during the activation of a particular registration. See the inner exception for details. Registration: Activator = KubernetesClient (ReflectionActivator), Services = [Microsoft.DevSpaces.Common.Kubernetes.IKubernetesClient], Lifetime = Autofac.Core.Lifetime.CurrentScopeLifetime, Sharing = None, Ownership = OwnedByLifetimeScope ---> An exception was thrown while invoking the constructor 'Void .ctor(Microsoft.DevSpaces.Common.Kubernetes.IK8sClientFactory, k8s.KubeConfigModels.K8SConfiguration, Microsoft.DevSpaces.Common.IO.IFileSystem, Microsoft.DevSpaces.Common.IO.IPlatform, Microsoft.DevSpaces.Common.IEnvironmentVariables, Microsoft.DevSpaces.Common.Logging.ILog)' on type 'KubernetesClient'. ---> User: [REDACTED] does not have appropriate auth credentials in kubeconfig (See inner exception for details.) (See inner exception for details.) (See inner exception for details.)",
      "Data": null,
      "InnerException": null,
      "HelpURL": null,
      "StackTraceString": "   at StreamJsonRpc.JsonRpc.<InvokeCoreAsync>d__118`1.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at Microsoft.VisualStudio.Kubernetes.Debugging.Core.LaunchProfiles.ConfigurationDialog.ConfigurationDialogCallbacks.<>c__DisplayClass14_0.<<RefreshNamespacesAsync>b__1>d.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at Microsoft.VisualStudio.Kubernetes.Debugging.Core.Services.ServiceHub.KubernetesManagementServiceBrokerClient.<ExecuteWithServiceAsync>d__4`1.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at Microsoft.VisualStudio.Kubernetes.Debugging.Core.LaunchProfiles.ConfigurationDialog.ConfigurationDialogCallbacks.<>c__DisplayClass14_0.<<RefreshNamespacesAsync>b__0>d.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at Microsoft.VisualStudio.Kubernetes.Debugging.Core.Services.Logging.PerfLoggerExtensions.<ExecuteOperationAsync>d__3`1.MoveNext()",
      "RemoteStackTraceString": null,
      "RemoteStackIndex": 0,
      "ExceptionMethod": "8\nMoveNext\nStreamJsonRpc, Version=2.5.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a\nStreamJsonRpc.JsonRpc+<InvokeCoreAsync>d__118`1\nVoid MoveNext()",
      "HResult": -2146233088,
      "Source": "StreamJsonRpc",
      "WatsonBuckets": null,
      "ErrorCode": -32000
    },
    "HelpURL": null,
    "StackTraceString": null,
    "RemoteStackTraceString": null,
    "RemoteStackIndex": 0,
    "ExceptionMethod": null,
    "HResult": -2146233088,
    "Source": null,
    "WatsonBuckets": null
  },

[daniv-msft]

Hi @rmcfarlane82,
This looks a lot like an issue we fixed recently, and for which the fix should be released in the coming days. I'm syncing with the team member who worked on this and I'll get back to you.

[daniv-msft]

@rmcfarlane82 > Unfortunately, this seems to be a different issue compared to the one I was mentioning above. In order to track this properly, I created this new issue in our Local Process with Kubernetes repository: microsoft/mindaro#26

I also added the results of our investigation so far, as well as a potential workaround (using VS Code). Could you please have a look at this and let us know if anything seems to be missing there?