Issues when interacting with azure devops repos

[iceaway]

Hello,

We are using pygit2 (1.15.0) against repositories hosted on azure devops, and have started seeing issues lately as Microsoft are phasing out ssh-rsa keys: https://devblogs.microsoft.com/devops/ssh-rsa-deprecation

The error message I get when I for example try to clone a repository is this:

_pygit2.GitError: remote: Command git-upload-pack: You’re using ssh-rsa that is about to be deprecated and your request has been blocked intentionally. Any SSH session using SSH-RSA is subject to brown out (failure during random time periods). Please use rsa-sha2-256 or rsa-sha2-512 instead. For more details see https://aka.ms/ado-ssh-rsa-deprecation.
remote: ERROR_SSH_UNSUPPORTED_CIPHER (7)

I have created a new ssh-key according to the specs that Microsoft advice (ssh-keygen -t rsa-sha2-512) and use this key when trying to clone. If I do a manual git clone using the same key, it works without any errors. I have tried to dig into the pygit2 code in order to figure out what is going on but I am stuck.

Any ideas what could be causing this?

Here is the tiny python program that I use for testing:

import pygit2
...
keypair = pygit2.Keypair(username, public_key, private_key, passphrase)
callbacks = pygit2.RemoteCallbacks(credentials=keypair)
pygit2.clone_repository(repo_url, "test", callbacks=callbacks)

[iceaway]

I have done some more digging and tried to follow the chain pygit2 -> libgit2 -> libssh2 backwards using the example programs to try and detect where in the chain it's failing, and the failure happens in libgit2. I do not think there is actually anything wrong though, but if anyone else runs into the same issue here is what seems to be happening and how to fix it:

libgit2/libssh2 checks the known hosts file for which algorithms the remote host supports/wants to use. When connecting to the azure devops host for the first time the known_hosts file is updated and ssh-rsa is put in there. This seems to cause this line of code to add ssh-rsa to the list of keys to use when connecting:

/* ssh_libssh2.c */
find_hostkey_preference(...
                       )
{
   ...
   add_hostkey_pref_if_avail(known_hosts, hostname, port, prefs, LIBSSH2_KNOWNHOST_KEY_SSHRSA, "ssh-rsa");
}

Commenting out that line makes it work again, but that is not a very good solution so I updated the known_hosts file instead, replacing ssh-rsa with rsa-sha2-512 for the lines related to azure devops and that works just as good.

I do not know why ssh-rsa is put in the known_hosts file in the first place when it is not actually supported, if that is something that must be solved on the azure devops end or elsewhere. Also noteworthy is that regular git seems to handle this, maybe that is due to git using openssh rather than libssh2?