davix version 0.8.0 cannot authenticate using username and password

[mrensan]

Hi,
I faced with an issue with davix-* commands when trying to authenticate by username and password. It was working before but suddenly stopped and constantly returns: (Davix::HttpRequest) Error: HTTP 401 : Authentification Error

So after investigation I found out the davix client version in the machine was upgraded to version 0.8.0 so I tested it with older version successfully.

We connect to a server with valid certificate over https, but to reproduce the issue here; I use a docker image as a webdav server:

1. Create a docker-compose.yml file with content:

version: '3'
services:
  webdav:
    image: bytemark/webdav
    restart: always
    ports:
      - "80:80"
    environment:
      AUTH_TYPE: Digest
      USERNAME: alice
      PASSWORD: secret1234
    volumes:
      - ./dav:/var/lib/dav

2. Run docker compose up -d to start the webdav server
3. Add some data files in ./dav/data path to have some thing to show
4. I have two versions of davix client installed on my machine 0.7.6 and 0.8.0 let's first try the older version that works:

/usr/local/Cellar/davix/0.7.6/bin/davix-ls -l --userlogin alice --userpass secret1234 http://localhost
-rwxrwxrwx 0     10         2021-10-28 19:17:29 data1.txt
-rwxrwxrwx 0     16         2021-10-28 19:44:54 data2.txt

5. Now try with new version that fails:

/usr/local/Cellar/davix/0.8.0/bin/davix-ls -l --userlogin alice --userpass secret1234 http://localhost
(Davix::HttpRequest) Error: HTTP 401 : Authentification Error

6. For more details we can run it of course with debug version:

/usr/local/Cellar/davix/0.8.0/bin/davix-ls -l --debug --userlogin alice --userpass secret1234 http://localhost
DAVIX(socket): ssl: Initialized OpenSSL thread-safety callbacks for 1 locks.

DAVIX(core): HTTP/SSL Session caching ENABLED
DAVIX(core): Redirection Session caching ENABLED
DAVIX(core): libdavix path /usr/local/Cellar/davix/0.8.0/lib/libdavix.0.8.0.dylib, version: 0.8.0
DAVIX(posix):  -> opendirpp
DAVIX(http): Create HttpRequest for http://localhost
DAVIX(http):  -> negotiateRequest
DAVIX(http): NEON start internal request
DAVIX(http): no cached ne_session, create a new one
HTTP session to http://localhost:80 begins.
DAVIX(http): configure session...
DAVIX(http): define connection timeout to 30
DAVIX(http): enable login/password authentication
DAVIX(http): enable client cert authentication by callback
DAVIX(ssl): ah_create, for WWW-Authenticate
DAVIX(socket): Running pre_send hooks
> PROPFIND / HTTP/1.1
> User-Agent: libdavix/0.8.0 neon/0.0.29
> Keep-Alive:
> Connection: Keep-Alive
> TE: trailers
> Host: localhost
> Depth: 1
> Content-Length: 303
>

DAVIX(socket): Sending request-line and headers:
DAVIX(socket): Doing DNS lookup on localhost...
DAVIX(socket): Sending request body:
DAVIX(socket): Request body sent successfully
DAVIX(socket): Request sent; retry is 0.
< HTTP/1.1 401 Unauthorized
< Date: Thu, 28 Oct 2021 18:01:19 GMT
< Server: Apache/2.4.37 (Unix)
< WWW-Authenticate: Digest realm="WebDAV", nonce="kv9pf23PBQA=87dcf2978519b77f6a33ecf1023df29d2c0c5cd6", algorithm=MD5, qop="auth"
< Content-Length: 381
< Keep-Alive: timeout=5, max=100
< Connection: Keep-Alive
< Content-Type: text/html; charset=iso-8859-1
<
DAVIX(socket): End of headers.
DAVIX(socket): Running post_headers hooks
DAVIX(ssl): Disable Session recycling
DAVIX(http):  <- negotiateRequest
DAVIX(core): Destroy HttpRequest
DAVIX(socket): sess: Closing connection.
DAVIX(socket): sess: Connection closed.
DAVIX(ssl): Disable Session recycling
DAVIX(socket): Running destroy hooks.
DAVIX(socket): Request ends.
DAVIX(socket): sess: Destroying session.
DAVIX(posix):  <- opendirpp
(Davix::HttpRequest) Error: HTTP 401 : Authentification Error

7. This scenario happens also for other commands like davix-get, etc.

Thanks a lot,
Shahram.

[mpatrascoiu]

Hello Shahram,

Great problem report!
I will look into why it fails. From the logs, it seems the client login callback is never invoked.

Cheers,
Mihai

[kelson42]

Seems I'm impacted by the same bug

[felix-egli]

I'm affected by this bug as well.

[nachter]

I came across this error too. With davix version 0.8.2. When I compare the debug-output of davix-version 0.7.6: In 0.7.6 there is this line:
creating session keys... httpsnextcloud.gbv.de
This line is completly missing in 0.8.2

Just a hint, perhaps this helps.

[nachter]

So, the message about creating session keys is just missing in the output for 0.8.x, it's just not present in the code of 0.8.x. So this doesnt seem to be the problem.

But for 0.8.x in the debug-log, I get the message
"DAVIX(ssl): Disable Session recycling"

Perhaps there is a case missing (so for username/passphrase authentication), when the session recycling should not be disabled?

[nachter]

Hello!
I now compiled and tested the login_callback_0.8.0 branch. It seems to work fine with this branch.
Thanks for having a look on this!

[crstmkt]

Hello,
I'm affected by this bug aswell.

[TGion]

Hi there,
I am not 100% sure this is the same issue / connected, but as far I could tell, there weren't any fixes about this issue yet. Sorry in advance if this is another issue / misconfiguration on my part.
I am trying to access my Nextcloud instance with davix 0.8.4. on my FreeBSD system:

[USERNAME@localhost.gion.io ~]$ davix-ls --debug --userlogin USERNAME --userpass PASSWORD davs://cloud.gion.io/remote.php/dav/files/USERNAME/
  DAVIX(socket): ssl: Initialized OpenSSL thread-safety callbacks for 1 locks.
  
  DAVIX(core): HTTP/SSL Session caching ENABLED
  DAVIX(core): Redirection Session caching ENABLED
  DAVIX(core): libdavix path /usr/local/lib/libdavix.so.0, version: 0.8.4
  DAVIX(posix):  -> opendirpp
  DAVIX(http): Create HttpRequest for davs://cloud.gion.io/remote.php/dav/files/USERNAME/
  DAVIX(http):  -> negotiateRequest
  DAVIX(http): NEON start internal request
  DAVIX(http): no cached ne_session, create a new one 
  DAVIX(http): HTTP session to https://cloud.gion.io:443 begins.
  DAVIX(ssl): ssl: SNI enabled by default.
  DAVIX(http): configure session...
  DAVIX(http): define connection timeout to 30
  DAVIX(http): enable login/password authentication
  DAVIX(http): enable client cert authentication by callback 
  DAVIX(ssl): ah_create, for WWW-Authenticate
  DAVIX(socket): Running pre_send hooks
  > PROPFIND /remote.php/dav/files/USERNAME/ HTTP/1.1
  > User-Agent: libdavix/0.8.4 neon/0.0.29
  > Keep-Alive: 
  > Connection: Keep-Alive
  > TE: trailers
  > Host: cloud.gion.io
  > Depth: 1
  > Content-Length: 303
  > 
  
  DAVIX(socket): Sending request-line and headers:
  DAVIX(socket): Doing DNS lookup on cloud.gion.io...
  DAVIX(ssl): Doing SSL negotiation.
  DAVIX(ssl): Chain depth: 2
  DAVIX(ssl): ssl: Match common name '*.gion.io' against ''
  DAVIX(ssl): ssl: Match common name 'gion.io' against ''
  DAVIX(ssl): Identity match for '': bad
  DAVIX(ssl): Cert #0:
  DAVIX(ssl): Certificate:
      Data:
          Version: 3 (0x2)
          Serial Number:
              04:b5:18:58:80:3d:d6:ec:2a:12:4d:db:84:25:11:78:0c:39
          Signature Algorithm: sha256WithRSAEncryption
          Issuer: C=US, O=Let's Encrypt, CN=R3
          Validity
              Not Before: May 22 10:48:47 2023 GMT
              Not After : Aug 20 10:48:46 2023 GMT
          Subject: CN=gion.io
          Subject Public Key Info:
              Public Key Algorithm: rsaEncryption
                  RSA Public-Key: (2048 bit)
                  Modulus:
                      00:f2:0a:8d:54:41:ce:9e:e9:83:a3:cd:6f:ae:08:
                      24:f0:b8:6f:b2:65:0e:53:e3:c3:3d:0f:1c:f2:fb:
                      ee:f6:cf:2c:eb:88:98:e9:14:10:83:34:00:9f:a9:
                      08:ee:88:e8:21:a2:71:cf:61:72:c5:d6:52:f5:6c:
                      d4:c1:27:d5:fe:f1:8d:d6:c3:b3:2f:1d:0f:f7:fc:
                      b8:a3:c0:53:6a:38:cd:0b:ed:c3:47:41:ed:08:50:
                      37:8a:f9:73:d2:76:a2:30:8f:4a:2e:b2:a5:b5:8e:
                      11:3b:4c:e0:1f:df:ad:67:23:1f:9a:79:f1:03:b1:
                      10:d4:1f:9b:7c:aa:1e:ba:97:06:78:8c:04:d4:4f:
                      56:3c:52:d7:f1:2e:0e:f4:61:c2:24:19:92:f0:0a:
                      de:0b:3a:5e:4a:6c:3c:d9:95:b5:7f:31:19:d4:cc:
                      82:bc:7b:0b:fe:76:03:34:42:dd:88:50:17:95:ba:
                      76:f5:66:95:6c:a8:e7:74:d5:8b:c0:72:ff:b3:94:
                      a8:eb:9a:ff:a3:2b:f9:58:1f:a9:62:0b:a1:35:3f:
                      a0:04:24:d0:42:7b:07:2a:9c:15:8b:ae:d5:0a:ea:
                      6c:d6:a4:68:86:b4:e6:ae:a2:0a:e3:f4:8d:be:63:
                      ac:5e:9c:2c:65:3b:16:b1:54:8a:3a:74:30:b4:50:
                      19:b1
                  Exponent: 65537 (0x10001)
          X509v3 extensions:
              X509v3 Key Usage: critical
                  Digital Signature, Key Encipherment
              X509v3 Extended Key Usage: 
                  TLS Web Server Authentication, TLS Web Client Authentication
              X509v3 Basic Constraints: critical
                  CA:FALSE
              X509v3 Subject Key Identifier: 
                  9F:AF:79:EA:A7:11:8C:28:EB:B2:5C:76:AF:97:E1:D3:2E:F4:8C:74
              X509v3 Authority Key Identifier: 
                  keyid:14:2E:B3:17:B7:58:56:CB:AE:50:09:40:E6:1F:AF:9D:8B:14:C2:C6
  
              Authority Information Access: 
                  OCSP - URI:http://r3.o.lencr.org
                  CA Issuers - URI:http://r3.i.lencr.org/
  
              X509v3 Subject Alternative Name: 
                  DNS:*.gion.io, DNS:gion.io
              X509v3 Certificate Policies: 
                  Policy: 2.23.140.1.2.1
                  Policy: 1.3.6.1.4.1.44947.1.1.1
                    CPS: http://cps.letsencrypt.org
  
              CT Precertificate SCTs: 
                  Signed Certificate Timestamp:
                      Version   : v1 (0x0)
                      Log ID    : 7A:32:8C:54:D8:B7:2D:B6:20:EA:38:E0:52:1E:E9:84:
                                  16:70:32:13:85:4D:3B:D2:2B:C1:3A:57:A3:52:EB:52
                      Timestamp : May 22 11:48:47.518 2023 GMT
                      Extensions: none
                      Signature : ecdsa-with-SHA256
                                  30:45:02:20:66:36:CF:29:95:A1:F7:54:E6:90:42:A1:
                                  F2:09:FD:50:A5:95:83:46:FA:64:18:23:61:69:4C:70:
                                  07:10:AE:9E:02:21:00:E2:3A:12:85:52:A5:6F:63:DE:
                                  EF:9E:EE:3B:46:DF:50:92:62:D2:9F:BC:DB:B1:5D:40:
                                  9C:DA:94:79:48:62:2D
                  Signed Certificate Timestamp:
                      Version   : v1 (0x0)
                      Log ID    : AD:F7:BE:FA:7C:FF:10:C8:8B:9D:3D:9C:1E:3E:18:6A:
                                  B4:67:29:5D:CF:B1:0C:24:CA:85:86:34:EB:DC:82:8A
                      Timestamp : May 22 11:48:47.541 2023 GMT
                      Extensions: none
                      Signature : ecdsa-with-SHA256
                                  30:45:02:20:27:1C:80:5D:5B:75:1B:87:7F:92:6A:57:
                                  DC:49:0C:54:CB:9B:50:20:66:84:56:41:52:A6:91:5D:
                                  B2:
  DAVIX(ssl): ssl: Match common name 'R3' against ''
  DAVIX(ssl): Identity match for '': bad
  DAVIX(ssl): Cert #1:
  DAVIX(ssl): Certificate:
      Data:
          Version: 3 (0x2)
          Serial Number:
              91:2b:08:4a:cf:0c:18:a7:53:f6:d6:2e:25:a7:5f:5a
          Signature Algorithm: sha256WithRSAEncryption
          Issuer: C=US, O=Internet Security Research Group, CN=ISRG Root X1
          Validity
              Not Before: Sep  4 00:00:00 2020 GMT
              Not After : Sep 15 16:00:00 2025 GMT
          Subject: C=US, O=Let's Encrypt, CN=R3
          Subject Public Key Info:
              Public Key Algorithm: rsaEncryption
                  RSA Public-Key: (2048 bit)
                  Modulus:
                      00:bb:02:15:28:cc:f6:a0:94:d3:0f:12:ec:8d:55:
                      92:c3:f8:82:f1:99:a6:7a:42:88:a7:5d:26:aa:b5:
                      2b:b9:c5:4c:b1:af:8e:6b:f9:75:c8:a3:d7:0f:47:
                      94:14:55:35:57:8c:9e:a8:a2:39:19:f5:82:3c:42:
                      a9:4e:6e:f5:3b:c3:2e:db:8d:c0:b0:5c:f3:59:38:
                      e7:ed:cf:69:f0:5a:0b:1b:be:c0:94:24:25:87:fa:
                      37:71:b3:13:e7:1c:ac:e1:9b:ef:db:e4:3b:45:52:
                      45:96:a9:c1:53:ce:34:c8:52:ee:b5:ae:ed:8f:de:
                      60:70:e2:a5:54:ab:b6:6d:0e:97:a5:40:34:6b:2b:
                      d3:bc:66:eb:66:34:7c:fa:6b:8b:8f:57:29:99:f8:
                      30:17:5d:ba:72:6f:fb:81:c5:ad:d2:86:58:3d:17:
                      c7:e7:09:bb:f1:2b:f7:86:dc:c1:da:71:5d:d4:46:
                      e3:cc:ad:25:c1:88:bc:60:67:75:66:b3:f1:18:f7:
                      a2:5c:e6:53:ff:3a:88:b6:47:a5:ff:13:18:ea:98:
                      09:77:3f:9d:53:f9:cf:01:e5:f5:a6:70:17:14:af:
                      63:a4:ff:99:b3:93:9d:dc:53:a7:06:fe:48:85:1d:
                      a1:69:ae:25:75:bb:13:cc:52:03:f5:ed:51:a1:8b:
                      db:15
                  Exponent: 65537 (0x10001)
          X509v3 extensions:
              X509v3 Key Usage: critical
                  Digital Signature, Certificate Sign, CRL Sign
              X509v3 Extended Key Usage: 
                  TLS Web Client Authentication, TLS Web Server Authentication
              X509v3 Basic Constraints: critical
                  CA:TRUE, pathlen:0
              X509v3 Subject Key Identifier: 
                  14:2E:B3:17:B7:58:56:CB:AE:50:09:40:E6:1F:AF:9D:8B:14:C2:C6
              X509v3 Authority Key Identifier: 
                  keyid:79:B4:59:E6:7B:B6:E5:E4:01:73:80:08:88:C8:1A:58:F6:E9:9B:6E
  
              Authority Information Access: 
                  CA Issuers - URI:http://x1.i.lencr.org/
  
              X509v3 CRL Distribution Points: 
  
                  Full Name:
                    URI:http://x1.c.lencr.org/
  
              X509v3 Certificate Policies: 
                  Policy: 2.23.140.1.2.1
                  Policy: 1.3.6.1.4.1.44947.1.1.1
  
      Signature Algorithm: sha256WithRSAEncryption
           85:ca:4e:47:3e:a3:f7:85:44:85:bc:d5:67:78:b2:98:63:ad:
           75:4d:1e:96:3d:33:65:72:54:2d:81:a0:ea:c3:ed:f8:20:bf:
           5f:cc:b7:70:00:b7:6e:3b:f6:5e:94:de:e4:20:9f:a6:ef:8b:
           b2:03:e7:a2:b5:16:3c:91:ce:b4:ed:39:02:e7:7c:25:8a:47:
           e6:65:6e:3f:46:f4:d9:f0:ce:94:2b:ee:54:ce:12:bc:8c:27:
           4b:b8:c1:98:2f:a2:af:cd:71:91:4a:08:b7:c8:b8:23:7b:04:
           2d:08:f9:08:57:3e:83:d9:04:33:0a:47:21:78:09:82:27:c3:
           2a:c8:9b:b9:ce:5c:f2:64:c8:c0:be:79:c0:4f:8e:6d:44:0c:
           5e:92:bb:2e:f7:8b:10:e1:e8:1d:44:29:db:59:20:ed:63:b9:
           21:f8:12:26:94:93:57:a0:1d:65:04:c1:0a:22:ae:10:0d:43:
           97:a1:18:1f:7e:e0:e0:86:37:b5:5a:b1:bd:30:bf:87:6e:2b:
           2a:ff:21:4e:1b:05:c3:f5:18:97:f0:5e:ac:c3:a5:b8:6a:f0:
           2e:bc:3b:33:b9:ee:4b:de:cc:fc:e4:af:84:0b:86:3f:c0:55:
           43:36:f6:68:e1:36:17:6a:8e:99:d1:ff:a5:40:a7:34:b7:c0:
           d0:63:39:35:39:75:6e:f2:ba:76:c8:93:02:e9:a9:4b:6c:17:
           ce:0c:02:d9:bd:81:fb:9f:b7:68:d4:06:65:b3:82:3d:77:53:
           f8:8e:79:03:ad:0a:31:07:75:2a:43:d8:55:97:72:c4:29:0e:
           f7:c4:5d:4e:c8:ae:46:84:30:d7:f2:85:5f:18:a1:79:bb:e7:
           5e:70:8b:07:e1:86:93:c3:b9:8f:dc:61:71:25:2a:af:df:ed:
           25:50:52:68:8b:92:dc:e5:d6:b5:e3:da:7d:d0:87:6c:84:21:
           31:ae:82:f5:fb:b9:ab:c8:89:17:3d:e1:4c:e5:38:0e:f6:bd:
           2b:bd:96:
  DAVIX(ssl): ssl: Match common name '*.gion.io' against 'cloud.gion.io'
  DAVIX(ssl): Identity match for 'cloud.gion.io': good
  DAVIX(socket): Sending request body:
  DAVIX(socket): Request body sent successfully
  DAVIX(socket): Request sent; retry is 0.
  < HTTP/1.1 401 Unauthorized
  < Server: nginx
  < Date: Sun, 04 Jun 2023 09:50:03 GMT
  < Content-Type: application/xml; charset=utf-8
  < Transfer-Encoding: chunked
  < Connection: keep-alive
  < Keep-Alive: timeout=5
  < Set-Cookie: [removed]
  < Expires: Thu, 19 Nov 1981 08:52:00 GMT
  < Cache-Control: no-store, no-cache, must-revalidate
  < Pragma: no-cache
  < Content-Security-Policy: default-src 'none';
  < WWW-Authenticate: Basic realm="cloud | gion . io", charset="UTF-8"
  < Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
  < Referrer-Policy: no-referrer
  < X-Content-Type-Options: nosniff
  < X-Download-Options: noopen
  < X-Permitted-Cross-Domain-Policies: none
  < X-XSS-Protection: 1; mode=block
  < Permissions-Policy: camera=(), microphone=(), geolocation=()
  < X-Frame-Options: SAMEORIGIN
  < X-Robots-Tag: noindex,nofollow
  < 
  DAVIX(socket): End of headers.
  DAVIX(socket): Running post_headers hooks
  DAVIX(ssl): Disable Session recycling
  DAVIX(http):  <- negotiateRequest
  DAVIX(core): Destroy HttpRequest
  DAVIX(socket): sess: Closing connection.
  DAVIX(socket): sess: Connection closed.
  DAVIX(ssl): Disable Session recycling
  DAVIX(socket): Running destroy hooks.
  DAVIX(socket): Request ends.
  DAVIX(socket): sess: Destroying session.
  DAVIX(posix):  <- opendirpp
  (Davix::HttpRequest) Error: HTTP 401 : Authentication Error 

What else has been tested:

• Successfully accessed the DAV server with cadaver
• Failed with https:// address too
• Reinstallation of davix

What has not been tested

• Earlier versions of davix with my Nextcloud instance
• Access via http:// or dav://
• Another server with davix

Help, a new version, or hints about any misconfiguration on my parts would be really appreciated. Thanks a lot in advance.
If I could be of any help with more testing, just let me know!

[mpatrascoiu]

Hello all,

This problem appeared with the davix v0.8.0 release, which introduced libcurl alongside libneon for the backend. A large refactoring was done to facilitate having two different backends, including introducing a new class hierarchy.

Unfortunately, during that refactoring, the hooks mechanism used with libneon was lost. Previously, hooks would be registered for different scenarios. One such hook would be triggered when the server requests Basic authentication via the response headers.

In branch login_callback_0.8.0, I'm trying to get the hooks mechanism to work again. However, this introduces a memory corruption in the session object management. The corruption does not show via the command line tool (e.g.: davix-get <url>), but if you were to reuse the same Davix object, it will show up sooner or later. For this reason, the branch is not ready to be merged.

Ultimately, we we plan to decommission the libneon backend. Due to this, I gave up on getting the hooks mechanism to work, especially since it is only tied to libneon and not libcurl.

For what concerns the username + password login, the davix + libcurl backend simply does not do it at all. If there's high demand for this functionality, I can add it to the libcurl backend. This means you'd have it only when activating libcurl:

$ DAVIX_USE_LIBCURL=1 davix-get <url>

[TGion]

Hey @mpatrascoiu ,

first of all thanks a lot for your effort and your clarification!

I am a bit confused about your last paragraph (most certainly because of my lack of knowledge):
How do you usually connect to a WebDAV Server, because I honestly don't know any other way then by basic authentification (username + password). Iam trying to access my Nextcloud instance via WebDAV and couldnt find another way then by username + password.
Long story short, I would think the demand for it is quite high.

If you have the time I would appreciate also some clarification on how other forms of authentification work with davix. I couldnt really find a lot in the documentation which would be useful for my appliance.

Thanks a lot again and please tell me if I could be of any help!

[spacehorst]

Hi @mpatrascoiu,

thanks for investigating on that issue.

Could you please clarify your statement of #81 (comment)
Do I understand correct, that it would not be possible to use a WebDAV Server with Basic Authentification by davix >= 0.8.0 any more?

This would be a big drawback for us. We have been using davix for more than 10 years in our CI/CD pipeline.
So we would like to encourage you to keep on with basic authentication.

[spacehorst]

As a workaround one could add the missing Authorization Header by hand:

echo -n "$username:$password" | base64
abc....xyz=
davix-get -H "Authorization: Basic abc....xyz=" http://your.web/dav

[TGion]

Thanks for your workaround @spacehorst

[mpatrascoiu]

Hello @spacehorst , that's correct. With davix >= 0.8.0, there is no more Basic Authentication.
And if we bring it back (I believe we should, it's more a question of effort and planning), it will only be available for the CURL backend.

Just to mention: your workaround should work well, cheers for that