Adding default "local-storage" as a tarball_backend

[santosjaimemeta]

Trying to make out tarball files to use Local-Storage (default plugin) and it seems it doesn't work.

at /usr/local/lib/node_modules/verdaccio/node_modules/verdaccio-storage-proxy/lib/plugin.js:39:48 at Array.forEach (<anonymous>) at new VerdaccioStoragePlugin (/usr/local/lib/node_modules/verdaccio/node_modules/verdaccio-storage-proxy/lib/plugin.js:33:39) at /usr/local/lib/node_modules/verdaccio/build/lib/plugin-loader.js:125:32 at Array.map (<anonymous>) at loadPlugin (/usr/local/lib/node_modules/verdaccio/build/lib/plugin-loader.js:62:37) at LocalStorage._loadStorePlugin (/usr/local/lib/node_modules/verdaccio/build/lib/local-storage.js:873:47) at LocalStorage._loadStorage (/usr/local/lib/node_modules/verdaccio/build/lib/local-storage.js:857:26) at new LocalStorage (/usr/local/lib/node_modules/verdaccio/build/lib/local-storage.js:49:31)

[favoyang]

To be honest I only tested the plugin with aws-s3-storage and redis-storage. Could you share more information to help me reproduce it?

1. The version of verdaccio
2. The verdaccio config's store section

store:
  storage-proxy:
    database_backend: ...
    search_backend: ...
    packument_backend: ...
    tarball_backend: ...
  backends:
    ...

[favoyang]

However, from the partial error message, it is a plugin loader issue.

I think to load an internal verdaccio plugin, you have to use the full name: local-storage => @verdaccio/local-storage

Example 1: use the storage-proxy as a wrap for the local-storage plugin:

store:
  storage-proxy:
    database_backend: '@verdaccio/local-storage'
    search_backend: '@verdaccio/local-storage'
    packument_backend: '@verdaccio/local-storage'
    tarball_backend: '@verdaccio/local-storage'
    backends:
      '@verdaccio/local-storage': {}

Example 2: use the storage-proxy to work with the local-storage plugin and redis-storage plugin

store:
  storage-proxy:
    database_backend: redis-storage
    search_backend: redis-storage
    packument_backend: redis-storage
    tarball_backend: '@verdaccio/local-storage'
    backends:
      '@verdaccio/local-storage': {}
      redis-storage:
        host: 127.0.0.1
        port: 6379

I didn't test the code, but it seems the way to go.

[santosjaimemeta]

Thanks for the quick response :)

I did try this before and i get this error whenever we try to update (or upload a new package).

[2023-05-11 15:36:16] error--- unexpected error: ENOENT: no such file or directory, open '/verdaccio/storage/com.epi.verdaccio-meta-storage/com.epi.verdaccio-meta-storage-0.0.2.tgz.tmp-8933205384693812'
Error: ENOENT: no such file or directory, open '/verdaccio/storage/com.verdaccio-storage/com.epi.verdaccio-storage-0.0.2.tgz.tmp-8933205384693812'

Now this issue is more on the local-storage plugin side of things but not sure specifically where is the issue :(.

For versions:

Verdaccio: 5.2.2
Storage Proxy: 0.0.9
Redis Storage: 0.2.4

Increasing the version of any of these causes issue with our infrastructure as other dependencies are attached.

[favoyang]

The verdaccio local-storage store directory structure:

storage/
  com.org.packagename/
    package.json # packument stores meta data and versions
    com.org.packagename-1.0.0.tgz
    com.org.packagename-1.0.1.tgz
    ... 

The package publishing follows below steps,

1. creates the package folder
2. create a temp file to stream the tarball
3. rename the temp file to -.tgz
4. update the package packument file

When mixing the Redis and local-storage like this, the writeTarball call (step 2) will fail because the folder doesn't exist yet.

store:
  storage-proxy:
    database_backend: redis-storage
    search_backend: redis-storage
    packument_backend: redis-storage
    tarball_backend: '@verdaccio/local-storage'
    backends:
      '@verdaccio/local-storage': {}
      redis-storage:
        host: 127.0.0.1
        port: 6379

https://github.com/verdaccio/verdaccio/blob/d52dbadae84edb1c34905bf09c3e8f8bf708eed0/packages/plugins/local-storage/src/local-fs.ts#L237

Potential solutions:

• Patch a custom local-storage version to create the folder before writing the temp file.
• Switch to aws-s3-storage, and install minio to provide local s3 storage. https://min.io/docs/minio/linux/index.html

[mbtools]

fyi, verdaccio/verdaccio#4594 should fix this problem with local storage.