Work item: come up with a set of security criteria for packages

[torgo]

As discussed on today's call - what are some key measurable security indicators we would like developers to see when they are selecting packages (e.g. NPM packages) to possibly bring into their web applications...

[torgo]

See https://github.com/w3c-cg/swag/blob/main/meetings/2024-08-12-minutes.md

[torgo]

Some suggested text:

When evaluating software packages for web development projects, OpenSSF Scorecard can provide a useful framework to evaluate the security health of the open source project behind the package. There are also additional factors not covered by Scorecard that should be taken into consideration.

The following elements of the OpenSSF Scorecard are particularly important for web developers to focus on when evaluating software packages:

1. Maintained

This score indicates whether the project is actively maintained. The theory is that an actively maintained project is more likely to address security vulnerabilities in a timely manner. Web developers should prefer packages with high maintenance scores to ensure prompt responses to security patches, bug fixes, and other updates.

• What to check: Regular commits, up-to-date dependency management, and responsiveness to security issues.

2. Dependency Management

This measures how well the project manages its dependencies. Vulnerabilities in dependencies are a common attack vector, and good dependency management practices can reduce risk. Web developers should prefer packages that regularly update their own dependencies and avoid outdated or insecure libraries.

• What to check: Use of automated tools like Dependabot, regular updates to package-lock.json, and avoidance of deprecated dependencies.

3. Security Policy

A clear and published security policy provides guidelines for how security vulnerabilities are handled. Projects with a defined security policy demonstrate a commitment to addressing security concerns and have clear processes for reporting and mitigating issues.

• What to check: Existence of a SECURITY.md file, which explains how users can report vulnerabilities and how they are handled.

4. Vulnerability Disclosure

This assesses whether there is a documented process for reporting vulnerabilities. It indicates the package’s transparency and ability to address security issues responsibly.

• What to check: The package should have a well-defined vulnerability disclosure process (often visible in the repository's documentation or as part of the security policy).

5. Branch Protection

Branch protection rules help prevent malicious code from being merged into critical branches (such as main). Strong protection policies, like requiring code reviews and passing automated tests, are essential for preventing security issues from being introduced into the codebase.

• What to check: Whether critical branches are protected by mandatory reviews, CI checks, or signed commits.

6. Signed Releases

This indicates whether the package maintains cryptographic signatures for releases. Signed releases ensure that the package hasn’t been tampered with during distribution and can be trusted as authentic.

• What to check: Whether release artifacts are signed to verify authenticity.

Additional Security Criteria to Consider When Selecting Packages:

Beyond the elements provided by OpenSSF Scorecard, there are additional security criteria web developers should pay attention to:

1. Package Popularity and Community Support

• Reason: Popular packages with large communities are more likely to be maintained and have security issues reported quickly.
• What to check: Number of stars, forks, contributors, and downloads over time. Also, consider how responsive the community is to issues.

2. Open Issues and Security Advisories

• Reason: A high number of unresolved issues, especially security vulnerabilities, can be a red flag. Security advisories published on platforms like GitHub's advisory database or npmjs.com are one indicator.
• What to check: Check for security advisories or reported vulnerabilities in the package's history.

3. Minimal Package Footprint

• Reason: Smaller packages with fewer dependencies have less attack surface and are easier to audit. Complex packages with many transitive dependencies can introduce vulnerabilities.
• What to check: Inspect the package and its dependencies for bloat and unnecessary code. Prefer minimalistic, modular packages that focus on one functionality.

4. License Compatibility

• Reason: Security isn't just about code vulnerabilities. Legal risks, such as license incompatibilities, can also affect project stability. Some licenses impose restrictions that could impact how you distribute your software.
• What to check: Ensure that the licenses of the NPM packages you use are compatible with your project’s license and don’t impose legal obligations that could be problematic.

5. Usage of Security Best Practices (e.g., Avoiding eval)

• Reason: Some packages might include dangerous or outdated practices like eval, which can lead to security vulnerabilities.
• What to check: Review the source code for use of unsafe JavaScript practices (e.g., avoid packages that use eval, innerHTML without sanitization, or untrusted data in DOM manipulation).

6. Reputation of Maintainers and Contributors

• Reason: The trustworthiness of the maintainers can be a critical security factor. Experienced and reputable maintainers are likely to follow best practices for security and package management.
• What to check: Evaluate reputation of the maintainers, including their contributions to other open-source projects, and ensure they have a strong track record in secure development.

7. Enforced Code of Conduct

• Reason: A project with a code of conduct and a documented enforcement process is more likely to be open to a wider and more diverse group of contributors.
• What to check: Existence of a CODE_OF_CONDUCT.md file, which explains expected conduct in the repo, how users can report violations, and how they are handled.

[simoneonofri]

For point 4. this can also be to "document it", e.g., in SECURITY.md file

[jkowalleck]

some remarks and thoughts on #1 (comment)

2. Dependency Management

[...] Web developers should prefer packages that regularly update their own dependencies and avoid outdated or insecure libraries.

I disagree. see the following use case.

As an author of a widely used library, I aim for compatibility and low friction, meaning i might support very old/deprecated versions of direct dependencies, like some-other-lib ^3.2.1 || ^4.0.2 || ^5 || ^6.
It is to the downstream user to pick the version, my library supports very old and very outdated versions, too.
This does not make my library a worse pick than something that supports some-other-lib ^6.1.3 only.

Also consider: new does not mean more secure.
it is usually a "change" to some existing package, that transforms a well-tested and secure library to a security-nightmare. Changing a direct dependency might introduce new security issues. Better use a library/version and its dependencies that were properly audited (by you, the downstream user), than some package/version that was released just some days ago.

• What to check: Use of automated tools like Dependabot, regular updates to package-lock.json, and avoidance of deprecated dependencies.

Disagree.
"Deprecated" does not mean obsolete nor abandoned nor end-of-life, it is just not future-safe with the aspect that the point from when it might be obsolete in unknown. The risk of using such a dependency is undefined, which should not have any meaning.

real world example: i remember bootstrap v4 or v3 being "deprecated" for some time, but then they removed the deprecation-flag after a year or so ...
A software that is flagged "deprecated" might be continued for several years, while a software that is not deprecated might be discontinued since some years ago ...

3. Security Policy

A clear and published security policy provides guidelines for how security vulnerabilities are handled. Projects with a defined security policy demonstrate a commitment to addressing security concerns and have clear processes for reporting and mitigating issues.

• What to check: Existence of a SECURITY.md file, which explains how users can report vulnerabilities and how they are handled.

what about github repo's security policy? like this one: https://github.com/CycloneDX/cyclonedx-javascript-library/security/policy

5. Branch Protection

Branch protection rules help prevent malicious code from being merged into critical branches (such as main). Strong protection policies, like requiring code reviews and passing automated tests, are essential for preventing security issues from being introduced into the codebase.

usually, such rules are not publicly disclosed, or are they?

6. Signed Releases

This indicates whether the package maintains cryptographic signatures for releases. Signed releases ensure that the package hasn’t been tampered with during distribution and can be trusted as authentic.

• What to check: Whether release artifacts are signed to verify authenticity.

such artifacts and signatures are not immutable, or are they?
if they are mutable, then there might not be much value to it, since they might be altered, tempered, etc

Additional Security Criteria to Consider When Selecting Packages:

Beyond the elements provided by OpenSSF Scorecard, there are additional security criteria web developers should pay attention to:

2. Open Issues and Security Advisories

• Reason: A high number of unresolved issues, especially security vulnerabilities, can be a red flag. Security advisories published on platforms like GitHub's advisory database or npmjs.com are one indicator.

A high number of issues might mean that this is a popular package, or that the project team behind the thing has a clear vision and is planning ahead, or many other things.
It does not help identigy the "quality" of the package, the documentation, or whatsoever.
I do not see how number of issues bring any value to the table.

3. Minimal Package Footprint

• Reason: Smaller packages with fewer dependencies have less attack surface and are easier to audit. Complex packages with many transitive dependencies can introduce vulnerabilities.

Mostly, transitive packages are under the control of the directly depended packages, not under control of the package itself.
How does this help anything when deciding for a package?
Furthermore, transitive dependencies are only resolved after resolving the direct ones, and these might not pinned to an exact version but to a whole range of versions - like some-direct-dep ^3.12.3 || ^4.0.2 || ^5 || ~6.0.0 -- the transitive deps might differ a lot depending on the actually used version of that some-direct-dep. I don't see how transitives might even be properly calculated for a library that does not come with a shrinkwrap-like pinned/narrowed dependency-policy.

4. License Compatibility

• Reason: Security isn't just about code vulnerabilities. Legal risks, such as license incompatibilities, can also affect project stability. Some licenses impose restrictions that could impact how you distribute your software.

Only relevant for software that is distributed in a bundled/packaged/assembled manner.
Hosting/distributing a software library unmodified is, to my understanding, irrelevant for any license incompatibility with other libraries or software that uses such. But I am not a lawyer.

7. Enforced Code of Conduct

• Reason: A project with a code of conduct and a documented enforcement process is more likely to be open to a wider and more diverse group of contributors.

• What to check: Existence of a CODE_OF_CONDUCT.md file, which explains expected conduct in the repo, how users can report violations, and how they are handled.

FYI: CODE_OF_CONDUCT.md can be positioned in many places.

• in the root of a project's root repository - like https://github.com/w3c-cg/swag/blob/main/CODE_OF_CONDUCT.md
• in a special repo of the organization that owns a project's root repository - like https://github.com/CycloneDX/.github/blob/master/CODE_OF_CONDUCT.md
• etc

[torgo]

related work: https://repos.openssf.org/principles-for-package-repository-security.html