Audit the diff in PRs?

[spenserblack]

Currently, PRs are getting marked as failing because the audit fails. This is happening a lot with the dependabot PRs. However, most of the time it is not the PRs that introduce the audit failure, so I believe it is inaccurate to mark many of these PRs as failing.

I think that, ideally, only the diff should be audited in PRs. That is, if the PR changes or adds a dependency, it should be that dependency only that is audited, not all dependencies.

Currently, I doubt this is reasonable to do in this repo without upstream changes to actions-rs/audit-check. It might be possible by generating temp files using the git diff, though.

If this makes sense, I can open an issue with audit-check.

[HallerPatrick]

As far as I know does cargo audit use the lock file to inspect for vulnerabilities. We could temporary generate a toml file with the changed dependencies, build it and audit that. But that also sounds like a hacky idea.

Filing an issue is probably the way to go. Maybe we are not the only one with failing PRs everywhere.

[spenserblack]

Filing an issue is probably the way to go

actions-rs/audit-check#216

[o2sh]

As a quick and easy fix, we could add a .cargo/audit.toml with:

[advisories]
ignore = [
    "RUSTSEC-2020-0071", # `time` localtime_r segfault
    "RUSTSEC-2020-0159", # `chrono` localtime_r segfault
]

[spenserblack]

I hadn't thought about that! That could work 👍

[spenserblack]

Hey, just want to note that I've stumbled on https://github.com/actions/dependency-review-action.
This might be an improvement over our currently used actions (it seems like it can be configured to fail PRs that introduce vulnerabilities, which is cool).

[o2sh]

Cool stuff, is this equivalent to a cargo audit + npm audit? In which case we could replace the audit.yml GH action and use this instead.

[spenserblack]

I think this uses GitHub's security advisory API, so it would be close to cargo audit + npm audit if not exact. I suppose it would behave similarly to dependabot's security advisories, which I believe also use GitHub's security advisory database.