Commits on Mar 8, 2024

1. Support test results output postprocessing by a child process. …

   Add the following two optional flags to `libtest` (rustc's built-in unit-test
   framework), in order to support postprocessing of the test results using a
   separate executable:
   
   *   `--output_postprocess_executable [PATH]`
   *   `--output_postprocess_args [ARGUMENT]` (can be repeated.)
   
   If you don't pass `--output_postprocess_executable [PATH]`, the behavior stays
   the same as before this commit. That is, the test results are sent to stdout.
   
   If you pass `--output_postprocess_executable [PATH]`, `libtest` will
   
   1.  Spawn a child process from the executable binary (aka *postprocessor*) at
       the given path.
   2.  Pass the arguments from the `--output_postprocess_args [ARGUMENT]` flags (if
       any) to the child process. If `--output_postprocess_args` was used multiple
       times, all listed arguments will be passed in the original order.
   3.  Propagate the environment variables to the child process.
   
   The *postprocessor* executable is expected to wait for the end of input (EOF)
   and then terminate.
   
   Usage example #1: Filter lines of the test results
   
   ```shell
   $ LD_LIBRARY_PATH=$(pwd) ./test-05daf44cb501aee6 --output_postprocess_executable=/usr/bin/grep --output_postprocess_args="test result"
   test result: ok. 59 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.31s
   ```
   
   Usage example rust-lang#2: Save test results into a file
   
   ```shell
   $ LD_LIBRARY_PATH=$(pwd) ./test-05daf44cb501aee6 --output_postprocess_executable=/usr/bin/sh --output_postprocess_args=-c --output_postprocess_args="cat > /tmp/output.txt"
   ```
   
   Usage example rust-lang#3: Save test results into a file while keeping the command line
   output
   
   ```shell
   $ LD_LIBRARY_PATH=$(pwd) ./test-05daf44cb501aee6 --output_postprocess_executable=/usr/bin/tee --output_postprocess_args="/tmp/output.txt"
   
   running 60 tests
   ...
   ```
   
   Usage example rust-lang#4: Prepend every line of test results with the value of an
   environment variable (to demonstrate environment variable propagation)
   
   ```shell
   $ LOG_PREFIX=">>>" LD_LIBRARY_PATH=$(pwd) ./test-05daf44cb501aee6 --output_postprocess_executable=/usr/bin/sh --output_postprocess_args=-c --output_postprocess_args="sed s/^/\$LOG_PREFIX/"
   >>>
   >>>running 60 tests
   ...
   ```
   
   Usage example rust-lang#5: Change format of JSON output (using
   https://jqlang.github.io/jq/)
   
   ```shell
   $ LD_LIBRARY_PATH=$(pwd) ./test-05daf44cb501aee6 -Zunstable-options --format=json --output_postprocess_executable=/usr/bin/jq
   ```
   
   Usage example rust-lang#6: Print test execution time in machine-readable format
   
   ```shell
   $ LD_LIBRARY_PATH=$(pwd) ./test-05daf44cb501aee6 -Zunstable-options --format=json --output_postprocess_executable=/usr/bin/jq --output_postprocess_args=".exec_time | numbers"
   0.234317996
   ```
   
   Rationale for adding this functionality:
   
   *   Bazel (build system) doesn't provide a way to process output from a binary
       (in this case, Rust test binary's output) other using a wrapper binary.
       However, using a wrapper binary potentially breaks debugging, because Bazel
       would suggest to debug the wrapper binary rather than the Rust test itself.
       *   See bazelbuild/rules_rust#1303.
       *   Cargo is not used in Rust Bazel rules.
       *   Although we could wait for rust-lang#96290
           and then modify Rust Bazel rules to pass `--logfile` on the command line
           to provisionally unblock
           bazelbuild/rules_rust#1303, that solution
           still wouldn't allow advanced test results postprocessing such as
           changing JSON/XML schema and injecting extra JUnit properties.
   *   Due to limitations of Rust libtest formatters, Rust developers often use a
       separate tool to postprocess the test results output (see comments to
       rust-lang#85563).
       *   Examples of existing postprocessing tools:
           *   https://crates.io/crates/cargo2junit
           *   https://crates.io/crates/gitlab-report
           *   https://crates.io/crates/cargo-suity
       *   For these use cases, it might be helpful to use the new flags
           `--output_postprocess_executable`, `--output_postprocess_args` instead
           of piping the test results explicitly, e.g. to more reliably separate
           test results from other output.
   
   Rationale for implementation details:
   
   *   Use platform-dependent scripts (.sh, .cmd) because it doesn't seem to be
       possible to enable unstable feature `bindeps`
       (https://rust-lang.github.io/rfcs/3028-cargo-binary-dependencies.html) in
       "x.py" by default.
       *   Here's a preexisting test that also uses per-platform specialization:
           `library/std/src/process/tests.rs`.

   

   aspotashev committed Mar 8, 2024
   Configuration menu

   • View commit details
   • Copy full SHA for b857882
   • Browse repository at this point

   Copy the full SHA

   b857882 View commit details

   Browse the repository at this point in the history