Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallelize Reporter in v2 #16

Open
aljones15 opened this issue May 21, 2022 · 1 comment
Open

Parallelize Reporter in v2 #16

aljones15 opened this issue May 21, 2022 · 1 comment
Assignees

Comments

@aljones15
Copy link
Contributor

aljones15 commented May 21, 2022

Running this reporter in parallel has turned out to be difficult as the reporter relies on mutating mocha's test object with meta data the reporter uses. My proposal is to remove the meta-data and instead create a new javascript file required by the reporter and potentially mocha that contains information on how to format and display tests.

  1. Create a new CLI option displayOptions=$PWD/tests/reportDisplay.js
  2. This file contains a list of Suite titles that are in turn going to be sections in the conformance part of the report
  3. Each entry in the reports Array should look like this:
export default [
  {
    suite: 'Data Integrity - Ed25519Signature2020',
    labels: {column: 'Issuer', row: 'Test'},
    ids: {
      // these can be filled in later
      rows: [],
      // these can be filled in later or supplied from implementations
      columns: []
    },
   template: 'matrix'
  }
]
  1. This then maps to a Suite formatted like this:
describe('Data Integrity - Ed25519Signature2020', function() {
  for(const [name, implementation] of implementations) {
    describe(implementationName, async function() {
      it('MUST have this property', async function() {
         // ...stuff happens here
      })
    })
  }
})
  1. This removes the need for meta-data inside of the tests itself and separate test logic from display logic
  2. If the reporter merges Suites from the display file by title then we can run multiple suites in a single loop or even have a single report run in multiple files aka parallel.

Other Goals:

  1. Switch over passing in packages with a full set of templates instead of individual templates
  2. Make the test stats their own template instead of being a part of body
@aljones15 aljones15 self-assigned this May 21, 2022
@aljones15
Copy link
Contributor Author

aljones15 commented Aug 12, 2022

Mocha has provided a high level overview of how their parallelization works as of Mocha ^10 here: https://mochajs.org/#parallel-mode

They also provide these instructions for porting reporters to parallel: https://mochajs.org/#parallel-tests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant