Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Rules updated for rdi #23

Closed
wants to merge 2 commits into from
Closed

Rules updated for rdi #23

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
394020b
Update mobile_inference_rules.adoc
peladodigital Mar 8, 2023
9cfe06d
Update mobile_inference_rules.adoc
peladodigital Mar 8, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions rules/mobile_inference_rules.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,12 @@ replication by MLCommons. This class of systems will be called RDI (research,
development, internal). RDI systems include reference hardware and software
modifications such as rooted phones.

RDI will be considered as any physical device that is not commercially available within 1 year. RDI submitter should submit that SoC under available category in the submission that comes immediately after its release. If submitter fails to do this submission, original RDI submission will be marked as invalid
SoC that later becomes available requires new submission under available or else the RDI results (made after v3.0 and beyond) will be marked as invalid

In order to be published, the RDI can be optionally integrated, depending on the vendor’s willingness, into the backend of the MLPerf app. Any other member can verify this integration giving credibility to this system.
RDI submitters must support such integration, the integration needs to be completed before presenting public results.

For Available Systems, devices used for testing should be publicly available as
commercial devices and device rooting is not allowed. For RDI Systesm, hardware and
software changes such as rooting should be explicitly mentioned in the device name.
Expand Down Expand Up @@ -129,6 +135,8 @@ All Closed/available submissions should make the device available for results re

Submitters must provide the device either as a gift/loan or reimburse MLCommons for the purchase of the test system.

RDI submissions will be audited with the backend code and the respective logs.

== Scenarios

In order to enable representative testing of a wide variety of inference
Expand Down