diff --git a/rules/mobile_inference_rules.adoc b/rules/mobile_inference_rules.adoc index 14d1a10..376964d 100644 --- a/rules/mobile_inference_rules.adoc +++ b/rules/mobile_inference_rules.adoc @@ -80,12 +80,22 @@ but excludes rooted devices. If you are measuring the performance of an experimental framework or system, you must make the system and framework you use available upon demand for -replication by MLCommons. This class of systems will be called RDI (research, -development, internal). RDI systems include reference hardware and software +replication by MLCommons. This class of systems will be called R&D SoC. R&D SoC systems include reference hardware and software modifications such as rooted phones. +R&D SoC will be considered as any physical device including SoC under test, where the device is not commercially available, and the SoC under test is currently not commercially available, but can be made commercially available at some point in the future. + +R&D SoC submitter should submit that SoC under Available category with a commercially available device in the next official submission after the commercial device is available. + +If submitter fails to do this submission, original R&D SoC submission will be marked as invalid. +The R&D SoC can be optionally integrated, depending on the vendor’s willingness, into the backend of the MLPerf app. Any other member can verify this integration giving credibility to this system. + +In order to be published R&D SoC submissions will be audited with the backend code and the respective logs by examining any modifications to the backend code integrated in the MLPerf app. R&D SoC submitters can also optionally send their engineering R&D device to MLCommons to reproduce their results + +R&D SoC submitter should submit that SoC under Available category with a commercially available device in the next official submission after the commercial device is available + For Available Systems, devices used for testing should be publicly available as -commercial devices and device rooting is not allowed. For RDI Systesm, hardware and +commercial devices and device rooting is not allowed. For R&D SoC Systems, hardware and software changes such as rooting should be explicitly mentioned in the device name. === Benchmark implementations must be shared