Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AR review updates #18

Merged
merged 7 commits into from
Nov 11, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions qos.bib
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@ @electronic{SSQOSID
title = {RISC-V Quality-of-Service (QoS) Identifiers},
url = {https://github.com/riscv/riscv-ssqosid}
}
@electronic{STATEEN,
title = {RISC-V State Enable Extension},
url = {https://drive.google.com/file/d/1dhI6OzVbejQbfwyBTuwK9U4VUmW8ii4o/view}
}
@INPROCEEDINGS{SSAMPLE,
author={Thornock, N.C. and Flanagan, J.K.},
booktitle={2000 Winter Simulation Conference Proceedings (Cat. No.00CH37165)},
Expand Down
12 changes: 10 additions & 2 deletions qos_bandwidth.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -49,10 +49,12 @@ bandwidth-controller capabilities.
{reg: [
{bits: 8, name: 'VER'},
{bits: 16, name: 'NBWBLKS'},
{bits: 8, name: 'WPRI'},
{bits: 1, name: 'RPFX'},
{bits: 4, name: 'P'},
{bits: 3, name: 'WPRI'},
{bits: 16, name: 'MRBWB'},
{bits: 16, name: 'WPRI'},
], config:{lanes: 2, hspace:1024}}
], config:{lanes: 4, hspace:1024}}
....

The `VER` field holds the version of the specification implemented by the
Expand Down Expand Up @@ -83,6 +85,12 @@ allowing all available bandwidth blocks to be reserved for allocation, the
bandwidth controller can handle such transient inaccuracies.
====

If `RPFX` is 1, the controller uses `RCID` in the requests along with `P` least
significant bits of the `MCID` carried in the request to compute an effective
`MCID` (<<EMCID>>) to identify the monitoring counter. Legal values of `P` range
from 0 to 12. If `RPFX` is 0, `P` is set to 0, and the effective `MCID` is the
same as the `MCID` in the request.

[[BC_MCTL]]
=== Bandwidth usage monitoring control (`bc_mon_ctl`)

Expand Down
38 changes: 34 additions & 4 deletions qos_capacity.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,30 @@ allocation by ways might define a _capacity block_ to be the cache blocks in one
way of the cache.
====

The capacity allocation affects the decision regarding which _capacity blocks_
to use when a new _capacity unit_ is requested by a workload but usually does
not affect other operations of the controller.

[NOTE]
====
For example, when a request is made to a cache controller, the request involves
scanning the entire cache to determine if the requested data is present. If the
data is located, then the request is fulfilled using this data, even if the
cache block containing the data was initially allocated for a different
workload. The data continues to reside in the same cache block. Consequently,
the cache lookup function remains unaffected by the capacity allocation
constraints set for the workload initiating the request. Conversely, if the data
is not found, a new cache block must be allocated. This allocation is executed
using the _capacity blocks_ assigned to the workload that made the request.
Hence, a workload may only trigger evictions within _capacity blocks_ designated
to it but can access shared data in _capacity blocks_ allocated to other
workloads.
====

[[CC_REG]]
.Capacity-controller QoS register layout
[width=100%]
[%header, cols="^3,10,^3, 18, 5"]
[%header, cols="^3,10,^4, 18, 5"]
|===
|Offset|Name |Size |Description | Optional?
|0 |`cc_capabilities` |8 |<<CC_CAP, Capabilities >> | No
Expand All @@ -37,7 +57,7 @@ way of the cache.
counter value>> | Yes
|24 |`cc_alloc_ctl` |8 |<<CC_ALLOC, Capacity
allocation control >> | Yes
|32 |`cc_block_mask` |`M` * 8 |<<CC_BMASK, Capacity
|32 |`cc_block_mask` |`BMW`/8 |<<CC_BMASK, Capacity
block mask >> | Yes
|N |`cc_cunits` |8 |<<CC_CUNITS, Capacity units
count>> | Yes
Expand All @@ -46,7 +66,9 @@ way of the cache.
The size and offset in <<CC_REG>> are specified in bytes.

The size of the `cc_block_mask` register is determined by the `NCBLKS` field
of the `cc_capabilities` register but is always a multiple of 8 bytes.
of the `cc_capabilities` register but is always a multiple of 8 bytes. The
formula for determination of `BMW` is defined in <<CC_BMASK>>. The offset `N` is
determined as `32 + BMW/8`.

The reset value is 0 for the following register fields.

Expand Down Expand Up @@ -77,7 +99,9 @@ capacity-controller capabilities.
{bits: 16, name: 'NCBLKS'},
{bits: 1, name: 'FRCID'},
{bits: 1, name: 'CUNITS'},
{bits: 38, name: 'WPRI'},
{bits: 1, name: 'RPFX'},
{bits: 4, name: 'P'},
{bits: 33, name: 'WPRI'},
], config:{lanes: 4, hspace:1024}}
....

Expand Down Expand Up @@ -105,6 +129,12 @@ units_ that can be occupied by an `RCID` in _capacity blocks_ allocated to it.
If `FRCID` is 1, the controller supports an operation to flush and deallocate
the _capacity blocks_ occupied by an `RCID`.

If `RPFX` is 1, the controller uses `RCID` in the requests along with `P` least
significant bits of the `MCID` carried in the request to compute an effective
`MCID` (<<EMCID>>) to identify the monitoring counter. Legal values of `P` range
from 0 to 12. If `RPFX` is 0, `P` is set to 0, and the effective `MCID` is the
same as the `MCID` in the request.

[[CC_MCTL]]
=== Capacity usage monitoring control (`cc_mon_ctl`)

Expand Down
2 changes: 1 addition & 1 deletion qos_header.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ endif::[]
:xrefstyle: short

= RISC-V Capacity and Bandwidth QoS Register Interface
RISC-V CMQRI Task Group
RISC-V CBQRI Task Group

// Preamble
[WARNING]
Expand Down
12 changes: 6 additions & 6 deletions qos_hw_guidelines.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -21,15 +21,15 @@ large pool of `MCID` speeds up this analysis.

[NOTE]
====
To achieve maximum flexibility in the allocation of QoS IDs to workloads, it is
To maximize flexibility in the allocation of QoS IDs to workloads, it is
recommended that all resource controllers in the system support an identical
number of `RCID` and `MCID`. Requests typically need to be processed by multiple
controllers, such as caches, fabrics, and memory controllers. In such cases, if
not all controllers accessed by a request support an identical number of IDs,
software will be constrained, using only the IDs supported by all controllers.
number of `RCID` and `MCID`, as well as a uniform mode of operation — either
direct or RCID-prefixed — for determining the effective `MCID`. Uniformity
ensures that software is not constrained by the lowest common denominator of ID
support when requests are processed by multiple controllers, such as caches,
fabrics, and memory controllers.
====


=== Sizing monitoring counters

Typically software samples the monitoring counters periodically to monitor
Expand Down
25 changes: 21 additions & 4 deletions qos_identifiers.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -20,19 +20,36 @@ controllers that control accesses to such shared resources.
Guidelines for sizing the QoS IDs and need for differentiated IDs for monitoring
is discussed in <<QOS_SIZING>>.

[[EMCID]]
=== Associating `RCID` and `MCID` with requests

The `RCID` in the request is used by the resource controllers to determine the
resource allocations (e.g., cache occupancy limits, memory bandwidth limits,
etc.) to enforce. The `MCID` in the request is used by the resource controllers
to identify the ID of a counter to monitor resource usage (e.g., cache
occupancy, memory bandwidth, etc.).
etc.) to enforce.

The `MCID` in the request is used by the resource controllers to identify the ID
of a counter to monitor resource usage (e.g., cache occupancy, memory bandwidth,
etc.). Two modes of operation are supported by CBQRI. In the direct mode, the
`MCID` carried with the request is directly used by the controller to identify
the counter and is the effective `MCID`. In the RCID-prefixed mode, the
controller identifies the counter for monitoring using an effective `MCID`
computed as:

----
Effective MCID = (RCID << P) | (MCID & ((1 << P) - 1))
----

Legal values of `P` range from 0 to 12 and are enumerated in the capability
register of the controller. Software should use the effective `MCID` as the
`MCID` operand to the controller for operations on the monitoring counters.

==== RISC-V hart initiated requests (Ssqosid)

The Ssqosid extension cite:[SSQOSID] introduces a read/write S/HS-mode register
(`sqoscfg`) to configure QoS Identifiers to be used with requests made by the
hart to shared resources.
hart to shared resources. If extension Smstateen cite:[STATEEN] is implemented
then bit (TBA) of `mstateen0` controls access to `sqoscfg` from privilege modes
less than M.

==== Device initiated requests

Expand Down
18 changes: 12 additions & 6 deletions qos_iommu.adoc
Original file line number Diff line number Diff line change
@@ -1,13 +1,17 @@
[[QOS_IOMMU]]
== IOMMU extension for QoS ID

Monitoring or allocation of resources accessed by the IOMMU and devices governed
by the IOMMU requires a way to associate QoS IDs with such requests. This
section specifies a RISC-V IOMMU cite:[IOMMU] extension to:
A method to associate QoS IDs with requests to access resources by the IOMMU, as
well as with devices governed by it, is required for effective monitoring and
allocation. This section specifies a RISC-V IOMMU cite:[IOMMU] extension to:

* Configure and associate QoS IDs for device-originated requests.
* Configure and associate QoS IDs for IOMMU-originated requests.

The size (or width) of `RCID` and `MCID`, as fields in registers or in data
structures, supported by the IOMMU must be at least as large as that supported
by any RISC-V application processor hart in the system.

=== IOMMU registers

The specified memory-mapped register layout defines a new IOMMU register named
Expand Down Expand Up @@ -35,7 +39,9 @@ If the reset value for `ddtp.iommu_mode` field is `Bare`, then the
At reset, it is required that the `RCID` field of `iommu_qosid` be set to 0 if
the IOMMU is in `Bare` mode, as typically the resource controllers in the
SoC default to a reset behavior of associating all capacity or bandwidth to the
`RCID` value of 0.
`RCID` value of 0. When the reset value of the `ddtp.iommu_mode` is not `Bare`,
the `iommu_qosid` register should be initialized by software prior to changing
the mode to allow DMA.
====

==== IOMMU capabilities (`capabilities`)
Expand Down Expand Up @@ -107,9 +113,9 @@ The `iommu_qosid` register fields are defined as follows:
[%header, cols="^1,2,^1,5"]
|===
|Bits |Field |Attribute | Description
|11:0 |`RCID` |WARL | RCID for IOMMU initiated requests.
|11:0 |`RCID` |WARL | `RCID` for IOMMU-initiated requests.
|15:12 |reserved |WPRI | Reserved for standard use.
|27:16 |`MCID` |WARL | MCID for IOMMU initiated requests.
|27:16 |`MCID` |WARL | `MCID` for IOMMU-initiated requests.
|31:28 |reserved |WPRI | Reserved for standard use.
|===

Expand Down
2 changes: 1 addition & 1 deletion readme.adoc
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
= RISC-V Capacity and Bandwidth Controller QoS Register Interface

This document is capturing discussions at the CMQRI TG and attempts to document the baseline.
This document is capturing discussions at the CBQRI TG and attempts to document the baseline.
This is not official specification and everything in this document may change.

Quality of Service (QoS) is the minimal end-to-end performance that is guaranteed in advance
Expand Down