Vector ACT #23
Comments
|
Update from Xi:
|
|
@allenjbaum and Xi need to work to find a time where the RIOS representative can meeting with key people from ACT group and begin to engage. This is the key next step, along with submitting PR for work. |
|
We really need to get someone from the Vector TG to be involved with this;
I don't know it well enough to understand all but the most basic corner
cases that need to be tested.
…On Mon, May 1, 2023 at 5:24 PM Jeff Scheel ***@***.***> wrote:
@allenjbaum <https://github.com/allenjbaum> and Xi need to work to find a
time where the RIOS representative can meeting with key people from ACT
group and begin to engage.
This is the key next step, along with submitting PR for work.
—
Reply to this email directly, view it on GitHub
<#23 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AHPXVJWBCD2POSYDHE4WAG3XEBH2HANCNFSM6AAAAAAV6UE5A4>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
I've just added @hushenwei2000 to this topic. Welcome Shenwei! @allenjbaum, I believe that Xi was working with Krste directly. Your question is a great one and we should see if Krste ever delegated. If not, I'd be inclined to ask the person helping @billmcspadden-riscv with the SAIL work to see if they'd help with ACT too. If not, we can ask Krste and Earl. Sound ok? |
|
I don't think the someone from the Sail team, or helping the Sail team, is
what we want.
We really want someone who understands the vector spec and its corner
cases.,
That could be someone working on the Sail code if their primary experience
is the vector spec,
but being conversant with all the intricacies of the vector spec is what we
are looking for, and I expect to find that expertise in the vector TG
…On Tue, May 2, 2023 at 8:56 AM Jeff Scheel ***@***.***> wrote:
I've just added @hushenwei2000 <https://github.com/hushenwei2000> to this
topic. Welcome Shenwei!
@allenjbaum <https://github.com/allenjbaum>, I believe that Xi was
working with Krste directly. Your question is a great one and we should see
if Krste ever delegated. If not, I'd be inclined to ask the person helping
@billmcspadden-riscv <https://github.com/billmcspadden-riscv> with the
SAIL work to see if they'd help with ACT too. If not, we can ask Krste and
Earl. Sound ok?
—
Reply to this email directly, view it on GitHub
<#23 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AHPXVJUUMQ2764PL6ROKAY3XEEVEXANCNFSM6AAAAAAV6UE5A4>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
Yes, @allenjbaum, the SAIL community asked a member of the Vector TG to attend their meetings and help review the SAIL code. Right, @billmcspadden-riscv? Perhaps this same person would do the same for ACT. |
|
Victor Moya ***@***.***) was our SME for the review of
the Sail vector implementation.
Should he be added to this thread?
Bill Mc.
…On Thu, May 4, 2023 at 9:57 AM Jeff Scheel ***@***.***> wrote:
Yes, @allenjbaum <https://github.com/allenjbaum>, the SAIL community
asked a member of the Vector TG to attend their meetings and help review
the SAIL code. Right, @billmcspadden-riscv
<https://github.com/billmcspadden-riscv>?
Perhaps this same person would do the same for ACT.
—
Reply to this email directly, view it on GitHub
<#23 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AXROLOBJET6P4K3RVIKXBQDXEO7VDANCNFSM6AAAAAAV6UE5A4>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
Bill McSpadden
Formal Verification Engineer
RISC-V International
mobile: 503-807-9309
|
|
Thanks, @billmcspadden-riscv. I'll reach out directly before adding him. |
|
@hushenwei2000, please work with @allenjbaum and the Arch Test SIG to begin integrating the Vector ACT. Let me know when you submit your first PR. |
|
@hushenwei2000, thanks for attending today's meeting. I'm happy to hear that you and allen are beginning to work together. Please let me know when that first PR is submitted. As discussed, there likely will be code re-work to split out common testing framework for all vector items. This will be needed by function currently being discussed in the Arch Test SIG such as Vector Crypto and other work not yet discussed, Zvfh/Zvfhmin. So, the key idea is to do integrate the new testing in such a way as to allow for commonality in future vector instructions. |
|
After the last meeting with test SIG, I learned that my previous tests had a self check and did not comply with the Test Format. Now, I am changing my test structure to allow the tests to run and compare signatures on RISCOF. I have completed the ACT with vsew=32, and other configurations will be completed as soon as possible. You can try in my repo: https://github.com/hushenwei2000/rvv-atg/tree/new_test_format |
|
Thanks, @hushenwei2000! It sounds like very positive work is underway. This is VERY good!!! |
|
I completely agree, naturally.
…On Wed, Jul 19, 2023 at 5:47 AM Jeff Scheel ***@***.***> wrote:
Thanks, @hushenwei2000 <https://github.com/hushenwei2000>! It sounds like
very positive work is underway. This is VERY good!!!
—
Reply to this email directly, view it on GitHub
<#23 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AHPXVJTNAIY5PRLFTBCDIRDXQ7JOHANCNFSM6AAAAAAV6UE5A4>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
I guess I have completed the code modifications to support all configurations. The tests I generated can now generate signature files, and I have tested them in the RISCOF framework. However, there are sometimes inconsistencies in the signatures between Sail and Spike, which may be a bug in Sail. You can try it according to the README in https://github.com/hushenwei2000/rvv-atg/tree/new_test_format I would like to know what I should do next? |
|
Could you give more details about the inconsistencies in the signatures
that you found? Can you isolate them to specific instructions?
…On Monday, August 14, 2023, HU Shenwei ***@***.***> wrote:
I guess I have completed the code modifications to support all
configurations. The tests I generated can now generate signature files, and
I have tested them in the RISCOF framework.
However, there are sometimes inconsistencies in the signatures between
Sail and Spike, which may be a bug in Sail.
You can try it according to the README in https://github.com/
hushenwei2000/rvv-atg/tree/new_test_format
I would like to know what I should do next?
—
Reply to this email directly, view it on GitHub
<#23 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AHPXVJQNWA66HVUXRMRFXP3XVIC2ZANCNFSM6AAAAAAV6UE5A4>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
Unfortunately, the Sail and Spike signatures of all instructions are inconsistent. But in my previous test with expected answers, Sail passed all the tests. Therefore, I suspect that there was an error in Sail about the load/store instruction used for reading/writing signatures. |
|
I was looking for more detail, such as what precisely the mismatch was.
Were the signature values offset (so elements were store at signature word
X+1 instead of X?)
or the signature values were only partially stored or not stored at all (so
the background canary wasn't change).
Could you cut&paste a bit of the test report so we can see where the
mismatches were?
…On Tue, Aug 15, 2023 at 6:58 AM HU Shenwei ***@***.***> wrote:
Unfortunately, the Sail and Spike signatures of all instructions are
inconsistent.
But in my previous test with expected answers, Sail passed all the tests.
Therefore, I suspect that there was an error in Sail about the load/store
instruction used for reading/writing signatures.
—
Reply to this email directly, view it on GitHub
<#23 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AHPXVJQFG2VZ3U7WMSGAVNLXVN563ANCNFSM6AAAAAAV6UE5A4>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
@hushenwei2000, would you kindly provide a written update here on your progress? Thanks! |
|
The issue of inconsistent signatures has been resolved by adding initial VLEN configuration in Sail model. And that means the functionality of the test generator is okay. We will continue to discuss the issue of coverage with Allen. |
|
@hushenwei2000, any updates? |
|
We are working on the coverage things, but they are quite complicated. For example, we need to decide on the numbers and configurations, which may take some time. Allen, James and I are collaborating on that. |
|
We are working on a formal coverage definition for all instructions. And deal with non-deterministic cases. |
|
I am kinda busy this month. Quswar Abid from 10xEngineers recently contacted me to help me complete the Vector ACT, much thanks! We are working together to ensure the entire framework correct, especially the signature part. |
|
I apologize for the inconvenience. Over the past few months, I have been dedicated to completing my master's research project. However, I intend to shift my focus back to ACT in December. While collaborating with Quswar, I identified some bugs in my signature. Therefore, the immediate priority is to address these issues to ensure the accuracy of the ACT. Subsequently, we will also need to address additional coverage points. |
|
It sounds like you're not putting your coverage into the .cgf format , so
we can't track it with ISAC spec, which is a bit of a problem we should
talk about in the arch-test meeting.
This would be the first coverage test that doesn't use ISAC to measure
coverage.
We allow test generation to to CTG or any other test generator, and even
manual tests, but this would be the first time that a different coverage
tool is used.
…On Tue, Mar 19, 2024 at 2:55 AM WU Xieyuan ***@***.***> wrote:
Updates: cover different mask value (walking_zeros, walking_ones,
checkerboard pattern)
—
Reply to this email directly, view it on GitHub
<#23 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AHPXVJQSSBLGCRAEMLWMTDDYZADPLAVCNFSM6AAAAAAV6UE5A6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBWGU4DOMJTGY>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
Yes, and I think it‘s also hard for covering different vstart to be put into cgf format.
|
|
I think that problem is getting fixed by adding all?some? CSR updates to
the log files,, both explicit and implicit.
On Wed, Mar 20, 2024 at 12:38 AM WU Xieyuan ***@***.***>
wrote:
… Yes, and I think it‘s also hard for covering different vstart to be put
into cgf format.
It sounds like you're not putting your coverage into the .cgf format , so
we can't track it with ISAC spec, which is a bit of a problem we should
talk about in the arch-test meeting. This would be the first coverage test
that doesn't use ISAC to measure coverage. We allow test generation to to
CTG or any other test generator, and even manual tests, but this would be
the first time that a different coverage tool is used.
… <#m_6940674792954379929_>
On Tue, Mar 19, 2024 at 2:55 AM WU Xieyuan *@*.*> wrote: Updates: cover
different mask value (walking_zeros, walking_ones, checkerboard pattern) —
Reply to this email directly, view it on GitHub <#23 (comment)
<#23 (comment)>>,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AHPXVJQSSBLGCRAEMLWMTDDYZADPLAVCNFSM6AAAAAAV6UE5A6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBWGU4DOMJTGY
<https://github.com/notifications/unsubscribe-auth/AHPXVJQSSBLGCRAEMLWMTDDYZADPLAVCNFSM6AAAAAAV6UE5A6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBWGU4DOMJTGY>
. You are receiving this because you were mentioned.Message ID: @.*>
—
Reply to this email directly, view it on GitHub
<#23 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AHPXVJQ4ZWRTDRCHQT4YHI3YZE4FZAVCNFSM6AAAAAAV6UE5A6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBYHE2TMMRWGM>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
updates: 1. compact some macro generator codes; 2. change all mask related tests from vlex to vlm for v0; 3. add floating point dataset |
|
updates: 1. extract 12 ibm fp datasets from rpt instead of hand-written in test generator; 2. modify tests for vwred |
|
Sounds like good progress. What work remains to have accepted PRs? |
|
In addition to the remaining tasks outlined in the coverage point draft, significant code refactoring is also needed. Currently, making minor changes requires modifications across a large portion of the code. The refactoring effort is substantial, and completing it may take some time if I'm working on it alone. |
|
Update: save vcsr in signature along with vd, fcsr and xstatus CSRs |
|
Saving xcsr requires eiable.ther running in M/S modes, or transitioning to them prior to saving. |
|
updates: fixes (vfwred; some macros) & code refactoring |
|
Thanks, @Oxyw. Do you have a date when you think you'll have a first PR for your work? |
I anticipate that it will be around July, as I will be quite busy over the next two weeks. |
|
@Oxyw, have you received the requested feedback from @allenjbaum on the cover points draft: RVV Test Coverage Points? |
Thank you for checking in. As of now, I haven't received the requested feedback on the draft. I'll follow up and inform you as soon as I have any updates. |
|
updates: checked for missing instructions, modified macros, and fixed discovered bugs. |
|
@Oxyw, thanks for the update. When do you believe you'll be ready to submit a PR? What works remains before doing so? |
I think I will be ready to submit the PR in the end of July or early August. Before doing so, I still need to check and fix bugs in the existing tests. Additionally, it is also necessary to add some tests for invalid situations. |
|
I'm sorry that I won't be able to submit the PR this week. I've recently identified some previously undetected issues with the load/store instructions that need to be addressed. After these modifications, all legal cases in the coverage points draft will be covered. However, testing for illegal cases still requires further discussion via email. |
|
Thanks, @Oxyw. I appreciate the update. Finding issues IS progress! |
|
still working on fixing and refactoring the tests |
Technical Group
Vector TG
ratification-pkg
Vector
Technical Liaison
Krste
Task Category
Arch Tests
Task Sub Category
Ratification Target
4Q2021
Statement of Work (SOW)
SOW: link
SOW Signoffs:
Waiver
Pull Request Details
No response
The text was updated successfully, but these errors were encountered: