-
Notifications
You must be signed in to change notification settings - Fork 239
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
About the confused relationship between DetA and predict IDs. #134
Comments
Your reported behavior is what I expected when I just read the related part of the code. In my opinion, DetA should not be based on HOTA_TP, HOTA_FN and HOTA_FP (which are determined by global_alignment_score matrix * similarity) but rather on regular TP, FN and FP that are purely determined by similarity. The paper also states clearly:
In that sense, the code would be inconsistent with the paper. |
Yes, in my understanding, I also think that the ID alignment result will not affect the But in the paper, they explained in page 555:
It seems like the matching results will be firstly affected by localization metrics (detection) and then affected by the association metrics. So, the association metrics DO affect the DetA results by influencing the matching results. But I'm confused about that. Will the ID results influence the DetA metric SO MUCH? As I discussed above, there almost 1.5 gaps on DetA metric when I use different ID results. |
In my understanding, if the same detector is used (that is, the position result of the boxes is fixed), no matter how I change the ID assignment of the targets, it will not change the result of DetA.
But in my experiment, I used the detection results of the same detector but utilized two different ID assignments.
I thought that the DetA metric of these two strategies should be exactly the same, and they should only differ on AssA, since they use the same detection boxes but different IDs. But I got two different DetA results instead, as follows:
I wonder to know that if different IDs will affect the calculation result of the DetA metric. Because in the paper HOTA: A Higher Order Metric for Evaluating Multi-object Tracking, I saw that the calculation of DetA is only related to TP, FN, and FP, which does not contain any ID information.
OR is there something wrong with my understanding?
The text was updated successfully, but these errors were encountered: