error saving curated volin plot

[carlocolantuoni]

hi guys,

im getting "Could not create Plotly plot for this dataset and parameters. Please contact the gEAR team."
when trying to save a violin plot for the following dataset

dataset_id=6b3f05b3-0885-4765-8d6b-c449753823b6

[carlocolantuoni]

also happening for dataset:
dataset_id=1cac4125-7083-4754-9aa6-6964c8d9ffec

for both these datsets i have static umap plots create no prob

[adkinsrs]

What conditions are you trying to plot?

[carlocolantuoni]

expression on Y-axis, "dev_state" on the X-axis - everything else is default settings
i think it is any violin for these 2 datasets - i have tried a lot of dif meta data on the X axis
there are "_" and "/" characters in some of the meta data, but those arent problem characters are they?
also cant make scatter plots

i all cases, im now getting this error instead of the original one i sent "Something went wrong. Please contact the gEAR team and provide steps to reproduce."

[carlocolantuoni]

in addition to the different error now, i am not evne getting to the screen where i sent the screen cap above - it gives the erros right when i hit the plot button

[adkinsrs]

Actually getting a chance to look at this. It's a Javascript error related to a warning that is supposed to show in addition when a data series (in this case "dev_state") has too many groups that could result in a crowded plot. Going to work on fixing this warning

[carlocolantuoni]

Cool

…

On Wed, Sep 4, 2024, 13:51 Shaun Adkins ***@***.***> wrote: Actually getting a chance to look at this. It's a Javascript error related to a warning that is supposed to show when a data series (in this case "dev_state") has too many groups that could result in a crowded plot. Going to work on fixing this warning — Reply to this email directly, view it on GitHub <#875 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AH7KC7XFTW643MIAQ2O5MPDZU5CBDAVCNFSM6AAAAABNBJOPXOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRZGY2TKNZYGI> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[adkinsrs]

Works now. Closing

[carlocolantuoni]

Awesome- thnx!

…

On Wed, Sep 4, 2024, 14:36 Shaun Adkins ***@***.***> wrote: Closed #875 <#875> as completed. — Reply to this email directly, view it on GitHub <#875 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AH7KC7WJVSUDH4YUEDCBYFTZU5HJNAVCNFSM6AAAAABNBJOPXOVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJUGEZTKNRTGA2TCMI> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[carlocolantuoni]

sorry shaun - still having issues. when i try this:

https://nemoanalytics.org/dataset_curator.html?dataset_id=6b3f05b3-0885-4765-8d6b-c449753823b6

or this

https://nemoanalytics.org/dataset_curator.html?dataset_id=1cac4125-7083-4754-9aa6-6964c8d9ffec

and then enter violin plot details i get an eternally loading symbol when i hit the PLOT button:

and

[carlocolantuoni]

when i tried to use different meta data for the violin "dev_state" instead of "stage.ord" as before, and hit the PLOT button, it progressed but then i got this error: "Could not create Plotly plot for this dataset and parameters. Please contact the gEAR team."

[adkinsrs]

Seems it's a separate issue with rendering the group series "sortable" component that lets you reorder facets, legends, etc. Pretty sure it's because the "stage.ord" series name has a period in it, which the HTML Sortable component does not like. However, it seems that the groups of sortable lists rendered does not reflect the chosen groups for the plot either (containing more groups than were specified)

[adkinsrs]

Ok fixed the issue where the plot will now render (forgot to escape a CSS selector ID). However I still need to work on the issue where old sortable series not currently applicable need to be removed.

[adkinsrs]

Closing ticket as things seem to be in working order now. I also resolved the issue from the previously mentioned comment

[carlocolantuoni]

Thnx shaun!

…

On Fri, Sep 6, 2024, 10:10 Shaun Adkins ***@***.***> wrote: Closed #875 <#875> as completed. — Reply to this email directly, view it on GitHub <#875 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AH7KC7US6OVIUKLOH7SE25DZVGZUBAVCNFSM6AAAAABNBJOPXOVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJUGE3DENZTGMYDANY> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>

[carlocolantuoni]

hey shaun - i was able to make a violin plot for both of the datasets, using the "stage.ord" meta data column, but when i try it with "dev_state" i am still getting this error for both datasets: "Could not create Plotly plot for this dataset and parameters. Please contact the gEAR team."

[adkinsrs]

I'm not able to replicate. The "dev_state" violin plot generates fine for me for both datasets

[adkinsrs]

Fixed some issues in #827 that may (or may not) also resolve your issues @carlocolantuoni

[carlocolantuoni]

Cool, thnx, i ll giv it a try again

…

On Mon, Sep 9, 2024, 10:38 Shaun Adkins ***@***.***> wrote: Fixed some issues in #827 <#827> that may (or may not) also resolve your issues @carlocolantuoni <https://github.com/carlocolantuoni> — Reply to this email directly, view it on GitHub <#875 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AH7KC7SW6NTMKANS3SNC6U3ZVW6FXAVCNFSM6AAAAABNBJOPXOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMZYGQ2TENBYGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[adkinsrs]

@carlocolantuoni are the issues still persisting?

[carlocolantuoni]

ya there are still when i try to make a violin plot for
https://nemoanalytics.org/dataset_curator.html?dataset_id=6b3f05b3-0885-4765-8d6b-c449753823b6
using the "dev_state" meta data, i get "Could not create Plotly plot for this dataset and parameters. Please contact the gEAR team."

super weird as it looks exactly like what you are doing without error

[carlocolantuoni]

when i plot by "dev_state" but DO NOT color by that meta data is does work!

[carlocolantuoni]

but if i plot and color by "dev_state" it fails with that error.
so for now its low priority because i can plot it withot color but maybe its becasue it cant pick >20 colors?

[adkinsrs]

I'd like to get it working though, so I'll tackle that after #886. There's a good chance others would encounter a similar issue under the same conditions (but different dataset and configuration). Thanks for the update.

[carlocolantuoni]

thnx shaun

…

On Mon, Sep 16, 2024 at 11:56 AM Shaun Adkins ***@***.***> wrote: I'd like to get it working though, so I'll tackle that after #886 <#886>. There's a good chance others would encounter a similar issue under the same conditions (but different dataset and configuration). Thanks for the update. — Reply to this email directly, view it on GitHub <#875 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AH7KC7WAGZD6NZZCOH3MQPLZW35T7AVCNFSM6AAAAABNBJOPXOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNJTGMYTCMJSHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Carlo

[adkinsrs]

Seems the issue is that there there is a missing value (nan) present in the color mapping which is causing type incompatibilities. I believe this is because the "NA" present as a "dev_state" category is being converted to "nan" (the pandas missing value indicator).

Reference -> https://stackoverflow.com/questions/33952142/prevent-pandas-from-interpreting-na-as-nan-in-a-string

[adkinsrs]

Unfortunately it seems like the AnnData reader has no way to remove "NA" as a default missing value, so it automatically gets converted into NaN. Normally I'd just say "ok, let's just turn them back into NA again", but I'm worried about edge cases where someone had both "NA" and missing values in a metadata category and then we may be incorrectly merging two bad properties. Honestly though, if that ever becomes an issue, I would attribute it to being a terrible dataset.

So I think I'm going to try and catch these missing values, and unify them if I can, at least to whatever is found in the color mapping.

[adkinsrs]

I pushed a fix that makes it work. However, the NA column does not show in the plot, but at least it is not erroring for me now.

[carlocolantuoni]

👍👍

…

On Mon, Sep 16, 2024 at 3:38 PM Shaun Adkins ***@***.***> wrote: I pushed a fix that makes it work. However, the NA column does not show in the plot, but at least it is not erroring for me now. — Reply to this email directly, view it on GitHub <#875 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AH7KC7SHFQAR55B22H3T4OTZW4XUDAVCNFSM6AAAAABNBJOPXOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNJTG43DGMJSGI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Carlo

[adkinsrs]

@carlocolantuoni I am closing this in favor of #888, since the issue is more focused now.

[carlocolantuoni]

👍

…

On Tue, Sep 17, 2024, 08:43 Shaun Adkins ***@***.***> wrote: @carlocolantuoni <https://github.com/carlocolantuoni> I am closing this in favor of #888 <#888>, since the issue is more focused now. — Reply to this email directly, view it on GitHub <#875 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AH7KC7QBXZTN6X6UBDYH6X3ZXAPX5AVCNFSM6AAAAABNBJOPXOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNJVGY2DQMJQHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>