Recommendations for triaging AppHang errors

[CraigSiemens]

After we updated to 8.0, we started getting a lot of app hang error. There were two error/traces that were logged for many users (100s) so we did some fixes to improve that. Unfortunately there were also 100s of unique traces that only one or two users encountered. That leads to a couple questions about the best way to think about and handle these.

I dove into Sentry's implantation and I would guess that for some users, the slow code ran in slightly under 2 seconds, but there was following work queued up on the main thread. Then when sentry detects the AppHangs, it reports a trace from some time after the slow code. Is this a correct understanding of how Sentry detects and reports the traces?

For all the AppHang errors in Sentry that each only have one or two users, would there be any harm in grouping them together? We mainly want to avoid the list getting filled up with one-off errors where the stack trace isn't particularly helpful.

This is probably more of a suggestion than a question, but would it be possible to auto-group the AppHang events together until the occurrences of a particular stack trace passes some threshold? Again, it's to try and keep the errors list manageable.

[philipphofmann]

Thanks for this excellent summary of your problems with AppHangs, @CraigSiemens.

Then when sentry detects the AppHangs, it reports a trace from some time after the slow code. Is this a correct understanding of how Sentry detects and reports the traces?

That is due to how our AppHang logic works. We enqueue an item to execute on the main threads loop and check if it doesn't get executed in a defined time we consider an app hang is ongoing. Getting the exact stacktrace works pretty well if the app is completely stuck, but if the run loop is extremely busy with different code locations, our logic might fire a bit too late and we give you a stacktrace that is not 100% correct. I think the proper solution would be some combination of AppHangs with profiling. So basically, if we detect too much work on the main thread, we start our profiler and link the profile to the AppHang event. Of course, you would have to pay for profiling in that scenario. Maybe we can also do something that works for errors only. We can also attach multiple stacktraces to the same event. I will discuss this with my team internally and get back to you. Thanks for the great input.

For all the AppHang errors in Sentry that each only have one or two users, would there be any harm in grouping them together?

I think we shouldn't group them together, instead I think it would make sense that we hide these until they reach some kind of threshold. The product should do this for you. Again, I will bring this to my team and get back to you.

Thanks again for the great input 👏

[markushi]

Excellent discussion!
We're facing the same issues on Android and recently built a prototype which links App Hangs (aka ANRs) with transactions and profiles. Whenever we detect an ANR, which usually means the app is about the get killed, we

1. finish any ongoing transaction/profiling, which will persist them and eventually send them to sentry.io
2. attach the traceContext to the ANR event, which links the ANR event, transaction and profile

It won't solve the grouping issue but at least the link to profiling allows you to better understand why an ANR is happening. Right now it also only works in ideal conditions, namely a transaction/profiling has to be active.

[philipphofmann]

I created a GH issue for linking app hangs to profiles #3493, but this will only be possible / useful once we implement continuous profiling, which will be somewhere in 2024 or maybe later.

[Lesmm]

App Hanging -[FlutterPlatformPlugin getClipboardData:]

App hanging for at least 2000 ms.

Happen in Every-Time App is resumed and asked for data from clipboard after copied some different text, the System(iOS 16+) alert to ask 【Allow paste】 permission. If this alert window won't disappear in 2 seconds, sentry upload this 【App Hanging】 event.

Is it possible the SentryANRTracker ignore event on this case?

xcrun atos -o $Flutter_DSYM/Contents/Resources/DWARF/Flutter --arch arm64 -offset 0x01aee0           
xcrun atos -o $Flutter_DSYM/Contents/Resources/DWARF/Flutter --arch arm64 -offset 0x01a584
xcrun atos -o $Flutter_DSYM/Contents/Resources/DWARF/Flutter --arch arm64 -offset 0x58e2fc
xcrun atos -o $Flutter_DSYM/Contents/Resources/DWARF/Flutter --arch arm64 -offset 0x043858

-[FlutterPlatformPlugin getClipboardData:] (in Flutter) (FlutterPlatformPlugin.mm:265)
-[FlutterPlatformPlugin handleMethodCall:result:] (in Flutter) (FlutterPlatformPlugin.mm:0)
__45-[FlutterMethodChannel setMethodCallHandler:]_block_invoke (in Flutter) (FlutterChannels.mm:300)
invocation function for block in flutter::PlatformMessageHandlerIos::HandlePlatformMessage(std::_LIBCPP_ABI_NAMESPACE::unique_ptr<flutter::PlatformMessage, std::_LIBCPP_ABI_NAMESPACE::default_delete<flutter::PlatformMessage>>) (in Flutter) (platform_message_handler_ios.mm:86)

[philipphofmann]

I'm sorry for not getting back to you sooner. I was on parental leave for two months. If you still experience the issue @Lesmm, this seems like a bug, so please open an issue for your problem.

[abdullahalamodi]

App Hanging -[FlutterPlatformPlugin getClipboardData:]

App hanging for at least 2000 ms.

Happen in Every-Time App is resumed and asked for data from clipboard after copied some different text, the System(iOS 16+) alert to ask 【Allow paste】 permission. If this alert window won't disappear in 2 seconds, sentry upload this 【App Hanging】 event.

Is it possible the SentryANRTracker ignore event on this case?

xcrun atos -o $Flutter_DSYM/Contents/Resources/DWARF/Flutter --arch arm64 -offset 0x01aee0           
xcrun atos -o $Flutter_DSYM/Contents/Resources/DWARF/Flutter --arch arm64 -offset 0x01a584
xcrun atos -o $Flutter_DSYM/Contents/Resources/DWARF/Flutter --arch arm64 -offset 0x58e2fc
xcrun atos -o $Flutter_DSYM/Contents/Resources/DWARF/Flutter --arch arm64 -offset 0x043858

-[FlutterPlatformPlugin getClipboardData:] (in Flutter) (FlutterPlatformPlugin.mm:265)
-[FlutterPlatformPlugin handleMethodCall:result:] (in Flutter) (FlutterPlatformPlugin.mm:0)
__45-[FlutterMethodChannel setMethodCallHandler:]_block_invoke (in Flutter) (FlutterChannels.mm:300)
invocation function for block in flutter::PlatformMessageHandlerIos::HandlePlatformMessage(std::_LIBCPP_ABI_NAMESPACE::unique_ptr<flutter::PlatformMessage, std::_LIBCPP_ABI_NAMESPACE::default_delete<flutter::PlatformMessage>>) (in Flutter) (platform_message_handler_ios.mm:86)

I have the same error. any update on this?

[kahest]

Hey @abdullahalamodi @Lesmm we identified that these reports can be caused by system dialogs (e.g. file selector, clipboard access, Face ID) and added an API to temporarily pause/resume App Hang tracking before/after invoking these dialogs.

• sentry-cocoa 8.30.0+: see docs
• sentry-dart 8.4.0+: see changelog

[julianD77]

This post helped us track down the issue we were seeing regarding new "App Hanging" issues - see related comments in #3472 (comment)