Best practices for clean-up after segfault / custom signal handler

[daniel-falk]

We have some peripheral clean-up that we need to do before the application is closed down. This should be done even if there is e.g. a SEGFAULT. We do this currently by adding a signal handler for SEGFAULT that sets an error flag, we then do clean-up and exits the application.

When using sentry-native it seems like the library is overriding our signal handlers to be able to send telemetry messages to sentry on SEGFAULTs. This handler seems to exit the application when done, meaning that we can't do our cleanup. We could manually override the sentry signal handler, but then we don't get the telemetry messages. Do we need to manually implement the transmission of a stacktrace to using sentry from our own signal handler, or is there a way we can get sentry to remove it's handler and raise the same (or another) signal after sending the stacktrace? Or is there any better way to design the error handling to achieve both clean-up and telemetry?

[supervacuus]

Hi, @daniel-falk! If you are asking for a solution specifically when a crash occurs, the preferred method would be to implement your shutdown by using our on_crash-hook which will be called from the crash handler before the crash snapshot is generated (not only for SIGSEGV but all hard-exceptions).

Here is the link to the docs: https://docs.sentry.io/platforms/native/configuration/filtering/#using-on_crash
And a bit more detailed in the header docs:

sentry-native/include/sentry.h

Lines 831 to 875 in 6c76f58

	* Type of the `on_crash` callback.
	*
	* The `on_crash` callback replaces the `before_send` callback for crash events.
	* The interface is analogous to `before_send` in that the callback takes
	* ownership of the `event`, and should usually return that same event. In case
	* the event should be discarded, the callback needs to call
	* `sentry_value_decref` on the provided event, and return a
	* `sentry_value_new_null()` instead.
	*
	* Only the `inproc` backend currently fills the passed-in event with useful
	* data and processes any modifications to the return value. Since both
	* `breakpad` and `crashpad` use minidumps to capture the crash state, the
	* passed-in event is empty when using these backends, and they ignore any
	* changes to the return value.
	*
	* If you set this callback in the options, it prevents a concurrently enabled
	* `before_send` callback from being invoked in the crash case. This allows for
	* better differentiation between crashes and other events and gradual migration
	* from existing `before_send` implementations:
	*
	* - if you have a `before_send` implementation and do not define an `on_crash`
	* callback your application will receive both normal and crash events as
	* before
	* - if you have a `before_send` implementation but only want to handle normal
	* events with it, then you can define an `on_crash` callback that returns
	* the passed-in event and does nothing else
	* - if you are not interested in normal events, but only want to act on
	* crashes (within the limits mentioned below), then only define an
	* `on_crash` callback with the option to filter (on all backends) or enrich
	* (only inproc) the crash event
	*
	* This function may be invoked inside of a signal handler and must be safe for
	* that purpose, see https://man7.org/linux/man-pages/man7/signal-safety.7.html.
	* On Windows, it may be called from inside of a `UnhandledExceptionFilter`, see
	* the documentation on SEH (structured exception handling) for more information
	* https://docs.microsoft.com/en-us/windows/win32/debug/structured-exception-handling
	*
	* Platform-specific behavior:
	*
	* - does not work with crashpad on macOS.
	* - for breakpad on Linux the `uctx` parameter is always NULL.
	* - on Windows the crashpad backend can capture fast-fail crashes which
	* by-pass SEH. Since `on_crash` is called by a local exception-handler, it will
	* not be invoked when such a crash happened, even though a minidump will be
	* sent.

[tian-yu-ocado]

Hi @supervacuus, I have a follow up question with respect to use sentry_options_set_on_crash. I have a cleanup callback inside which I would like to change the title of the event, as in my Sentry web portal the reported event title is . Is there a way to do this inside the cleanup callback function? What I did was like this:

static sentry_value_t crash_cleanup(
        const sentry_ucontext_t* uctx,  // provides the user-space context of the crash
        sentry_value_t event,           // used the same way as in `before_send`
        void* closure                   // user-data that you can provide at configuration time
) {
    sentry_value_set_by_key(event, "title", sentry_value_new_string("crash-title"));
    sentry_value_set_by_key(event, "message", sentry_value_new_string("stack crashed"));       
    return event;
}

However, setting the message seem to work properly, but setting the event title does not.

[supervacuus]

AFAIK, there is no way to change the "title" from the client because it is a computed property of the backend. "message" OTOH is part of the Message Interface and can be sent from any client.

[tian-yu-ocado]

Thanks! If the event title is computed by the backend, do you happen to know what information is used to do the calculation? For my case if I trigger a hard crash by nullptr dereferncing, the event title would just be . Would it because it is missing the debug information?

[supervacuus]

The "title" typically reflects the function/symbol of the top-level frame in the exception stack trace. So, if that symbol cannot be resolved, it might be empty (but I don't know the details of the current implementation).

[tian-yu-ocado]

Thanks a lot! @supervacuus