Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: Segmentation fault in Lambda when importing event_parser #4140

Closed
wevonosky opened this issue Apr 16, 2024 · 7 comments
Closed

Bug: Segmentation fault in Lambda when importing event_parser #4140

wevonosky opened this issue Apr 16, 2024 · 7 comments
Labels
bug Something isn't working need-customer-feedback Requires more customers feedback before making or revisiting a decision

Comments

@wevonosky
Copy link

Expected Behaviour

I expect that running from aws_lambda_powertools.utilities.parser import event_parser wouldn't cause a segmentation fault

Current Behaviour

First, thank you to all the contributors and maintainers of an awesome package! Appreciate all of you!

Currently for an extremely minimal lambda definition I am getting a Invoke failed error=Runtime exited with error: signal: segmentation fault InvokeID=76ea8eec-920e-428d-ba40-b53e05d9b869 when the only code in the lambda imports event_parser with a lambda handler function that returns "Hello".

Code snippet

from aws_lambda_powertools.utilities.parser import event_parser

def lambda_handler(event: dict, _) -> dict:
    """blah"""
    print(event)
    return "Hello"

Possible Solution

No response

Steps to Reproduce

  1. Build an image using the following lambda base image:

public.ecr.aws/lambda/python
digest: sha256:4d7703751e75e999b80787888496c131e782491320a2e25ec84d6bda5e1f2438

  1. docker run --rm --platform linux/amd64 -p 9000:8080 name:tag

  2. curl "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{}'

Powertools for AWS Lambda (Python) version

2.36.0

AWS Lambda function runtime

3.10

Packaging format used

PyPi

Debugging logs

No response

@wevonosky wevonosky added bug Something isn't working triage Pending triage from maintainers labels Apr 16, 2024
Copy link

boring-cyborg bot commented Apr 16, 2024

Thanks for opening your first issue here! We'll come back to you as soon as we can.
In the meantime, check out the #python channel on our Powertools for AWS Lambda Discord: Invite link

@wevonosky
Copy link
Author

I may have traced this to pydantic causing the problem instead of powertools. Will report back!

@rubenfonseca
Copy link
Contributor

Thank you for opening this and for your nice words! It's awesome that you could narrow it down. Can you tell me a little further how did you build the image locally? (point 1). I just wanted to test it locally too.

@rubenfonseca rubenfonseca added need-customer-feedback Requires more customers feedback before making or revisiting a decision and removed triage Pending triage from maintainers labels Apr 16, 2024
@wevonosky
Copy link
Author

Its a little complicated since I am using bazel and rules_oci. I'll see if I can find a way to write a minimal docker file that reproduces the issue.

@wevonosky
Copy link
Author

I am actually still not sure what is causing the problem but I don't think its powertools. I removed that import and still get a seg fault but I can't figure out how to trace it.

In case this does end up still being relevant here, here are some more complete logs:

16 Apr 2024 19:57:53,050 [INFO] (rapid) exec '/var/runtime/bootstrap' (cwd=/var/task, handler=)
START RequestId: 22eec31d-9307-40ef-8a8c-206a844e6cbc Version: $LATEST
16 Apr 2024 19:57:55,092 [INFO] (rapid) INIT START(type: on-demand, phase: init)
16 Apr 2024 19:57:55,092 [INFO] (rapid) The extension's directory "/opt/extensions" does not exist, assuming no extensions to be loaded.
16 Apr 2024 19:57:55,093 [INFO] (rapid) Starting runtime without AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN , Expected?: false
/var/lang/bin/python3.10: Relink `/var/task/pydantic_core/_pydantic_core.cpython-310-x86_64-linux-gnu.so' with `/lib64/librt.so.1' for IFUNC symbol `clock_gettime'
16 Apr 2024 19:58:02,657 [WARNING] (rapid) First fatal error stored in appctx: Runtime.ExitError
16 Apr 2024 19:58:02,657 [WARNING] (rapid) Process runtime-1 exited: signal: segmentation fault
16 Apr 2024 19:58:02,657 [INFO] (rapid) INIT RTDONE(status: error)
16 Apr 2024 19:58:02,657 [INFO] (rapid) INIT REPORT(durationMs: 7564.567000)
16 Apr 2024 19:58:02,657 [ERROR] (rapid) Init failed error=Runtime exited with error: signal: segmentation fault InvokeID=
16 Apr 2024 19:58:02,657 [WARNING] (rapid) Shutdown initiated: spindown
16 Apr 2024 19:58:02,657 [INFO] (rapid) Waiting for runtime domain processes termination
16 Apr 2024 19:58:02,657 [INFO] (rapid) INIT START(type: on-demand, phase: invoke)
16 Apr 2024 19:58:02,657 [INFO] (rapid) The extension's directory "/opt/extensions" does not exist, assuming no extensions to be loaded.
16 Apr 2024 19:58:02,657 [INFO] (rapid) INIT REPORT(durationMs: 0.051000)
16 Apr 2024 19:58:02,657 [INFO] (rapid) INVOKE START(requestId: f2a25232-59c2-41e7-bf07-c85a89d24fc4)
16 Apr 2024 19:58:02,657 [ERROR] (rapid) Invoke failed error=Runtime exited with error: signal: segmentation fault InvokeID=f2a25232-59c2-41e7-bf07-c85a89d24fc4
16 Apr 2024 19:58:02,657 [ERROR] (rapid) Invoke DONE failed: Sandbox.Failure
16 Apr 2024 19:58:02,657 [WARNING] (rapid) Reset initiated: ReleaseFail
16 Apr 2024 19:58:02,657 [INFO] (rapid) Waiting for runtime domain processes termination

@wevonosky
Copy link
Author

Ok I can't reproduce this problem with a simple dockerfile like this:

FROM public.ecr.aws/lambda/python:3.10@sha256:4d7703751e75e999b80787888496c131e782491320a2e25ec84d6bda5e1f2438

RUN pip install aws_lambda_powertools
RUN pip install pydantic


COPY test.py ${LAMBDA_TASK_ROOT}

CMD [ "test.lambda_handler" ]

where test.py contains the code I posted. I'm thinking now that this is a problem with my internal build tools. Going to close since I don't think this is a problem with powertools!

Copy link
Contributor

⚠️COMMENT VISIBILITY WARNING⚠️

This issue is now closed. Please be mindful that future comments are hard for our team to see.

If you need more assistance, please either tag a team member or open a new issue that references this one.

If you wish to keep having a conversation with other community members under this issue feel free to do so.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working need-customer-feedback Requires more customers feedback before making or revisiting a decision
Projects
Status: Shipped
Development

No branches or pull requests

2 participants