ImportError masks original exception

[rittneje]

newrelic-lambda-layers/python/newrelic_lambda_wrapper.py

Line 65 in 48654ab

raise ImportError("Failed to import module '%s': %s" % (module_path, e))

If importing the main module fails, this takes the original exception, converts to to a string, and then appends to the ImportError, which masks details about the original exception, including its actual stack trace, and the type of exception.

As an example, suppose my main module does os.environ["ABC"], where there is no ABC environment variable. With the current code, I get:

Runtime.ImportModuleError: Unable to import module 'newrelic_lambda_wrapper': Failed to import module '<name>': 'ABC'

That does not make it clear at all what the problem is, or even that it was a KeyError.

Ideally, this wrapper would instead do something like:

raise ImportError("Failed to import module '%s'" % module_path) from e

Unfortunately, it sounds like the Lambda runtime neglects to output the exception chain properly. https://stackoverflow.com/questions/62112585/aws-lambda-not-showing-cause-exception-stacktrace-in-python-3-8

So this wrapper may instead need to collect the traceback (and exception type) itself, and glue all of that into the ImportModuleError.

[workato-integration]

https://new-relic.atlassian.net/browse/NR-336930

[aavinash-nr]

Hello @rittneje ,
We have checked all the error messages which are being shown from NR Extension and we found that we are capturing the trace for all the errors.

[ERROR] Runtime.ImportModuleError: Unable to import module 'newrelic_lambda_wrapper': Failed to import module 'app': No module named 'ABC'

The above error message correctly indicates that module named ABC could not be imported. Let us know if any other issue is present.

Thank you!

[rittneje]

@aavinash-nr That is not a trace. As you can see, it did not output the original exception type or traceback. As I mentioned in the issue description, it only outputs the text of the original exception, which is not helpful in general. Please re-open this issue.

[rittneje]

@aavinash-nr Following up on this. Please re-open this issue.

[chaudharysaket]

Hi @rittneje
I have re-opened the issue.
While testing, we did not encounter the issue of the ImportModuleError not being propagated.
Could you please share some examples.
Thank you!

[rittneje]

@chaudharysaket Please re-read the issue description. The problem is not ImportModuleError itself, but rather that if my module raises an exception while initializing global variables (say), the original exception type and stack trace get lost.

For example, if my main module does:

import os

ABC = os.environ["ABC"]

and there is no ABC environment variable, that throws a KeyError, but the only thing that gets logged in the Lambda logs is

Runtime.ImportModuleError: Unable to import module 'newrelic_lambda_wrapper': Failed to import module '<name>': 'ABC'