Include more detailed error response messages for debugging

[vsquared10]

More Verbose Errors for RequestError type:
While developing new features for search templates or updating synonyms, often times a developer will receive a 400 error, (RequestError) without any other additional context or ability to view the full body of the response. This opacity prevents automated error handling/synonym adjustment without further detail into the actual error['caused_by']['caused_by']['reason'] message that would appear in a Kibana response window. For example, the below response is the full amount of verbosity included while using Kibana when making a PUT <index_name>/ call:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "illegal_argument_exception",
        "reason" : "failed to build synonyms"
      }
    ],
    "type" : "illegal_argument_exception",
    "reason" : "failed to build synonyms",
    "caused_by" : {
      "type" : "parse_exception",
      "reason" : "Invalid synonym rule at line 1912",
      "caused_by" : {
        "type" : "illegal_argument_exception",
        "reason" : "term: Content management analyzed to a token (content management systems) with position increment != 1 (got: 0)"
      }
    }
  },
  "status" : 400
}

When making a call to es.indices.create call, I get the following error message:

RequestError(400, 'illegal_argument_exception', 'failed to build synonyms')

, which from the stack trace, seems to be coming from connection/base.py at line 322. Unfortunately, the "additional info" argument is not enough for drilling into the aforementioned detail that comes with Kibana.

I'm trying to synchronize several massive synonyms files, and have some additional logic that would take the corresponding, troublesome token (in the above case: "Content management"), and join associated skills into one line to avoid running into this error again. I would like to automatically handle these types of errors, parse out the token from error['caused_by']['caused_by']['reason'] in the error response, and retry a es.indices.create call to keep retrying the PUT until all synonym errors are cleared.

I would do this manually, but I'm dealing with 100's to 1000's of conflicts in my synonyms, so a manual update is overly cumbersome.

In reviewing the exceptions.py file in this repo, I see that some of the other error types have more robust messaging built-in. It would be super helpful for more support here.

[vsquared10]

While looking more into the source code, I discovered the entire error message is, in fact, included in the TransportError. For a RequestError in particular, I was able to capture the full error content as normally in Kibana by accessing the info property of the RequestError:

from elasticsearch import RequestError

try:
    es.indices.create('index_11', body=<some_error_generating_json>)
except RequestError as e:
    print(e.info['error']['caused_by']['caused_by']['reason'])

This results in a more verbose error message:

term: Content management analyzed to a token (content management systems) with position increment != 1 (got: 0)

From there, I can parse out the necessary text for my purposes.

I would still suggest revisiting the verbosity of the error handling here as debugging with ES is already somewhat troublesome at times. Including the lower-level reason would help in the development process/googling for answers 😄

I was able to test out changes like this with 3 edits to exceptions.py in the __str__ method of the TransportError class

[florianvazelle]

Any updates on this subjects ?

I would also like to have all the causes of the error in the exception, it would help a lot for debugging.

I would be happy to work on this feature.

(This ticket is related to elastic/elasticsearch-dsl-py#1367.)