[DROP] Added listener command group to pywbemcli

[andy-maier]

DROP: This PR has been abandoned in favor of PR #971, but we keep it around until that PR is merged.

Initial stab at listener support. This PR adds a 'listener' command group to pywbemcli. At this point, the listener does nothing with any received indications.

Ready for a concept review and early test.

TODO:

• Add documentation
• Add tests
• Add support for using the general logging options to log the output of 'listener run'. (see discussion item, below)
• Change --certfile and --keyfile to use the corresponding global options instead. (see discussion item, below)
• Add support for processing indications: display (subject to being logged), log to file (separate from logging), call Python function. (see discussion item, below)

DISCUSSION:

• How to treat stdout/stderr of the running listeners. Should they log to a file?
  KS/COMMENT: I think we have a real discussion about how we log/display/count, etc. received indications.
  Result: We go with the general logging options, for now.
  Status: OPEN

• The general options --certfile, --keyfile and --ca-certs could be used instead of adding similar options at the level of the listener run/start commands.
  Status: OPEN

• What types of indication processing would we support: display, log to file, call Python function, execute command?
  Result: For now, we do: display (subject to being logged), log to file (separate from logging), call Python function.
  Status: OPEN

• Default listener protocol? (currently http)
  KS/COMMENT: This one does not really bother me other than the default for the server is https so this is "different".
  Andy: Changed the default to https for security by default and for consistency.
  Result: Agreed to https as a default.
  Status: DONE

• Default listener port? (currently 5988)
  KS/COMMENT: I think that Jim did set it up that way and certainly we did not claim any standard port. However 5988 always bothered me because there was always the real chance that the server was on the same system. That is why I always pick some high number like 50000.
  Andy: Changed to 25989 to have a more unique value with less chance of collisions.
  Result: Agreed to that port as a default.
  Status: DONE

• Need for a certificate trust store: Somehow the single (!) server certificate to be presented during SSL/TLS handshake needs to be determined. How would that be done with multiple certificates in a trust store? The underlying WBEMListener class takes certfile and keyfile and it passes them on to ssl.SSLContext.load_cert_chain(). That is straight forward. What would a trust store improve?
  KS/COMMENTS: Our listener may be more primitive but the listener should be capable of handling certs from a CA tree, possibly even multiple CAs since, in all probably the same cert store as the server(s) with which it is defining subscriptions. For our listener we need to determine how much of that we want to do since we do not have an implemented multiple certificate store that we can simply attach to. I should have recognized this much earlier in the project but (to be honest) I tried to avoid the whole certificate subject as long as possible (i.e. until I had to fix pegasus).
  Andy: This does not answer the question why a server would need more than one server certificate?
  Result: We agree that for the server certificate presented by the listener, only one certificate is needed. However, when supporting mutual TLS, a set of CA certs is needed, which Karl meant with "trust store". This item is now about adding support for mutual TLS verification.
  Status: DEFERRED to issue Add support for mutual TLS to listener commands #965

• Do we want a send or test command? KS: I like the idea of an indication sender that we can use in tests but is part of pywbemcli.
  Result: For now, we do only "send". It is not clear what the addiitonal value of test would be given that we cannot really test the indication was processed.
  Status: DEFERRED to issue Add 'listener send' command #966

• Do we want to show the general options in the 'show' command? They are not generically available but are properties on the context object.
  Result: We want to show them. We have to, if we use the general options --certfile etc. It is also easier than I thought because they come from parsing the command line of the run commands, not from the context of the currently running list/show command.
  Status: DEFERRED to issue Show general options in 'listener show' command #967

• Does the WBEMListener class need to be extended with:

  • support for keyfile password? (see ssl.SSLContext.load_cert_chain())
    Result: Yes
    Status: DONE - it already did that, by letting OpenSSL perform the prompt. PR Improved the running of indication listeners when using HTTPS pywbem#2695 improves that a little.
  • support for restricting version of TLS protocol? (see Protocol versions)
    Result: Yes
    Status: DEFERRED to issue Add support to listener run for restricting TLS protocol version and ciphers #964
  • support for setting the ciphers? (see Cipher selection)
    Result: Yes
    Status: DEFERRED to issue Add support to listener run for restricting TLS protocol version and ciphers #964

• Idea: During startup of the "run" command, have its stdout/err/in redirected to/from the parent process, in order to get success and error messages in the parent process, and also the potential prompt for keyfile password. Needs investigation.
  Status: DONE - all messages of the run process are now displayed on the correct stream before the start process terminates, and the start process exit code is also correct (i.e. shows error if the child process had an error). The password prompt is also properly displayed and is not subject to redirection.

[coveralls]

Coverage decreased (-4.8%) to 88.108% when pulling 6d7d15d on andy/add-listener-commands into e66b1b0 on master.

[KSchopmeyer]

Made some particular comments and a couple of general thoughts. Generally I like this. Fairly simple and straight forward but covers the options.

[KSchopmeyer]

Made first overview review but have not yet run tests against OpenPegasus. Will do that Wednesday.

[andy-maier]

I resolved most of your comments, some by adding an item to the TODO list. Please review that, and particularly the comments not yet marked as resolved. Also, please review the new help text of the listener group again.

Also, I added support for honoring the output format in the show and list commands, and removed the general options field from the output since there is no generic way to get to the general options (they can be accessed one by one as context attributes).

[andy-maier]

DISCUSSION: Should we move the listener functionality into a separate command?

A major consideration for this are the general options. It turns out the majority of options of pywbemcli is only for targeting a WBEM server, or has different meaning (caused by opposite direction of server/client role in TLS handshake).

Options only for targeting a WBEM server:

  -n, --name NAME
  -m, --mock-server FILE
  -s, --server URL
  -u, --user TEXT
  -p, --password TEXT
  -t, --timeout INT
  -U, --use-pull [yes|no|either]
  --pull-max-cnt INT
  -d, --default-namespace NAMESPACE
  -T, --timestats / --no-timestats
  -C, --connections-file FILE PATH

Common options with different meanings:

  --verify / --no-verify
  --ca-certs CACERTS
  -c, --certfile FILE
  -k, --keyfile FILE

Common options with same meaning:

  -o, --output-format FORMAT
  -l, --log COMP[=DEST[:DETAIL]],...
  -v, --verbose / --no-verbose
  --warn / --no-warn
  --pdb
  --version
  -h, --help

[andy-maier]

DISCUSSION: If a new 'pywbemlistener' command is the direction (see PR #971), we can abandon this PR.