Improve startup logging

README.md

@@ -21,6 +21,15 @@ Place a `config.yml` file in the cwd, or specify a config file with `-c`.
 
     rtpl -c /etc/retemplate.yml
 
+rtpl also supports the `-l` or `--logfile` option, which sets the initial log file for the software.
+A very small number of startup messages are logged before the log configuration is processed. On
+Python versions prior to 3.8, the `basicConfig` function does not accept the `force` option, which
+allows rtpl to reconfigure an existing logger. So if you run Python 3.8 or later, you can just
+configure logging in the config file, and you'll only lose a couple mostly insignificant messages.
+On older Pythons, all logging will be lost if you don't establish the filename at launch time. Older
+Pythons also will not allow custom log formats, etc. **It is strongly recommended that you run
+Retemplate on Python 3.8 or later.**
+
 ## Use Case
 Let's say you have an instance of [PyHiAPI](https://github.com/ryanjjung/pyhiapi) running out of `/opt/hiapi` and it is kept alive by a [supervisord](http://supervisord.org/) configuration. Furthermore, let's say the message returned by HiAPI should match the value stored in a Redis server in a key called `hiapi.message`. So you feed `-c /opt/hiapi/config.txt` to hiapi in your supervisord config so it cares about the content of that file. Then you configure Retemplate to generate that file based on a template. You might create a template at `/etc/retemplate/hiapi.config.j2` that looks like this:
 

rtpl

@@ -23,6 +23,9 @@ def parseargs():
     parser.add_argument('-c', '--config',
                         help='Config file to read in',
                         default='config.yml')
+    parser.add_argument('-l', '--logfile',
+                        help='The file to direct log output to (see readme for details)',
+                        default=None)
     parser.add_argument('-v', '--values',
                         help='Log all values retrieved from data stores',
                         default=False,
@@ -36,30 +39,40 @@ def main():
     '''
 
     args = parseargs()
+
+    # First set up basic logging so we can emit events before the user's custom configs can be read
+    logcfg = {
+        'level': 'INFO',
+        'filename': args.logfile,
+        'format': '%(levelname)s %(asctime)s %(message)s',
+        'style': '%'
+    }
+
+    # Forcible reconfiguration of live logging is only available after Python 3.8
+    if sys.version_info.minor >= 8:
+        logcfg['force'] = True
+    logging.basicConfig(**logcfg)
+
+    logging.info('Starting up Retemplate')
     config = dict()
     stores = dict()
     templates = list()
     threads = list()
 
     # Parse the config file
+    logging.info('Parsing config file at {}'.format(args.config))
     try:
         with open(args.config, 'r') as fh:
             config = yaml.load(fh.read(), Loader=yaml.FullLoader)
     except IOError:
         logging.error('Could not read config file: {}'.format(args.config))
         sys.exit(1)
 
-    # Configure logging
-    logcfg = {
-        'level': 'INFO',
-        'filename': None,
-        'format': '%(levelname)s %(asctime)s %(message)s',
-        'style': '%'
-    }
-    if 'retemplate' in config and 'logging' in config['retemplate']:
+    # Update logging config based on user configs if Python allows it
+    if 'retemplate' in config and 'logging' in config['retemplate'] and sys.version_info.minor >= 8:
         logcfg.update(config['retemplate']['logging'])
         logging.basicConfig(**logcfg)
-    logging.debug('Logging configured')
+        logging.debug('Logging configured')
 
     # Configure DataStores
     for store in config['stores']:
@@ -94,5 +107,7 @@ def main():
     for thread in threads:
         thread.join()
 
+    logging.info('Shutting down Retemplate')
+
 if __name__ == '__main__':
     main()

setup.py

@@ -2,7 +2,7 @@
 
 setup(
     name='retemplate',
-    version='0.0.8',
+    version='0.0.9',
     description="A module to execute a Jinja template on a schedule, supporting several backends for value storage",
     url='https://github.com/ryanjjung/retemplate',
     author='Ryan Jung',