UnicodeDecodeError raised by log view

[kmierzeje]

UnicodeDecodeError at /admin/logs/get-log-line/1/ 'ascii' codec can't decode byte

Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/usr/local/lib/python3.8/site-packages/django/core/handlers/base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/usr/local/lib/python3.8/site-packages/django/core/handlers/base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/local/lib/python3.8/site-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view
return view_func(request, *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/logtailer/views.py", line 56, in get_log_lines
content = get_history(file, history)
File "/usr/local/lib/python3.8/site-packages/logtailer/views.py", line 31, in get_history
data.append(f.read(buffer_size))
File "/usr/lib64/python3.8/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]

The error makes it impossible to view log.
Please make the encoding configurable is settings or in LogFile model or most preferably in both places.

Even if correct encoding is configured, the error might still appear if log contains invalid codepoints. logtailer should be resilient to such characters - replace them with '?' or something...

Django Version: 3.0.8
Python Version: 3.8.3

[fireantology]

Can you upload a log file that reproduces the error ?

Thanks
Regards

[kmierzeje]

test.log

I reproduced it on Windows, so default codec is cp1250 not ascii, but effect is the same. I got a different call stack:

UnicodeDecodeError: 'charmap' codec can't decode byte 0x98 in position 5: character maps to

Traceback (most recent call last):
File "C:\apps\Python37\lib\site-packages\django\core\handlers\exception.py", line 47, in inner
response = get_response(request)
File "C:\apps\Python37\lib\site-packages\django\core\handlers\base.py", line 179, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "C:\apps\Python37\lib\site-packages\django\contrib\auth\decorators.py", line 21, in _wrapped_view
return view_func(request, *args, **kwargs)
File "C:\apps\Python37\lib\site-packages\logtailer\views.py", line 64, in get_log_lines
for line in file:
File "C:\apps\Python37\lib\encodings\cp1250.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]