improve ByteUtil printable char validation of multi-byte charset #1001
Conversation
fbruton
changed the title
| * @param bytes the bytes to be scanned. | ||
| * @return whether or not there were non-printable values. | ||
| * @param utf8Bytes the bytes to be scanned. | ||
| * @return regardless of whether there were non-printable values. |
There was a problem hiding this comment.
@return true if the byte array contains non-printable bytes as defined as .....
| @Test | ||
| void testHasnonPrintableValues() { | ||
| String newLineCarriageTab = "\tThis is line one\r\nThis is line two\nThis is line three\n\nEnding with a tab"; | ||
| assertFalse(ByteUtil.hasNonPrintableValues(newLineCarriageTab.getBytes(StandardCharsets.UTF_8))); |
There was a problem hiding this comment.
It's possible there is more than one definition, but I thought non-printing characters were tab, newline, etc. Maybe I need more concise definition of what we're trying to identify.
There was a problem hiding this comment.
Also, is it supposed to end with a tab? \t?
There was a problem hiding this comment.
Maybe it should be called binary or valid text.
There was a problem hiding this comment.
The previous code allowed tabs, new lines and carriage returns, which makes sense.
There was a problem hiding this comment.
Also, is it supposed to end with a tab? \t?
The tab was at the beginning of the sentence, and now it aligns with the description.
|
I'm thinking we should merge the ideas from this pr and #1006 |
Planning to add additional unit-tests.