decodeBase64 doesn't decode valid values

[skoshx]

What version of Effect is running?

3.7.0

What steps can reproduce the bug?

const base64 = `bmFtZTogYmVuY2gKCm9uOgogIHB1bGxfcmVxdWVzdDoKICAgIGJyYW5jaGVz
OgogICAgICAtIG1haW4KICBwdXNoOgogICAgYnJhbmNoZXM6CiAgICAgIC0g
bWFpbgoKam9iczoKICBsaW50OgogICAgcnVucy1vbjogdWJ1bnR1LWxhdGVz
dAoKICAgIHN0ZXBzOgogICAgICAtIHVzZXM6IGFjdGlvbnMvY2hlY2tvdXRA
djMKICAgICAgLSB1c2VzOiBkZW5vbGFuZC9zZXR1cC1kZW5vQHYxCiAgICAg
ICAgd2l0aDoKICAgICAgICAgIGRlbm8tdmVyc2lvbjogdjEueAogICAgICAg
ICAgCiAgICAgIC0gbmFtZTog8J+PhyBSdW4gYmVuY2htYXJrcwogICAgICAg
IHJ1bjogZGVubyB0YXNrIGJlbmNoCg==`


import { decodeBase64 } from "effect/Encoding"
import { decodeBase64 as denoDecodeBase64 } from "@std/encoding"

// Produces Either.Left
console.log(decodeBase64(base64))

// Produces Uint8Array
console.log("deno decoded", denoDecodeBase64(base64))

Outputs:

{
  _id: 'Either',
  _tag: 'Left',
  left: {
    _tag: 'DecodeException',
    input: 'bmFtZTogYmVuY2gKCm9uOgogIHB1bGxfcmVxdWVzdDoKICAgIGJyYW5jaGVz\n' +
      'OgogICAgICAtIG1haW4KICBwdXNoOgogICAgYnJhbmNoZXM6CiAgICAgIC0g\n' +
      'bWFpbgoKam9iczoKICBsaW50OgogICAgcnVucy1vbjogdWJ1bnR1LWxhdGVz\n' +
      'dAoKICAgIHN0ZXBzOgogICAgICAtIHVzZXM6IGFjdGlvbnMvY2hlY2tvdXRA\n' +
      'djMKICAgICAgLSB1c2VzOiBkZW5vbGFuZC9zZXR1cC1kZW5vQHYxCiAgICAg\n' +
      'ICAgd2l0aDoKICAgICAgICAgIGRlbm8tdmVyc2lvbjogdjEueAogICAgICAg\n' +
      'ICAgCiAgICAgIC0gbmFtZTog8J+PhyBSdW4gYmVuY2htYXJrcwogICAgICAg\n' +
      'IHJ1bjogZGVubyB0YXNrIGJlbmNoCg==',
    message: 'Length must be a multiple of 4, but is 459',
    [Symbol(effect/Encoding/errors/Decode)]: Symbol(effect/Encoding/errors/Decode)
  }
}
deno decoded Uint8Array(337) [
  110,  97, 109, 101,  58,  32,  98, 101, 110,  99, 104, 10,
   10, 111, 110,  58,  10,  32,  32, 112, 117, 108, 108, 95,
  114, 101, 113, 117, 101, 115, 116,  58,  10,  32,  32, 32,
   32,  98, 114,  97, 110,  99, 104, 101, 115,  58,  10, 32,
   32,  32,  32,  32,  32,  45,  32, 109,  97, 105, 110, 10,
   32,  32, 112, 117, 115, 104,  58,  10,  32,  32,  32, 32,
   98, 114,  97, 110,  99, 104, 101, 115,  58,  10,  32, 32,
   32,  32,  32,  32,  45,  32, 109,  97, 105, 110,  10, 10,
  106, 111,  98, 115,
  ... 237 more items
]

As you can see, Deno's standard library implementation correctly returns, where the effect/Encoding doesn't.

What is the expected behavior?

Expected to decode the string correctly.

What do you see instead?

left: {
_tag: 'DecodeException',
input: 'bmFtZTogYmVuY2gKCm9uOgogIHB1bGxfcmVxdWVzdDoKICAgIGJyYW5jaGVz\n' +
'OgogICAgICAtIG1haW4KICBwdXNoOgogICAgYnJhbmNoZXM6CiAgICAgIC0g\n' +
'bWFpbgoKam9iczoKICBsaW50OgogICAgcnVucy1vbjogdWJ1bnR1LWxhdGVz\n' +
'dAoKICAgIHN0ZXBzOgogICAgICAtIHVzZXM6IGFjdGlvbnMvY2hlY2tvdXRA\n' +
'djMKICAgICAgLSB1c2VzOiBkZW5vbGFuZC9zZXR1cC1kZW5vQHYxCiAgICAg\n' +
'ICAgd2l0aDoKICAgICAgICAgIGRlbm8tdmVyc2lvbjogdjEueAogICAgICAg\n' +
'ICAgCiAgICAgIC0gbmFtZTog8J+PhyBSdW4gYmVuY2htYXJrcwogICAgICAg\n' +
'IHJ1bjogZGVubyB0YXNrIGJlbmNoCg==',
message: 'Length must be a multiple of 4, but is 459',
[Symbol(effect/Encoding/errors/Decode)]: Symbol(effect/Encoding/errors/Decode)
}

Additional information

No response

[fubhy]

We do not currently support newline characters in base64 strings. The spec technically allows for newline characters but does not require implementations to support those. It specifically states that that's a a decision of the implementation: https://datatracker.ietf.org/doc/html/rfc4648#section-3.3

Hence, this is not a bug and instead a matter of choice. A lot of implementations ignore \n and \r\n though so maybe we should do too.

In order to fix your problem here you can just do this for now:

decodeBase64(base64.replaceAll("\n", ""))