How to represent "ANY IDENTIFIED BY ..."

[NathanReb]

First of all I'd like to thank you for your work, asn1-combinators is a great library and
I'm using it every day.

At cryptosense we're open sourcing some the of the asn1 cryptographic key parsers we've wrote
so far and we're facing an issue.

I'm trying to express an ASN grammar for algorithmIdentifier which is defined in
RFC 5280 as:

AlgorithmIdentifier  ::=  SEQUENCE  {
    algorithm               OBJECT IDENTIFIER,
    parameters              ANY DEFINED BY algorithm OPTIONAL  }

When only considering RSA and DSA, it works well because RSA params are NULL and DSA ones are
a 3 integer sequence. The thing is when adding EC keys to the equation the grammar becomes
ambiguous because EC parameters can also be a sequence, even though it's completely
different from the DSA one.

Currently we have a different grammar for each key types and the generic decode function
is a very ugly doubly nested try ... with Asn.Parse_error _ ->.

I wrote a wildcard grammar with choice and fix allowing me to accept "almost" everything
but then I have to manipulate the recursive type and to write converters from and to it which
adds a lot of code and doesn't really look prettier in the end.

ASN.1 construction like:

Something ::= SEQUENCE {
    id                  OID,
    params              ANY IDENTIFIED BY id }

is pretty usual and I guess I'm not the only one that could run into this kind of problem.

I'd really like to know what you suggest to adress this issue.

Thank you :)

[hannesm]

I'd use a choice and postprocessing, similar to how we do it in X.509: https://github.com/mirleft/ocaml-x509/blob/master/lib/asn_grammars.ml#L332-L418 (no EC support there yet). Maybe the parsing code (and esp. the ASN OID registry) should be a separate OPAM package, so that we do not have to duplicate this information in all the packages?

[pqwy]

@NathanReb

I see what you mean. You cannot encode different params as CHOICE, because CHOICE components have to be internally disambiguated, whereas ANY IDENTIFIED BY doesn't.

I had a combinator in mind would would allow expressing this cleanly. I'll hack it in the next few days, and get back to you.

In the meantime, would you mind checking how does the other branch work for you? The parser changed and has different performance tradeoffs, but that's the branch I intend to continue working from.

[NathanReb]

Sure ! I'll give it a try

[NathanReb]

Ok so I installed the new asn1-combinators you gave me yesterday. I've tried it a bit on our open
source lib and it works fine.
We're currently still using some old parsers, waiting for the elliptic curve key part of
the key-parsers lib to be released so I ran the Cryptosense Analyzer test suite.
I found out that one of the test now fails. The good thing is it fails because of a
mistake within an EC private key grammar.
The question is why doesn't it fail with the current asn1-combinators version ?

The following is the base64 encoding of the DER-encoded EC private key we use as test
data:

MHQCAQEEIB7LbYFLdWg6VfF9/Xnc8AAOSoWJiB+5qtQvwtC2NCU/oAcGBSuBBAAKoUQDQgAE2RHVNu5/Opj8LzhZ+V4FoWM7JyzMMo3surH1iTOArw3GSVuCYaWYKhTBaUfiiy2IRD/HlHsCI1xBFtwUvrGCKw==

This is how the grammar was expressed:

sequence4
  (required ~label:"version" asn_version)
  (required ~label:"privateKey" octet_string)
  (optional ~label:"parameters" @@ explicit 0 oid)
  (optional ~label:"publicKey" @@ explicit 1 octet_string)

The test passes with the current version and fails with the one you gave me.

And this is how it should be:

sequence4
  (required ~label:"version" asn_version)
  (required ~label:"privateKey" octet_string)
  (optional ~label:"parameters" @@ explicit 0 oid)
  (optional ~label:"publicKey" @@ explicit 1 bit_string_cs)

This passes the test with both the current and the future version.

[NightBlues]

@NathanReb

I see what you mean. You cannot encode different params as CHOICE, because CHOICE components have to be internally disambiguated, whereas ANY IDENTIFIED BY doesn't.

I had a combinator in mind would would allow expressing this cleanly. I'll hack it in the next few days, and get back to you.

In the meantime, would you mind checking how does the other branch work for you? The parser changed and has different performance tradeoffs, but that's the branch I intend to continue working from.

Guys, I'm trying to implement PKCS12 (https://tools.ietf.org/html/rfc7292) and I ran into the same problem:

 SafeBag ::= SEQUENCE {
     bagId         BAG-TYPE.&id ({PKCS12BagSet}),
     bagValue      [0] EXPLICIT BAG-TYPE.&Type({PKCS12BagSet}{@bagId}),
     bagAttributes SET OF PKCS12Attribute OPTIONAL
 }
...
 keyBag BAG-TYPE ::=
     {KeyBag              IDENTIFIED BY {bagtypes 1}}
 certBag BAG-TYPE ::=
     {CertBag             IDENTIFIED BY {bagtypes 3}}
...
-- KeyBag
 KeyBag ::= PrivateKeyInfo
-- CertBag
 CertBag ::= SEQUENCE {
...

KeyBag and CertBag are both sequences and I tried to implement them through choice.
How do you hack it?

[hannesm]

FWIW, I took a look on PKCS12 some time ago at mirleft/ocaml-x509#114 and avoided the ANY issue (though the asn1 looks pretty ugly)

[NightBlues]

Oh, it would be better for me, if I had found this PR before started :) (NightBlues/ocaml-x509#1)
Thank you, I'll steal some code from your PR:)