-
Notifications
You must be signed in to change notification settings - Fork 129
Using the pycrate asn1 runtime
In this section, we will see how to use the ASN.1 runtime with already compiled specifications from the pycrate_asn1dir/ directory, with basic examples.
Check the few limitations that actually exist in the ASN.1 runtime on the ASN.1 compiler wiki page.
Each ASN.1 specification defines objects with specific structures made of the ASN.1 basic types (INTEGER, BIT STRING, ...) and constructed types (CHOICE, SEQUENCE, ...). The purpose of the ASN.1 compiler is to transform an ASN.1 specification to a target programming language and runtime. The ASN.1 runtime is then in charge, together with the compiled specification, to encode values for those ASN.1 objects to buffers, and decode buffers to values.
The pycrate ASN.1 runtime, as defined in pycrate_asn1rt, currently implements the following encoders / decoders:
- the ASN.1 textual syntax, with to_asn1() / from_asn1() methods
- the unaligned Packed Encoding Rules (UPER), with to_uper() / from_uper() methods
- the aligned Packed Encoding Rules (APER), with to_aper() / from_aper() methods
- the Basic Encoding Rules (BER), with to_ber() / from_ber() methods
- the Canonical Encoding Rules (CER), with to_cer() / from_cer() methods
- the Distinguished Encoding Rules (DER), with to_der() / from_der() methods
When decoding a bytes' buffer for a given object with one of those from_() methods, or setting a value in it with the set_val() method, the value is put in the _val attribute of the object. We must know however how the pycrate runtime handle values for each basic and constructed ASN.1 objects. All those information are detailed in the doc strings of each objects, here is a summary about each ASN.1 type and the corresponding Python type for values expected or returned by the runtime:
- NULL: int value 0
- BOOLEAN: bool
- INTEGER: int
- REAL: 3-tuple of int (mantissa, base, exponent)
- ENUMERATED: str, must be a key in the enumerated content (available in the _cont attribute)
- OBJECT IDENTIFIER: tuple of positive int
- RELATIVE-OID: tuple of positive int
- BIT STRING: 2-tuple of positive int (bit string uint value, bit string length), alternatively a CHOICE-like value in case a CONTAINING object is defined in the _const_cont attribute
- OCTET STRING: bytes, alternatively a CHOICE-like value in case a CONTAINING object is defined in the _const_cont attribute
- *String (UTF8String, IA5String, ...): str
- UTCTime: 7-tuple of str or None (YY, MM, DD, HH, MM, [SS,] Z)
- GeneralizedTime: 8-tuple of str or None (YYYY, MM, DD, HH, [MM, [SS,]] [{.,}F*,] [Z])
- CHOICE: 2-tuple, 1st item is a str (identifier of the choice as defined in the _cont attribute), 2nd item is the chosen object's value
- SEQUENCE, SET: dict, keys are str (identifier of the component as defined in the _cont attribute), values are component's values
- SEQUENCE OF, SET OF: list of component's values (as defined in the the _cont attribute)
- OPEN, ANY: bytes, alternatively a CHOICE-like value in case some constraints are defined (use _get_const_tr() method to list all potential objects in those constraints)
- EXTERNAL, EMBEDDED PDV and CHARACTER STRING are just defined like SEQUENCE objects
While encoding and decoding ASN.1 values, we are converting abstract values to bytes' buffer and vice versa: we will see in the several examples below how to use the to_[bcdp]er() and from_[bcdp]er() methods to make such encoding, respectively decoding. There is also the possibility to generate the exact binary structures corresponding to those encoding and decoding, making use of basic pycrate core objects (mostly Int, Uint, Buf and Envelope). This is done by adding the _ws suffixes to encoding and decoding methods: to_[bcdp]er_ws() and from_[bcdp]er_ws().
This will generate a master Envelope object under the attribute _struct of the encoded or decoded object, which will demonstrate the exact structure of the transfer syntax. Here is an example with the TCAP-MAP case:
>>> from pycrate_asn1dir import TCAP_MAP
>>> M = TCAP_MAP.TCAP_MAP_Messages.TCAP_MAP_Message
>>> M
<TCAP-MAP-Message ([TCMessage] CHOICE)>
>>> from binascii import hexlify, unhexlify
>>> # this is the way to go for a simple decoding:
>>> M.from_ber(unhexlify('626a48042f3b46026b3a2838060700118605010101a02d602b80020780a109060704000001001302be1a2818060704000001010101a00da00b80099656051124006913f66c26a12402010102013b301c04010f040eaa180da682dd6c31192d36bbdd468007917267415827f2'))
>>> # this is the way to go if we want to additionnally generate the binary structure:
>>> M.from_ber_ws(unhexlify('626a48042f3b46026b3a2838060700118605010101a02d602b80020780a109060704000001001302be1a2818060704000001010101a00da00b80099656051124006913f66c26a12402010102013b301c04010f040eaa180da682dd6c31192d36bbdd468007917267415827f2'))
>>> M() # we have the decoded value, as usual
(u'begin', {u'dialoguePortion': {u'direct-reference': (0, 0, 17, 773, 1, 1, 1), u'encoding': (u'single-ASN1-type', ('DialoguePDU', (u'dialogueRequest', {u'user-information': [{u'direct-reference': (0, 4, 0, 0, 1, 1, 1, 1), u'encoding': (u'single-ASN1-type', ('MAP-DialoguePDU', (u'map-open', {u'destinationReference': '\x96V\x05\x11$\x00i\x13\xf6'})))}], u'application-context-name': (0, 4, 0, 0, 1, 0, 19, 2), u'protocol-version': (1, 1)})))}, u'components': [(u'basicROS', (u'invoke', {u'opcode': (u'local', 59), u'argument': ('USSD-Arg', {u'ussd-DataCodingScheme': '\x0f', u'msisdn': "\x91rgAX'\xf2", u'ussd-String': '\xaa\x18\r\xa6\x82\xddl1\x19-6\xbb\xddF'}), u'invokeId': (u'present', 1)}))], u'otid': '/;F\x02'})
>>> print(M.to_asn1())
begin : {
otid '2F3B4602'H,
dialoguePortion {
direct-reference {0 0 17 773 1 1 1} -- dialogue-as-id --,
encoding single-ASN1-type : DialoguePDU: dialogueRequest : {
protocol-version '1'B -- version1 --,
application-context-name {0 4 0 0 1 0 19 2} -- networkUnstructuredSsContext-v2 --,
user-information {
{
direct-reference {0 4 0 0 1 1 1 1} -- map-DialogueAS --,
encoding single-ASN1-type : MAP-DialoguePDU: map-open : {
destinationReference '9656051124006913F6'H
}
}
}
}
},
components {
basicROS : invoke : {
invokeId present : 1,
opcode local : 59,
argument USSD-Arg: {
ussd-DataCodingScheme '0F'H,
ussd-String 'AA180DA682DD6C31192D36BBDD46'H,
msisdn '917267415827F2'H
}
}
}
}
>>> M._struct # we have additionnally the binary structure
<TCAP-MAP-Message : <T : <Class : 1 (APPLICATION)> [...] >
>>> print(M._struct.show())
### TCAP-MAP-Message ###
### T ###
<Class : 1 (APPLICATION)>
<PC : 1 (constructed)>
<Val : 2>
### L ###
<Form : 0 (short)>
<Val : 106>
### V ###
### otid ###
### T ###
<Class : 1 (APPLICATION)>
<PC : 0 (primitive)>
<Val : 8>
### L ###
<Form : 0 (short)>
<Val : 4>
<V : 0x2f3b4602>
### dialoguePortion ###
### T ###
<Class : 1 (APPLICATION)>
<PC : 1 (constructed)>
<Val : 11>
### L ###
<Form : 0 (short)>
<Val : 58>
### V ###
### T ###
<Class : 0 (UNIVERSAL)>
<PC : 1 (constructed)>
<Val : 8 (INSTANCE OF, EXTERNAL)>
### L ###
<Form : 0 (short)>
<Val : 56>
### V ###
### direct-reference ###
### T ###
<Class : 0 (UNIVERSAL)>
<PC : 0 (primitive)>
<Val : 6 (OBJECT IDENTIFIER)>
### L ###
<Form : 0 (short)>
<Val : 7>
<V : '\x00\x11\x86\x05\x01\x01\x01'>
### encoding ###
### T ###
<Class : 2 (CONTEXT-SPECIFIC)>
<PC : 1 (constructed)>
<Val : 0>
### L ###
<Form : 0 (short)>
<Val : 45>
### Type ###
### T ###
<Class : 1 (APPLICATION)>
<PC : 1 (constructed)>
<Val : 0>
### L ###
<Form : 0 (short)>
<Val : 43>
### V ###
### protocol-version ###
### T ###
<Class : 2 (CONTEXT-SPECIFIC)>
<PC : 0 (primitive)>
<Val : 0>
### L ###
<Form : 0 (short)>
<Val : 2>
### V ###
<BU : 7>
<BS : 0x80>
### application-context-name ###
### T ###
<Class : 2 (CONTEXT-SPECIFIC)>
<PC : 1 (constructed)>
<Val : 1>
### L ###
<Form : 0 (short)>
<Val : 9>
### V ###
### T ###
<Class : 0 (UNIVERSAL)>
<PC : 0 (primitive)>
<Val : 6 (OBJECT IDENTIFIER)>
### L ###
<Form : 0 (short)>
<Val : 7>
<V : '\x04\x00\x00\x01\x00\x13\x02'>
### user-information ###
### T ###
<Class : 2 (CONTEXT-SPECIFIC)>
<PC : 1 (constructed)>
<Val : 30>
### L ###
<Form : 0 (short)>
<Val : 26>
### V ###
### _item_ ###
### T ###
<Class : 0 (UNIVERSAL)>
<PC : 1 (constructed)>
<Val : 8 (INSTANCE OF, EXTERNAL)>
### L ###
<Form : 0 (short)>
<Val : 24>
### V ###
### direct-reference ###
### T ###
<Class : 0 (UNIVERSAL)>
<PC : 0 (primitive)>
<Val : 6 (OBJECT IDENTIFIER)>
### L ###
<Form : 0 (short)>
<Val : 7>
<V : '\x04\x00\x00\x01\x01\x01\x01'>
### encoding ###
### T ###
<Class : 2 (CONTEXT-SPECIFIC)>
<PC : 1 (constructed)>
<Val : 0>
### L ###
<Form : 0 (short)>
<Val : 13>
### Type ###
### T ###
<Class : 2 (CONTEXT-SPECIFIC)>
<PC : 1 (constructed)>
<Val : 0>
### L ###
<Form : 0 (short)>
<Val : 11>
### V ###
### destinationReference ###
### T ###
<Class : 2 (CONTEXT-SPECIFIC)>
<PC : 0 (primitive)>
<Val : 0>
### L ###
<Form : 0 (short)>
<Val : 9>
<V : 0x9656051124006913f6>
### components ###
### T ###
<Class : 1 (APPLICATION)>
<PC : 1 (constructed)>
<Val : 12>
### L ###
<Form : 0 (short)>
<Val : 38>
### V ###
### _item_ ###
### T ###
<Class : 2 (CONTEXT-SPECIFIC)>
<PC : 1 (constructed)>
<Val : 1>
### L ###
<Form : 0 (short)>
<Val : 36>
### V ###
### invokeId ###
### T ###
<Class : 0 (UNIVERSAL)>
<PC : 0 (primitive)>
<Val : 2 (INTEGER)>
### L ###
<Form : 0 (short)>
<Val : 1>
<V : 1>
### opcode ###
### T ###
<Class : 0 (UNIVERSAL)>
<PC : 0 (primitive)>
<Val : 2 (INTEGER)>
### L ###
<Form : 0 (short)>
<Val : 1>
<V : 59>
### argument ###
### T ###
<Class : 0 (UNIVERSAL)>
<PC : 1 (constructed)>
<Val : 16 (SEQUENCE, SEQUENCE OF)>
### L ###
<Form : 0 (short)>
<Val : 28>
### V ###
### ussd-DataCodingScheme ###
### T ###
<Class : 0 (UNIVERSAL)>
<PC : 0 (primitive)>
<Val : 4 (OCTET STRING)>
### L ###
<Form : 0 (short)>
<Val : 1>
<V : 0x0f>
### ussd-String ###
### T ###
<Class : 0 (UNIVERSAL)>
<PC : 0 (primitive)>
<Val : 4 (OCTET STRING)>
### L ###
<Form : 0 (short)>
<Val : 14>
<V : 0xaa180da682dd6c31192d36bbdd46>
### msisdn ###
### T ###
<Class : 2 (CONTEXT-SPECIFIC)>
<PC : 0 (primitive)>
<Val : 0>
### L ###
<Form : 0 (short)>
<Val : 7>
<V : 0x917267415827f2>
You should not use those extended methods for standard ASN.1 processing, but only for debugging purpose, as they are making the encoding and decoding process much slower. You have been warned !
In the UMTS and LTE cellular technologies, the Radio Ressources Configuration protocol used between handsets and the radio access network is defined in ASN.1. It makes use of the unaligned Packed Encoding Rules (UPER) to serialize the data exchanged according to the defined data model.
Pycrate provides two different modules to handle both protocols:
- the UMTS RRC protocol is defined in the TS 25.331 3GPP specification; the ASN.1 definition is available in the directory pycrate_asn1dir/3GPP_UTRAN_RRC_25331/ and the corresponding Python modules are compiled in the file pycrate_asn1dir/RRC3G.py.
- the LTE RRC protocol is defined in the TS 36.331 3GPP specification; the ASN.1 definition is available in the directory pycrate_asn1dir/3GPP_EUTRAN_RRC_36331/ and the corresponding Python modules are compiled in the file pycrate_asn1dir/RRCLTE.py.
Both specifications provided are from the 3GPP release 13 (0xd), however, it is possible to easily upgrade or downgrade them thanks to the extract.py scripts provided in the ASN.1 definition directories (just read those scripts to see how to do exactly).
We can simply load the RRC3G Python module, which will load the corresponding ASN.1 objects and the ASN.1 runtime together:
>>> from pycrate_asn1dir import RRC3G # this can take few seconds
>>> list(RRC3G.GLOBAL.MOD.keys())
['Constant-definitions', 'InformationElements', 'PDU-definitions', 'Class-definitions', 'Internode-definitions']
The GLOBAL.MOD dictionnary contains all ASN.1 modules and objects defined in the 3GPP specification. The modules in GLOBAL.MOD are actually dictionnaries listing all objects defined in them. They are also directly available under the RRC3G module as classes and attributes, with a slight adaptation of their names (all - are replaced with _).
>>> RRC3G.GLOBAL.MOD['PDU-definitions']['System-Information-Container']
<System-Information-Container (SEQUENCE)>
>>> RRC3G.PDU_definitions.System_Information_Container == RRC3G.GLOBAL.MOD['PDU-definitions']['System-Information-Container']
True
Each ASN.1 object is of a certain ASN.1 type (e.g. INTEGER, OCTET STRING, SEQUENCE, ...), which has different attributes. Each ASN.1 object has a generous doc string which can be read to get all the tiny details about the available attributes.
>>> SIC = RRC3G.PDU_definitions.System_Information_Container
>>> help(SIC)
Help on SEQ in module pycrate_asn1rt.asnobj_construct object:
class SEQ(_CONSTRUCT)
| ASN.1 constructed type SEQUENCE object
|
| Single value: Python dict
| keys are Python str, components' identifier, must be key in _cont
| values are ASN1Obj single value specific to components object
[...]
One important attribute for constructed objects is the _cont one, which stores the content of the object:
>>> SIC._cont
{
mib: <mib (OCTET STRING)>,
sysInfoTypeSB1: <sysInfoTypeSB1 (OCTET STRING)>,
sysInfoTypeSB2: <sysInfoTypeSB2 (OCTET STRING)>,
sysInfoType1: <sysInfoType1 (OCTET STRING)>,
sysInfoType3: <sysInfoType3 (OCTET STRING)>,
sysInfoType5: <sysInfoType5 (OCTET STRING)>,
sysInfoType7: <sysInfoType7 (OCTET STRING)>,
sysInfoType11: <sysInfoType11 (OCTET STRING)>,
sysInfoType11bis: <sysInfoType11bis (OCTET STRING)>,
sysInfoType12: <sysInfoType12 (OCTET STRING)>,
vb50NonCriticalExtensions: <vb50NonCriticalExtensions (SEQUENCE)>
}
Here, only the first level of content is returned. The get_proto() method returns the complete content by entering each constructed content recursively. Be careful about some limitations of this method: its call on a recursive object will trigger a Python recursion exception... Moreover it does not provides hints on optional components (those marked OPTIONAL or with a DEFAULT value), neither on component in the root or extension part of the content.
>>> SIC.get_proto()
{
mib: 'OCTET STRING',
sysInfoTypeSB1: 'OCTET STRING',
sysInfoTypeSB2: 'OCTET STRING',
sysInfoType1: 'OCTET STRING',
sysInfoType3: 'OCTET STRING',
sysInfoType5: 'OCTET STRING',
sysInfoType7: 'OCTET STRING',
sysInfoType11: 'OCTET STRING',
sysInfoType11bis: 'OCTET STRING',
sysInfoType12: 'OCTET STRING',
vb50NonCriticalExtensions: {
system-Information-Container-vb50ext: {
sysInfoType22: 'OCTET STRING'
},
vc50NonCriticalExtensions: {
system-Information-Container-vc50ext: {
sysInfoType11ter: 'OCTET STRING'
},
nonCriticalExtensions: {}
}
}
}
Because the UMTS RRC protocol is using the UPER encoding, we will use the from_uper() method to deserialize buffers. The corresponding value will be available in the _val attribute, and will also be returned when calling the ASN.1 object itself. Finally, the to_asn1() method returns a printable representation of the ASN.1 value, which conforms to the ASN.1 syntax (and hence would be compilable again...).
>>> from binascii import unhexlify
>>> pcch = RRC3G.Class_definitions.PCCH_Message
>>> pcch.from_uper(unhexlify('4455c803999055c601b95855aa06b09e'))
>>> pcch()
{'message': ('pagingType1', {'pagingRecordList': [...]})}
>>> print(pcch.to_asn1())
{
message pagingType1 : {
pagingRecordList {
cn-Identity : {
pagingCause terminatingInteractiveCall,
cn-DomainIdentity ps-domain,
cn-pagedUE-Identity p-TMSI-GSM-MAP : 'E401CCC8'H
},
cn-Identity : {
pagingCause terminatingInteractiveCall,
cn-DomainIdentity ps-domain,
cn-pagedUE-Identity p-TMSI-GSM-MAP : 'E300DCAC'H
},
cn-Identity : {
pagingCause terminatingInteractiveCall,
cn-DomainIdentity ps-domain,
cn-pagedUE-Identity p-TMSI-GSM-MAP : 'D503584F'H
}
}
}
}
To serialize some values, one need to set the value into the selected ASN.1 object. The method set_val() is here for that purpose. Then, we can call the to_uper() method to serialize the value into a bytes' buffer. With the previous example about the PCCH message, we can take its value and change the first identity paging record to see the effect on the encoding:
>>> v = pcch()
>>> v['message'][1]['pagingRecordList'][0][1]
{'cn-pagedUE-Identity': ('p-TMSI-GSM-MAP', (3825323208, 32)), 'cn-DomainIdentity': 'ps-domain', 'pagingCause': 'terminatingInteractiveCall'}
>>> v['message'][1]['pagingRecordList'][0][1]['cn-pagedUE-Identity'] = ('p-TMSI-GSM-MAP', (0xf0000001, 32))
>>> v['message'][1]['pagingRecordList'][0][1]['cn-DomainIdentity'] = 'cs-domain'
>>> v['message'][1]['pagingRecordList'][0][1]['pagingCause'] = 'terminatingHighPrioritySignalling'
>>> pcch.set_val(v)
>>> hexlify(pcch.to_uper())
b'4485e000000255c601b95855aa06b09e'
We can also appreciate how the UPER encoding is more compact than the BER one. This is one of the main reason why it is used by many protocols transported over radio interfaces: it saves bandwidth.
>>> hexlify(pcch.to_ber())
b'3039a037a035a033a00f800104810100a207820500f0000001a00f800102810101a207820500e300dcaca00f800102810101a207820500d503584f'
As seen in the previous example, it is required to know the content of constructed objects to be able to access their internals, and also to be able to set values. Two utility functions are provided to help with that:
- get_obj_at(Obj, path)
- get_val_at(Obj, path)
Both take an ASN.1 object and the path to one of its internal component or value and return it. Here are some examples with the previous PCCH_Message object:
>>> pcch.get_proto()
{
message: {
pagingType1: {
pagingRecordList: [{
[...]
},
spare: 'NULL'
}
}
>>> get_obj_at(pcch, ['message', 'pagingType1', 'pagingRecordList', None, 'utran-Identity'])
<utran-Identity (SEQUENCE)>
>>> # pagingRecordList being a SEQUENCE OF, no need to provide any name to select its content
>>> get_val_at(pcch, ['message', 'pagingType1', 'pagingRecordList', 2])
('cn-Identity', {'pagingCause': 'terminatingInteractiveCall', 'cn-DomainIdentity': 'ps-domain', 'cn-pagedUE-Identity': ('p-TMSI-GSM-MAP', (3573766223, 32))})
>>> # this returns the 3rd identity from the paging message
Let's see another example with an LTE RRC protocol message. The following message is broadcasted by an LTE eNodeB within its downlink shared channel over the broadcast control channel, it contains a SIB2 which provides many parameters of the radio interface:
>>> from pycrate_asn1dir import RRCLTE
>>> from binascii import unhexlify, hexlify
>>> sch = RRCLTE.EUTRA_RRC_Definitions.BCCH_DL_SCH_Message
>>> sch.from_uper(unhexlify('00800c61bc8c8cc11609ba020004100193394c52d5425c700708518b613a9690'))
>>> print(sch.to_asn1())
{
message c1 : systemInformation : {
criticalExtensions systemInformation-r8 : {
sib-TypeAndInfo {
sib2 : {
radioResourceConfigCommon {
rach-ConfigCommon {
preambleInfo {
numberOfRA-Preambles n52
},
[...]
},
intraFreqCellReselectionInfo {
q-RxLevMin -61,
p-Max 23,
s-IntraSearch 5,
presenceAntennaPort1 TRUE,
neighCellConfig '01'B,
t-ReselectionEUTRA 1
}
}
}
}
}
}
>>> hexlify(sch.to_uper())
b'00800c61bc8c8cc11609ba020004100193394c52d5425c700708518b613a9690'
In order to understand the complexity of those RRC protocols, one can simply check the structure of a dedicated signaling channel downlink message (expand your scrollback buffer if you want to see it all!):
>>> RRCLTE.EUTRA_RRC_Definitions.DL_DCCH_Message.get_proto()
{
message: {
c1: {
csfbParametersResponseCDMA2000: {
rrc-TransactionIdentifier: 'INTEGER',
criticalExtensions: {
csfbParametersResponseCDMA2000-r8: {
rand: 'BIT STRING',
mobilityParameters: 'OCTET STRING',
nonCriticalExtension: {
lateNonCriticalExtension: 'OCTET STRING',
nonCriticalExtension: {}
}
},
criticalExtensionsFuture: {}
}
},
dlInformationTransfer: {
rrc-TransactionIdentifier: 'INTEGER',
criticalExtensions: {
c1: {
[...]
antennaInfoDedicatedPCell-r13: {
maxLayersMIMO-r10: 'ENUMERATED'
},
drb-ContinueROHC-r13: 'ENUMERATED',
lateNonCriticalExtension: 'OCTET STRING',
nonCriticalExtension: {}
},
spare3: 'NULL',
spare2: 'NULL',
spare1: 'NULL'
},
criticalExtensionsFuture: {}
}
},
spare3: 'NULL',
spare2: 'NULL',
spare1: 'NULL'
},
messageClassExtension: {}
}
}
The ASN.1 top-level messages are those which make reference to others ASN.1 objects, but are not referenced themselves. They are often corresponding to the RRC messages exchanged on the different transport channels over the radio interface between mobile phones and the radio access network.
The function get_top_level() from the *pycrate_asn1rt/utils.py" module works over the json files produced by the pycrate_asn1c compiler and returns those exact top-level objects from an ASN.1 specification.
For the LTE RRC specification, we get the following objects of the EUTRA-RRC-Definitions ASN.1 module:
- EUTRA-RRC-Definitions.BCCH-DL-SCH-Message
- EUTRA-RRC-Definitions.BCCH-DL-SCH-Message-BR
- EUTRA-RRC-Definitions.MCCH-Message
- EUTRA-RRC-Definitions.PCCH-Message
- EUTRA-RRC-Definitions.DL-CCCH-Message
- EUTRA-RRC-Definitions.UL-CCCH-Message
- EUTRA-RRC-Definitions.UL-DCCH-Message
- EUTRA-RRC-Definitions.SC-MCCH-Message-r13
- EUTRA-RRC-Definitions.UE-EUTRA-Capability
- EUTRA-RRC-Definitions.BCCH-BCH-Message
- EUTRA-RRC-Definitions.SL-TxPoolIdentity-r13
We can see that the ASN.1 object EUTRA-RRC-Definitions.DL-DCCH-Message is not a top-level one, it is actually referenced by object EUTRA-InterNodeDefinitions.HandoverCommand-r8-IEs. However, it is still to be used for decoding downlink RRC message on dedicated signaling channels.
For the UMTS RRC specification, we get the following ASN.1 top-level objects of the Class-definitions ASN.1 module:
- Class-definitions.DL-DCCH-Message
- Class-definitions.UL-DCCH-Message
- Class-definitions.DL-CCCH-Message
- Class-definitions.UL-CCCH-Message
- Class-definitions.PCCH-Message
- Class-definitions.DL-SHCCH-Message
- Class-definitions.UL-SHCCH-Message
- Class-definitions.BCCH-FACH-Message
- Class-definitions.BCCH-BCH-Message
- Class-definitions.BCCH-BCH2-Message
- Class-definitions.MCCH-Message
- Class-definitions.MSCH-Message
There are many protocols defined by the 3GPP to inter-connect equipments in radio access networks, and with core networks. In the 3G RAN specifications (TS 25 serie), we can find the following protocols corresponding to different signaling interfaces:
- NBAP used between a NodeB and an RNC
- RNSAP used to inter-connect two RNCs
- RANAP used between an RNC and a core network (both CS and PS domains core)
- PCAP used between an RNC and a location server (for positionning services)
- SABP used between an RNC and a cell-broadcasting server (for multicast / broadcast services)
- HNBAP and RUA used between a femtocell and a core network
- RNA used to inter-connect two femtocells
In the LTE RAN specifications (TS 36 serie), we can find the following ones:
- S1AP used between an eNodeB and an MME
- X2AP used to inter-connect two eNodeBs
- M2AP and M3AP used between an eNodeB and a multicast server (for multicast / broadcast services)
- LPP and LPPa used between a mobile phone and a location server (for positionning services)
- SLmAP used between an eNodeB and a location server
- XwAP used between an eNodeB and a Wi-Fi access point
All these protocols are defined with ASN.1, and most of them are using the aligned packed encoding rules (APER) to serialize the data exchanged according to the defined data model.
RANAP is a signaling protocol used between the CS core network (MSC-VLR) or the PS core network (SGSN) and the 3G radio access network (RNC). It enables merely to drive mobile phones connecting to the 3G radio access network, from the core network stand-point. The protocol is specified by the 3GPP standard TS 25.413, the ASN.1 definition is available in the directory pycrate_asn1dir/3GPP_UTRAN_RANAP_25413/ and the corresponding Python modules are in the compiled file pycrate_asn1dir/RANAP.py.
Contrary to RRC protocols, there is a single RANAP ASN.1 object to encode and decode all the messages that can be exchanged with it: the RANAP-PDU defined in the RANAP-PDU-Descriptions ASN.1 module. Also contrary to RRC protocols, where all constructed types are all defined in a straightforward way, the RANAP protocol uses the ASN.1 parameterization to define modular and extendable messages, and defines also procedures through sets of message types and values. In this way, it can be seen as a sort of remote-procedure-call protocol.
>>> from pycrate_asn1dir import RANAP
>>> PDU = RANAP.RANAP_PDU_Descriptions.RANAP_PDU
>>> PDU
<RANAP-PDU (CHOICE)>
>>> help(PDU)
Help on CHOICE in module pycrate_asn1rt.asnobj_construct object:
class CHOICE(pycrate_asn1rt.asnobj.ASN1Obj)
| ASN.1 constructed type CHOICE object
|
| Single value: Python 2-tuple
| 1st item is a Python str, choice identifier, must be a key in _cont
| 2nd item is the ASN1Obj single value specific to the chosen object
[...]
>>> PDU.get_proto()
{
initiatingMessage: {
procedureCode: 'INTEGER',
criticality: 'ENUMERATED',
value: 'OPEN_TYPE'
},
successfulOutcome: {
procedureCode: 'INTEGER',
criticality: 'ENUMERATED',
value: 'OPEN_TYPE'
},
unsuccessfulOutcome: {
procedureCode: 'INTEGER',
criticality: 'ENUMERATED',
value: 'OPEN_TYPE'
},
outcome: {
procedureCode: 'INTEGER',
criticality: 'ENUMERATED',
value: 'OPEN_TYPE'
}
}
The structure of a RANAP-PDU looks very simple at first sight. It is however not... Each message can be one of the fourth types defined. It is then built with one of the PDU content as defined in the RANAP-PDU-Contents ASN.1 module. Let's see how a RANAP Relocation Command is made:
>>> RelCmd = RANAP.RANAP_PDU_Contents.RelocationCommand
>>> RelCmd
<RelocationCommand (SEQUENCE)>
>>> RelCmd.get_proto()
{
protocolIEs: [{
id: 'INTEGER',
criticality: 'ENUMERATED',
value: 'OPEN_TYPE'
}],
protocolExtensions: [{
id: 'INTEGER',
criticality: 'ENUMERATED',
extensionValue: 'OPEN_TYPE'
}]
}
Here again, things look simple, but are not... Each RANAP PDU can contain a sequence of protocolIE and another sequence of protocolExtension. Those IEs and Extensions definition can be found in another ASN.1 set of values, which lists every possible IE, respectively Extension, defined by the specification. An ASN.1 set is a specific object within the pycrate ASN.1 runtime. It has a root part and an extended part. Moreover, it is callable with some built-in filtering features.
>>> RelCmdIEs = RANAP.RANAP_PDU_Contents.RelocationCommandIEs
>>> RelCmdIEs
<RelocationCommandIEs ([RANAP-PROTOCOL-IES] CLASS): ASN1Set(root=[...], ext=[])>
>>> RelCmdIEs().root
[{'Value': <Value ([Target-ToSource-TransparentContainer] OCTET STRING)>, [...]]
>>> RelCmdIEs().ext
[]
>>> RelCmdIEs('id') # listing all IE's id
[63, 14, 46, 28, 9]
>>> RelCmdIEs('id', 14) # filtering the content according to the id's value 14
{'Value': <Value ([L3-Information] OCTET STRING)>, 'presence': 'optional', 'criticality': 'ignore', 'id': 14}
To summerize, a RANAP PDU is one of the four choices possible for the RANAP-PDU object, the procedureCode links to one of the procedure message defined in the RANAP-PDU-Contents ASN.1 module, which itself links to a serie of protocolIEs and protocolExtensions. We can see those links by checking the table constraints applied to specific parts of those objects (OPEN types, actually):
>>> PDU._cont
{
initiatingMessage: <initiatingMessage ([InitiatingMessage] SEQUENCE)>,
successfulOutcome: <successfulOutcome ([SuccessfulOutcome] SEQUENCE)>,
unsuccessfulOutcome: <unsuccessfulOutcome ([UnsuccessfulOutcome] SEQUENCE)>,
outcome: <outcome ([Outcome] SEQUENCE)>
}
>>> PDU._cont['initiatingMessage']._cont
{
procedureCode: <procedureCode ([RANAP-ELEMENTARY-PROCEDURE.&procedureCode] INTEGER)>,
criticality: <criticality ([RANAP-ELEMENTARY-PROCEDURE.&criticality] ENUMERATED)>,
value: <value ([RANAP-ELEMENTARY-PROCEDURE.&InitiatingMessage] OPEN_TYPE)>
}
>>> PDU._cont['initiatingMessage']._cont['value']._const_tab
<_tab_RANAP-ELEMENTARY-PROCEDURE ([RANAP-ELEMENTARY-PROCEDURE] CLASS): ASN1Set(root=[...], ext=[...])
>>> # this ASN1Set contains all the values defined in the RANAP-PDU-Descriptions.RANAP-ELEMENTARY-PROCEDURES set
>>>
>>> RelCmd._cont
{
protocolIEs: <protocolIEs ([ProtocolIE-Container] SEQUENCE OF)>,
protocolExtensions: <protocolExtensions ([ProtocolExtensionContainer] SEQUENCE OF)>
}
>>> RelCmd._cont['protocolIEs']._cont._cont
{
id: <id ([RANAP-PROTOCOL-IES.&id] INTEGER)>,
criticality: <criticality ([RANAP-PROTOCOL-IES.&criticality] ENUMERATED)>,
value: <value ([RANAP-PROTOCOL-IES.&Value] OPEN_TYPE)>
}
>>> RelCmd._cont['protocolIEs']._cont._cont['value']._const_tab
<_tab_RANAP-PROTOCOL-IES ([RANAP-PROTOCOL-IES] CLASS): ASN1Set(root=[...], ext=None)>
>>> # this ASN1Set contains all the values defined in the RANAP-PDU-Contents.RelocationCommandIEs set
It is also possible to list all possible objects that can be embedded into an OPEN one with the _get_const_tr() method:
>>> from pprint import PrettyPrinter
>>> PP = PrettyPrinter()
>>> PP.pprint(RANAP.RANAP_PDU_Descriptions.RANAP_PDU._cont['initiatingMessage']._cont['value']._get_const_tr())
{'CN-DeactivateTrace': <InitiatingMessage ([CN-DeactivateTrace] SEQUENCE)>,
'CN-InvokeTrace': <InitiatingMessage ([CN-InvokeTrace] SEQUENCE)>,
[...]
'UeRegistrationQueryRequest': <InitiatingMessage ([UeRegistrationQueryRequest] SEQUENCE)>,
'UplinkInformationExchangeRequest': <InitiatingMessage ([UplinkInformationExchangeRequest] SEQUENCE)>}
>>> PP.pprint(RelCmd._cont['protocolIEs']._cont._cont['value']._get_const_tr())
{'CriticalityDiagnostics': <Value ([CriticalityDiagnostics] SEQUENCE)>,
'L3-Information': <Value ([L3-Information] OCTET STRING)>,
'RAB-DataForwardingList': <Value ([RAB-DataForwardingList] SEQUENCE OF)>,
'RAB-RelocationReleaseList': <Value ([RAB-RelocationReleaseList] SEQUENCE OF)>,
'Target-ToSource-TransparentContainer': <Value ([Target-ToSource-TransparentContainer] OCTET STRING)>}
Now that we understand the structure of RANAP messages, we can simply use the runtime to encode and decode them. Let's see first how to decode a bytes' buffer corresponding to a downlink RANAP Security Mode Command message, with the APER codec:
>>> from binascii import unhexlify
>>> PDU.from_aper(unhexlify('00060035000003004b000140000b4013110800e03d3da3fc2ee693d0232a6d366a685f000c00120880a261c3cbf6b885745e95e56890586e60'))
>>> print(PDU.to_asn1())
initiatingMessage : {
procedureCode 6,
criticality reject,
value SecurityModeCommand: {
protocolIEs {
{
id 75,
criticality reject,
value KeyStatus: new
},
{
id 11,
criticality ignore,
value EncryptionInformation: {
permittedAlgorithms {
2 -- standard-UMTS-encryption-algorithm-UEA2 --,
1 -- standard-UMTS-encryption-algorith-UEA1 --,
0 -- no-encryption --
},
key 'E03D3DA3FC2EE693D0232A6D366A685F'H
}
},
{
id 12,
criticality reject,
value IntegrityProtectionInformation: {
permittedAlgorithms {
1 -- standard-UMTS-integrity-algorithm-UIA2 --,
0 -- standard-UMTS-integrity-algorithm-UIA1 --
},
key 'A261C3CBF6B885745E95E56890586E60'H
}
}
}
}
}
>>> from pycrate_asn1rt.utils import *
>>> for ie in get_val_at(PDU, ['initiatingMessage', 'value', 'SecurityModeCommand', 'protocolIEs']): print(ie)
...
{'value': ('KeyStatus', 'new'), 'criticality': 'reject', 'id': 75}
{'value': ('EncryptionInformation', {'key': (298065051383415014526417577255883139167, 128), 'permittedAlgorithms': [2, 1, 0]}), 'criticality': 'ignore', 'id': 11}
{'value': ('IntegrityProtectionInformation', {'key': (215842559341980337150017521848692534880, 128), 'permittedAlgorithms': [1, 0]}), 'criticality': 'reject', 'id': 12}
>>> intkey = get_val_at(PDU, ['initiatingMessage', 'value', 'SecurityModeCommand', 'protocolIEs', 2, 'value', 'IntegrityProtectionInformation', 'key'])
>>> intkey
(215842559341980337150017521848692534880, 128)
>>> uint_to_bytes(*intkey)
b'\xa2a\xc3\xcb\xf6\xb8\x85t^\x95\xe5h\x90Xn`'
>>> uint_to_hex(*intkey)
'a261c3cbf6b885745e95e56890586e60'
And here is an example on how to encode an uplink RANAP Direct Transfer message:
>>> for ie in RANAP.RANAP_PDU_Contents.DirectTransferIEs().root: print(ie)
...
{'Value': <Value ([NAS-PDU] OCTET STRING)>, 'presence': 'mandatory', 'criticality': 'ignore', 'id': 16}
{'Value': <Value ([LAI] SEQUENCE)>, 'presence': 'optional', 'criticality': 'ignore', 'id': 15}
{'Value': <Value ([RAC] OCTET STRING)>, 'presence': 'optional', 'criticality': 'ignore', 'id': 55}
{'Value': <Value ([SAI] SEQUENCE)>, 'presence': 'optional', 'criticality': 'ignore', 'id': 58}
{'Value': <Value ([SAPI] ENUMERATED)>, 'presence': 'optional', 'criticality': 'ignore', 'id': 59}
>>> IEs = [] # let's build the list of IEs values
>>> IEs.append({'id': 16, 'criticality': 'ignore', 'value': ('NAS-PDU', b'\x08\x13\x00"\x00I%\xaf)\x04u\xb2\x86B')})
>>> IEs.append({'id': 15, 'criticality': 'ignore', 'value': ('LAI', {'pLMNidentity': b'\x00\x01\xf1', 'lAC': b'\x00\x01'})})
>>> IEs.append({'id': 55, 'criticality': 'ignore', 'value': ('RAC', b'\x10')})
>>> IEs.append({'id': 58, 'criticality': 'ignore', 'value': ('SAI', {'sAC': b'\xff\xff', 'pLMNidentity': b'\x00\x01\xf1', 'lAC': b'\x00\x01'})})
>>> val = ('initiatingMessage', {'procedureCode': 20, 'value': ('DirectTransfer', {'protocolIEs': IEs}), 'criticality': 'ignore'})
>>> PDU.set_val(val)
>>> print(PDU.to_asn1())
initiatingMessage : {
procedureCode 20,
criticality ignore,
value DirectTransfer: {
protocolIEs {
{
id 16,
criticality ignore,
value NAS-PDU: '08130022004925AF290475B28642'H
},
{
id 15,
criticality ignore,
value LAI: {
pLMNidentity '0001F1'H,
lAC '0001'H
}
},
{
id 55,
criticality ignore,
value RAC: '10'H
},
{
id 58,
criticality ignore,
value SAI: {
pLMNidentity '0001F1'H,
lAC '0001'H,
sAC 'FFFF'H
}
}
}
}
}
>>> from binascii import hexlify
>>> hexlify(PDU.to_aper())
b'001440310000040010400f0e08130022004925af290475b28642000f4006000001f100010037400110003a4008000001f10001ffff'
In order to go further, we can read the source code of the ongoing effort to build a 3G core network, available in the directory pycrate_corenet. There is a generic Python module LinkSigProc to help with the procedures defined in this kind of RPC-like protocol: pycrate_corenet/ProcProto.py, and some initial RANAP procedures defined here: pycrate_corenet/ProcCNRanap.py.
S1AP is the signaling protocol used between the MME in an LTE core network (also called an EPC) and eNodeBs (4G base-stations). It enables to drive the eNodeB with few specific procedures and also mobile phones connecting to the 4G radio access network, from the core network stand-point. The protocol is specified by the 3GPP standard TS 36.413, the ASN.1 definition is available in the directory pycrate_asn1dir/3GPP_EUTRAN_S1AP_36413/ and the corresponding Python modules are in the file pycrate_asn1dir/S1AP.py.
S1AP and X2AP protocols are very similar to the existing 3G protocols (like RANAP). There is a single top-level object to encode and decode all protocol's messages: the S1AP-PDU defined in the S1AP-PDU-Descriptions ASN.1 module. The same concept of modular messages containing protocolIEs and protocolExtensions elements is used.
Here are an example with the decoding of an S1 Setup Request message:
>>> from pycrate_asn1dir import S1AP
>>> from pycrate_asn1rt.utils import *
>>> from binascii import hexlify, unhexlify
>>> PDU.from_aper(unhexlify('00110034000004003b0008000001f100000010003c40110700656e62303030312d636f72656e657400400007000000400001f10089400140'))
>>> print(PDU.to_asn1())
initiatingMessage : {
procedureCode 17,
criticality reject,
value S1SetupRequest: {
protocolIEs {
{
id 59,
criticality reject,
value Global-ENB-ID: {
pLMNidentity '0001F1'H,
eNB-ID macroENB-ID : '00001'H
}
},
{
id 60,
criticality ignore,
value ENBname: "enb0001-corenet"
},
{
id 64,
criticality reject,
value SupportedTAs: {
{
tAC '0001'H,
broadcastPLMNs {
'0001F1'H
}
}
}
},
{
id 137,
criticality ignore,
value PagingDRX: v128
}
}
}
}
>>> IEs = get_val_at(PDU, ['initiatingMessage', 'value', 'S1SetupRequest', 'protocolIEs'])
>>> for ie in IEs: print(ie['value'])
...
('Global-ENB-ID', {'pLMNidentity': b'\x00\x01\xf1', 'eNB-ID': ('macroENB-ID', (1, 20))})
('ENBname', 'enb0001-corenet')
('SupportedTAs', [{'tAC': b'\x00\x01', 'broadcastPLMNs': [b'\x00\x01\xf1']}])
('PagingDRX', 'v128')
And here is an example with the encoding of an S1 Initial UE message:
>>> for ie in S1AP.S1AP_PDU_Contents.InitialUEMessage_IEs().root: print(ie)
...
{'Value': <Value ([ENB-UE-S1AP-ID] INTEGER)>, 'id': 8, 'criticality': 'reject', 'presence': 'mandatory'}
{'Value': <Value ([NAS-PDU] OCTET STRING)>, 'id': 26, 'criticality': 'reject', 'presence': 'mandatory'}
{'Value': <Value ([TAI] SEQUENCE)>, 'id': 67, 'criticality': 'reject', 'presence': 'mandatory'}
{'Value': <Value ([EUTRAN-CGI] SEQUENCE)>, 'id': 100, 'criticality': 'ignore', 'presence': 'mandatory'}
{'Value': <Value ([RRC-Establishment-Cause] ENUMERATED)>, 'id': 134, 'criticality': 'ignore', 'presence': 'mandatory'}
{'Value': <Value ([S-TMSI] SEQUENCE)>, 'id': 96, 'criticality': 'reject', 'presence': 'optional'}
{'Value': <Value ([CSG-Id] BIT STRING)>, 'id': 127, 'criticality': 'reject', 'presence': 'optional'}
{'Value': <Value ([GUMMEI] SEQUENCE)>, 'id': 75, 'criticality': 'reject', 'presence': 'optional'}
{'Value': <Value ([CellAccessMode] ENUMERATED)>, 'id': 145, 'criticality': 'reject', 'presence': 'optional'}
{'Value': <Value ([TransportLayerAddress] BIT STRING)>, 'id': 155, 'criticality': 'ignore', 'presence': 'optional'}
{'Value': <Value ([RelayNode-Indicator] ENUMERATED)>, 'id': 160, 'criticality': 'reject', 'presence': 'optional'}
{'Value': <Value ([GUMMEIType] ENUMERATED)>, 'id': 170, 'criticality': 'ignore', 'presence': 'optional'}
{'Value': <Value ([TunnelInformation] SEQUENCE)>, 'id': 176, 'criticality': 'ignore', 'presence': 'optional'}
{'Value': <Value ([TransportLayerAddress] BIT STRING)>, 'id': 184, 'criticality': 'ignore', 'presence': 'optional'}
{'Value': <Value ([LHN-ID] OCTET STRING)>, 'id': 186, 'criticality': 'ignore', 'presence': 'optional'}
{'Value': <Value ([MME-Group-ID] OCTET STRING)>, 'id': 223, 'criticality': 'ignore', 'presence': 'optional'}
{'Value': <Value ([UE-Usage-Type] INTEGER)>, 'id': 230, 'criticality': 'ignore', 'presence': 'optional'
>>> IEs = []
>>> IEs.append({'id': 8, 'value': ('ENB-UE-S1AP-ID', 1202), 'criticality': 'reject'})
>>> IEs.append({'id': 26, 'value': ('NAS-PDU', unhexlify('0741720bf600f11040000af910512604e060c04000240205d011d1271d8080211001000010810600000000830600000000000d00000a00001000500bf600f110000101c8d595065200f11000015c0a003103e5e0341300f110400011035758a65d0100c1')), 'criticality': 'reject'})
>>> IEs.append({'id': 67, 'value': ('TAI', {'pLMNidentity': b'\x00\x01\xf1', 'tAC': b'\x00\x01'}), 'criticality': 'reject'})
>>> IEs.append({'id': 100, 'value': ('EUTRAN-CGI', {'cell-ID': (1, 28), 'pLMNidentity': b'\x00\x01\xf1'}), 'criticality': 'ignore'})
>>> IEs.append({'id': 134, 'value': ('RRC-Establishment-Cause', 'highPriorityAccess'), 'criticality': 'ignore'})
>>> val = ('initiatingMessage', {'procedureCode': 12, 'value': ('InitialUEMessage', {'protocolIEs': IEs}), 'criticality': 'ignore'})
>>> PDU.set_val(val)
>>> print(PDU.to_asn1())
initiatingMessage : {
procedureCode 12,
criticality ignore,
value InitialUEMessage: {
protocolIEs {
{
id 8,
criticality reject,
value ENB-UE-S1AP-ID: 1202
},
{
id 26,
criticality reject,
value NAS-PDU: '0741720BF600F11040000AF910512604E060C04000240205D011D1271D8080211001000010810600000000830600000000000D00000A00001000500BF600F110000101C8D595065200F11000015C0A003103E5E0341300F110400011035758A65D0100C1'H
},
{
id 67,
criticality reject,
value TAI: {
pLMNidentity '0001F1'H,
tAC '0001'H
}
},
{
id 100,
criticality ignore,
value EUTRAN-CGI: {
pLMNidentity '0001F1'H,
cell-ID '0000001'H
}
},
{
id 134,
criticality ignore,
value RRC-Establishment-Cause: highPriorityAccess
}
}
}
}
>>> hexlify(PDU.to_aper())
b'000c40808e000005000800034004b2001a0065640741720bf600f11040000af910512604e060c04000240205d011d1271d8080211001000010810600000000830600000000000d00000a00001000500bf600f110000101c8d595065200f11000015c0a003103e5e0341300f110400011035758a65d0100c100430006000001f1000100644008000001f1000000100086400110'
The MAP (Mobile Application Part) and CAP (Camel Application Part) protocols are used within mobile core networks, they were initially specified by ETSI and are now maintained by the 3GPP. They are transported over the TCAP protocol, which is specified by ITU-T under the tiny name Q.773; it is quite an old protocol, the initial version dates back from 1988, and the current one is from 1997.
The MAP specification can be found in the 3GPP standard TS 29.002, the ASN.1 definition is available in the directory pycrate_asn1dir/3GPP_MAP_29002/ and the corresponding Python modules are in the compiled file pycrate_asn1dir/MAP.py.
The CAP specification can be found in the 3GPP standard TS 29.078, the ASN.1 definition is available in the directory pycrate_asn1dir/3GPP_CAP_29078/ and the corresponding Python modules are in the compiled file pycrate_asn1dir/CAP.py.
The TCAP specification can be found in the ITU-T standard Q.773, the ASN.1 definition is available in the directory pycrate_asn1dir/ITUT_Q773_1997-06/ and the corresponding Python modules are in the compiled file pycrate_asn1dir/TCAP.py.
The TCMessage object which corresponds to the structure of a TCAP message transporting MAP or CAP data needs actually to be parameterized with higher level protocol definition. Moreover, the EXTERNAL object used within the TCAP specification is from an older ASN.1 standard (the one from 1988) and does not correspond to the EXTERNAL structure defined in the pycrate ASN.1 runtime. For this reasons, two custom modules have been created:
- pycrate_asn1dir/Pycrate-TCAP-MAP with the corresponding compiled file pycrate_asn1dir/TCAP_MAP.py and the top-level object TCAP-MAP-Message
- pycrate_asn1dir/Pycrate-TCAP-CAP with the corresponding compiled file pycrate_asn1dir/TCAP_CAP.py; CAP has multiple top-level objects defined in the CAP-gsm and CAP-gprs modules.
All those protocols are using the ASN.1 BER encoding.
MAP is used between mobile core network equipments (MSC/VLR, SGSN, HLR, ...) to handle technical information about connected subscribers. The MAP specification defines several MAP procedures in multiple ASN.1 modules, all of them are grouped into a single ASN.1 set: the MAP-Protocol.Supported-MAP-Operations object. On the TCAP side, a TCAP message is defined by the TCAPMessages.TCMessage object, which needs to be parameterized with a set of upper layer procedures. This is exactly what is accomplished with the custom object TCAP-MAP-Messages.TCAP-MAP-Message. It can be used to encode and decode any TCAP-MAP messages.
Here is what happens when decoding the TCAP MAP message containing an USSD request, as proposed on the Wireshark wiki, and re-encoding it with a null USSD string:
>>> from pycrate_asn1dir import TCAP_MAP
>>> M = TCAP_MAP.TCAP_MAP_Messages.TCAP_MAP_Message
>>> M
<TCAP-MAP-Message ([TCMessage] CHOICE)>
>>> M.get_proto()
{
unidirectional: {
dialoguePortion: {
direct-reference: 'OBJECT IDENTIFIER',
indirect-reference: 'INTEGER',
data-value-descriptor: 'ObjectDescriptor',
[...]
single-ASN1-type: 'OPEN_TYPE',
octet-aligned: 'OCTET STRING',
arbitrary: 'BIT STRING'
}
}
}
}
}
>>> from binascii import hexlify, unhexlify
>>> M.from_ber(unhexlify('626a48042f3b46026b3a2838060700118605010101a02d602b80020780a109060704000001001302be1a2818060704000001010101a00da00b80099656051124006913f66c26a12402010102013b301c04010f040eaa180da682dd6c31192d36bbdd468007917267415827f2'))
>>> print(M.to_asn1())
begin : {
otid '2F3B4602'H,
dialoguePortion {
direct-reference {0 0 17 773 1 1 1} -- dialogue-as-id --,
encoding single-ASN1-type : DialoguePDU: dialogueRequest : {
protocol-version '1'B -- version1 --,
application-context-name {0 4 0 0 1 0 19 2} -- networkUnstructuredSsContext-v2 --,
user-information {
{
direct-reference {0 4 0 0 1 1 1 1} -- map-DialogueAS --,
encoding single-ASN1-type : MAP-DialoguePDU: map-open : {
destinationReference '9656051124006913F6'H
}
}
}
}
},
components {
basicROS : invoke : {
invokeId present : 1,
opcode local : 59,
argument USSD-Arg: {
ussd-DataCodingScheme '0F'H,
ussd-String 'AA180DA682DD6C31192D36BBDD46'H,
msisdn '917267415827F2'H
}
}
}
}
>>> from pycrate_asn1rt.utils import *
>>> get_val_at(M, ['begin', 'components', 0, 'basicROS', 'invoke', 'argument', 'USSD-Arg'])
{'ussd-String': b'\xaa\x18\r\xa6\x82\xddl1\x19-6\xbb\xddF', 'msisdn': b"\x91rgAX'\xf2", 'ussd-DataCodingScheme': b'\x0f'}
>>> get_val_at(M, ['begin', 'components', 0, 'basicROS', 'invoke', 'argument', 'USSD-Arg'])['ussd-String'] = b'\x00'
>>> print(M.to_asn1())
begin : {
otid '2F3B4602'H,
[...]
ussd-DataCodingScheme '0F'H,
ussd-String '00'H,
msisdn '917267415827F2'H
}
}
}
}
>>> hexlify(M.to_ber())
b'625d48042f3b46026b3a2838060700118605010101a02d602b80020780a109060704000001001302be1a2818060704000001010101a00da00b80099656051124006913f66c19a11702010102013b300f04010f0401008007917267415827f2'
Camel is mainly used for messaging and subscription management services in mobile core networks. The CAP specification defines several CAP procedures and corresponding TCAP messages in different ASN.1 modules:
- module CAP-gprsSSF-gsmSCF-pkgs-contracts-acs defines the top-level objects GenericGprsSSF-gsmSCF-PDUs and GenericGsmSCF-gprsSSF-PDUs
- module CAP-gsmSCF-gsmSRF-pkgs-contracts-acs defines the top-level object BASIC-gsmSRF-gsmSCF-PDUs
- module CAP-gsmSSF-gsmSCF-pkgs-contracts-acs defines the top-level objects GenericSSF-gsmSCF-PDUs, AssistHandoffsSF-gsmSCF-PDUs and GenericSCF-gsmSSF-PDUs
- module CAP-smsSSF-gsmSCF-pkgs-contracts-acs defines the top-level objects Generic-sms-PDUs
Taking again an example from the Wireshark wiki, here is what happens when decoding and re-encoding an InitialDP operation CAP message, that is handled by the ASN.1 object CAP-gsmSSF-gsmSCF-pkgs-contracts-acs.GenericSSF-gsmSCF-PDUs:
>>> M = TCAP_CAP.CAP_gsmSSF_gsmSCF_pkgs_contracts_acs.GenericSSF_gsmSCF_PDUs
>>> M
<GenericSSF-gsmSCF-PDUs ([TCMessage] CHOICE)>
>>> from binascii import unhexlify, hexlify
>>> M.from_ber(unhexlify('628187480206f76b1e281c060700118605010101a011600f80020780a1090607040000010032016c61a15f020101020100305780012a830884111487095040f79c01029f32061487572586f9bf34148107913366020000f0a3098007313233343536379f3605a12345678f9f3707913366020000f09f3807111487085040f79f39080230900211223370'))
>>> print(M.to_asn1())
begin : {
otid '06F7'H,
dialoguePortion {
direct-reference {0 0 17 773 1 1 1} -- dialogue-as-id --,
encoding single-ASN1-type : DialoguePDU: dialogueRequest : {
protocol-version '1'B -- version1 --,
application-context-name {0 4 0 0 1 0 50 1}
}
},
components {
basicROS : invoke : {
invokeId present : 1,
opcode local : 0,
argument InitialDPArg: {
serviceKey 42,
callingPartyNumber '84111487095040F7'H,
eventTypeBCSM collectedInfo,
iMSI '1487572586F9'H,
locationInformation {
vlr-number '913366020000F0'H,
cellGlobalIdOrServiceAreaIdOrLAI cellGlobalIdOrServiceAreaIdFixedLength : '31323334353637'H -- 1234567 --
},
callReferenceNumber 'A12345678F'H,
mscAddress '913366020000F0'H,
calledPartyBCDNumber '111487085040F7'H,
timeAndTimezone '0230900211223370'H
}
}
}
}
>>> from pycrate_asn1rt.utils import *
>>> get_val_at(M, ['begin', 'components', 0, 'basicROS', 'invoke', 'argument', 'InitialDPArg', 'locationInformation'])
{'vlr-number': b'\x913f\x02\x00\x00\xf0', 'cellGlobalIdOrServiceAreaIdOrLAI': ('cellGlobalIdOrServiceAreaIdFixedLength', b'1234567')}
>>> # let's modify the locationInformation
>>> get_obj_at(M, ['begin', 'components', 0, 'basicROS', 'invoke', 'argument', 'InitialDPArg', 'locationInformation', 'cellGlobalIdOrServiceAreaIdOrLAI'])
<cellGlobalIdOrServiceAreaIdOrLAI ([CellGlobalIdOrServiceAreaIdOrLAI] CHOICE)>
>>> get_obj_at(M, ['begin', 'components', 0, 'basicROS', 'invoke', 'argument', 'InitialDPArg', 'locationInformation', 'cellGlobalIdOrServiceAreaIdOrLAI'])._cont
{
cellGlobalIdOrServiceAreaIdFixedLength: <cellGlobalIdOrServiceAreaIdFixedLength ([CellGlobalIdOrServiceAreaIdFixedLength] OCTET STRING)>,
laiFixedLength: <laiFixedLength ([LAIFixedLength] OCTET STRING)>
}
>>> get_val_at(M, ['begin', 'components', 0, 'basicROS', 'invoke', 'argument', 'InitialDPArg', 'locationInformation'])['cellGlobalIdOrServiceAreaIdOrLAI'] = ('laiFixedLength', b'myLai01234')
>>> print(M.to_asn1())
begin : {
otid '06F7'H,
dialoguePortion {
direct-reference {0 0 17 773 1 1 1} -- dialogue-as-id --,
encoding single-ASN1-type : DialoguePDU: dialogueRequest : {
protocol-version '1'B -- version1 --,
application-context-name {0 4 0 0 1 0 50 1}
}
},
components {
basicROS : invoke : {
invokeId present : 1,
opcode local : 0,
argument InitialDPArg: {
serviceKey 42,
callingPartyNumber '84111487095040F7'H,
eventTypeBCSM collectedInfo,
iMSI '1487572586F9'H,
locationInformation {
vlr-number '913366020000F0'H,
cellGlobalIdOrServiceAreaIdOrLAI laiFixedLength : '6D794C61693031323334'H -- myLai01234 --
},
callReferenceNumber 'A12345678F'H,
mscAddress '913366020000F0'H,
calledPartyBCDNumber '111487085040F7'H,
timeAndTimezone '0230900211223370'H
}
}
}
}
>>> hexlify(M.to_ber())
b'62818a480206f76b1e281c060700118605010101a011600f80020780a1090607040000010032016c64a162020101020100305a80012a830884111487095040f79c01029f32061487572586f9bf34178107913366020000f0a30c810a6d794c616930313233349f3605a12345678f9f3707913366020000f09f3807111487085040f79f39080230900211223370'
X.509 is an ITU-T recommendation. The last version of this document dates from 2016 and is unfortunately behind a paywall. It is however possible to access the PDF version of 2012 for free: X.509-2012-10 This recommendation has actually quite a broad scope, from public-key certificate, attribute certificates and authentication services, and also quite old: the 1st version was published in 1988. The corresponding ASN.1 specification can be found in the fomal description section of this web page. It is a set of 8 ASN.1 modules, which actually requires many other modules from other ITU-T specifications to be compiled.
The X.509 2016 ASN.1 module from ITU-T is available in the directory pycrate_asn1dir/ITUT_X509_2016-10 and the corresponding Python module are in the compiled file pycrate_asn1dir/X509_2016.py Few errors have been corrected compared to the archive of ASN.1 files provided by the ITU-T web site for the specification to compile correctly.
The definition of an X.509 certificate is provided by the object Certificate in the AuthenticationFramework module. Certificates are known to be DER-encoded.
Let's see how the X.509 cetificate of the ITU-T website is structured, according to their ASN.1 specification. With any good web browser, we can export the certificate to the PEM format: the certificate is base64-encoded between two specific lines: BEGIN CERTIFICATE and END CERTIFICATE. The PEM-encoded certificate from the ITU-T web site is 2718 bytes long.
Let's see what happen when we load the compiled module and decode the certificate from the ITU-T website:
>>> from pycrate_asn1dir.X509_2016 import *
init_modules: different OID objects (id-oc, objectClass) with same OID value ((2, 5, 6))
init_modules: different OID objects (id-at, attributeType) with same OID value ((2, 5, 4))
[...]
>>> Cert = AuthenticationFramework.Certificate
>>> Cert
<Certificate ([SIGNED] SEQUENCE)>
>>> Cert.get_proto()
{
toBeSigned: {
version: 'INTEGER',
serialNumber: 'INTEGER',
signature: {
algorithm: 'OBJECT IDENTIFIER',
parameters: 'OPEN_TYPE'
},
issuer: {
rdnSequence: [[{
type: 'OBJECT IDENTIFIER',
value: 'OPEN_TYPE'
}]]
},
validity: {
notBefore: {
utcTime: 'UTCTime',
generalizedTime: 'GeneralizedTime'
},
notAfter: {
utcTime: 'UTCTime',
generalizedTime: 'GeneralizedTime'
}
},
subject: {
rdnSequence: [[{
type: 'OBJECT IDENTIFIER',
value: 'OPEN_TYPE'
}]]
},
subjectPublicKeyInfo: {
algorithm: {
algorithm: 'OBJECT IDENTIFIER',
parameters: 'OPEN_TYPE'
},
subjectPublicKey: 'BIT STRING'
},
issuerUniqueIdentifier: 'BIT STRING',
subjectUniqueIdentifier: 'BIT STRING',
extensions: [{
extnId: 'OBJECT IDENTIFIER',
critical: 'BOOLEAN',
extnValue: 'OCTET STRING'
}]
},
algorithmIdentifier: {
algorithm: 'OBJECT IDENTIFIER',
parameters: 'OPEN_TYPE'
},
signature: 'BIT STRING'
}
>>> cl = open('~/Downloads/ituint.crt').readlines()
>>> import base64
>>> cb = base64.b64decode(''.join(cl[1:-1]))
>>> Cert.from_der(cb)
SEQUENCE._decode_ber_cont_ws: Certificate.toBeSigned.issuer.rdnSequence._item_._item_, unable to determine if component value is present (err 3)
OPEN._decode_ber_cont: Certificate.toBeSigned.issuer.rdnSequence._item_._item_.value, unable to retrieve an object in the table constraint (Certificate.toBeSigned.issuer.rdnSequence._item_._item_.value: non-existent value (2, 5, 4, 6) for identifier id in the table constraint)
SEQUENCE._decode_ber_cont_ws: Certificate.toBeSigned.issuer.rdnSequence._item_._item_, unable to determine if component value is present (err 3)
OPEN._decode_ber_cont: Certificate.toBeSigned.issuer.rdnSequence._item_._item_.value, unable to retrieve an object in the table constraint (Certificate.toBeSigned.issuer.rdnSequence._item_._item_.value: non-existent value (2, 5, 4, 8) for identifier id in the table constraint)
SEQUENCE._decode_ber_cont_ws: Certificate.toBeSigned.issuer.rdnSequence._item_._item_, unable to determine if component value is present (err 3)
OPEN._decode_ber_cont: Certificate.toBeSigned.issuer.rdnSequence._item_._item_.value, unable to retrieve an object in the table constraint (Certificate.toBeSigned.issuer.rdnSequence._item_._item_.value: non-existent value (2, 5, 4, 7) for identifier id in the table constraint)
SEQUENCE._decode_ber_cont_ws: Certificate.toBeSigned.issuer.rdnSequence._item_._item_, unable to determine if component value is present (err 3)
OPEN._decode_ber_cont: Certificate.toBeSigned.issuer.rdnSequence._item_._item_.value, unable to retrieve an object in the table constraint (Certificate.toBeSigned.issuer.rdnSequence._item_._item_.value: non-existent value (2, 5, 4, 10) for identifier id in the table constraint)
SEQUENCE._decode_ber_cont_ws: Certificate.toBeSigned.issuer.rdnSequence._item_._item_, unable to determine if component value is present (err 3)
OPEN._decode_ber_cont: Certificate.toBeSigned.issuer.rdnSequence._item_._item_.value, unable to retrieve an object in the table constraint (Certificate.toBeSigned.issuer.rdnSequence._item_._item_.value: non-existent value (2, 5, 4, 3) for identifier id in the table constraint)
SEQUENCE._decode_ber_cont_ws: Certificate.toBeSigned.subject.rdnSequence._item_._item_, unable to determine if component value is present (err 3)
OPEN._decode_ber_cont: Certificate.toBeSigned.subject.rdnSequence._item_._item_.value, unable to retrieve an object in the table constraint (Certificate.toBeSigned.subject.rdnSequence._item_._item_.value: non-existent value (2, 5, 4, 5) for identifier id in the table constraint)
SEQUENCE._decode_ber_cont_ws: Certificate.toBeSigned.subject.rdnSequence._item_._item_, unable to determine if component value is present (err 3)
OPEN._decode_ber_cont: Certificate.toBeSigned.subject.rdnSequence._item_._item_.value, unable to retrieve an object in the table constraint (Certificate.toBeSigned.subject.rdnSequence._item_._item_.value: non-existent value (1, 3, 6, 1, 4, 1, 311, 60, 2, 1, 3) for identifier id in the table constraint)
SEQUENCE._decode_ber_cont_ws: Certificate.toBeSigned.subject.rdnSequence._item_._item_, unable to determine if component value is present (err 3)
OPEN._decode_ber_cont: Certificate.toBeSigned.subject.rdnSequence._item_._item_.value, unable to retrieve an object in the table constraint (Certificate.toBeSigned.subject.rdnSequence._item_._item_.value: non-existent value (2, 5, 4, 15) for identifier id in the table constraint)
SEQUENCE._decode_ber_cont_ws: Certificate.toBeSigned.subject.rdnSequence._item_._item_, unable to determine if component value is present (err 3)
OPEN._decode_ber_cont: Certificate.toBeSigned.subject.rdnSequence._item_._item_.value, unable to retrieve an object in the table constraint (Certificate.toBeSigned.subject.rdnSequence._item_._item_.value: non-existent value (2, 5, 4, 6) for identifier id in the table constraint)
SEQUENCE._decode_ber_cont_ws: Certificate.toBeSigned.subject.rdnSequence._item_._item_, unable to determine if component value is present (err 3)
OPEN._decode_ber_cont: Certificate.toBeSigned.subject.rdnSequence._item_._item_.value, unable to retrieve an object in the table constraint (Certificate.toBeSigned.subject.rdnSequence._item_._item_.value: non-existent value (2, 5, 4, 17) for identifier id in the table constraint)
SEQUENCE._decode_ber_cont_ws: Certificate.toBeSigned.subject.rdnSequence._item_._item_, unable to determine if component value is present (err 3)
OPEN._decode_ber_cont: Certificate.toBeSigned.subject.rdnSequence._item_._item_.value, unable to retrieve an object in the table constraint (Certificate.toBeSigned.subject.rdnSequence._item_._item_.value: non-existent value (2, 5, 4, 8) for identifier id in the table constraint)
SEQUENCE._decode_ber_cont_ws: Certificate.toBeSigned.subject.rdnSequence._item_._item_, unable to determine if component value is present (err 3)
OPEN._decode_ber_cont: Certificate.toBeSigned.subject.rdnSequence._item_._item_.value, unable to retrieve an object in the table constraint (Certificate.toBeSigned.subject.rdnSequence._item_._item_.value: non-existent value (2, 5, 4, 7) for identifier id in the table constraint)
SEQUENCE._decode_ber_cont_ws: Certificate.toBeSigned.subject.rdnSequence._item_._item_, unable to determine if component value is present (err 3)
OPEN._decode_ber_cont: Certificate.toBeSigned.subject.rdnSequence._item_._item_.value, unable to retrieve an object in the table constraint (Certificate.toBeSigned.subject.rdnSequence._item_._item_.value: non-existent value (2, 5, 4, 9) for identifier id in the table constraint)
SEQUENCE._decode_ber_cont_ws: Certificate.toBeSigned.subject.rdnSequence._item_._item_, unable to determine if component value is present (err 3)
OPEN._decode_ber_cont: Certificate.toBeSigned.subject.rdnSequence._item_._item_.value, unable to retrieve an object in the table constraint (Certificate.toBeSigned.subject.rdnSequence._item_._item_.value: non-existent value (2, 5, 4, 10) for identifier id in the table constraint)
SEQUENCE._decode_ber_cont_ws: Certificate.toBeSigned.subject.rdnSequence._item_._item_, unable to determine if component value is present (err 3)
OPEN._decode_ber_cont: Certificate.toBeSigned.subject.rdnSequence._item_._item_.value, unable to retrieve an object in the table constraint (Certificate.toBeSigned.subject.rdnSequence._item_._item_.value: non-existent value (2, 5, 4, 11) for identifier id in the table constraint)
SEQUENCE._decode_ber_cont_ws: Certificate.toBeSigned.subject.rdnSequence._item_._item_, unable to determine if component value is present (err 3)
OPEN._decode_ber_cont: Certificate.toBeSigned.subject.rdnSequence._item_._item_.value, unable to retrieve an object in the table constraint (Certificate.toBeSigned.subject.rdnSequence._item_._item_.value: non-existent value (2, 5, 4, 3) for identifier id in the table constraint)
>>> print(Cert.to_asn1())
{
toBeSigned {
version 2 -- v3 --,
serialNumber 163212335596803956569165623251943599291,
signature {
algorithm {1 2 840 113549 1 1 11} -- sha256WithRSAEncryption --,
parameters NULL: NULL
},
issuer rdnSequence : {
{
{
type {2 5 4 6} -- id-at-countryName --,
value '4742'H
}
},
{
{
type {2 5 4 8} -- id-at-stateOrProvinceName --,
value '47726561746572204D616E63686573746572'H
}
},
{
{
type {2 5 4 7} -- id-at-localityName --,
value '53616C666F7264'H
}
},
{
{
type {2 5 4 10} -- id-at-organizationName --,
value '434F4D4F444F204341204C696D69746564'H
}
},
{
{
type {2 5 4 3} -- id-at-commonName --,
value '434F4D4F444F2052534120457874656E6465642056616C69646174696F6E2053656375726520536572766572204341'H
}
}
},
validity {
notBefore utcTime : "170328000000Z" -- Tue Mar 28 00:00:00 2017 --,
notAfter utcTime : "190328235959Z" -- Thu Mar 28 23:59:59 2019 --
},
subject rdnSequence : {
{
{
type {2 5 4 5} -- id-at-serialNumber --,
value '5265736F6C7574696F6E204E6F20393020412F333730'H
}
},
{
{
type {1 3 6 1 4 1 311 60 2 1 3},
value '4348'H
}
},
{
{
type {2 5 4 15} -- id-at-businessCategory --,
value '4E6F6E2D436F6D6D65726369616C20456E74697479'H
}
},
{
{
type {2 5 4 6} -- id-at-countryName --,
value '4348'H
}
},
{
{
type {2 5 4 17} -- id-at-postalCode --,
value '31323131'H
}
},
{
{
type {2 5 4 8} -- id-at-stateOrProvinceName --,
value '47656EC3A87665203230'H
}
},
{
{
type {2 5 4 7} -- id-at-localityName --,
value '47656EC3A87665'H
}
},
{
{
type {2 5 4 9} -- id-at-streetAddress --,
value '506C61636520646573204E6174696F6E73'H
}
},
{
{
type {2 5 4 10} -- id-at-organizationName --,
value '496E7465726E6174696F6E616C2054656C65636F6D6D756E69636174696F6E7320556E696F6E202849545529'H
}
},
{
{
type {2 5 4 11} -- id-at-organizationalUnitName --,
value '434F4D4F444F2045562053534C'H
}
},
{
{
type {2 5 4 3} -- id-at-commonName --,
value '7777772E6974752E696E74'H
}
}
},
subjectPublicKeyInfo {
algorithm {
algorithm {1 2 840 113549 1 1 1} -- rsaEncryption --,
parameters NULL: NULL
},
subjectPublicKey '3082010A0282010100C5C4341CF4363220289A51E75B53333105216FB691124A6AB2E8D595525FA682BB9A257737F4140F06281D310DB7BB005986FF17088F61362A8A9A1727D64AF36B281E4D932E07717BF50B8305BFD36F18F73B1154BB6FF6D9E2DF53A3C71DD66F94829BE1613151C3B729482A713112F6912773A62564F551693D1310DB3076CDE24F4556A51DCADC27E537E1E2AB53354F8DCC0441A706328720126AD8AA7AA455E3D93BAD77039E9B73596A68112C83909E524DD9AF48100A610E27FD5C419AC4F4565A9E438A9B5E466E8137D9C144A1C81A93F0493C38040E7B9144F6F3EC4E6DF29ABDE2716F762856BF5D60A43645B4493FDB0754CEFCBA037EA694DB0203010001'H
},
extensions {
{
extnId {2 5 29 35} -- id-ce-authorityKeyIdentifier --,
extnValue '3016801439DAFFCA28148AA8741308B9E40EA9D2FA7E9D69'H
},
{
extnId {2 5 29 14} -- id-ce-subjectKeyIdentifier --,
extnValue '04144CCAF046405EC0DAF417682805C993E47B16CCEA'H
},
{
extnId {2 5 29 15} -- id-ce-keyUsage --,
critical TRUE,
extnValue '030205A0'H
},
{
extnId {2 5 29 19} -- id-ce-basicConstraints --,
critical TRUE,
extnValue '3000'H
},
{
extnId {2 5 29 37} -- id-ce-extKeyUsage --,
extnValue '301406082B0601050507030106082B06010505070302'H
},
{
extnId {2 5 29 32} -- id-ce-certificatePolicies --,
extnValue '303D303B060C2B06010401B2310102010501302B302906082B06010505070201161D68747470733A2F2F7365637572652E636F6D6F646F2E636F6D2F435053'H
},
{
extnId {2 5 29 31} -- id-ce-cRLDistributionPoints --,
extnValue '304D304BA049A0478645687474703A2F2F63726C2E636F6D6F646F63612E636F6D2F434F4D4F444F525341457874656E64656456616C69646174696F6E53656375726553657276657243412E63726C'H
},
{
extnId {1 3 6 1 5 5 7 1 1} -- id-pe-authorityInfoAccess --,
extnValue '3079305106082B060105050730028645687474703A2F2F6372742E636F6D6F646F63612E636F6D2F434F4D4F444F525341457874656E64656456616C69646174696F6E53656375726553657276657243412E637274302406082B060105050730018618687474703A2F2F6F6373702E636F6D6F646F63612E636F6D'H
},
{
extnId {2 5 29 17} -- id-ce-subjectAltName --,
extnValue '3016820B7777772E6974752E696E7482076974752E696E74'H
},
{
extnId {1 3 6 1 4 1 11129 2 4 2},
extnValue '0482016C016A007600A4B90990B418581487BB13A2CC67700A3C359804F91BDFB8E377CD0EC80DDC100000015B157198BC000004030047304502207B8D96679302EFB061C71EF0762BD9FF1938FEB604DFAB820B00684F35E90A69022100DAE8F2A04DFF20536CDF0B9FA56FCE19D4DE8227FC9BB17E0DC7383DF0F8772A0077005614069A2FD7C2ECD3F5E1BD44B23EC74676B9BC99115CC0EF949855D689D0DD0000015B1571964F0000040300483046022100D6B0C8B655FA18E02521178AD3F2AB2D41672C7177A853B4F8124218F0788270022100A9827EAAE294130A3D23D224511533B2BEAA972EF59B592EEC126A045202A978007700EE4BBDB775CE60BAE142691FABE19E66A30F7E5FB072D88300C47B897AA8FDCB0000015B1571988600000403004830460221008451A06AB517D32A0DA9408E28C50E05DB7B35576A188B80F567A6A09C4D7E49022100CE45402927D318B911F922F5F856DEB05DC36CC30B71DFCD7E12E8FCC0B86587'H
}
}
},
algorithmIdentifier {
algorithm {1 2 840 113549 1 1 11} -- sha256WithRSAEncryption --,
parameters NULL: NULL
},
signature '4ABE511D378ECD7FB8B69BED9588E11B1550D84E626E0391C56D6780B15819AE18248259399A72DAEFADE86B6C26F6C47ACCC44E43B4D142EFEB55A14C478056204A902A2759CF0F1CE5CD74F08D257055A89BCA627A5B9D19DDA1E516346E323E43F7B613935E4605BA8AFCB6DFB52D7ADC4CCB0029A7E9FFD62ED19738D8531B048C97ECA51082FD50974958165D9E6C1D5279DDE1F0019C80E773E2E801C8FAF7FE72FBFB8FA4AEBFE96CB38B1E88940255BCB1C8E578DAE9A188AD5F38D626E73A61E37DC63686789066CAF732FE4C3B1CCDF98B80F5A0BFC54B4DDB5B7041F471DC795941816CD5949E8ED0D47C127BE9D1B182EE8C4B42CD711CC6EA89'H
}
We get lots of warnings from the runtime. This is because the X.509 specification makes use of many OPEN types (e.g. for attributes, extensions, algorithm parameters), but the ITU-T specification does not define complete look-up tables to resolve all those open types. For instance, in the AuthenticationFramework module, the set of said supported algorithms is empty:
SupportedAlgorithms ALGORITHM ::= {...}
This means that, when parsing a certificate, the runtime will not be able to link an algorithm OID to the type of its parameters.
This has been manually completed in the pycrate X509 ASN.1 specification, with the list of basic algorithms defined in the AlgorithmObjectIdentifiers, to create the set AllAlgorithmsOID. Thanks to this, the runtime is actually capable to link algorithm OID to their corresponding parameters (which are all NULL, in our case). You can see it here:
>>> get_val_at(Cert, ['toBeSigned', 'signature'])
{'algorithm': (1, 2, 840, 113549, 1, 1, 11), 'parameters': ('NULL', 0)}
>>> get_val_at(Cert, ['toBeSigned', 'subjectPublicKeyInfo', 'algorithm'])
{'algorithm': (1, 2, 840, 113549, 1, 1, 1), 'parameters': ('NULL', 0)}
>>> get_val_at(Cert, ['algorithmIdentifier'])
{'algorithm': (1, 2, 840, 113549, 1, 1, 11), 'parameters': ('NULL', 0)}
If you want to translate OID tuple values into their names, you can have a look at the GLOBAL.OID dictionnary created when loading the Python module:
>>> GLOBAL.OID
{(0, 9, 2342, 19200300, 100): 'cosine',
[...]
(2, 16, 840, 1, 101, 3, 4, 2): 'hashAlgs',
(2, 16, 840, 1, 101, 3, 4, 2, 1): 'id-sha256',
(2, 16, 840, 1, 101, 3, 4, 2, 2): 'id-sha384',
(2, 16, 840, 1, 101, 3, 4, 2, 3): 'id-sha512',
(2, 16, 840, 1, 101, 3, 4, 2, 4): 'id-sha224',
(2, 16, 840, 1, 101, 3, 4, 2, 5): 'id-sha512-224',
(2, 16, 840, 1, 101, 3, 4, 2, 6): 'id-sha512-256',
[...]
}
This same problem happens also with the set SupportedAttributes defined in the module InformationFramework, which is very incomplete. See this extract from the ITU-T ASN.1 specification:
-- Definition of the following information object set is deferred, perhaps to
-- standardized profiles or to protocol implementation conformance statements. The set
-- is required to specify a table constraint on the values component of Attribute, the
-- value component of AttributeTypeAndValue, and the assertion component of
-- AttributeValueAssertion.
SupportedAttributes ATTRIBUTE ::= {objectClass | aliasedEntryName, ...}
This is why all the certificate's issuer information are not decoded properly (and the runtime prints warnings). This is again the case with the set ExtensionSet defined in the module AuthenticationFramework, which is completely empty:
EXTENSION ::= {...}
For this reason, the runtime is unable to decode properly the list of extensions in the certificate. It seems that finally the ITU-T X.509 ASN.1 definition is not a readily usable specification for decoding certificates.
Let's see what is provided by the IETF.
Lots of RFC have been published by the IETF about public key infrastructures, which contain many ASN.1 specifications about tons of structures to deal with digital certificate, cryptographic signature and so on... Two majors RFC for ASN.1 definitions are RFC 5911 and 5912, which gather ASN.1 definitions from several others RFC.
The digital certificate object, equivalent to the one we just used from the ITU-T documents, is located in the PKIX1Explicit-2009, taken from the RFC 5912. Let's see how it decodes the previous certificate blob:
>>> Cert = PKIX1Explicit_2009.Certificate
>>> Cert
<Certificate ([SIGNED] SEQUENCE)>
>>> Cert.from_der(cb)
OPEN._decode_ber_cont: Certificate.toBeSigned.signature.parameters, unable to retrieve an object in the table constraint (Certificate.toBeSigned.signature.parameters: non-existent value (1, 2, 840, 113549, 1, 1, 11) for identifier id in the table constraint)
OPEN._decode_ber_cont: Certificate.toBeSigned.subject.rdnSequence._item_._item_.value, unable to retrieve an object in the table constraint (Certificate.toBeSigned.subject.rdnSequence._item_._item_.value: non-existent value (1, 3, 6, 1, 4, 1, 311, 60, 2, 1, 3) for identifier id in the table constraint)
OPEN._decode_ber_cont: Certificate.toBeSigned.subject.rdnSequence._item_._item_.value, unable to retrieve an object in the table constraint (Certificate.toBeSigned.subject.rdnSequence._item_._item_.value: non-existent value (2, 5, 4, 15) for identifier id in the table constraint)
OPEN._decode_ber_cont: Certificate.toBeSigned.subject.rdnSequence._item_._item_.value, unable to retrieve an object in the table constraint (Certificate.toBeSigned.subject.rdnSequence._item_._item_.value: non-existent value (2, 5, 4, 17) for identifier id in the table constraint)
OPEN._decode_ber_cont: Certificate.toBeSigned.subject.rdnSequence._item_._item_.value, unable to retrieve an object in the table constraint (Certificate.toBeSigned.subject.rdnSequence._item_._item_.value: non-existent value (2, 5, 4, 9) for identifier id in the table constraint)
OPEN._decode_ber_cont: Certificate.toBeSigned.extensions._item_._cont_extnValue, unable to retrieve an object in the table constraint (Certificate.toBeSigned.extensions._item_._cont_extnValue: non-existent value (1, 3, 6, 1, 4, 1, 11129, 2, 4, 2) for identifier id in the table constraint)
OPEN._decode_ber_cont: Certificate.algorithmIdentifier.parameters, unable to retrieve an object in the table constraint (Certificate.algorithmIdentifier.parameters: non-existent value (1, 2, 840, 113549, 1, 1, 11) for identifier id in the table constraint)
BIT_STR.__from_ber_buf: signature, CONTAINING object decoding failed
>>> print(Cert.to_asn1())
{
toBeSigned {
version 2 -- v3 --,
serialNumber 163212335596803956569165623251943599291,
signature {
algorithm {1 2 840 113549 1 1 11} -- sha256WithRSAEncryption --,
parameters ''H
},
issuer rdnSequence : {
{
{
type {2 5 4 6} -- id-at-countryName --,
value PrintableString: "GB"
}
},
{
{
type {2 5 4 8} -- id-at-stateOrProvinceName --,
value DirectoryString: printableString : "Greater Manchester"
}
},
{
{
type {2 5 4 7} -- id-at-localityName --,
value X520LocalityName: printableString : "Salford"
}
},
{
{
type {2 5 4 10} -- id-at-organizationName --,
value DirectoryString: printableString : "COMODO CA Limited"
}
},
{
{
type {2 5 4 3} -- id-at-commonName --,
value X520CommonName: printableString : "COMODO RSA Extended Validation Secure Server CA"
}
}
},
validity {
notBefore utcTime : "170328000000Z" -- Tue Mar 28 00:00:00 2017 --,
notAfter utcTime : "190328235959Z" -- Thu Mar 28 23:59:59 2019 --
},
subject rdnSequence : {
{
{
type {2 5 4 5} -- id-at-serialNumber --,
value PrintableString: "Resolution No 90 A/370"
}
},
{
{
type {1 3 6 1 4 1 311 60 2 1 3},
value '4348'H
}
},
{
{
type {2 5 4 15} -- id-at-businessCategory --,
value '4E6F6E2D436F6D6D65726369616C20456E74697479'H
}
},
{
{
type {2 5 4 6} -- id-at-countryName --,
value PrintableString: "CH"
}
},
{
{
type {2 5 4 17} -- id-at-postalCode --,
value '31323131'H
}
},
{
{
type {2 5 4 8} -- id-at-stateOrProvinceName --,
value DirectoryString: uTF8String : "Genève 20"
}
},
{
{
type {2 5 4 7} -- id-at-localityName --,
value X520LocalityName: uTF8String : "Genève"
}
},
{
{
type {2 5 4 9} -- id-at-streetAddress --,
value '506C61636520646573204E6174696F6E73'H
}
},
{
{
type {2 5 4 10} -- id-at-organizationName --,
value DirectoryString: printableString : "International Telecommunications Union (ITU)"
}
},
{
{
type {2 5 4 11} -- id-at-organizationalUnitName --,
value DirectoryString: printableString : "COMODO EV SSL"
}
},
{
{
type {2 5 4 3} -- id-at-commonName --,
value X520CommonName: printableString : "www.itu.int"
}
}
},
subjectPublicKeyInfo {
algorithm {
algorithm {1 2 840 113549 1 1 1} -- rsaEncryption --,
parameters NULL: NULL
},
subjectPublicKey '3082010A0282010100C5C4341CF4363220289A51E75B53333105216FB691124A6AB2E8D595525FA682BB9A257737F4140F06281D310DB7BB005986FF17088F61362A8A9A1727D64AF36B281E4D932E07717BF50B8305BFD36F18F73B1154BB6FF6D9E2DF53A3C71DD66F94829BE1613151C3B729482A713112F6912773A62564F551693D1310DB3076CDE24F4556A51DCADC27E537E1E2AB53354F8DCC0441A706328720126AD8AA7AA455E3D93BAD77039E9B73596A68112C83909E524DD9AF48100A610E27FD5C419AC4F4565A9E438A9B5E466E8137D9C144A1C81A93F0493C38040E7B9144F6F3EC4E6DF29ABDE2716F762856BF5D60A43645B4493FDB0754CEFCBA037EA694DB0203010001'H
},
extensions {
{
extnID {2 5 29 35} -- id-ce-authorityKeyIdentifier --,
extnValue EXTENSION: AuthorityKeyIdentifier: {
keyIdentifier '39DAFFCA28148AA8741308B9E40EA9D2FA7E9D69'H
}
},
{
extnID {2 5 29 14} -- id-ce-subjectKeyIdentifier --,
extnValue EXTENSION: KeyIdentifier: '4CCAF046405EC0DAF417682805C993E47B16CCEA'H
},
{
extnID {2 5 29 15} -- id-ce-keyUsage --,
critical TRUE,
extnValue EXTENSION: KeyUsage: '101'B -- digitalSignature | keyEncipherment --
},
{
extnID {2 5 29 19} -- id-ce-basicConstraints --,
critical TRUE,
extnValue EXTENSION: BasicConstraints: { }
},
{
extnID {2 5 29 37} -- id-ce-extKeyUsage --,
extnValue EXTENSION: ExtKeyUsageSyntax: {
{1 3 6 1 5 5 7 3 1} -- id-kp-serverAuth --,
{1 3 6 1 5 5 7 3 2} -- id-kp-clientAuth --
}
},
{
extnID {2 5 29 32} -- id-ce-certificatePolicies --,
extnValue EXTENSION: CertificatePolicies: {
{
policyIdentifier {1 3 6 1 4 1 6449 1 2 1 5 1},
policyQualifiers {
{
policyQualifierId {1 3 6 1 5 5 7 2 1} -- id-qt-cps --,
qualifier CPSuri: "https://secure.comodo.com/CPS"
}
}
}
}
},
{
extnID {2 5 29 31} -- id-ce-cRLDistributionPoints --,
extnValue EXTENSION: CRLDistributionPoints: {
{
distributionPoint fullName : {
uniformResourceIdentifier : "http://crl.comodoca.com/COMODORSAExtendedValidationSecureServerCA.crl"
}
}
}
},
{
extnID {1 3 6 1 5 5 7 1 1} -- id-pe-authorityInfoAccess --,
extnValue EXTENSION: AuthorityInfoAccessSyntax: {
{
accessMethod {1 3 6 1 5 5 7 48 2} -- id-ad-caIssuers --,
accessLocation uniformResourceIdentifier : "http://crt.comodoca.com/COMODORSAExtendedValidationSecureServerCA.crt"
},
{
accessMethod {1 3 6 1 5 5 7 48 1} -- id-ad-ocsp --,
accessLocation uniformResourceIdentifier : "http://ocsp.comodoca.com"
}
}
},
{
extnID {2 5 29 17} -- id-ce-subjectAltName --,
extnValue EXTENSION: GeneralNames: {
dNSName : "www.itu.int",
dNSName : "itu.int"
}
},
{
extnID {1 3 6 1 4 1 11129 2 4 2},
extnValue EXTENSION: '016A007600A4B90990B418581487BB13A2CC67700A3C359804F91BDFB8E377CD0EC80DDC100000015B157198BC000004030047304502207B8D96679302EFB061C71EF0762BD9FF1938FEB604DFAB820B00684F35E90A69022100DAE8F2A04DFF20536CDF0B9FA56FCE19D4DE8227FC9BB17E0DC7383DF0F8772A0077005614069A2FD7C2ECD3F5E1BD44B23EC74676B9BC99115CC0EF949855D689D0DD0000015B1571964F0000040300483046022100D6B0C8B655FA18E02521178AD3F2AB2D41672C7177A853B4F8124218F0788270022100A9827EAAE294130A3D23D224511533B2BEAA972EF59B592EEC126A045202A978007700EE4BBDB775CE60BAE142691FABE19E66A30F7E5FB072D88300C47B897AA8FDCB0000015B1571988600000403004830460221008451A06AB517D32A0DA9408E28C50E05DB7B35576A188B80F567A6A09C4D7E49022100CE45402927D318B911F922F5F856DEB05DC36CC30B71DFCD7E12E8FCC0B86587'H
}
}
},
algorithmIdentifier {
algorithm {1 2 840 113549 1 1 11} -- sha256WithRSAEncryption --,
parameters ''H
},
signature '4ABE511D378ECD7FB8B69BED9588E11B1550D84E626E0391C56D6780B15819AE18248259399A72DAEFADE86B6C26F6C47ACCC44E43B4D142EFEB55A14C478056204A902A2759CF0F1CE5CD74F08D257055A89BCA627A5B9D19DDA1E516346E323E43F7B613935E4605BA8AFCB6DFB52D7ADC4CCB0029A7E9FFD62ED19738D8531B048C97ECA51082FD50974958165D9E6C1D5279DDE1F0019C80E773E2E801C8FAF7FE72FBFB8FA4AEBFE96CB38B1E88940255BCB1C8E578DAE9A188AD5F38D626E73A61E37DC63686789066CAF732FE4C3B1CCDF98B80F5A0BFC54B4DDB5B7041F471DC795941816CD5949E8ED0D47C127BE9D1B182EE8C4B42CD711CC6EA89'H
}
This is a little bit better... but not entirely satisfying ! The specification from the RFC5912 enables to decode properly all issuer information. There are still some subject and extension information which are not decoded entirely. The algorithm parameters are not decoded too.
To conclude here, we see that we will need in anyway to extend manually the ASN.1 specification in order to include structures describing all existing issuer, subject, extension and algorithm parameters' structures, to be able to use the pycrate ASN.1 runtime to properly decode X.509 certificate.