diff --git a/doc/SONiC_OC_NTP_HLD.md b/doc/SONiC_OC_NTP_HLD.md new file mode 100644 index 0000000000..b29cb8d564 --- /dev/null +++ b/doc/SONiC_OC_NTP_HLD.md @@ -0,0 +1,1014 @@ +# Feature Name +NTP Support in Management Framework + +# High Level Design Document + +#### Rev 0.5 + +# Table of Contents + * [List of Tables](#list-of-tables) + * [Revision](#revision) + * [About This Manual](#about-this-manual) + * [Scope](#scope) + * [Definition/Abbreviation](#definitionabbreviation) + +# List of Tables +[Table 1: Abbreviations](#table-1-abbreviations) + +# Revision +| Rev | Date | Author | Change Description | +|:---:|:-----------:|:------------------:|-----------------------------------| +| 0.1 | 05/03/2020 | Bing Sun | Initial version | +| 0.2 | 06/15/2020 | Bing Sun | Update based on comments | +| 0.3 | 09/21/2020 | Bing Sun | Add dhcp behavior | +| 0.4 | 11/02/2020 | Bing Sun | Add support for NTP authentication| +| 0.5 | 11/08/2020 | Bing Sun | Allow configuration of multiple NTP source interfaces| +| 0.6 | 12/07/2020 | Bing Sun | Updated section 3.2 configDB changes | +| | | | Minor change for Yang models | + + +# About this Manual + +This document introduces the support of NTP configuration using management framework. It also describes the corresponding backend NTP configuration changes(/etc/ntp.conf and /etc/ntp.keys) as well as ntp service restart upon configuration changes. + +# Scope + +This document covers NTP "configuration" and "show" commands based on the OpenConfig YANG model. In addition, it decribes the backend mechanism required to support each command. +A summary of NTP unit test cases is presented at the end. + +# Definition/Abbreviation + +### Table 1: Abbreviations +| **Term** | **Meaning** | +|--------------------------|-------------------------------------| +| NTP | Network Time Protocol | +| ntpd | NTP Daemon | +| mgmt VRF | Management VRF | + + +# 1 Feature Overview + +NTP stands for Network Time Protocol. It is used to synchronize the time of a computer or server to another server or reference time source. + +Today, SONiC click CLI provides commands to +- add and delete remote NTP servers with IPv4 or IPv6 addresses +- display NTP synchronization status with show command (output of "ntpq -pn") + +This feature provides the the same above mentioned capabilities via Management CLI, REST and gNMI using OpenConfig YANG models. +In addition, it provides the following configuration, +- add remote NTP server with hostname +- NTP source interfaces +- NTP vrf +- NTP authentication + +## 1.1 Requirements + +### 1.1.1 Front end configuration and get capabilities + +#### 1.1.1.1 add/delete NTP server +``` +ntp server 99.1.1.1 +ntp server pool.ntp.org +``` +Add/delete NTP server information in the Redis ConfigDB and in /etc/ntp.conf. +The NTP server can be IPv4 address, IPv6 address , or hostname. +Mutliple NTP servers can be configured. + +#### 1.1.1.2 add/delete NTP source interface +``` +ntp source-interface Ethernet36 +``` + +``` +ntp source-interface PortChannel 100 +``` + +``` +ntp source-interface Vlan 100 +``` + +``` +ntp source-interface Management 0 +``` + + +Add/delete the global NTP source interface in the Redis ConfigDB and in /etc/ntp.conf. The ip address of this interface will be used by ntpd as source ip for all NTP packets. +Multiple NTP source interfaces can be configured. +Following interface types can be used as NTP source interface, +- Ethernet interface +- PortChannel +- Vlan interface +- Loopback interfacee +- eth0(management interface) + +#### 1.1.1.3 add/delete VRF name +``` +ntp vrf default +``` + +``` +ntp vrf mgmt +``` + +Add/delete the global NTP VRF information in the Redis ConfigDB. It is used by /etc/init.d/ntp script to start the ntpd in a specific VRF context. +For this release, only Management VRF and default instance are supported. + +#### 1.1.1.4 Get NTP association +``` +show ntp association +``` + +This command displays the output of "ntpq -np" command. + +#### 1.1.1.5 Overall Behavior related to NTP source interface and NTP vrf +##### When mgmt VRF is configured +a.if no ntp vrf is configured, ntp service starts in mgmt VRF context by default +b.if "mgmt" is configured as NTP vrf, ntp service starts in mgmt VRF context +c.if "default" is again configured as NTP vrf, ntp service starts in default vrf context + +##### When mgmt VRF is not configured +ntp service always starts in default vrf context + +##### NTP source interface related +a.if a NTP source interface has IP address configured, the IP address will be used as source ip for all NTP packets exchanged with the respective NTP servers/clients +b.if a NTP source interface has no IP address configured, it is not being considered as an NTP source interface + +#### 1.1.1.6 NTP authentication configuration +NTP authentication enables an NTP client or peer to authenticate time received from their servers and peers. + +##### ntp authenticate +``` +ntp authenticate +``` + +NTP server and NTP client use this command to enable the NTP authentication feature. + +##### ntp authentication-key +``` +ntp authentication-key 1 md5 "ntp client 1" + +ntp authentication-key 2 md5 ntp_client2 +``` + +NTP client uses this command to define an authentication key with key number, authentication type and password. +The key number is from 1 to 65535. +The authentication type supported is MD5, SHA1 and SHA2-256. +The password is configured with plaintext the first time. In runnning-configuration, it is encrypted and is indicated with "encrypted" at the end. Authentication key can then be configured with the encrypted format and "encrypted" flag. + + +##### ntp trusted key +``` +ntp trusted-key 1 + +ntp trusted-key 2 +``` + +NTP server uses this command to add a list of key numbers that the NTP server must provide in its NTP packets in order for the NTP clients to synchronize to it. +NTP client must configure a list with this command if it desires to authenticate the NTP server with any of the key number. + +##### ntp server key +``` +ntp server 99.1.1.1 key 1 +``` + +NTP client uses this command to configure the key expected from a specific NTP server. + +### 1.1.2 Backend mechanisms to support configuration and get + +#### 1.1.2.1 add/delete NTP server +This creates or deletes a NTP server entry in the Redis ConfigDB. + +``` + "NTP_SERVER|10.11.0.1": { + "type": "hash", + "value": { + "NULL": "NULL" + } + }, + "NTP_SERVER|2001:aa:aa::a": { + "type": "hash", + "value": { + "NULL": "NULL" + } + }, + "NTP_SERVER|pool.ntp.org": { + "type": "hash", + "value": { + "NULL": "NULL" + } + } +``` + +A change in the NTP_SERVER entry triggers hostcfgd to start the NTP configuration script, which in turn writes each NTP server to the ntp.conf and then restart the ntp service. + + +#### 1.1.2.2 add/delete NTP source + +This creates or deletes a global NTP source interface entry in the Redis ConfigDB. + +``` + "NTP|global": { + "type": "hash", + "value": { + "src_intf": "Ethernet36" + } + } +``` + +A change in this entry triggers hostcfgd to start the NTP configuration script, which in turn writes the ntp source interface to the ntp.conf and then restart the ntp service. + +SONiC click CLI can be extended to include this configuration. + + +#### 1.1.2.3 add/delete NTP VRF + +This creates or deletes a global NTP vrf entry in the Redis ConfigDB. For this release, it can only be "mgmt" or "default". + +``` + "NTP|global": { + "type": "hash", + "value": { + "vrf": "mgmt" + } + } +``` + +A change in this DB entry triggers hostcfgd to restart ntp service. + +SONiC click CLI can be extended to include this configuration. + + +#### 1.1.2.4 get NTP associations + +Transformer function issues "ntpq -pn" command, parses the response and maps the outputs to the OpenConfig system YANG NTP states. + +#### 1.1.2.5 NTP authentication + +##### 1.1.2.5.1 Enable or disable ntp authenticate + +When "authenticate" is enabled, "enable-ntp-auth" field is set to "true" in the NTP global entry, + +``` +"NTP|global": { + "type": "hash", + "value": { + "enable-ntp-auth": "true", + } +} +``` + +This change triggers /etc/ntp.conf to get generated with the line indicating where to find the configured keys +``` +key /etc/ntp.keys +``` + +When "authenticate" is removed, the same attribute is set to "false". The file /etc/ntp.conf is generated without "key /etc/ntp.keys" but with the line +``` +disable auth +``` + +The file /etc/ntp.keys will be created with the configured authentication keys if "authenticate" is enabled, and removed if "authenticate" is disabled. + +##### 1.1.2.5.2 Add or delete ntp authentication key + +When an authentication key is configured with a key number, authentication type and password in plaintext, a transformer function will change the plaintext password to the encrypted format and puts the key in an NTP_AUTHENTICATION_KEY ConfigDB entry. A boolean "key_encrypted" is set to true and added in the same entry as well. This is done so that "show running-configuration" from CLI or GET from REST/gNMI will be able to display the password in encrypted format. For example, +``` + "NTP_AUTHENTICATION_KEY|1": { + "type": "hash", + "value": { + "key_encrypted": "true", + "key_type": "MD5", + "key_value": "3b88c0eb8406a9e76722b84baf1d94e5e185eb7f64f8dd46c759719c33557876" + } + } +``` + +If "authenticate" is enabled, the file /etc/ntp.keys is populated with the configured authentication keys. The password in this file is in the plaintext format. Only root user can read /etc/ntp.keys. + +When an ntp authentication key is removed, the ConfigDb and /etc/ntp.keys will be updated accordingly. + +##### 1.1.2.5.3 Add or delete ntp trusted key + +When a ntp trusted key number is configured, the key number is added to the "trustedkeys" list in the NTP global entry, e.g +``` + "NTP|global": { + "type": "hash", + "value": { + "authenticat": enabled, + "trustedkeys@": "1,2" + } + } +``` + +/etc/ntp.conf will be generated with the line +``` +trustedkey 1 2 +``` + +When a ntp trusted key number is removed, the key number is removed from the "trustedkey" list. + + +##### 1.1.2.5.4 Add a key for NTP server + +When a ntp server is created with a key number, the "key_id" with the key number will be added as a field for the NTP server ConfigDb entry, e.g +``` +"NTP_SERVER|99.1.1.1": { + "type": "hash", + "value": { + "key_id": "1" + } + } +``` + +The file /etc/ntp.conf will be generated with the same key number for that NTP server, e.g +``` +server 99.1.1.1 iburst key 1 +``` + +##### 1.1.2.5.5 Sample NTP authentication CLI commands on NTP server and NTP client +``` +NTP master ------------------------- SONiC switch ----------------------------------server +Mgmt. IP: 100.94.121.15 mgmt. IP: 100.94.122.16 + Loopback100: 2001:aa:aa::1 + +``` + +Here the SONiC switch is a NTP client to the NTP master. It is also a NTP server to the downstream servers. +As a NTP client, SONiC switch uses NTP authentication to validate its NTP server. +As a NTP server, the downstream servers are the NTP client. It is up to the NTP client (server) whether NTP authentication is desired with its NTP server (SONiC switch). +SONiC switch can server as a NTP client and a NTP server simultaneously, with or without NTP authentication with either a remote NTP server or NTP client. +The NTP master reaches SONiC switch via its management interface. The downstream servers reach the SONiC switch via its front panel ports. + +###### Relevant CLI commmands on SONiC switch as NTP server + +``` +sonic(config)# ntp source-interface Loopback 100 +sonic(config)#ip vrf mgmt +``` + +###### Relevant CLI commands on SONiC switch as NTP client +``` +sonic(config)#ntp authenticate +sonic(config)#ntp authentication-key 1 MD5 force +sonic(config)#ntp trusted-key 1 +sonic(config)#ntp server 100.94.121.15 key 1 +sonic(config)# ntp source-interface Management 0 +``` + +###### Relevant CLI commands on the server as NTP client without NTP authentication +``` +sonic(config)#ntp server 2001:aa:aa::1 +sonic(config)# ntp source-interface Vlan 100 +``` + +###### Relevant CLI commands for the server and SONiC switch with NTP authentication +``` +On the SONiC switch + +sonic(config)#ntp authenticate +sonic(config)#ntp authentication-key 2 MD5 jungle +sonic(config)#ntp trusted-key 2 +sonic(config)# ntp source-interface Loopback 100 +sonic(config)#ip vrf mgmt +``` + +``` +On the server + +sonic(config)#ntp authenticate +sonic(config)#ntp authentication-key 2 MD5 jungle +sonic(config)#ntp trusted-key 2 +sonic(config)#ntp server 2001:aa:aa::1 key 2 +sonic(config)# ntp source-interface Vlan 100 + +``` + +### 1.1.3 Functional Requirements + +Provide management framework support to +- configure NTP server +- configure NTP source interface +- configure NTP vrf +- configure NTP authentication + +### 1.1.4 Configuration and Management Requirements +- CLI style configuration and show commands +- REST API support +- gNMI Support + +Details described in Section 3. + +### 1.1.5 Configurations not supported by this feature using management framework: +- configure local server as a NTP master +- broadcast mode + +### 1.1.6 Scalability Requirements +Interface listening will not be enabled on all L3 interfaces. User can configure NTP source interfaces which are only a few. +Ntpd runs in one VRF context, default vrf or mgmt vrf. +Multiple ntp servers supported, also only a few. + +### 1.1.7 Warm Boot Requirements +NA + +## 1.2 Design Overview + +### 1.2.1 Basic Approach +Implement NTP support using transformer in sonic-mgmt-framework. + +### 1.2.2 Container +The front end code change will be done in management-framework container including: +- XML file for the CLI +- Python script to handle CLI request (actioner) +- Jinja template to render CLI output (renderer) +- front-end code to support "show running-configuration" +- OpenConfig YANG model for NTP openconfig-system.yang and openconfig-system-ext.yang +- SONiC NTP model for NTP based on Redis DB schema of NTP +- transformer functions to + * convert OpenConfig YANG model to SONiC YANG model for NTP related configurations + * convert from Linux command "ntpq -p" output to OpenConfig NTP state YANG model + +### 1.2.3 SAI Overview + +# 2 Functionality + +## 2.1 Target Deployment Use Cases +Manage/configure NTP via gNMI, REST and CLI interfaces. + +## 2.2 Functional Description +Provide CLI, gNMI and REST supports for NTP related configurations. + +## 2.3 Backend change to support new configurations +Provide changes in hostcfgd, ntp.conf.j2, ntp.keys.j2 and /etc/init.d/ntp. +SONiC click CLI enhancement if possible. + +## 2.4 Behavior when Management IP Address is acquired via DHCP +If the management IP address is acquired via DHCP, and if the NTP server option specifies the NTP server, /etc/dhcp/dhclient-exit-hooks.d/ntp script will generate the file /var/lib/ntp/ntp.conf.dhcp. This file is a copy of the default /etc/ntp.conf with a modified server list from the DHCP server. +NTP daemon only uses one of the 2 files, and /var/lib/ntp/ntp.conf.dhcp takes precedence over the default /etc/ntp.conf. It is the existing behavior and is out of the scope of this HLD. + +NTP source-interface, NTP vrf and NTP authentication discussed in the HLD are only guaranteed to take effect on the static configured NTP servers. +For acquired NTP servers from DHCP server, NTP source-interface, NTP vrf and NTP authentication will only take effect if /var/lib/ntp/ntp.conf.dhcp is generated based on the /etc/ntp.conf with user configured NTP source-interface and NTP authentication. + +Applying the configured NTP source-interface, NTP vrf and NTP authentication to acquired NTP servers from the DHCP server is not a requirement for this release. + +# 3 Design + +## 3.1 Overview + +Enhancing the management framework backend code and transformer methods to add support for NTP. + +## 3.2 DB Changes + +### 3.2.1 CONFIG DB +This feature will allow users to make NTP configuration changes to CONFIG DB, and get NTP configurations. + +``` +NTP server + + "NTP_SERVER": { + "2.2.2.2": { + "key_id": "1" + }, + "3.3.3.3": { + "key_id": "2" + }, + "4.4.4.4": { + "key_id": "3" + }, + "10.14.8.140": {} + } +``` + +``` +NTP authentication key + + "NTP_AUTHENTICATION_KEY": { + "1": { + "encrypted": "true", + "type": "MD5", + "value": "U2FsdGVkX18LP3kIv47lRKCboUop/+0YyacH2UT2WJ0=" + }, + "2": { + "encrypted": "true", + "type": "SHA1", + "value": "U2FsdGVkX1+DU7geMDXVvCOJjZQyP1zTT4vRbHFqsZo=" + }, + "3": { + "encrypted": "true", + "type": "SHA2_256", + "value": "U2FsdGVkX19yHcvrGFDKJb80FRY+cnmO1+yv6SGkao8=" + } + } +``` + +``` +NTP global configuration + + "NTP": { + "global": { + "auth_enabled": "true", + "src_intf": [ + "eth0", + "Loopback99" + ], + "trusted_key": [ + "1", + "2", + "3" + ], + "vrf": "mgmt" + } + } +``` + + +### 3.2.2 APP DB + +### 3.2.3 STATE DB + +### 3.2.4 ASIC DB + +### 3.2.5 COUNTER DB + +## 3.3 Switch State Service Design + +### 3.3.1 Orchestration Agent + +### 3.3.2 Other Process + +## 3.4 SyncD + +## 3.5 SAI + +## 3.6 User Interface + +### 3.6.1 Data Models + +YANG models needed for NTP handling in the management framework: +1. **openconfig-system.yang** + +2. **openconfig-system-ext.yang** + +3. **sonic-system-ntp.yang** + +Supported yang objects and attributes: +```diff + +--rw ntp + | +--rw config + | | +--rw enabled? boolean ++ | | +--rw enable-ntp-auth? boolean ++ | | +--rw oc-sys-ext:ntp-source-interface* oc-if:base-interface-ref ++ | | +--rw oc-sys-ext:vrf? string ++ | | +--rw oc-sys-ext:trusted-key* uint16 + | +--ro state + | | +--ro enabled? boolean ++ | | +--ro enable-ntp-auth? boolean + | | +--ro auth-mismatch? oc-yang:counter64 ++ | | +--ro oc-sys-ext:ntp-source-interface? oc-if:base-interface-ref ++ | | +--ro oc-sys-ext:vrf? string ++ | | +--rw oc-sys-ext:trusted-key* uint16 ++ | +--rw ntp-keys ++ | | +--rw ntp-key* [key-id] ++ | | +--rw key-id -> ../config/key-id ++ | | +--rw config ++ | | | +--rw key-id? uint16 ++ | | | +--rw key-type? identityref ++ | | | +--rw key-value? string ++ | | | +--rw oc-sys-ext:encrypted? boolean ++ | | +--ro state ++ | | +--ro key-id? uint16 ++ | | +--ro key-type? identityref ++ | | +--ro key-value? string ++ | | +--rw oc-sys-ext:encrypted? boolean + | +--rw servers + | +--rw server* [address] ++ | +--rw address -> ../config/address ++ | +--rw config ++ | | +--rw address? oc-inet:host + | | +--rw port? oc-inet:port-number + | | +--rw version? uint8 + | | +--rw association-type? enumeration + | | +--rw iburst? boolean + | | +--rw prefer? boolean ++ | | +--rw oc-sys-ext:key-id? uint16 ++ | +--ro state ++ | +--ro address? oc-inet:host + | +--ro port? oc-inet:port-number + | +--ro version? uint8 + | +--ro association-type? enumeration + | +--ro iburst? boolean + | +--ro prefer? boolean ++ | +--ro stratum? uint8 + | +--ro root-delay? uint32 + | +--ro root-dispersion? uint64 + | +--ro offset? uint64 ++ | +--ro poll-interval? uint32 ++ | +--rw oc-sys-ext:key-id? uint16 ++ | +--ro oc-sys-ext:peerdelay? decimal64 ++ | +--ro oc-sys-ext:peeroffset? decimal64 ++ | +--ro oc-sys-ext:peerjitter? decimal64 ++ | +--ro oc-sys-ext:selmode? string ++ | +--ro oc-sys-ext:refid? inet:host ++ | +--ro oc-sys-ext:peertype? string ++ | +--ro oc-sys-ext:now? uint32 ++ | +--ro oc-sys-ext:reach? uint8 +``` + +```diff +module: sonic-system-ntp ++ +--rw NTP ++ | +--rw NTP_LIST* [global_key] ++ | +--rw global_key enumeration ++ | +--rw src_intf* union ++ | +--rw vrf? union ++ | +--rw auth_enabled? boolean ++ | +--rw trusted_keys* -> /sonic-system-ntp/NTP_AUTHENTICATION_KEY/NTP_AUTHENTICATION_KEY_LIST/id ++ +--rw NTP_AUTHENTICATION_KEY ++ | +--rw NTP_AUTHENTICATION_KEY_LIST* [id] ++ | +--rw id uint16 ++ | +--rw type enumeration ++ | +--rw value string ++ | +--rw encrypted? boolean ++ +--rw NTP_SERVER ++ +--rw NTP_SERVER_LIST* [server_address] ++ +--rw server_address inet:host ++ +--rw key_id? -> /sonic-system-ntp/NTP_AUTHENTICATION_KEY/NTP_AUTHENTICATION_KEY_LIST/id +``` + +### 3.6.2 CLI + + +#### 3.6.2.1 Configuration Commands +All commands are executed in `configuration-view`: +``` +sonic# configure terminal +sonic(config)# + +sonic(config)# ntp + authenticate Authenticate time sources + authentication-key Authentication key for trusted time sources + server Configure NTP server + source-interface Configure NTP source interface to pick the source IP, used for the NTP packets + trusted-key Key numbers for trusted time sources + vrf Enable NTP on VRF + +``` + +##### 3.6.2.1.1 Configure NTP server +``` +sonic(config)#ntp + server Configure NTP server +sonic(config)#ntp server +String NTP server address or name + +sonic(config)# ntp server 10.11.0.1 + +sonic(config)# ntp server 2001:aa:aa::a + +sonic(config)# ntp server pool.ntp.org + +``` + +##### 3.6.2.1.2 Delete NTP server + +``` +sonic(config)# no ntp server + String NTP server address or name + +sonic(config)# no ntp server 10.11.0.1 + +sonic(config)# no ntp server 2001:aa:aa::a + +sonic(config)# no ntp server pool.ntp.org + +``` + +##### 3.6.2.1.3 Configure NTP source interface + +``` +sonic(config)# ntp source-interface + Ethernet Ethernet interface + Loopback Loopback interface + Management Management Interface + PortChannel PortChannel interface + Vlan Vlan interface + +sonic(config)# ntp source-interface Ethernet 48 +sonic(config)# +sonic(config)# + +sonic(config)# ntp source-interface Loopback 100 +sonic(config)# +sonic(config)# +sonic(config)# +sonic(config)# ntp source-interface Management 0 +sonic(config)# +sonic(config)# +sonic(config)# +sonic(config)# ntp source-interface PortChannel 100 +sonic(config)# +sonic(config)# +sonic(config)# +sonic(config)# ntp source-interface Vlan 100 +sonic(config)# + +``` + +##### 3.6.2.1.4 Delete NTP source interface + +``` +sonic(config)# no ntp source-interface PortChannel 100 + +``` + +``` +sonic(config)# no ntp source-interface +``` + +##### 3.6.2.1.5 Configure NTP vrf + +``` +sonic(config)# + vrf Enabling NTP on a VRF + +sonic(config)#ntp vrf + mgmt Enable NTP on management VRF + default Enable NTP on default VRF + +``` + +##### 3.6.2.1.6 Delete NTP vrf + +``` +sonic(config)# no ntp + vrf Disable NTP on a VRF + +sonic(config)# no ntp vrf + +``` + +##### 3.6.2.1.7 Enable NTP authentication +``` +sonic(config)#ntp + authenticate Authenticate time sources +sonic(config)#ntp authenticate +``` + +##### 3.6.2.1.8 Disable NTP authentication +``` +sonic(config)#no ntp authenticate +``` + +##### 3.6.2.1.9 Configure NTP authentication-key +``` +sonic(config)#ntp authentication-key + <1-65535> Key number + +sonic(config)#ntp authentication-key 1 + md5 MD5 authentication + sha1 SHA1 authentication + sha2-256 SHA2-256 authentication + +sonic(config)#ntp authentication-key 1 md5 + String Authentication key (max 64 chars, keys longer than 20 chars must be hex) + +sonic(config)#ntp authentication-key 1 md5 "ntp client 1" + +``` + +##### 3.6.2.1.10 Delete NTP authentication-key +``` +sonic(config)#no ntp authentication-key 1 +``` + +##### 3.6.2.1.11 Configure NTP trusted-key +``` +sonic(config)#ntp trusted-key + <1-65535> Key number + +sonic(config)#ntp trusted-key 1 +``` + +##### 3.6.2.1.12 Delete NTP trusted-key +``` +sonic(config)no ntp trusted-key 1 +``` + +##### 3.6.2.1.13 Add NTP server with key +``` +sonic(config)#ntp server 99.1.1.1 + key Configure peer authentication key + +sonic(config)#ntp server 99.1.1.1 key 1 +``` + +##### 3.6.2.1.14 Delete NTP server with key +``` +sonic(config)#no ntp server 99.1.1.1 +``` + +#### 3.6.2.2 Show ntp +``` +sonic# show ntp + associations Display NTP associations + global Display NTP global configuration + server Display NTP server configuration + +``` + +##### 3.6.2.2.1 show ntp associations + +``` +sonic# show ntp associations + remote refid st t when poll reach delay offset jitter +============================================================================== +*10.11.0.1 10.11.8.1 4 u 28 64 1 0.183 1.499 2.625 ++2001:aa:aa::b 60.39.129.68 10 u 27 64 1 0.638 2171.31 0.411 ++10.11.0.2 10.11.8.1 4 u 24 64 1 0.240 -13.957 12.786 +* master (synced), # master (unsynced), + selected, - candidate, ~ configured + +``` + +##### 3.6.2.2.2 Show configured ntp servers +``` +sonic# show ntp server +-------------------------------- +NTP Servers +-------------------------------- +10.11.0.1 +10.11.0.2 +``` + +##### 3.6.2.2.3 Show global ntp configurations +``` +sonic# show ntp global +---------------------------------------------- +NTP Global Configuration +---------------------------------------------- +NTP source-interface: eth0 + Loopback100 + +NTP vrf: mgmt + +``` + +##### 3.6.2.2.4 Show running-configuration + +``` +sonic(config)#ntp authenticate +sonic(config)#ntp authentication-key 1 md5 "ntp client 1" +sonic(config)#ntp authentication-key 1 md5 ntp_client_2 +sonic(config)#ntp server 99.1.1.1 key 1 +sonic(config)#ntp trusted-keys 1 +sonic(config)#ntp trusted-keys 2 +sonic(config)# do show running-configuration +! +ntp authenticate +ntp authentication-key 1 md5 3b88c0eb8406a9e76722b84baf1d94e5e185eb7f64f8dd46c759719c33557876 encrypted +ntp authentication-key 2 md5 771de7710005c5d6aa5b3313812b721d5d0d4a93fb1548572994464495476c4e encrypted +ntp server 99.1.1.1 key 1 +ntp trusted-keys 1 +ntp trusted-keys 2 +! + +sonic(config)#ntp server 10.11.0.1 +sonic(config)#ntp server pool.ntp.org +sonic(config)#ntp source-interface Management 0 +sonic(config)#ntp source-interface Loopback 100 +sonic(config)#do show running-configuration +! +ntp server 10.11.0.1 +ntp server pool.ntp.org +ntp source-interface Management 0 +ntp source-interface Loopback 100 +! + +sonic(config)# no ntp source-interface Loopback 100 +sonic(config)# ntp vrf mgmt +sonic(config)# do show running-configuration +! +ntp server 10.11.0.1 +ntp server pool.ntp.org +ntp source-interface Management 0 +ntp vrf mgmt +! + +sonic(config)# ntp vrf default +sonic(config)# do show running-configuration +! +ntp server 10.11.0.1 +ntp server pool.ntp.org +ntp source-interface Management 0 +ntp vrf default +! + +``` + +#### 3.6.2.3 Debug Commands +``` +From KLISH: + +show ntp associations + +show ntp server + +show ntp global + +``` + +``` +From shell: + +servcie ntp status + +check /etc/ntp.conf + +check /var/log/syslog and look for ntp + +check "docker exec -it mgmt tail -f /var/log/rest_server/rest_server.log" for rest logs + +ifconfig lo + +ifconfig lo-m + +show mgmt-vrf + +ntpq -pn + +``` + +#### 3.6.2.4 IS-CLI Compliance + +### 3.6.3 REST API Support +``` +GET - Get existing NTP configuration information from CONFIG DB. + Get NTP peer states +PUT - Create NTP configuration into CONFIG DB. +POST - Add NTP configuration into CONFIG DB. +PATCH - Update existing NTP configuraiton information in CONFIG DB. +DELETE - Delete a existing NTP configuration from CONFIG DB. +``` + +# 4 Flow Diagrams + +# 5 Error Handling + +# 6 Serviceability and Debug + +# 7 Warm Boot Support +NA + +# 8 Scalability + +# 9 Unit Test + +The unit-test for this feature will include: +#### Configuration via CLI + +| Test Name | Test Description | +| :-------- | :----- | +| Configure NTP server | Verify NTP servers are installed correctly in the configDB and reflected in the NTP peer status | +| Delete NTP server | Verify NTP server is deleted from the configDB and reflected in the NTP peer status | +| Configure NTP source interface| Verify NTP source interface is installed correctly in the configDB, NTP packets are transmitted and received over this source | +| | Verify that multiple NTP source interfaces can be configured, for both default and mgmt vrf cases| +| Delete NTP source interface| Verify that NTP source interface is removed from the configDB, NTP packets are transmitted and received over the default interface| +| Configure NTP vrf| Verify that NTP vrf is installed correctly in the configDB and ntp service is running in the specified VRF| +| | Verify that only default and mgmt can be configured as NTP vrf| +| Delete NTP vrf| Verify that NTP vrf is removed from the configDB and ntp service is running in the default instance| +| Configure NTP authentication for NTP server| Verify that NTP authentication-key can be created correctly| +| | Verify that NTP trusted-keys can be added correctly| +| | Verify that NTP authentication can be enabled and disabled| +| Configure NTP authentication for NTP client| Verify that NTP authentication-key can be created correctly| +| | Verify that NTP trusted-keys can be added correctly| +| | Verify that key number can be added to a NTP server | +| | Verify that NTP authentication can be enabled and disabled| +| | Verify NTP server is accepted if authentication keys match on NTP server and NTP client| +| | Verify NTP server is rejected if authentication keys mismatch on NTP server and NTP client| +| show ntp associations | Verify ntp associations are displayed correctly | +| show ntp server | Verify ntp servers are displayed correctly | +| show ntp global | Verify ntp global configurations are displayed correctly | + +#### Configuration via gNMI + +Same test as CLI configuration Test but using gNMI request. +Additional tests will be done to set NTP configuration at different levels of YANG models. + +#### Get configuration via gNMI + +Same as CLI show test but with gNMI request, will verify the JSON response is correct. +Additional tests will be done to get NTP configuration and NTP states at different levels of YANG models. + +#### Configuration via REST (POST/PUT/PATCH) + +Same test as CLI configuration Test but using REST POST/PUT/PATCH request. +Additional tests will be done to set NTP configuration at different levels of YANG models. + + +#### Get configuration via REST (GET) + +Same as CLI show test but with REST GET request, will verify the JSON response is correct. +Additional tests will be done to get NTP configuration and NTP states at different levels of YANG models. + + +# 10 Internal Design Information + + diff --git a/doc/VRRP_HLD_Buzznik.md b/doc/VRRP_HLD_Buzznik.md new file mode 100755 index 0000000000..6981680e84 --- /dev/null +++ b/doc/VRRP_HLD_Buzznik.md @@ -0,0 +1,730 @@ +# Virtual Router Redundency Protocol (VRRP) HLD + +#### Rev 0.1 + + + +[TOC] + + + +# List of Tables + +[Table 1: Abbreviations](#table-1-abbreviations) + +# Revision +| Rev | Date | Author | Change Description | +|:--:|:--------:|:-----------------:|:------------------------------------------------------------:| +| 0.1 | 09/05/2019 | Dilip Kumar | Initial version | +| 0.2 | 09/11/2019 | Dilip Kumar | Addressed review comments from Ben and others. Minor edits to complete the unifished sections. | +| 0.3 | 10/10/2019 | Vijay Kumar | Added VRRP version 3 and VARP support | +| | | | | +| | | | | + +# Definition/Abbreviation + +### Table 1: Abbreviations + +| **Term** | **Meaning** | +| -------- | ------------------------------------------- | +| ARP | Address Resolution Protocol | +| IP | Internet Protocol | +| LAG | Link Aggregation Group | +| LAN | Local Area Network | +| MAC | Media Access Control addresses | +| VIP | Virtual IP address | +| VLAN | Virtual Local Area Network | +| VMAC | Virtual MAC address | +| VRF | Virtual Routing and Forwarding | +| VRID | Virtual Router Identifier | +| VRRP | Virtual Router Redundency Protocol | +| VRRPv3 | Virtual Router Redundency Protocol verson 3 | +| IPv6 | Internet Protocol version 6 | +| VARP | Virtual Address Resolution Protocol | + +# About this Manual + +This document provides general overview of VRRP feature implementation based on RFC 5798 and VARP feature implementation in SONiC. + + + +# 1 Introduction and Scope + +Virtual Router Redundancy Protocol (VRRP) functionality is designed to eliminate the single point of +failure inherent in the static default routed environment. VRRP specifies an election protocol that +dynamically assigns responsibility of gateway router to one of the VRRP routers on a LAN. The VRRP +router controlling the IP(v6) address(es) associated with a virtual router is called the Master, and routes the +traffic. The election process provides dynamic fail-over in the forwarding responsibility should the Master +become unavailable. Any of the virtual router's IP(v6) addresses on a LAN can then be used as the default first +hop router by end-hosts. The advantage gained from using VRRP is a higher availability default path +without requiring configuration of dynamic routing or router discovery protocols on every end-host. + +Virtual-ARP (VARP) allows multiple switches to simultaneously route packets from a common IP address in an active-active router configuration. Each switch is configured with the same set of virtual IP addresses on corresponding VLAN interfaces and a common virtual MAC address. + + +# 2 Feature Requirements + +## 2.1 Functional Requirements + +Following requirements are addressed by the design presented in this document: + +1. Support VRRPv2 (IPv4), VRRPv3(IPv4, IPv6), VARP(IPv4, IPv6) + +2. Support multiple VRRP instances (groups) per interface + +3. Support VRRP on VLAN, PortChannel and Ethernet interfaces + +4. Support Uplink interface tracking + +5. Support preemption of a Master when a high priority VRRP node comes up + +6. Support configurable priority for VRRP instance + +7. Support configuration and management of various VRRP parameters + +8. VRRP support on non-default VRF + +9. Support REST access to VRRP objects + + + +Following requirements are beyond scope of this release. + +2. SNMP and gNMI access support to VRRP objects + + + +## 2.2 Configuration and Management Requirements + +This feature will support configuration and display CLIs to control and monitor VRRP parameters + +1. Support configuration of VRRP instances per interface + +2. Support configuration of VIP, priority, hello interval and preemption for VRRP instance + +3. Support configuration of uplink interface track + +4. Support configuration of VRRP protocol version + +5. Allow users to configure track interface weight to provide flexible policy of Master to Backup switchover + +6. Support display of various VRRP parameters as well as states using CLIs. + +7. Support configuration of VARP on interface + + + +## 2.3 Scalability Requirements + +1. Max number of VRRP instances: 128 +2. Max number of VRRP enabled interfaces: 128 +3. Max number of VRRP instances per interface: 16 +4. Max number of tracked interfaces per VRRP Instance: 8 +5. Max IP addresses per VRRP instance: 4 +6. Max number of VIP for VARP instance on an interface: 4 + + + +## 2.4 Warm Boot Requirements + +VRRP module is warm reboot compliant. That is, VRRP docker will be restarted as part of warm-reboot but will come up afresh and will build the state from scratch. The behavior of VRRP will be like Cold reboot. + +VARP module does not store any states. During warm-reboot router will be restarted. After restart VIP will be programmed in forwarding as per configuration. + + + +# 3 Feature Description + +## 3.1 Target Deployment use cases + +The following are some of the deployment use cases for VRRP and VARP + +- The Leaf nodes of the Clos network to provide first hop redundency to connected devices + + + +## 3.2 Functional Description of VRRP + +VRRP specifies an election protocol to provide the virtual router function described earlier. All protocol +messaging is performed using IP multicast datagrams, thus the protocol can operate over a variety of +multiaccess LAN technologies supporting IP multicast. Each VRRP virtual router has a single well-known +MAC address allocated to it. This document currently only details the mapping to networks using the IEEE +802 48-bit MAC address. The virtual router MAC address is used as the source in all periodic VRRP +messages sent by the Master router to enable bridge learning in an extended LAN. +A virtual router is defined by its virtual router identifier (VRID) and a set of IP addresses. A VRRP router +may associate a virtual router with its real addresses on an interface, and may also be configured with +additional virtual router mappings and priority for virtual routers it is willing to backup. The mapping +between VRID and addresses must be coordinated among all VRRP routers on a LAN. However, there is +no restriction against reusing a VRID with a different address mapping on different LANs. The scope of +each virtual router is restricted to a single LAN. +To minimize network traffic, only the Master for each virtual router sends periodic VRRP Advertisement +messages. A Backup router will not attempt to preempt the Master unless it has higher priority. This +eliminates service disruption unless a more preferred path becomes available. It's also possible to +administratively prohibit all preemption attempts. The only exception is that a VRRP router will always +become Master of any virtual router associated with addresses it owns. If the Master becomes unavailable +then the highest priority Backup will transition to Master after a short delay, providing a controlled +transition of the virtual router responsibility with minimal service interruption. +The VRRP protocol design provides rapid transition from Backup to Master to minimize service +interruption, and incorporates optimizations that reduce protocol complexity while guaranteeing +controlled Master transition for typical operational scenarios. The optimizations result in an election +protocol with minimal runtime state requirements, minimal active protocol states, and a single message +type and sender. The typical operational scenarios are defined to be two redundant routers and/or distinct +path preferences among each router. A side effect when these assumptions are violated (i.e., more than two redundant paths all with equal preference) is that duplicate packets may be forwarded for a brief period during Master election. However, the typical scenario assumptions are likely to cover the vast majority of deployments, loss of the Master router is infrequent, and the expected duration in Master election convergence is quite small ( << 1 second ). Thus the VRRP optimizations represent significant +simplifications in the protocol design while incurring an insignificant probability of brief network +degradation. +Though VRRP standard protocol present in RFC 3768 and RFC 5798 are complete in itself, there are few +limitations/drawbacks of the protocol: + +1. The point of failure which VRRP safeguards against is the interface on which VRRP instance is +present and the router as a whole. Even if all of the connectivity of master to the external network +fails, VRRP will still not trigger the failover to the backup gateway. Uplink interface tracking feature has been introduced to overcome this limitation +2. Only master router sends the advertisements, and backups just listen to them. Since backups do +not send any periodic message, it is difficult to ascertain that the backups are still active, and the +healthiness of the backup routers can not be known until the failover happens. +3. VRRP owner - if a VRRP instance's virtual IP address is same as the real interface IP address. VRRP owner has special privileges (priority is 255) and can preempt any other router acting as master. This usually causes unwanted temporary network disruption after the non-owner master had stabilized +after the failover from owner. +4. A non-owner master can not accept packets destined to the virtual IP address. Only the owner can +accept and respond to such packets. +5. Two virtual routers with same VRIDs but on different IP subnets can not co-exist on the same +VLAN because of the virtual MAC address clash. + +### 3.2.1 Virtual Router Owner + +VRRP instance whose virtual IP address (VIP) is same as real interface address is called the owner of virtual router and has the highest priority. VRRP owner is supported in SONIC's VRRP implementation. + +### 3.2.2 Virtual MAC Address + +Following virtual MAC addresses is used by the protocol (as per RFC 5798) + +IPv4 case: **00-00-5e-00-01-{vrid}** + +IPv6 case: **00-00-5e-00-02-{vrid}** + +where, vrid is user configured 1-byte virtual router identifier. VRID has interface scope; that is, VRID has to be unique among the VRRP instances on an interface. However, same VRID can be used for two or more virtual router instances across different interfaces. + +### 3.2.3 Preemption + +Preemption is turned on by default. Even if preemption is disabled, it does not affect the owner router since owner preempts the active master. Mastership switchover causes unnecessary temporary network disruption. + +### 3.2.4 VRRP Advertisement Frame + +VRRP control packets have IP protocol type as 112 (reserved for VRRP). IPv4 and IPv6 VRRP keepalives are sent to VRRP multicast address 224.0.0.18 and FF02::12 respectively. Source MAC in VRRP control packets is virtual MAC address + +### 3.2.5 ARP Request Handling + +Only master responds to the ARP requests for virtual IP address. In ARP replies sent by master, the source MAC in Ethernet header and ARP payload is virtual MAC address. + +### 3.2.6 Uplink Interface Tracking + +Interfaces other than the VRRP instance interface can be tracked for up/down events. When interface-tracking is enabled in the VRRP instance configuration, the tracked interface's operational status will be monitored. When a interface operational down event is detected on a tracked-interface, the track-priority/weight is subtracted from the current router’s priority value. Similarly, when interface operational up event is detected on the tracked-interface, the track-priority/weight is added to the router’s current priority value. + +The dynamic change of router priority can trigger mastership switchover if the preemption is enabled. **However, if the router is an owner, the mastership switchover will not happen**. + +Maximum number of interfaces that can be tracked for a virtual router instance is 8. + +## 3.3 Functional Description of VARP + +In most of Leaf-Spine deployments, redundancy in Spine layer is required to achieve high availability and to prevent network service disruption. Modern layer 2 networks adopted loop-free and balanced path networks using Multi Chassis Link Aggregation topologies with LACP port channels, leaving loop control methods (STP) as second protection layer. Spines also supports layer 3 networks, using ECMP in a scalable network topology. For unicast redundancy in layer 3, a common method is use Virtual Router Redundancy Protocol (VRRP) to provide a simple and unique gateway for Leaf level. Although VRRP provides redundancy, it is active-standby protocol and do not provide a balanced data traffic distribution over Multi Chassis Link Aggregated topologies. + +VARP provides better data traffic balancing and faster redundancy convergence, implementing active-active First Hop Router Redundancy to provide active/active unicast IP routing. + +The primary benefit of using VARP is that all configured routers are active and are able to perform routing. VARP also provides rapid failover in the event of a link or switch failure, while enabling the sharing of IP forwarding load between both switches. VARP requires configuring the same virtual-router IP address on the appropriate VLAN interfaces of both peers, as well as a global unique virtual-router MAC address. VARP functions by having both switches respond to ARP requests and GARP for a configured IP address with the “virtual-router” MAC address. This address is receive-only MAC address and no packet is ever sent with this address as its source. If IP routing is enabled, received packets will be routed as follows: when the DMAC of a packet destined to a remote network matches the configured “virtual-router” MAC address, each MLAG peer locally forwards the traffic to its next hop destination. + +### 3.3.1 Virtual MAC Address + +Administrator assigns virtual MAC address to the switch. The switch maps all virtual router IP addresses to this MAC address. The address is receive-only; the switch never sends packets with this address as the source. + +# 4 Feature Design + + + +## 4.1 Design Overview + +### 4.1.1 Basic Approach + +Keepalived (https://www.keepalived.org/) open source code is chosen for VRRP control plane code. + +![VRRP Keepalived Design](images/VRRP_Keepalived_design.PNG "Figure : Design") + + +### 4.1.2 Container + +A new container "vrrp" has been added to host VRRP protocol operation. VRRP source code is not maintained in SONIC repos. Instead, VRRP code is downloaded from keeplaived project repo at the compile time, patches are applied to it (for any fixes by us) and then compiled/linked with SONIC binary. + +New container "vrrp" handles both VRRP and VARP functionality. For VARP the configuration is programmed in kernel through 'vrrpmgrd' module and 'vrrpsyncd' programs the hardware entry in SAI. + +### 4.1.3 SAI Overview + +VRRP virtual router interface can be created by setting the attribute “SAI_ROUTER_INTERFACE_ATTR_IS_VIRTUAL” with **create_router_interface** API. + + * @brief RIF creation is a virtual RIF. + * + * Create a Virtual RIF object, which only programs the ingress router MAC. + * This simplifies the management of VRRP master router's configuration in + * SAI adapter, as defined by RFC 5798 (or similar proprietary protocols). + * Using a Virtual RIF allows SAI to optimize resources, so neighbor entries + * cannot be learned on a Virtual RIF. On a virtual RIF following attributes + * are invalid: ADMIN state, MTU size, packet action and multicast enable. + * Alternatively VRRP can also be configured using native RIF objects without + * using VIRTUAL attribute, with the expectation that SAI adapter will consume + * resources that will not be used. + * + * @type bool + * @flags CREATE_ONLY + * @default false + */ + + SAI_ROUTER_INTERFACE_ATTR_IS_VIRTUAL, +## 4.2 DB Changes + +At a high level below are some of the interactions between relevant components and the DB involved for VRRP support in SONiC architecture. + + + +![VRRP Arch](images/VRRP_architecture.PNG "Figure : Arch") +__Figure 1: VRRP Architecture__ + +### 4.2.1 CONFIG_DB changes + +**VRRP_TABLE** + +Producer: config manager + +Consumer: VrrpMgr + +Description: New table that stores VRRP configuration for per interface + VRID. + +Schema: + +``` +;New table +;holds the VRRP configuration per interface and VRID + +key = VRRP_TABLE:interface_name:vrid:address_family + ; Interface name string like Vlan1 or PortChannel002 or Ethernet4 + ; vrid is an integer +; field = value +vrid = 1*3DIGIT ; VRRP Instance Identifier +address_family = "IPv4"/"IPv6"; Address Family of VRRP instances +vip = ip_address ; Virtual IPv4/IPv6 address. This is a list of IPv4/IPv6 addresses +priority = vrrp_priority ; Priority of VRRP instance +adv_interval = 1*3DIGITS ; Advertisement interval for VRRP. Default = 1sec +state = vrrp_state ; String denoting the state of VRRP instance +version = vrrp_version ; VRRP version. VRRP for IPv6 will always be version 3 +pre_empt = "true"/"false" ; VRRP premption is enabled? Default is True +track_interface = track_interface ; List of interfaces tracked by a VRRP instance + |weight|; This is repeated for the configured tracking interfaces +``` + +Example:- + +**Key**: VRRP_TABLE:Vlan11:1 + +**Value**: + +​ 'vrid': '1', +​ 'vip': '4.1.1.100,', + +​ 'priority': '50', + +​ 'adv_interval': '1', +​ 'state': '', + +​ 'version': '2', +​ 'pre_empt': 'True', +​ 'track_interface': +​ 'Ethernet7|weight|20, + +​ PortChannel001|weight|40 + + + +Example:- Entery with multiple virtual IPs + +**Key**: VRRP_TABLE:Vlan11:1 + +**Value**: + +​ 'vrid': '1', +​ 'vip': '4.1.1.100,4.1.1.200,4.1.1.201,4.1.1.202,', + +​ 'priority': '50', + +​ 'adv_interval': '1', +​ 'state': '', + +​ 'version': '2', +​ 'pre_empt': 'True', +​ 'track_interface': +​ 'Ethernet7|weight|20, + +​ PortChannel001|weight|40 + + + +**Key**: VRRP_TABLE:Vlan12:2 + +**Value**: + +​ 'vrid': '2', +​ 'vip': 'fe80::100,4::100', + +​ 'priority': '50', + +​ 'adv_interval': '1', +​ 'state': '', + +​ 'version': '3', +​ 'pre_empt': 'True', +​ 'track_interface': +​ 'Ethernet8|weight|20, + +​ PortChannel002|weight|40 + + + +Example:- VRRP for IPv6 address + +**VARP_TABLE** + +Producer: config manager + +Consumer: VrrpMgr + +Description: New table that stores VARP configuration for per interface. + +Schema: + +``` +;New table +;holds the VARP configuration per interface + +key = VARP_TABLE:interface_name:address_family + ; Interface name string like Vlan1 or PortChannel002 or Ethernet4 +address_family = "IPv4"/"IPv6"; Address Family of VRRP instances +vip = ip_address ; Virtual IPv4/IPv6 address. This is a list of IPv4/IPv6 addresses +``` + + + +``` +;New table +;holds the VARP_GLOBAL configuration per router + +key = VARP_GLOBAL_TABLE:address_family + ; Interface name string like Vlan1 or PortChannel002 or Ethernet4 +vmac = mac ; Virtual mac address for all the VARP Virtual IP +``` + + + +Example:- + +**Key**: VRRP_TABLE:Vlan15 + +**Value**: +​ 'vip': '15.1.1.100,16.1.1.100', + +### 4.2.2 APP_DB Changes + +**VRRP_TABLE** + +Producer: VrrpMgr + +Consumer: VrrpOrch + +Description: This is a new table that contains VRRP and VARP state information + +Schema: + +``` +; New table +; holds the VRRP state and VMAC information + +key = VRRP_TABLE:interface_name:vip:type + interface_name ; interface name as a string. Vlan, Ethernet or PortChannel + vip ; virtual IP address in a.b.c.d/32 or a:b:c::d format + type ; IP(v6) address type string. + +; field = value +vmac = virtual_mac_address ; Virtual MAC address associated with VRRP instance +``` + +Example:- + +**Key**: VRRP_TABLE:Vlan1000:[40.10.8.101/32:ipv4] + +**Value**: "vmac":"00:00:5e:00:01:08" + +## 4.3 Modules Design and Flows + + + + + + +## 5 CLI + +SONIC Click based configuration and monitoring CLIs have been introduced in SONIC for VRRP + +### 5.1 Configuration Commands + +#### 5.1.1 Configuration Commands for IPv4 VRRP + +``` +configure interface vrrp add +This command adds/enables a IPv4 VRRP instance on an interface. +- interface_name - name of interface (Ethernet or Vlan or PortChannel) over which VRRP is to be enabled. +- vrid - VRRP instance identifier. + +configure interface vrrp remove +This command removes a IPv4 VRRP instance from an interface. + +configure interface vrrp vip add +This command adds a virtual IP address for a VRRP instance on an interface. User is allowed to create multiuple VIPs for a IPv4 VRRP instance. VIP must fall in the interface's subnet. +- interface_name - name of interface (Ethernet or Vlan or PortChannel) over which VIP is being added +- vrid - VRRP instance identifier. +- virtual_ip_address - VIP address in dotted decimal IPv4 address + +configure interface vrrp vip remove +This command deletes a already configured VIP from a IPv4 VRRP instance + +configure interface vrrp priority +This command configures priority for a VRRP instance +- priority - a number between 1 and 254 with 1 being the lowest and 254 being the highest priority. Default is 100. Priority 255 is reserved for owner VRRP router. + +configure interface vrrp adv_interval +This command configures VRRP periodic advertisement interval for a VRRP instance +- interval - a number between 1 and 255. Unit is in seconds. Default is 1sec. + +configure interface vrrp pre_empt enable +This command enables premeption of a Master when a higher priority VRRP router arrives +- enable - Enable premeption. Default is enabled. + +configure interface vrrp pre_empt disable +This command disables premeption of a Master when a higher priority VRRP router arrives +- disable - Disable premeption. Default is enabled. + +configure interface vrrp version +This command configures VRRP protocol version for IPv4 VRRP instances + +configure interface vrrp track_interface add +This command adds a track interface to a VRRP Instance. A maximum of 8 track interfaces can be added to a VRRP instance. +- track_interface - Interface to track. Interface can be Ethernet, Vlan or PortChannel +- weight - weight or importance assigned to the track_interface. When track interface goes down, the priority of VRRP instance will be reduced by weight + +configure interface vrrp track_interface remove +This command removes an already configured track interface from a IPv4 VRRP Instance. +``` + +#### 5.1.1 Configuration Commands for IPv6 VRRP + +``` +configure interface vrrp6 add +This command adds/enables a IPv6 VRRP instance on an interface. +- interface_name - name of interface (Ethernet or Vlan or PortChannel) over which VRRP is to be enabled. +- vrid - VRRP instance identifier. + +configure interface vrrp6 remove +This command removes a IPv6 VRRP instance from an interface. + +configure interface vrrp6 vip add +This command adds a virtual IPv6 address for a IPv6 VRRP instance on an interface. User is allowed to create multiuple VIPs for a VRRP instance. VIP must fall in the interface's subnet. +- interface_name - name of interface (Ethernet or Vlan or PortChannel) over which VIP is being added +- vrid - VRRP instance identifier. +- virtual_ip_address - VIP address in dotted decimal IPv4 address + +configure interface vrrp6 vip remove +This command deletes a already configured VIP from a IPv6 VRRP instance + +configure interface vrrp6 priority +This command configures priority for a VRRP instance +- priority - a number between 1 and 254 with 1 being the lowest and 254 being the highest priority. Default is 100. Priority 255 is reserved for owner VRRP router. + +configure interface vrrp6 adv_interval +This command configures VRRP periodic advertisement interval for a VRRP instance +- interval - a number between 1 and 255. Unit is in seconds. Default is 1sec. + +configure interface vrrp6 pre_empt enable +This command enables premeption of a Master when a higher priority VRRP router arrives +- enable - Enable premeption. Default is enabled. + +configure interface vrrp6 pre_empt disable +This command disables premeption of a Master when a higher priority VRRP router arrives +- disable - Disable premeption. Default is enabled. + +configure interface vrrp6 version +This command configures VRRP protocol version for IPv4 VRRP instances + +configure interface vrrp6 track_interface add +This command adds a track interface to a IPv6 VRRP Instance. A maximum of 8 track interfaces can be added to a VRRP instance. +- track_interface - Interface to track. Interface can be Ethernet, Vlan or PortChannel +- weight - weight or importance assigned to the track_interface. When track interface goes down, the priority of VRRP instance will be reduced by weight + +configure interface vrrp6 track_interface remove +This command removes an already configured track interface from a VRRP Instance. +``` + +SONIC kLISH based configuration and monitoring CLIs have been introduced in SONIC for VRRP + +``` +[no] configure vrrp address-family {ipv4 | ipv6} +This command configures/removes VRRP instance in IPv4 or IPv6 address-family. +- vrid - VRRP instance identifier. + +[no] vip +This command configures/removes a virtual IP address for a VRRP instance on an interface. User is allowed to create multiuple VIPs for a IPv4 VRRP instance. VIP must fall in the interface's subnet. For IPv6 VRRP instance VIP could be any link-local address. +- virtual_ip_address - VIP address + +priority +This command configures priority for a VRRP instance +- priority_value - a number between 1 and 254 with 1 being the lowest and 254 being the highest priority. Default is 100. Priority 255 is reserved for owner VRRP router. + +adv_interval +This command configures VRRP periodic advertisement interval for a VRRP instance +- int_value - a number between 1 and 255. Unit is in seconds. Default is 1sec. + +[no] pre_empt +This command enables/disables premeption of a Master when a higher priority VRRP router arrives. + +[no] vrrpv3 +This command configures/removes VRRP version 3 for a IPv4 VRRP instance. IPv4 VRRP instances are version 2 by default. + +[no] track_interface + +This command adds/removes a track interface to a VRRP Instance. A maximum of 8 track interfaces can be added to a VRRP instance. +- track_ifname - Interface to track. Interface can be Ethernet, Vlan or PortChannel +- weight - weight or importance assigned to the track_interface. When track interface goes down, the priority of VRRP instance will be reduced by weight + +``` + +SONIC kLISH based configuration and monitoring CLIs have been introduced in SONIC for VARP + +``` +[no] ip virtual-router address +This command configures/removes virtual IPv4 on an interface. +- vip - IPv4 virtual address. + +[no] ipv6 virtual-router address +This command configures/removes virtual IPv6 on an interface. +- vip - IPv6 virtual address. + +[no] ip virtual-router mac-address +This command configures/removes virtual mac address for all the IPv4/IPv6 virtual addresses. + + +``` + +#### + +### 5.2 Show Commands + +``` +show vrrp + - lists all the VRRP instances including their current state + Sample output:- +admin@sonic:~$ show vrrp +Interface_Name VRID State VIP Cfg_Prio Curr_Prio + Vlan1 1 Backup 4.1.1.100 100 120 + Vlan2 2 Master 4.1.2.100 100 100 + Vlan3 3 Backup 4.1.3.100 100 100 + Vlan4 4 Backup 4.1.4.100 100 100 + Vlan5 5 Master 4.1.5.100 100 100 + + +show vrrp + - This command displays data about a VRRP instance in detail + Sample output:- +admin@sonic:~$ show vrrp Vlan1 1 +Vlan1, VRID 1 +Version is 2 +State is Backup +Virtual IP address: + 4.1.1.100 +Virtual MAC address is 0000.5e00.0101 +Track interface: + Intfname State Priority + Ethernet7 Up 10 + PortChannel001 Up 10 +Configured Priority is 100, Current Priority is 120 +Advertisement interval is 1 sec +Preemption is enabled +``` + +## + +``` +show varp + - lists all the VARP instances including their current state + Sample output:- +IP virtual router is configured with MAC address: 24cd.5a29.cc31 +Interface IP Address Virtual IP Address Status Protocol +Vlan15 10.1.1.3/24 10.1.1.15 up up +Vlan15 10.1.1.3/24 10.1.1.16 up up +Vlan15 10.1.1.3/24 10.1.1.17 up up +Vlan20 10.12.1.6/24 10.12.1.51 up up +Vlan20 10.12.1.6/24 10.12.1.53 up up +Vlan20 10.12.1.6/24 10.12.1.55 up up + + +``` + + +## 6 Serviceability and Debug + +The existing logging mechanisms shall be used. Proposed debug framework shall be used for internal state dump. + +## 7 Warm Reboot Support + +Currently, warm-reboot is not supported for VRRP. That is, warm-reboot will simply restart the VRRP docker without VRRP storing any data for warm restart. + +VARP is stateless, during warm-reboot router will go down and restart with new configuration. + +## 8 Unit Test cases + +| Test Cases | UT Action and observation | Remarks | +| ---------------------------- | ------------------------------------------------------------ | ---------------------------------------------------------- | +| Configuration | | | +| | Validate IPv4 VRRP config on Physical Interface | | +| | Validate IPv4 VRRP config on Ve Interface | | +| | Ensure that VMAC is generated based on VRID for IPv4 VRRP | | +| | Ensure VMAC is programmed in FDB and VIP in LPM | | +| | Validate IPv6 VRRP config on Physical Interface | | +| | Validate IPv6 VRRP config on Ve Interface | | +| | Ensure that VMAC is generated based on VRID for IPv6 VRRP | | +| | Ensure VMAC is programmed in FDB and VIP in LPM for IPv6 VRRP | | +| | Validate VRRP on more than one interface | | +| | Validate VRRP with more than one VIP per instance | | +| | Validate multiple VRRP instance per interface | | +| Failover | | | +| | Validate VRRP master failover with priority change | | +| | Validate VRRP master failover by disabling a vrrp group | Add 'disable' feature if needed and feasible | +| | Validate VRRP master failover by removing all virtual-ip | | +| | Validate VRRP master failover by removing interface IPv4/IPv6 address | | +| | Validate VRRP master failover by deleting master VRRP session | | +| | Validate VRRP master failover by bringing down interface | | +| | Validate VRRP master failover by deleting interface | | +| VRRP parameter changes | | | +| | Validate gratutious ARP | | +| | Validate adv interval | | +| | Validate version change for IPv4 VRRP instance | | +| ARP/ND resolve to VIP | | | +| | validate ARP resolution from host to VRRP session VIP | Hosts should get arp response with VMAC as source MAC | +| | Perform Master failover and check that ARP resolution to VIP will happen with VMAC | | +| Ping to VIP | | | +| | Validate ping to IVRRP from host, backup & non vrrp router works | | +| L3 Forwarding with VIP as GW | | | +| | Validate that IPv4/IPv6 traffic forwarding with VIP as gateway works from source to destination host | | +| | Perform VRRP master failover and check if traffic forwarding continues with new master | | +| VRRP Owner | | | +| | Configure VRRP for IPv4/IPv6 session with VIP same as interface IP and ensure this session becomes master and priority of session becomes max=255 | | +| | Ensure ping to VIP works in owner case | | +| | Ensure arp/nd resolution with VIP as gateway in owner case | ARP/ND should get resolved with VMAC and not interface MAC | +| | Ensure ping and traffic forwarding continues after owner failover and comes back | | +| | Traffic forwarding works with VIP as gateway | | +| Interface tracking | | | +| | Run below tests with Ping and traffic running | | +| | Enable interface tracking in a VRRP group with priority, bring down the port and ensure VRRP session priority comes down and session transition from Master to Backup State | | +| | Enable the tracked interface and check if session priority gets restored and session becomes master | | +| | Repeat this test on Port-channel interface | | +| | Repeat this test by deleting and re-adding port-channel interface | | \ No newline at end of file diff --git a/doc/aaa/Persistent HTTP Connections from CLI.md b/doc/aaa/Persistent HTTP Connections from CLI.md new file mode 100644 index 0000000000..4d455c44f7 --- /dev/null +++ b/doc/aaa/Persistent HTTP Connections from CLI.md @@ -0,0 +1,60 @@ +Persistent HTTP connections from CLI +==================================== + +The SONiC CLI provided by the management framework container runs an instance of +klish, which provides a fixed set of commands. The CLI is simply a front-end to +the REST server, and each command is mapped to a corresponding REST endpoint. + +# Command Flow + +When the user enters a specific command on the CLI, the corresponding `ACTION` +tag in the CLI XML specification shells out to a Python script with any +arguments and an optional template to format the returned values. This script +connects to the REST server on the local machine over HTTPS, retrieves and +formats the JSON response. The Python script then exits, terminating any HTTP +connection that had been set up. + +This is the current behavior, even without RBAC support, which means that every +command will need to set up a new HTTPS connection. However, when RBAC is +enabled, it is not likely to cause a noticeable performance impact, since the +system is already incurring the TLS overhead. + +As can be seen from the flow above, it is not possible to set up a persistent +HTTP connection, since every command spawns a new connection. + +# Alternative Designs + +This section describes some alternative designs that will enable the CLI to +create a persistent connection. + +## Proxy service + +As part of the management framework, we can add a "proxy" service that is +spawned with the CLI. This service will set up a secure HTTP connection for the +authenticated user, and create a local unix socket that is accessible only by +that user. The CLI XML will remain unchanged, but the Python ApiClient class +will be changed to connect to the local socket. + +This will still create independent HTTP connections, but they can be insecure +connections, while the proxy service will transfer the connections from the +insecure unix socket to the secure tunnel, which will reduce the TLS connection +time. + +**Note:** The security considerations have not been completely mapped out, and +this may open the system up to security holes. + +## Klish modification + +This approach considers modifying the Klish executable. When Klish is spawned, +it will set up the HTTPS connection and keep it alive as long as the CLI is +active. Each `ACTION` tag will call into klish functions that will connect to +the existing HTTPS connection. + +This approach is the most secure option, however, it needs heavy modification to +klish, and there are several unknowns at this time. + +## No modification - Buzznik + +This approach leaves the design as is for the Buzznik release. Every command +will continue to create a new HTTPS connection, as it does today, and will tear +down the connection on completion of the request. diff --git a/doc/aaa/SONiC RBAC HLD.md b/doc/aaa/SONiC RBAC HLD.md new file mode 100644 index 0000000000..2f6d4a3e37 --- /dev/null +++ b/doc/aaa/SONiC RBAC HLD.md @@ -0,0 +1,460 @@ + + +# Authentication and Role-Based Access Control + +# High Level Design Document +#### Rev 0.1 + +# Table of Contents +- [Revision History](#revision-history) +- [About this Manual](#about-this-manual) +- [Scope](#scope) +- [Definitions/Abbreviations](#definitionsabbreviations) +- [1 Feature Overview](#1-feature-overview) + * [1.1 Requirements](#11-requirements) + + [1.1.1 Functional Requirements](#111-functional-requirements) + - [1.1.1.1 NBI Authentication](#1111-nbi-authentication) + - [1.1.1.2 CLI Authentication to REST server](#1112-cli-authentication-to-rest-server) + - [1.1.1.3 Translib Enforcement of RBAC](#1113-translib-enforcement-of-rbac) + - [1.1.1.4 Linux Groups](#1114-linux-groups) + - [1.1.1.5 Certificate-based Authentication for gNMI and REST](#1115-certificate-based-authentication-for-gnmi-and-rest) + - [1.1.1.6 Local User Management and UserDB Sync](#1116-local-user-management-and-userdb-sync) + + [1.1.2 Configuration and Management Requirements](#112-configuration-and-management-requirements) + + [1.1.3 Scalability Requirements](#113-scalability-requirements) + - [1.1.3.1 REST Server](#1131-rest-server) + - [1.1.3.2 gNMI Server](#1132-gnmi-server) + - [1.1.3.3 Translib](#1133-translib) + + [1.1.4 Warm Boot Requirements](#114-warm-boot-requirements) + * [1.2 Design Overview](#12-design-overview) + + [1.2.1 Basic Approach](#121-basic-approach) + + [1.2.2 Container](#122-container) + + [1.2.3 SAI Overview](#123-sai-overview) +- [2 Functionality](#2-functionality) + * [2.1 Target Deployment Use Cases](#21-target-deployment-use-cases) + * [2.2 Functional Description](#22-functional-description) +- [3 Design](#3-design) + * [3.1 Overview](#31-overview) + * [3.2 DB Changes](#32-db-changes) + + [3.2.1 CONFIG DB](#321-config-db) + + [3.2.2 APP DB](#322-app-db) + + [3.2.3 STATE DB](#323-state-db) + + [3.2.4 ASIC DB](#324-asic-db) + + [3.2.5 COUNTER DB](#325-counter-db) + * [3.3 Switch State Service Design](#33-switch-state-service-design) + + [3.3.1 Orchestration Agent](#331-orchestration-agent) + + [3.3.2 Other Process](#332-other-process) + * [3.4 SyncD](#34-syncd) + * [3.5 SAI](#35-sai) + * [3.6 User Interface](#36-user-interface) + + [3.6.1 Data Models](#361-data-models) + + [3.6.2 CLI](#362-cli) + - [3.6.2.1 Configuration Commands for User Management](#3621-configuration-commands-for-user-management) + - [3.6.2.2 Show Commands](#3622-show-commands) + - [3.6.2.3 Debug Commands](#3623-debug-commands) + - [3.6.2.4 IS-CLI Compliance](#3624-is-cli-compliance) + + [3.6.3 REST API Support](#363-rest-api-support) +- [4 Flow Diagrams](#4-flow-diagrams) +- [5 Error Handling](#5-error-handling) + * [5.1 REST Server](#51-rest-server) + * [5.2 gNMI server](#52-gnmi-server) + * [5.3 CLI](#53-cli) + * [5.4 Translib](#54-translib) +- [6 Serviceability and Debug](#6-serviceability-and-debug) +- [7 Warm Boot Support](#7-warm-boot-support) +- [8 Scalability](#8-scalability) +- [9 Unit Test](#9-unit-test) +- [10 Internal Design Information](#10-internal-design-information) + +# Revision History +| Rev | Date | Author | Change Description | +|:---:|:-----------:|:------------------:|-----------------------------------| +| 0.1 | 10/22/2019 | Jeff Yin | Initial version | +| 0.2 | 10/30/2019 | Jeff Yin | Revision after joint review with Broadcom/Dell | + +# About this Manual +This document provides a high-level design approach for authentication and RBAC in the SONiC Management Framework. + +For authentication, this document describes how the CLI and programmatic interfaces (REST, gNMI) -- collectively referred to in this document as the northbound interfaces (NBIs) -- will authenticate users and the supported credentials and methods. + +For authorization, this document describes a centralized authorization approach to be implemented in the Translib component of the SONiC Management Framework. + +# Scope +This document covers the interfaces and mechanisms by which NBIs will authenticate users who wish to access and configure the SONiC system via the Management Framework. It will also cover RBAC enforcement. + +This document will NOT extensively cover (or assumes the pre-existence of): +- Implementation of remote authentication and authorization (RADIUS, TACACS+, etc.) +- Public Key Infrastructure management: X.509v3 certificate installation, deletion, trust store management, etc. + + +# Definitions/Abbreviations + +| **Term** | **Meaning** | +|:--------------------------|:-------------------------------------| +| RBAC | Role-Based Access Control | +| AAA | Authentication, Authorization, Accounting | +| CLI | Command-Line Interface | +| REST | REpresentational State Transfer | +| gNMI | gRPC Network Management Interface | +| NBI | North-bound Interfaces (CLI, REST, gNMI) | + + + +# 1 Feature Overview + +## 1.1 Requirements + +- The SONiC Management Framework must support authenticated access for the various supported northbound interfaces (NBIs): CLI, REST, and gNMI. Since the CLI works with the REST server, it must also be authenticated with the REST server. +- The NBIs must pass along the username info to Translib, so that Translib can enforce role-based access (RBAC). +- For RBAC, Linux Groups will facilitate authorization. Initially, two roles will be supported: read/write and read-only. Additionally, remotely-authenticated users who map to a defined role will be authenticated as a static global user on the system. These limitations should be revisited at a later date. +- Local user management: CLIs and APIs for creating and managing local users on the system -- their usernames, passwords, and roles. + +### 1.1.1 Functional Requirements + +#### 1.1.1.1 NBI Authentication +A variety of authentication methods must be supported: +* **CLI** authentication is handled via the same mechanisms supported by SSH + - Password-based Authentication + - Public-key Authentication +* **REST** authentication + - Password-based Authentication with JWT token-based authentication + - Certificate-based Authentication with JWT token-based authentication + - The REST server must be enhanced to accept all types of authentication concurrently +* **gNMI** Authentication + - Password-based Authentication with Token-based authentication + - Certificate-based Authentication + +#### 1.1.1.2 CLI Authentication to REST server +Given that the CLI module works by issuing REST transactions to the Management Framework's REST server, the CLI module must also be able to authenticate with the REST server when REST authentication is enabled. This can be accomplished via the following steps: +1. When a user is created on the system, a self-signed certificate is generated with the username embedded into the Subject field. That self-signed certificate will be stored in the user's home directory, with access permissions limited to that user only. The self-signed certificate will be used for mutual authentication from the KLISH shell to the REST server. +**Design note:** Certificate-based authentication is desired over password-based authentication for the CLI-to-REST connection because it prevents the need to store and forward any plaintext passwords. Per-user certificates with strict file permissions are desired so as to ensure that users cannot drop into the Linux shell and perform operations as other than themselves (e.g., via `curl` with a shared certificate, or someone else's certificate). +2. When a user logs into the switch, the switch will authenticate the user by using the username and password credentials. +3. When the KLISH process is started, its UID and GID are set to those of the user that was authenticated by `sshd` or `login` (as through the console). +4. When the KLISH shell is spawned, it will create a persistent, local HTTPS connection over an internal socket, and send the REST request to authenticate this KLISH session as the logged-in user, which can be looked up through NSS by using `getpwuid()`. This authentication request will contain the user's client certificate. +5. The REST server will authenticate the user and return a token with the username encoded in it. The KLISH CLI must cache this token and use it for all future requests from this KLISH session. The REST server will also maintain this token to username mapping with it so as to identify all future requests as well. This will also allow the REST server in creating audit logs for all the requests sent from KLISH, as well as pass the username to Translib for RBAC enforcement. +6. KLISH CLI session will store the authentication token, and from then on, KLISH CLI will send REST requests using the persistent connection with the authentication token in the HTTP header to the REST server for all the CLI commands. +7. The KLISH session must be able to cleanly handle ctrl-c breaks while it is busy waiting on a response from the REST server. +8. When the user exits the KLISH CLI session, the HTTP persistent connection is closed. The REST server will clean up the corresponding authentication token for its corresponding KLISH CLI Client. + +#### 1.1.1.3 Translib Enforcement of RBAC +The Translib module in the [management framework](https://github.com/project-arlo/SONiC/blob/master/doc/mgmt/Management%20Framework.md) is a central point for all commands, regardless of the interface (CLI, REST, gNMI). Therefore, the RBAC enforcement will be done in Translib library. + +The CLI, REST, and gNMI will result in Translib calls with xpath and payload. Additionally, the REST and gNMI server modules must pass the username to the Translib. The RBAC checks will be enforced in the **Request Handler** of the Translib. The Request Handler processes the entire transaction atomically. Therefore, if even one operation in the transaction is not authorized, the entire transaction is rejected with the appropriate "Not authorized" error code. If the authorization succeeds, the transaction is passed to the Common Application module for further ‘transformation’ into the ABNF format. The rest of the flow of the transaction through the management framework is unmodified. + +At bootup time of the manageability framework (or gNMI container), it is recommended to cache the Privilege, Tenant and Resource Tables(RBAC tables) stored in the Config DB. This is because every command needs to access the information in the RBAC tables in order to authorize the access for the information in other tables. Alternately, instead of caching the entire RBAC tables, the information can be cached once the record is read from the DB. Additionally, the Translib must listen to change notifications on these RBAC Tables in order to keep its cache current. + +As described in section [1.1.1.4 Linux Groups](#1114-linux-groups), the enforcement of the users and roles in Linux will be done via the Linux groups. A user can use Linux commands on the Linux shell and create users and Linux groups (which represent the roles). This will mean that the information in the RBAC tables is no longer current. In order to keep the information in the RBAC tables and the Linux `/etc/passwd` file in sync, a service must run on the host (not the container) to keep the databases in sync. + +Since Translib is the main authority on authorized operations, this means that the NBIs cannot render to the user what they are and are not allowed to do. The CLI, therefore, renders the entire command tree to a user, even the commands they are not authorized to execute. + +#### 1.1.1.4 Linux Groups +Initially, only two roles will be supported: +- `admin` -- can perform read and write functions across all attributes (i.e., GET/PUT/PATCH/etc.) +- `operator` -- can only perform read functions on all attributes (i.e., GET only) +These privileges will be enforced by Translib. + +RBAC will be facilitated via Linux Groups: +- Local users + - Local user with `Operator` role is added into `operator` group + - Local user with `Admin` role is added into `admin` group and is a `sudoer`. +- Remote users + - Remote users with `Operator` role are mapped to the same global `remote-user` user who is part of `operator` group + - Remote users with `Admin` role are mapped to the same global `remote-user-su` user who is part of `admin` group and is a `sudoer` + - This means that all remote users will share the same accounts on the system, which also means they will share the same /home directory and certificate to log into the REST server. + + In the future, this "global user" approach will be revisited so that remote users are authenticated with their own username so that their activities may be properly audited. + +#### 1.1.1.5 Certificate-based Authentication for gNMI and REST +For the initial release, it will be assumed that certificates will be managed outside of the NBIs. That is, no CLIs or REST/gNMI interfaces will be implemented to support public key infrastructure management for certificates and certificate authorities. Certificates will be manually generated and copied into the system via Linux utilities. + +The exception to this is the self-signed certificates used for CLI authentication to the REST server. Those certificates will be auto-generated when a user is created on the system. These certificates should be copied to a user's home directory _as well as_ the trust store so that the REST server can use them for authenticating local CLI sessions. + +User certificates must be stored in a user's home directory. Their corresponding private keys must also be stored in the user's home directory, albeit with restricted permissions so that they are not readable by other users. + +The gNMI server will use a trust store for CA certificates from a location such as `/usr/local/share/ca-certificates`. The trust store itself must be managed by [existing Linux tools](https://manpages.debian.org/jessie/ca-certificates/update-ca-certificates.8.en.html). + +The gNMI server must implement a method by which a username can be determined from the presented client certificate, so that the username can thus be passed to Translib for RBAC enforcement. The username will be derived from the Subject field of the X.509v3 certificate. + +Users must be informed by way of documentation so that they know how to manage their certificate infrastructure in order to properly facilitate gNMI communication. + +The REST server must use the same certificate scheme as the gNMI server to validate client certificates. + +#### 1.1.1.6 Local User Management and UserDB Sync +An interface must be developed for local user management, so that administrators can add users and assign passwords and roles to them. Administrators with the appropriate role must be able to add/delete users, modify user passwords, and modify user roles. They must be able to do so through all of the NBIs, meaning that a YANG model and CLI tree must be developed. + +Users must be added to the Linux database (`/etc/passwd`, `/etc/group`, and optionally `/etc/shadow`). That is where a user is mapped to a Linux [User Identifier](https://en.wikipedia.org/wiki/User_identifier) (UID) and primary [Group Identifier](https://en.wikipedia.org/wiki/Group_identifier) (GID). When users are created they also need to be assigned roles. Roles are simply defined as Linux groups (`/etc/group`) and assigned to users as [Supplementary GIDs](https://en.wikipedia.org/wiki/Group_identifier#Supplementary_groups). + +When a user is created it also needs to be assigned certificates that will allow them to communicate with the REST server. Finally, all users need to be added to the REDIS database (see [section 3.2.6]()) where additional information about each user can be stored (e.g. *tenant*). + +Since these operations (i.e. creating Linux users, assigning certificates, etc.) are non-trivial, the process of creating users will be entrusted to the Host Account Management Daemon (**hamd**). + +##### 1.1.1.6.1 Host Account Management Daemon (hamd) + +The **hamd** process runs on the host. It is accessed via a DBus interface that provides the ability to access and/or modify the host's Linux database (`/etc/passwd`, `/etc/group`, and optionally `/etc/shadow`). Since DBus is a secured interface we can control which processes will be allowed to access **hamd**. + +The **hamd** process will provide the following DBus APIs to create/modify/delete user and group (role) accounts: + +- **useradd**: To create new users (similar to GNU [useradd](http://man7.org/linux/man-pages/man8/useradd.8.html)) +- **userdel**: To delete a user (similar to GNU [userdel](http://man7.org/linux/man-pages/man8/userdel.8.html)) +- **passwd**: To change a user password (similar to GNU [passwd](http://man7.org/linux/man-pages/man1/passwd.1.html)) +- **set_roles**: To set a user's roles (similar to GNU [usermod](http://man7.org/linux/man-pages/man8/usermod.8.html)) +- **groupadd**: To create new groups/roles (similar to GNU [groupadd](http://man7.org/linux/man-pages/man8/groupadd.8.html)) +- **groupdel**: To delete groups/roles (similar to GNU [groupdel](http://man7.org/linux/man-pages/man8/groupdel.8.html)) + +##### 1.1.1.6.2 User management with hamd + +Applications that need to manage users (for example **click** or **klish**) can do so by using **hamd**'s DBus **ham.accounts** interface. DBus services such as **hamd** publish their interfaces. This can be retrieved and analyzed at runtime in order to understand the used implementation. The resulting introspection data is in XML format. DBus debug tools such as **qdbus** can be used to retrieve this data. At the time this document was written, the DBus XML definition for the APIs defined in the previous section was: + +> ```xml +> +> +> +> +> +> +> +> +> +> +> +> +> +> +> +> +> +> +> +> +> +> +> +> +> +> +> +> +> +> +> ``` + +##### 1.1.1.6.3 The hamctl shell program + +A utility program, **hamctl**, is provided to make it easier for operators to interact with **hamd** from a Linux shell (e.g. bash). This is primarily a debug tool and *should not be invoked from other programs*. Programs should use the DBus interface described above. + +Users logged in at a shell terminal can control **hamd** (e.g. ask it to create or delete a user) with **hamctl**. **hamctl** is sell-documented. Simply invoke "**hamctl --help**" to get the list of commands available. + +##### 1.1.1.6.4 Name Service Switch + +In addition to providing APIs to create/modify/delete user and group (role) accounts, **hamd** also provides APIs to simply read user and group (role) accounts. Here's the list: + +- **getpwnam**: To retrieve user credentials (similar to POSIX [getpwnam](http://man7.org/linux/man-pages/man3/getpwnam.3.html)) +- **getpwuid**: To retrieve user credentials (similar to POSIX [getpwuid](http://man7.org/linux/man-pages/man3/getpwnam.3.html)) +- **getgrnam**: To retrieve group/role credentials (similar to POSIX [getgrnam](http://man7.org/linux/man-pages/man3/getgrnam.3.html)) +- **getgrgid**: To retrieve group/role credentials (similar to POSIX [getgrgid](http://man7.org/linux/man-pages/man3/getgrnam.3.html)) + +These APIs, however, are meant to be invoked through [NSS](https://en.wikipedia.org/wiki/Name_Service_Switch) (name service switch). That is, applications running in containers can simply continue invoking the standard POSIX APIs (`getpwnam()`, `getgrnam()`, etc) and a Host Account Management NSS module will ensure that the credentials get retrieved from **hamd** running on the host. The HAM NSS module (`libnss_ham.so.2`) need be installed and configured (`/etc/nsswitch.conf`) in the containers that require access to the host's Linux database. + +### 1.1.2 Configuration and Management Requirements +An interface and accompanying CLI must be developed for local user management. Local users should be configurable like any other feature: via CLI, REST, and gNMI. Additionally, users may also be created and managed via Linux commands in the Bash shell. This will add additional complexity and require a service to sync between the Redis DB and the Linux user database. + + +### 1.1.3 Scalability Requirements +Adding authentication to NBIs will result in some performance overhead, especially when doing operations involving asymmetric cryptography. Care should be taken to leverage performance-enhancing features of a protocol wherever possible. + +#### 1.1.3.1 REST Server +- Persistent HTTP connections can be used to preserve TCP sessions, thereby avoiding handshake overhead. +- TLS session resumption can be used to preserve the TLS session layer, thereby avoiding TLS handshake overhead and repeated authentication operations (which can involve expensive asymmetric cryptographic operations) +- Token-based authentication via JSON Web Tokens (JWT) will be used to preserve sessions for users who have already authenticated with password-based authentication, so that they do not need to constantly re-use their passwords. + +#### 1.1.3.2 gNMI Server +- TLS session resumption can be used to preserve the TLS session layer, thereby avoiding TLS handshake overhead and repeated authentication operations (which can involve expensive asymmetric cryptographic operations) + +#### 1.1.3.3 Translib +- Translib will cache all the user information along with the privilege and resource information to avoid the overhead of querying them every time we receive a request. +- Will rely on notification to update any change in the user information, privilege or resource information + +### 1.1.4 Warm Boot Requirements +N/A + +## 1.2 Design Overview +### 1.2.1 Basic Approach +The code will extend the existing Klish (CLI) and REST Server modules in the sonic-mgmt-framework repository. Klish will be extended to enable authentication to the REST server (depending on the ultimately chosen approach), and the REST Server will need to be extended to map transactions to a user and pass that username data to the Translib. + +The gNMI server, which currently exists in the sonic-telemetry repository, needs to support passing the username down to Translib as well. + +The Translib code (also in sonic-mgmt-framework) will be extended to support RBAC via Linux Groups. It will receive username data from the REST/gNMI NBIs and perform the Group lookup for a given user. + +For user management, a service must run on the host to sync the Redis RBAC tables with the Linux user database and vice-versa. + +### 1.2.2 Container +SONiC Management Framework, gNMI Telemetry containers + +### 1.2.3 SAI Overview +N/A + +# 2 Functionality +## 2.1 Target Deployment Use Cases +Enterprise networks that enforce authentication for their management interfaces. + +## 2.2 Functional Description +This feature enables authentication and Role-Based Access Control (RBAC) on the REST and gNMI programmatic interfaces that are provided by the SONiC Management Framework and Telemetry containers. With respect to authentication, these programmatic interfaces will support password-based authentication with tokens, and certificate-based authentication. + +Since the Klish CLI in the management framework communicates with the REST server in the back-end, the solution will also be extended to support REST authentication. + +RBAC will be enforced centrally in the management framework, so that users accessing the system through varying interfaces will be limited to the same, consistent set of operations and objects depending on their role. Users' roles will be mapped using Linux Groups. + +Users and their role (group) assignments may be managed via Linux tools or the NBIs. + +# 3 Design +## 3.1 Overview +(TODO/DELL: Draw a picture) + +## 3.2 DB Changes +### 3.2.1 CONFIG DB +To support this the following tables will be introduced in the CONFIG DB : +* **UserTable** + + This table contains the username to role mapping needed for enforcing the authorization checks. It has the following columns : + * *user* : This is the username being authorized. This is a string. + * *tenant* : This contains the tenant with which the user is associated. This is a string + * *role* : This specifies the role associated with the username in the tenant. This is a comma separated list of strings. + The UserTable is keyed on <***user, tenant***>. + + **Note**: The UserTable will _not_ store users' salted+hashed passwords due to security concerns surrounding access restrictions to the DB; instead, that information will be maintained in `/etc/shadow` as per Linux convention. + +* **PrivilegeTable** + + This table has provides the information about the type of operations that a particular role is authorized to perform. The authorization can be performed at the granularity of a feature, feature group, or the entire system. The table has the following columns : + * *role* : The role associated with the user that is being authorized. This is a string. + * *feature* : This is feature that the role is being authorized to access. The granularity of the feature can be : + * *feature* - A logical grouping of multiple commands. If the user is authorized to access a particular feature, the column contains the tag associated with that feature. (More on tagging later. This will be implemented in Phase 2 of RBAC.) + * *feature-group* - A logical grouping of multiple features. If the user is authorized to a feature-group, the column contains the name of the feature-group. (More on feature-group later. This will be implemented in Phase 2 of RBAC.) + * *entire-system* - If the user is being granted access to the entire system, the column contains *all* + * *permissions* : Defines the permissions associated with the role. This is a string. + * *none* - This is the default permissions that a role is created with. A role associated with *none* permission cannot access any resources on the system to read, or to modify them. + * *read-only* - The role only has read access to the resources associated with the *feature*. + * *read-write* - The role has permissions to read and write (create, modify and delete) the resources associated with the *feature*. + The PrivilegeTable is keyed on <***role, feature***> + +* **ResourceTable** + (To be implemented in Phase 2) + Though the resources are statically tagged with the features that they belong to, a ResourceTable is still needed so as to allow for future extensibility. It is possible that in the future, a customer wants a more granular control over the authorization and wants to either sub partition the features or override the default tagging associated with a feature. The ResourceTable will allow for this support in the future. In the Phase 2, this table will be create using the default tagging associated with the resources. + * *resource* : The xpath associated with the resource being accessed. This is a string. + * *feature-name* : The tag of the feature this resource belongs to. + The ResourceTable is keyed on <***resource, feature***> + +* **TenantTable** + (To be implemented in Phase 3 or when Multi-tenancy is introduced) + In most systems today, a single SONiC system will serve multiple tenants. A tenant is a group of users who have a different privileges for resource instances. As SONiC becomes multitenant, RBAC needs to account for this when authorizing users. The TenantTable is needed to enable this and has the following columns : + * *resource* : The xpath associated with the resource being accessed. This is a string. + * *tenant* : The tenant for which the resource partitioning is being done. This is a string. + * *instances* : The instances of the *resource* allocated to this *tenant*. This is a list of instances. + The TenantTable is keyed on <***resource, tenant***> + +### 3.2.2 APP DB +N/A + +### 3.2.3 STATE DB +N/A + +### 3.2.4 ASIC DB +N/A + +### 3.2.5 COUNTER DB +N/A + +## 3.3 Switch State Service Design +### 3.3.1 Orchestration Agent +N/A + +### 3.3.2 Other Process +N/A + +## 3.4 SyncD +To facilitate the sync between users in the UserDB and with users in Linux, a service must run on the host that listens for both changes to `/etc/passwd` and changes to UserDB, since users can be created via either interface (UserDB via NBIs / Linux commands). + +This service runs a process that uses the POSIX [inotify APIs](http://man7.org/linux/man-pages/man7/inotify.7.html) to register for file system events like changes to `/etc/passwd` and/or `/etc/shadow`. +Another approach would be to have a process started by `systemd` on changes to `/etc/passwd` or `/etc/group`, and that process would simply reconcile the Redis DB with what is found in those files. `systemd` allows starting processes based on file create/delete/modify. + +A user can be created either via CLI or REST. It is the responsibility of this service to ensure that when the user information is added to the User DB, the appropriate user and groups are also created in the `/etc/passwd` and `/etc/groups` files. +This way, the User DB information and the Linux groups information is always in sync. + +## 3.5 SAI +N/A + +## 3.6 User Interface +### 3.6.1 Data Models +TBD from developer +(TODO/DELL) + +### 3.6.2 CLI +#### 3.6.2.1 Configuration Commands for User Management +Users may be managed via Linux tools like `useradd`, `usermod`, `passwd`, etc. They may also be managed via configuration. + +##### username +`username password role ` -- Configures a user on the system with a given name, password, and role. +* **name** is a text string of 1-32 alphanumeric characters +* **password-string** is a text string of 1-32 alphanumeric characters +* **role-string** is a text string consisting of a role name. In the initial release, the user is recommended to use "admin" and "operator" roles, as other roles will not be supported. A text string is desired instead of keywords so that in the future, more roles may be implemented and expanded. +* Configuring another a user with the same **name** should result in modification of the existing user. + +`no username ` -- Deletes a user from the system. +* **name** is a text string of 1-32 alphanumeric characters + +#### 3.6.2.2 Show Commands +N/A + +#### 3.6.2.3 Debug Commands +N/A + +#### 3.6.2.4 IS-CLI Compliance +N/A + +### 3.6.3 REST API Support +Ability to configure local users via REST API. + +# 4 Flow Diagrams +N/A + +# 5 Error Handling +## 5.1 REST Server +The REST server should return standard HTTP errors when authentication fails or if the user tries to access a forbidden resource or perform an unauthorized activity. + +## 5.2 gNMI server +The gNMI server should return standard gRPC errors when authentication fails. + +## 5.3 CLI +Authentication errors will be handled by SSH. However, the CLI must gracefully handle authorization failures from the REST server (the authorization failure would originate from Translib of course). While the CLI will render all of the available commands to a user, the user will actually only be able to execute a subset of them. This limitation is a result of the design decision to centralize RBAC in Translib. Nevertheless, the CLI must inform the user when they attempt to execute an unauthorized command. + +## 5.4 Translib +Translib will authorize the user and when the authorization fails will return appropriate error string to the REST/gNMI server. + +If a user authenticates but is not part of one of the pre-defined groups, they will not be allowed to do anything at all on the system. + +# 6 Serviceability and Debug +All operations performed by NBIs (CLI commands, REST/gNMI operations) should be logged/audited with usernames attached to the given operation(s) performed. + +Initially, users who are remotely authenticated will share a common role-specific username, so there will be a limitation here. + +# 7 Warm Boot Support +N/A + +# 8 Scalability +See previous section 1.1.3: Scalability Requirements + +# 9 Unit Test + +### Table 3: Test Cases +| **Test Case** | **Description** | +|:--------------------------|:-------------------------------------| +| REST with password | Authenticate to REST server with username/password and perform some operations | +| REST with token | Perform subsequent operations with token, ensure username/password are not re-prompted | +| REST authorized RBAC | Perform authorized operations as both `Admin` and `Operator` via REST | +| REST unauthorized RBAC | Attempt unauthorized operations as both `Admin` and `Operator` via REST | +| CLI with password | SSH to the system with username/password and execute some commands | +| CLI with RSA | SSH to the system with pubkey and execute some commands | +| CLI authorized RBAC | SSH to the system and perform authorized commands | +| CLI unauthorized RBAC | SSH to the system and perform unauthorized commands | +| RBAC no-group | Create a user and assign them to a non-predefined group; make sure they can't perform any operations | +| gNMI authentication | Test the same authentication methods as REST, but for gNMI instead | +| gNMI authorization | Test the same authorization as REST, but for gNMI instead | diff --git a/doc/aaa/SONiC RBAC.md b/doc/aaa/SONiC RBAC.md new file mode 100644 index 0000000000..194032fdaa --- /dev/null +++ b/doc/aaa/SONiC RBAC.md @@ -0,0 +1,481 @@ +# Authentication and Role-Based Access Control +# High Level Design Document +#### Rev 0.1 + +# Table of Contents +- [Revision History](#revision-history) +- [About this Manual](#about-this-manual) +- [Scope](#scope) +- [Definitions/Abbreviations](#definitionsabbreviations) +- [1 Feature Overview](#1-feature-overview) + * [1.1 Requirements](#11-requirements) + + [1.1.1 Functional Requirements](#111-functional-requirements) + - [1.1.1.1 NBI Authentication](#1111-nbi-authentication) + - [1.1.1.2 CLI Authentication to REST server](#1112-cli-authentication-to-rest-server) + - [1.1.1.3 REST/gNMI server authentication methods CLIs](#1113-restgnmi-server-authentication-methods-clis) + - [1.1.1.4 Linux Groups](#1114-linux-groups) + - [1.1.1.5 Customizable roles](#1115-customizable-roles) + - [1.1.1.6 Local User Management and UserDB Sync](#1116-local-user-management-and-userdb-sync) + + [1.1.2 Configuration and Management Requirements](#112-configuration-and-management-requirements) + + [1.1.3 Scalability Requirements](#113-scalability-requirements) + - [1.1.3.1 Translib](#1131-translib) + + [1.1.4 Warm Boot Requirements](#114-warm-boot-requirements) + * [1.2 Design Overview](#12-design-overview) + + [1.2.1 Basic Approach](#121-basic-approach) + + [1.2.2 Container](#122-container) + + [1.2.3 SAI Overview](#123-sai-overview) +- [2 Functionality](#2-functionality) + * [2.1 Target Deployment Use Cases](#21-target-deployment-use-cases) + * [2.2 Functional Description](#22-functional-description) +- [3 Design](#3-design) + * [3.1 Overview](#31-overview) + * [3.2 DB Changes](#32-db-changes) + + [3.2.1 CONFIG DB](#321-config-db) + + [3.2.2 APP DB](#322-app-db) + + [3.2.3 STATE DB](#323-state-db) + + [3.2.4 ASIC DB](#324-asic-db) + + [3.2.5 COUNTER DB](#325-counter-db) + * [3.3 Switch State Service Design](#33-switch-state-service-design) + + [3.3.1 Orchestration Agent](#331-orchestration-agent) + + [3.3.2 Other Process](#332-other-process) + * [3.4 SyncD](#34-syncd) + * [3.5 SAI](#35-sai) + * [3.6 User Interface](#36-user-interface) + + [3.6.1 Data Models](#361-data-models) + + [3.6.2 CLI](#362-cli) + - [3.6.2.1 Configuration Commands for User Management](#3621-configuration-commands-for-user-management) + - [3.6.2.2 Show Commands](#3622-show-commands) + - [3.6.2.3 Debug Commands](#3623-debug-commands) + - [3.6.2.4 IS-CLI Compliance](#3624-is-cli-compliance) + + [3.6.3 REST API Support](#363-rest-api-support) +- [4 Flow Diagrams](#4-flow-diagrams) +- [5 Error Handling](#5-error-handling) + * [5.1 REST Server](#51-rest-server) + * [5.2 gNMI server](#52-gnmi-server) + * [5.3 CLI](#53-cli) + * [5.4 Translib](#54-translib) +- [6 Serviceability and Debug](#6-serviceability-and-debug) +- [7 Warm Boot Support](#7-warm-boot-support) +- [8 Scalability](#8-scalability) +- [9 Unit Test](#9-unit-test) + +# Revision History +| Rev | Date | Author | Change Description | +|:---:|:-----------:|:------------------:|-----------------------------------| +| 0.1 | 04/07/2020 | Nirenjan Krishnan | Initial version | +| 0.2 | 06/12/2020 | Nirenjan Krishnan | Update with review comments | + +# About this Manual +This document provides a high-level design approach for authentication and RBAC in the SONiC Management Framework. + + +# Scope + +TBD + +# Definitions/Abbreviations + +| **Term** | **Meaning** | +|:--------------------------|:-------------------------------------| +| RBAC | Role-Based Access Control | +| AAA | Authentication, Authorization, Accounting | +| CLI | Command-Line Interface | +| REST | REpresentational State Transfer | +| gNMI | gRPC Network Management Interface | +| NBI | North-bound Interfaces (CLI, REST, gNMI) | +| HAM | Host Account Manager | + + + +# 1 Feature Overview + +## 1.1 Requirements + +- For RBAC, the roles will be defined in the configuration database. Initially, + two roles will be supported; admin (with read/write privileges), and operator + (with read-only privileges); however, it shall be possible for the users with + admin role to define additional roles. +- HAM will become the single source of truth for user management on SONiC + systems. +- Local user management: CLIs and APIs for creating and managing local users on + the system -- their usernames, passwords, and roles. +- Authentication control: CLIs and APIs to modify the allowed authentication + modes on the REST and gNMI interfaces. +- CLI authentication mechanisms to REST server. + +### 1.1.1 Functional Requirements + +#### 1.1.1.1 NBI Authentication +A variety of authentication methods must be supported: +* **CLI** authentication is handled via user detection on the Unix Socket. +* **REST** authentication + - Password-based Authentication + - JWT token-based authentication + - Certificate-based Authentication +* **gNMI** Authentication + - Password-based Authentication + - JWT token-based authentication + - Certificate-based Authentication + +#### 1.1.1.2 CLI Authentication to REST server +Given that the CLI module works by issuing REST transactions to the Management +Framework's REST server, the CLI module must also be able to authenticate with +the REST server when REST authentication is enabled. This can be accomplished +via the following steps: +1. KLISH must be launched as the corresponding user + - This is performed by the `-u` argument to `docker exec` +2. REST will listen in on a Unix socket. Listener will detect the user at the + remote end of the connection, and use that for authentication/authorization. +3. The REST server will authenticate the user and return a token with the + username and role encoded in it. The KLISH CLI must cache this token and use + it for all future requests from this KLISH session. The REST server will + validate this token to username/role mapping with it so as to identify all + future requests as well. This will also allow the REST server in creating + audit logs for all the requests sent from KLISH, as well as pass the username + and role to Translib for RBAC enforcement. +4. KLISH CLI session will store the authentication token, and from then on, + KLISH CLI will send REST requests using the persistent connection with the + authentication token in the HTTP header to the REST server for all the CLI + commands. +5. The KLISH session must be able to cleanly handle ctrl-c breaks while it is + busy waiting on a response from the REST server. +6. When the user exits the KLISH CLI session, the HTTP persistent connection is + closed. The REST server will clean up the corresponding authentication token + for its corresponding KLISH CLI Client. + +#### 1.1.1.3 REST/gNMI server authentication methods CLIs + +The CLI shall provide commands to modify the authentication methods of both the +REST and gNMI interfaces. + +#### 1.1.1.4 Linux Groups +Roles will not have a direct map to a primary Linux group, however, +supplementary groups are used to allow access to additional functionalities. +- `admin` users will be added to `sudo` and `docker` groups +- `operator` users will be added to `docker` group only. +- All non-admin users will be added to the `docker` group only. + +The default shell for all users is set to a script. This script will determine +if the user is a member of the admin group, and if so, drop them into a Bash +shell, otherwise, it will spawn KLISH to drop them into the CLI. Non-admin users +will only have access to the CLI and not be able to access Bash. Users cannot +run `system` commands from KLISH. + +#### 1.1.1.5 Customizable roles + +**This is planned for future support.** + +Admin users shall be able to create customizable roles for further granularity beyond read-only and read-write. For instance, a `secadmin` role shall allow the users to configure security parameters on the switch. The custom role shall have a default deny policy. Custom roles shall map a set of features, and their access level to the feature. A feature is a set of YANG paths. The default policy is no access to a feature, if it is not specified in the role. + +``` +[ + { + "name": "secadmin", + "features": [ + { + "feature": "aaa", + "access": "rw" + }, + { + "feature": "net", + "access": "ro" + } + ] + } +] +``` + +Features map to a set of YANG paths. The features are saved in the +ResourceTable, but a default feature set is defined in the system to allow users +to start from a known state. Sample feature description is shown below. + +``` +[ + { + "name": "aaa", + "paths": [ + "openconfig-aaa:aaa/system/users/user={username}", + "openconfig-aaa:aaa/system/users" + ] + }, + { + "name": "net", + "paths": [ + "openconfig-net:net/xyz" + ] + } +] +``` + + +#### 1.1.1.6 Local User Management and UserDB Sync +An interface must be developed for local user management, so that administrators can add users and assign passwords and roles to them. Administrators with the appropriate role must be able to add/delete users, modify user passwords, and modify user roles. They must be able to do so through all of the NBIs, meaning that a YANG model and CLI tree must be developed. + +Users must be added to the Linux database (`/etc/passwd`, `/etc/group`, and optionally `/etc/shadow`). That's where a user is mapped to a Linux [User Identifier](https://en.wikipedia.org/wiki/User_identifier) (UID) and primary [Group Identifier](https://en.wikipedia.org/wiki/Group_identifier) (GID). When users are created they also need to be assigned roles. Roles are defined in the REDIS database. Users may also be assigned to supplementary Linux groups (`/etc/group`) in order to allow functionality required for their roles (e.g. Docker group membership in order to spawn the CLI). + +Since these operations (i.e. creating Linux users, assigning roles, supplementary groups, etc.) are non-trivial, the process of creating users will be entrusted to the Host Account Management Daemon (**hamd**). + +##### 1.1.1.6.1 Host Account Management Daemon (hamd) + +The hamd process runs on the host. It is accessed via a DBus interface that provides the ability to access and/or modify the host's Linux database (`/etc/passwd`, `/etc/group`, and optionally `/etc/shadow`). Since DBus is a secured interface we can control which processes will be allowed to access hamd. + +The hamd process will provide the following APIs to create/modify/delete user and group (role) accounts: + +- **useradd**: To create new users (similar to GNU [useradd](http://man7.org/linux/man-pages/man8/useradd.8.html)) +- **userdel**: To delete a user (similar to GNU [userdel](http://man7.org/linux/man-pages/man8/userdel.8.html)) +- **passwd**: To change a user password (similar to GNU [passwd](http://man7.org/linux/man-pages/man1/passwd.1.html)) +- **setroles**: To set a user's roles (similar to GNU [usermod](http://man7.org/linux/man-pages/man8/usermod.8.html)) +- **groupadd**: To create new groups/roles (similar to GNU [groupadd](http://man7.org/linux/man-pages/man8/groupadd.8.html)) +- **groupdel**: To delete groups/roles (similar to GNU [groupdel](http://man7.org/linux/man-pages/man8/groupdel.8.html)) +- **usermod**: To create or modify users. This method will accept a username, + hashed password, and a list of strings corresponding to the roles. + +The hamd process will add the users to the User Table in the Config DB. + +##### 1.1.1.6.2 Name Service Switch + +In addition to providing APIs to create/modify/delete user and group (role) accounts, hamd also provides APIs to simply read user and group (role) accounts. Here's the list: + +- **getpwnam**: To retrieve user credentials (similar to POSIX [getpwnam](http://man7.org/linux/man-pages/man3/getpwnam.3.html)) +- **getpwuid**: To retrieve user credentials (similar to POSIX [getpwuid](http://man7.org/linux/man-pages/man3/getpwnam.3.html)) +- **getgrnam**: To retrieve group/role credentials (similar to POSIX [getgrnam](http://man7.org/linux/man-pages/man3/getgrnam.3.html)) +- **getgrgid**: To retrieve group/role credentials (similar to POSIX [getgrgid](http://man7.org/linux/man-pages/man3/getgrnam.3.html)) + +These APIs, however, are meant to be invoked through [NSS](https://en.wikipedia.org/wiki/Name_Service_Switch) (name service switch). That is, applications running in containers can simply continue invoking the standard POSIX APIs (`getpwnam()`, `getgrnam()`, etc) and a Host Account Management NSS module will ensure that the credentials get retrieved from hamd running on the host. The HAM NSS module (`libnss_ham.so.2`) need be installed and configured (`/etc/nsswitch.conf`) in the containers that require access to the host's Linux database. + +### 1.1.2 Configuration and Management Requirements + +Local users are managed via all the NBIs, i.e., CLI, REST and gNMI. The admin +users may also create or modify local users from the Bash shell. All requests +will be proxied to HAM to perform the actual user management. + +### 1.1.3 Scalability Requirements + +#### 1.1.3.1 Translib +- Translib will cache all the user information along with the privilege and resource information to avoid the overhead of querying them every time we receive a request. +- Will rely on notification to update any change in the user information, privilege or resource information + +### 1.1.4 Warm Boot Requirements +N/A + +## 1.2 Design Overview +### 1.2.1 Basic Approach + +The Translib code (also in sonic-mgmt-framework) will be modified to support RBAC via Roles, rather than Groups. It will receive username data from the REST/gNMI NBIs and perform the role lookup for a given user. + +Translib shall cache the user table and role list to amortize the cost of a +database lookup over several transactions. + +### 1.2.2 Container +SONiC Management Framework, gNMI Telemetry containers + +### 1.2.3 SAI Overview +N/A + +# 2 Functionality +## 2.1 Target Deployment Use Cases +Enterprise networks that enforce authentication for their management interfaces. + +## 2.2 Functional Description +This feature enables authentication and Role-Based Access Control (RBAC) on the REST and gNMI programmatic interfaces that are provided by the SONiC Management Framework and Telemetry containers. With respect to authentication, these programmatic interfaces will support password-based authentication with tokens, and certificate-based authentication. + +Since the Klish CLI in the management framework communicates with the REST server in the back-end, the solution will also be extended to support REST authentication. + +RBAC will be enforced centrally in the management framework, so that users accessing the system through varying interfaces will be limited to the same, consistent set of operations and objects depending on their role. Users' roles will be mapped using Linux Groups. + +Users and their role (group) assignments may be managed via the NBIs. HAM shall +provide tools to allow administrators to manage the users and roles from the +shell using the hamctl command. + +# 3 Design +## 3.1 Overview +(TODO/DELL: Draw a picture) + +## 3.2 DB Changes +### 3.2.1 CONFIG DB +* **UserTable** + + This table contains the username to role mapping needed for enforcing the authorization checks. It has the following columns : + * *user* : This is the username being authorized. This is a string. + * *tenant* : This contains the tenant with which the user is associated. This is a string + * *role* : This specifies the role associated with the username in the tenant. This is a comma separated list of strings. + The UserTable is keyed on <***user, tenant***>. + + **Note**: The UserTable will _not_ store users' salted+hashed passwords due to security concerns surrounding access restrictions to the DB; instead, that information will be maintained in `/etc/shadow` as per Linux convention. + +**Future Support** + +* **PrivilegeTable** + + This table provides the information about the type of operations that a particular role is authorized to perform. The authorization can be performed at the granularity of a feature, feature group, or the entire system. The table has the following columns : + * *role* : The role associated with the user that is being authorized. This is a string. + * *feature* : This is feature that the role is being authorized to access. The granularity of the feature can be : + * *feature* - A logical grouping of multiple commands. If the user is authorized to access a particular feature, the column contains the tag associated with that feature. + * *entire-system* - If the user is being granted access to the entire system, the column contains *all* + * *permissions* : Defines the permissions associated with the role. This is a string. + * *none* - This is the default permissions that a role is created with. A role associated with *none* permission cannot access any resources on the system to read, or to modify them. + * *read-only* - The role only has read access to the resources associated with the *feature*. + * *read-write* - The role has permissions to read and write (create, modify and delete) the resources associated with the *feature*. + The PrivilegeTable is keyed on <***role, feature***> + +* **ResourceTable** + Though the resources are statically tagged with the features that they belong to, a ResourceTable is still needed so as to allow for future extensibility. It is possible that in the future, a customer wants a more granular control over the authorization and wants to either sub partition the features or override the default tagging associated with a feature. The ResourceTable will allow for this support in the future. In the Phase 2, this table will be create using the default tagging associated with the resources. + * *resource* : The xpath associated with the resource being accessed. This is a string. + * *feature-name* : The tag of the feature this resource belongs to. + The ResourceTable is keyed on <***resource, feature***> + +* **TenantTable** + (To be implemented in Phase 3 or when Multi-tenancy is introduced) + In most systems today, a single SONiC system will serve multiple tenants. A tenant is a group of users who have a different privileges for resource instances. As SONiC becomes multitenant, RBAC needs to account for this when authorizing users. The TenantTable is needed to enable this and has the following columns : + * *resource* : The xpath associated with the resource being accessed. This is a string. + * *tenant* : The tenant for which the resource partitioning is being done. This is a string. + * *instances* : The instances of the *resource* allocated to this *tenant*. This is a list of instances. + The TenantTable is keyed on <***resource, tenant***> + +### 3.2.2 APP DB +N/A + +### 3.2.3 STATE DB +N/A + +### 3.2.4 ASIC DB +N/A + +### 3.2.5 COUNTER DB +N/A + +## 3.3 Switch State Service Design +### 3.3.1 Orchestration Agent +N/A + +### 3.3.2 Other Process +N/A + +## 3.4 SyncD + +N/A. HAM is going to be the single source of truth, and it will take care of +synchronizing the User Table and /etc/passwd + +## 3.5 SAI +N/A + +## 3.6 User Interface +### 3.6.1 Data Models + +TBD from developer +(TODO/DELL) + +### 3.6.2 CLI +#### 3.6.2.1 Configuration Commands for User Management +Users may be managed via Linux tools like `sonic-useradd`, `sonic-usermod`, `passwd`, etc. They may also be managed via configuration. + +##### username +`username password role ` -- Configures a user on the system with a given name, password, and role. +* **name** is a text string of 1-32 alphanumeric characters +* **password-string** is a text string of 1-32 alphanumeric characters +* **role-string** is a text string consisting of a role name. In the initial release, the user is recommended to use "admin" and "operator" roles, as other roles will not be supported. A text string is desired instead of keywords so that in the future, more roles may be implemented and expanded. +* Configuring another a user with the same **name** should result in modification of the existing user. + +`no username ` -- Deletes a user from the system. +* **name** is a text string of 1-32 alphanumeric characters + +##### aaa authentication +``` +aaa authentication {rest-server | gnmi-server} {[password] [token] [certificate]} +aaa authentication {rest-server | gnmi-server} certificate no-authorization +no aaa authentication {rest-server | gnmi-server} +``` + +The `certificate no-authorization` mode will force users to authenticate via +certificate, however, there will be no role lookup performed. All authenticated +users will be treated as having full administrator access. + +The `no aaa ...` command will disable authentication altogether, and will allow +all users to execute admin level commands via the REST/gNMI interfaces. The CLI +however, will still require a valid password based login, due to it being +accessed via SSH. + +##### userrole +`userrole ` - Creates a new custom role and enters userrole config to +enable/disable individual features. +`(config-userrole)# feature {read-only | read-write}` - Enable +feature `` with read-only or read-write access. +`(config-userrole)# no feature ` - Disable access to feature +``. + +`no userrole ` - Deletes a user role from the system. + +#### 3.6.2.2 Show Commands + +##### show users + +`show users` + +This will display a list of configured users on the system. + +##### show roles + +`show roles` + +This will display a list of configured roles on the system. +#### 3.6.2.3 Debug Commands +N/A + +#### 3.6.2.4 IS-CLI Compliance +N/A + +### 3.6.3 REST API Support +Ability to configure local users via REST API. + +# 4 Flow Diagrams +N/A + +# 5 Error Handling +## 5.1 REST Server +The REST server should return standard HTTP errors when authentication fails or if the user tries to access a forbidden resource or perform an unauthorized activity. + +## 5.2 gNMI server +The gNMI server should return standard gRPC errors when authentication fails. + +## 5.3 CLI +Authentication errors will be handled by user detection on Unix sockets. However, the CLI must gracefully handle authorization failures from the REST server. While the CLI will render all of the available commands to a user, the user will actually only be able to execute a subset of them. This limitation is a result of the design decision to centralize RBAC in Translib. Nevertheless, the CLI must inform the user when they attempt to execute an unauthorized command. + + +## 5.4 Translib +Translib will authorize the user and when the authorization fails will return appropriate error string to the REST/gNMI server. Translib will also log an audit message with the username and the command that was attempted. + +If a user authenticates but is not part of one of the pre-defined roles, they will not be allowed to do anything at all on the system. + +# 6 Serviceability and Debug +All operations performed by NBIs (CLI commands, REST/gNMI operations) should be logged/audited with usernames attached to the given operation(s) performed. + +Initially, users who are remotely authenticated will share a common role-specific username, so there will be a limitation here. + +# 7 Warm Boot Support +N/A + +# 8 Scalability +See previous section 1.1.3: Scalability Requirements + +# 9 Unit Test + +### Table 3: Test Cases +| **Test Case** | **Description** | +|:--------------------------|:-------------------------------------| +| REST with password | Authenticate to REST server with username/password and perform some operations | +| REST with token | Perform subsequent operations with token, ensure username/password are not re-prompted | +| REST authorized RBAC | Perform authorized operations as both `Admin` and `Operator` via REST | +| REST unauthorized RBAC | Attempt unauthorized operations as both `Admin` and `Operator` via REST | +| CLI with password | SSH to the system with username/password and execute some commands | +| CLI with RSA | SSH to the system with pubkey and execute some commands | +| CLI authorized RBAC | SSH to the system and perform authorized commands | +| CLI unauthorized RBAC | SSH to the system and perform unauthorized commands | +| RBAC no-group | Create a user and assign them to a non-predefined group; make sure they can't perform any operations | +| gNMI authentication | Test the same authentication methods as REST, but for gNMI instead | +| gNMI authorization | Test the same authorization as REST, but for gNMI instead | +| Create custom role | Create a custom role on the system with individual features | +| Delete custom role | Delete custom role from system | +| REST with custom role authorized | Perform authorized operations with custom role user via REST | +| REST with custom role unauthorized | Perform unauthorized operations with custom role user via REST | +| gNMI with custom role authorized | Perform authorized operations with custom role user via gNMI | +| gNMI with custom role unauthorized | Perform unauthorized operations with custom role user via gNMI | diff --git a/doc/aaa/ldap.md b/doc/aaa/ldap.md new file mode 100644 index 0000000000..288508a467 --- /dev/null +++ b/doc/aaa/ldap.md @@ -0,0 +1,2626 @@ +# LDAP Name Service + +## High Level Design Document +#### Rev 0.13 + +# Table of Contents + * [List of Tables](#list-of-tables) + * [Overview](#overview) + * [Revision](#revision) + * [Requirements](#requirements) + * [Functional Description](#functional-description) + * [LDAP NSS](#ldap-nss) + * [LDAP Authentication](#ldap-authentication) + * [LDAP Authorization](#ldap-authorization) + * [LDAP Logon Authorization](#ldap-logon-authorization) + * [LDAP Sudoers Authorization](#ldap-sudoers-authorization) + * [Session Setup](#session-setup) + * [Design](#design) + * [Block Diagram of PAM and NSS Configuration](#block-diagram-of-pam-and-nss-configuration) + * [Sequence Diagram of TACPLUS or RADIUS Authentication and LDAP Name Service](#sequence-diagram-of-tacplus-or-radius-authentication-and-ldap-name-service) + * [ConfigDB Schema](#configdb-schema) + * [OC YANG](#oc-yang) + * [Klish](#klish) + * [Click](#click) + * [Known Issues and Notes](#known-issues-and-notes) + * [Use Case](#use-case) + * [Flow Diagrams](#flow-diagrams) + * [Error Handling](#error-handling) + * [Serviceability and Debug](#serviceability-and-debug) + * [Warm Boot Support](#warm-boot-support) + * [Scalability](#scalability) + * [Unit Test](#unit-test) + * [Prerequisites](#prerequisites) + * [UT Authentication LDAP](#ut-authentication-ldap) + * [UT Authentication TACPLUS with Name Service LDAP](#ut-authentication-tacplus-with-name-service-ldap) + * [UT Authentication RADIUS with Name Service LDAP](#ut-authentication-radius-with-name-service-ldap) + * [UT Authentication RADIUS with Logon Authorization LDAP and Name Service LDAP](#ut-authentication-radius-with-logon-authorization-ldap-and-name-service-ldap) + * [UT Authentication RADIUS with Name Service LDAP and sudo Authorization LDAP](#ut-authentication-radius-with-name-service-ldap-and-sudo-authorization-ldap) + * [UT Authentication Non Global NSS PAM SUDO options](#ut-authentication-non-global-nss-pam-sudo-options) + * [References](#references) + * [nss_ldap](#nss_ldap) + * [pam_ldap](#pam_ldap) + * [sudo_ldap](#sudo_ldap) + * [Linux-PAM](#Linux-PAM) + * [NSS Reference](#NSS-Reference) + * [LDAP client configuration file](#LDAP-client-configuration-file) + * [TACACS+ Authentication](#TACPLUS-Authentication) + * [RADIUS Management User Authentication](#RADIUS-Management-User-Authentication) + +# List of Tables + +# Revision + +| Rev | Date | Author | Change Description | +|:---:|:------------:|:------------------:|-----------------------------------| +| 0.1 | 03/20/2020 | Arun Barboza | Initial version | +| 0.2 | 03/23/2020 | Arun Barboza | After Internal Review | +| 0.3 | 04/03/2020 | Arun Barboza | First External Review | +| 0.4 | 04/29/2020 | Arun Barboza | Detail Config DB Table schema | +| 0.5 | 04/30/2020 | Arun Barboza | Updated Unit Test Details | +| 0.6 | 05/04/2020 | Arun Barboza | Fixed typos, omissions, examples | +| 0.7 | 05/06/2020 | Arun Barboza | Updates for Future Release | +| 0.8 | 05/07/2020 | Arun Barboza | Annotations in LDAP table | +| 0.9 | 05/11/2020 | Arun Barboza | LDAP New Mgmt if- Balachandar M. | +| | | | Update for VRF, SRC IP - Suresh R.| +| 0.10| 05/17/2020 | Arun Barboza | KLISH show aaa|ldap output draft. | +| | | | Added missing retry to ldap host. | +| 0.11| 05/17/2020 | Arun Barboza | Update the UT with KLISH examples.| +| 0.12| 05/20/2020 | Arun Barboza | IIFR, clarifications, OCYang, Maps| +| 0.13| 06/22/2020 | Arun Barboza | project-arlo review updates | +| | | | | + +# Overview + +Lightweight Directory Access Protocol (LDAP) allows a client to access +information stored in distributed directory services. + +Name Service information typically includes users, hosts, groups. This +information is historically stored in flat files or other information +services (like a directory service). + +This document describes the high level design of LDAP Name service feature +in SONiC. + +# Requirements + +- Obtain User Information from an LDAP server. + +- Enable LDAP to be used as an authentication service. + +- Enable LDAP to be used as an authorization service. + +- Allow an authentication service, authorization service, separate from + Name Service. + +# Functional Description + +## LDAP NSS + +LDAP can be used as an option in the Name Services Switch(NSS) configuration. +The NSS configuration enables various programming APIs to use other sources +than the default files (e.g., Use LDAP directory information instead of +/etc/passwd for user and group information). User information includes uid, +gid, and home directory. + +## LDAP Authentication + +The LDAP Pluggable Authentication Module (PAM) can be used to authenticate +a CLI (SSH, or console) user to a Linux device like SONiC. + +## LDAP Authorization + +### LDAP Logon Authorization + +The LDAP PAM can be used to authorize a user at login. (e.g., ensure the +user logging on is a member of a group aka group-based logon authorization). +This group can represent any membership (e.g., group of users who are +permitted to logon to a host). In PAM, authorization task is done by account +management modules. + +### LDAP Sudoers Authorization + +The sudoers policy plugin determines a user's sudo privileges. This policy +is driven by the /etc/sudoers file. LDAP can be used to drive this policy +by being a source in the NSS configuration. (e.g. can a user be allowed to +execute the click config command?) + +## Session Setup + +When a user logs into a SONiC device, there would need to be some session +environment (e.g. home directory). This can also be provided by PAM session +modules. The pam_open_session(3) mentions tasks such as creating or +mounting a home directory. + +As of writing, the KLISH certificate to authenticate with the REST server +may not be necessary if the move to UNIX domain sockets from TCP sockets +takes place. + +In order to perform RBAC (Role Based Access Control), role(s) may be +needed for the user. The PAM session module could retrieve the same and +store it in the redis ConfigDB UserTable. Some options to retrieve roles. + +- Infer from the user's group membership. A possible way would be to use membership in a group to indicate the role membership (say by using a map). +- Have a role attribute in the user's LDAP entry. This might require an LDAP schema change so could be less desirable. + +The session modules could be made extensible by passing configuration options. + + +# Design + +Publicly available pre-built debian packages for libpam-ldap, libnss-ldap, +and sudo-ldap, will be planned on being used for providing the backend. +Configuration will take place through the Host Config Daemon using +configuration stored in ConfigDB. The ConfigDB tables can be set using +REST/gNMI, KLISH or click. + + +## Block Diagram of PAM and NSS Configuration + + +``` + +-------+ +---------+ + | SSH | | Console | + +---+---+ +----+----+ + | | + | (4) | + | | + | | + V V ++---------------------------------------------+ +| AUTHENTICATION, Name | +| AUTHORIZATION, Service | +| & SESSION SETUP | +| +-------------------+ +-------------------+ | +| | PAM Configuration | | NSS Configuration | | +| | Files | | Files | | +| +-------------------+ +-------------------+ | +| ^ ^ | +---------------------+ +| | | | | | +| | | | (3) | +------------+ | +| +---------------------+-----------+--------+--+ AAA Config | | +| | | +------------+ | +| | | | +| +--------------+ +------------+ | | Host Config Daemon | +| | PAM | | NSS | | +----------^----------+ +| | Libraries | | Libraries | | | +| +------------- + +------------+ | | ++---------------------------------------------+ (2) | + | + | + +---------+ +-------+--------+ + | | (1) | | + | CLI +------------------------------------> ConfigDB | + | | | | + +---------+ +----------------+ + +``` + +1. The CLI is used to update the ConfigDB with the LDAP and AAA configuration. +2. The Host Config Daemon(HCD) reads data from ConfigDB. +3. The HCD configures the PAM & NSS configuration files. (The AAA config + module in the HCD is responsible for modifying the configuration files.) +4. All new CLI sessions (SSH, Console), now use the new PAM & NSS + configurations to obtain name service information, authenticate, and + authorize users. + +## Sequence Diagram of TACPLUS or RADIUS Authentication and LDAP Name Service + +Following is the sequence of events during TACACS+/RADIUS authenticated login +for an SSH client, when LDAP is used for name service: + +``` +SSH Client SONiC Device TACACS+ Server + SSH Server NSS LDAP Plugin or RADIUS Server +---------------------------------------------------------------------- + + | | | | + | | | | + | Login(1) | | | + +---------------->| | | + | | getpwnam_r()(2) | | + | +---------------->| Retrieve | + | | |(3) data from LDAP | + | | user info.(4) | or name service | + | |<----------------+ cache daemon | + | | | | + | (5) | | | + | pam_auth_rqst. | | | + | to PAM library | | | + | | | + | | TACACS+ Authentication START (6) | + | | or RADIUS Access-Request | + | +------------------------------------->| + | | | + | | Authentication REPLY - PASS (7) | + | | or RADIUS Access-Accept | + | |<-------------------------------------+ + | | | + | | | | + | Success(8) | | | + |<----------------+ | | + | | | | +``` + +1. A user tries to login through a SSH client. +2. A getpwnam_r() call is made to obtain user information. +3. If the user is not found in the local /etc/passwd file, the NSS + LDAP plugin retreives the data from the LDAP server (or name service + cache daemon). If not found, login fails. +4. User information is returned to the SSH server. +5. The SSH server makes a PAM Authentication request to the PAM library. +6. The PAM configuration causes a Authentication-Start to TACACS+ server. + (or RADIUS Access-Request to RADIUS server). +7. TACACS+ returns Authentication-Reply (PASS) (or RADIUS returns Access- + Accept) -- (Goto 8.), + If Authentication-Reply (FAIL)/Access-Reject, the user authentication has + failed. End. +8. The user has authenticated successfully. + +Instead of TACACS+, RADIUS (or LDAP) could be used as the authentication +service as well. + +## ConfigDB Schema + + +There will be New Config DB Table schema for holding LDAP Configuration information. + +- LDAP (keys: 'global' [|'nss' | 'sudo' | 'pam']) + +- LDAP_SERVER (keys: .... ) + +- LDAP_MAP (keys[0]: ATTRIBUTE, + OBJECTCLASS, + DEFAULT_ATTRIBUTE_VALUE, + OVERRIDE_ATTRIBUTE_VALUE + keys[1]: )) + +An existing table will be modified to allow for LDAP Authentication, +Authorization, Name Service, and Session Setup. + +- AAA (new keys: 'authorization', 'nss' [, 'open_session']?) + + ? The 'open_session' key will be reserved for Implementation In a Future + Release(IIFR) + + +``` + #--------------------------------------------------# + # # + # LDAP LDAP_SERVER # + # # + #--------------------------------------------------# + # # + # ____ nss .... server_1 # + # | .... # + # global -----+--- sudo .... # + # |___ .... # + # pam .... server_n # + # # + #--------------------------------------------------# + + #--------------------------------------------------# + # # + # LDAP_MAP_xyz # + # # + #--------------------------------------------------# + + #--------------------------------------------------# + # # + # AAA # + # # + #--------------------------------------------------# + # # + # authentication # + # # + # authorization # + # # + # nss # + # # + # [open_session]? # + # # + # ? For Implementation In a Future Release (IIFR) # + # # + #--------------------------------------------------# + +``` + +### LDAP Table Schema + +This table holds some global LDAP configuration values common to NSS, PAM, +and sudoers. (e.g. search base in LDAP) + +``` +; LDAP configuration attributes global to the device. Upto 4 rows can exist in +; the table. Any of the "global" row LDAP attributes can be overriden by the +; server use_type row settings in the same table. +; Only a few settings can be overriden in the per server Table (please see the +; next table) +; Key +global_key = "global" / "nss" / "sudo" / "pam"; + ; + ; Attributes supported in all use_types + ; unless noted Eg: + ; n --> "nss" only + ; s --> "sudo" only + ; p --> "pam" only + ; - --> "global" support only. + ; np --> "nss" and "pam" only + ; + ; (Units,Default[ - meaning])[([snp]*)] + ; Eg: (seconds,(0 - indefinite)) + ; means that + ; the units are in seconds, + ; the default when the attribute is + ; absent is 0, which means wait + ; indefinitely. + ; the attribute is supported on all + ; use_types. + +; Attributes +timelimit = 1*nDIGIT ; Search time limit (secs,(0 - indefinite)) +bind_timelimit = 1*nDIGIT ; Connect time limit (secs,(10)) +idle_timelimit = 1*nDIGIT ; NSS idle time limit (secs,(0 - indefinite))(n) +retry = 1*2DIGIT ; Retry (attempts,(0,none)) +port = 1*5DIGIT ; Port (number,(389)) +scope = 1*nVCHAR ; Search scope ("sub"|"one"|"base",("sub"))(np) +ldap_version = 1DIGIT ; LDAP protocol version (2|3,(3)) +base = 1*nVCHAR ; Search base DN +ssl = 1*nVCHAR ; Use TLS? ("on"|"off"|"start_tls",("off")) +binddn = 1*nVCHAR ; Bind DN (,( - bind anonymously)) +bindpw = 1*nVCHAR ; Bind credentials (,( - bind anonymously)) +pam_filter = 1*nVCHAR ; Filter for retrieving user info. (,)(p) +pam_login_attribute = 1*nVCHAR ; Attribute for retrieving user info. + ; (,("uid"))(p) +pam_group_dn = 1*nVCHAR ; DN of a group for login Authorization (,)(p) +pam_member_attribute = 1*nVCHAR ; Attribute for login Authorization (,)(p) +sudoers_base = 1*nVCHAR ; Sudo LDAP queries search base DN (,)(s) +nss_base_passwd = 1*nVCHAR ; ... (,)(np) +nss_base_group = 1*nVCHAR ; ... (,)(n) +nss_base_shadow = 1*nVCHAR ; ... (,)(n) +nss_base_netgroup = 1*nVCHAR ; ... (,)(n) +nss_base_sudoers = 1*nVCHAR ; ... (,)(n) +nss_initgroups_ignoreusers = 1*nVCHAR ; NOT_FOUND for initgroups(3) (,)(n) +src_ip = IPAddress ; source IPv4|IPv6 addr for LDAP client (,)(-) +vrf = 1*nVCHAR ; vrf_name (,)(-) + +``` + +### LDAP_SERVER Table Schema + +This table holds per server configuration (e.g. connection retries) +``` +; LDAP per server configuration on the device. +; Key +server_key = NameOrIPAddress; LDAP server's DNS name or IPv4|6 addr +; Attributes +port = 1*5DIGIT ; per server Port +use_type = 1*nVCHAR ; "all" / "nss" / "sudo" / "pam" ("all") +priority = 1*2DIGIT ; specify LDAP server's priority (1) +ssl = 1*nVCHAR ; Use TLS? ("on"|"off"|"start_tls",("off")) +retry = 1*2DIGIT ; Retry (attempts,(0,none)) + +``` + +### LDAP_MAP Table Schema + +This table holds the maps for LDAP configuration. + +- LDAP_MAP_ATTRIBUTE +- LDAP_MAP_OBJECTCLASS +- LDAP_MAP_DEFAULT_ATTRIBUTE_VALUE +- LDAP_MAP_OVERRIDE_ATTRIBUTE_VALUE + +``` +; LDAP_MAP +; Key +map_name = "ATTRIBUTE"/ + "OBJECTCLASS"/ + "DEFAULT_ATTRIBUTE_VALUE"/ + "LDAP_MAP_OVERRIDE_ATTRIBUTE_VALUE" ; Map Name +from_key = 1*nVCHAR ; Map key +; Attributes +to = 1*nVCHAR ; Map value +``` + +### AAA Table Schema + +This is an existing table which will hold additional keys for configuring +NSS, and Authorization. For Authentication a key exists already. + +``` +; Key +aaa_key = "authentication" / "authorization" / "nss" ; AAA type +; Attributes +; For "authentication" / "authorization" Key only +login = LIST(1*32VCHAR) ; "local"/"ldap"/"radius"/"tacacs+" ; PAM modules +failthrough = "True" / "False" ; failthrough mechanism for pam modules +debug = "True" / "False" ; Debugging (Developer) +trace = "True" / "False" ; Packet Trace (Developer) +; For "nss" Key only +passwd = "login" / "ldap" / "radius" / "tacacs+" +group = "login" / "local" / "ldap" +shadow = "login" / "local" / "ldap" +netgroup = "local" / "ldap" +sudoers = "ldap" +; For "nss" Key: "login" ==> Name Service db is based on authentication login +``` + +## OC YANG + +OC-Yang extensions need to be written for LDAP for the following: + +- LDAP global configurations. +- LDAP per server configuration. +- LDAP Maps. +- AAA login authorization. +- AAA nss configuration. + +The OC-Yang tree would need to be extended, and modified in the following +areas as follows: [ All new leaves ] +``` +module: openconfig-system + +--rw system +... + +--rw aaa +... + | +--rw authentication +... + | | +--rw config + | | | +--rw authentication-method* (This is existing leaf-list that + needs to be extended for "ldap" + option) +... + | +--rw oc-sys-ext:authorization +... + | | +--rw oc-aaa-ldap-ext:login (New container: LDAP logon authorization) + | | +--rw oc-aaa-ldap-ext:config + | | | +--rw oc-aaa-ldap-ext:authorization-method* union +... + | +--rw oc-aaa-ldap-ext:name-service (New container) + | +--rw oc-aaa-ldap-ext:passwd + | +--rw config + | +--rw oc-aaa-ldap-ext:name-service-method* +... + | +--rw oc-aaa-ldap-ext:shadow + | +--rw config + | +--rw oc-aaa-ldap-ext:name-service-method* +... + | +--rw oc-aaa-ldap-ext:group + | +--rw config + | +--rw oc-aaa-ldap-ext:name-service-method* +... + | +--rw oc-aaa-ldap-ext:netgroup + | +--rw config + | +--rw oc-aaa-ldap-ext:name-service-method* +... + | +--rw oc-aaa-ldap-ext:sudoers + | +--rw config + | +--rw oc-aaa-ldap-ext:name-service-method* +... + | +--rw server-groups + | +--rw server-group* [name] (new LDAP_ALL group) + (new LDAP_SUDO|PAM|NSS groups) +... + | +--rw servers + | +--rw server* [address] + | | | + | | | +... + | | | + | | +--rw oc-aaa-ldap-ext:ldap (New container) + | | +--rw oc-aaa-ldap-ext:config + | | | +--rw oc-aaa-ldap-ext:port? oc-inet:port-number + | | | +--rw oc-aaa-ldap-ext:use-type? enumeration + | | | +--rw oc-aaa-ldap-ext:priority? uint8 + | | | +--rw oc-aaa-ldap-ext:ssl? ldap-ssl-type + | | | +--rw oc-aaa-ldap-ext:retransmit-attempts? uint8 + | | +--ro oc-aaa-ldap-ext:state + | | +--ro oc-aaa-ldap-ext:port? oc-inet:port-number + | | +--ro oc-aaa-ldap-ext:use-type? enumeration + | | +--ro oc-aaa-ldap-ext:priority? uint8 + | | +--ro oc-aaa-ldap-ext:ssl? ldap-ssl-type + | | +--ro oc-aaa-ldap-ext:retransmit-attempts? uint8 + | +--rw oc-aaa-ldap-ext:ldap (New Container) + | +--rw oc-aaa-ldap-ext:config + | | +--rw oc-aaa-ldap-ext:source-interface? -> /oc-if:interfaces/interface/name + | | +--rw oc-aaa-ldap-ext:vrf-name? -> /oc-ni:network-instances/network-instance/name + | | +--rw oc-aaa-ldap-ext:search-time-limit? uint32 + | | +--rw oc-aaa-ldap-ext:bind-time-limit? uint32 + | | +--rw oc-aaa-ldap-ext:retransmit-attempts? uint8 + | | +--rw oc-aaa-ldap-ext:port? oc-inet:port-number + | | +--rw oc-aaa-ldap-ext:version? uint8 + | | +--rw oc-aaa-ldap-ext:base? string + | | +--rw oc-aaa-ldap-ext:ssl? ldap-ssl-type + | | +--rw oc-aaa-ldap-ext:bind-dn? string + | | +--rw oc-aaa-ldap-ext:bind-pw? string + | | +--rw oc-aaa-ldap-ext:source-address? oc-inet:ip-address + | | +--rw oc-aaa-ldap-ext:vrf-name? string + | | +--rw oc-aaa-ldap-ext:idle-time-limit? uint32 + | | +--rw oc-aaa-ldap-ext:nss-base-group? string + | | +--rw oc-aaa-ldap-ext:nss-base-shadow? string + | | +--rw oc-aaa-ldap-ext:nss-base-netgroup? string + | | +--rw oc-aaa-ldap-ext:nss-base-sudoers? string + | | +--rw oc-aaa-ldap-ext:nss-initgroups-ignoreusers? string + | | +--rw oc-aaa-ldap-ext:pam-filter? string + | | +--rw oc-aaa-ldap-ext:pam-login-attribute? string + | | +--rw oc-aaa-ldap-ext:pam-group-dn? string + | | +--rw oc-aaa-ldap-ext:pam-member-attribute? string + | | +--rw oc-aaa-ldap-ext:sudoers-base? string + | | +--rw oc-aaa-ldap-ext:scope? enumeration + | | +--rw oc-aaa-ldap-ext:nss-base-passwd? string +... + | | +--rw oc-aaa-ldap-ext:maps + | | +--rw oc-aaa-ldap-ext:map* [name from] + | | +--rw oc-aaa-ldap-ext:name -> ../config/name + | | +--rw oc-aaa-ldap-ext:from -> ../config/from + | | +--rw oc-aaa-ldap-ext:config + | | | +--rw oc-aaa-ldap-ext:name? enumeration + | | | +--rw oc-aaa-ldap-ext:from? string + | | | +--rw oc-aaa-ldap-ext:to? string + | | +--ro oc-aaa-ldap-ext:state + | | +--ro oc-aaa-ldap-ext:name? enumeration + | | +--ro oc-aaa-ldap-ext:from? string + | | +--ro oc-aaa-ldap-ext:to? string + +... + + +``` + +## REST/gNMI + +REST, and gNMI support would be needed by writing transformer annotation, +overloads, and SONiC Yang. + +## KLISH + +KLISH commands (XML, actioners, renderers) needs to be written for + +- AAA authentication ldap option. + +``` +sonic(config)# [no] aaa authentication login default local \ + [ | group { radius | tacacs+ | ldap } ] +sonic(config)# [no] aaa authentication login default \ + group { radius | tacacs+ | ldap } [ local ] +sonic(config)# [no] aaa authentication login default +``` + +- AAA authorization login ldap option.. + +``` +sonic(config)# [no] aaa authorization login default { group ldap | local } +``` + +- AAA name-service. + +``` +sonic(config)# [no] aaa name-service passwd { group ldap | login | local } +sonic(config)# [no] aaa name-service shadow { group ldap | login | local } +sonic(config)# [no] aaa name-service group { group ldap | login | local } +sonic(config)# [no] aaa name-service netgroup { group ldap | local } +sonic(config)# [no] aaa name-service sudoers { group ldap | local } +``` + +- LDAP global|nss|pam|sudo server authentication, authorization, name-service parameters. + + +``` +ldap-server timelimit <0 - 65535> +- The default value is 0 seconds. + +ldap-server bind-timelimit <0 - 65535> +- The default value is 10 seconds. + +ldap-server idle-timelimit <0 - 65535> +- The default value is 0 seconds. + +ldap-server retry <0 - 10> +- Default value is 0. + +ldap-server port <0 - 65535> +- Default value is 389. + +ldap-server scope [sub|one|base] +- Default value is "sub". + +ldap-server ldap-version [2|3] +- Default value is 3. + +ldap-server base + +ldap-server ssl [on|off|start_tls] +-Default value is "off" + +ldap-server binddn + +ldap-server bindpw + +ldap-server pam-filter + +ldap-server pam-login-attribute + -Default value is "uid". + +ldap-server pam-group-dn + +ldap-server pam-member-attribute + +ldap-server sudoers-base + +ldap-server nss-base-passwd + +ldap-server nss-base-group + +ldap-server nss-base-shadow + +ldap-server nss-base-netgroup + +ldap-server nss-base-sudoers + +ldap-server nss-initgroups-ignoreusers + +ldap-server source-ip + +ldap-server vrf + +==================================================================== + +LDAP server NSS commands: + +ldap-server nss timelimit <0 - 65535> +- The default value is 0 seconds. + +ldap-server nss bind-timelimit <0 - 65535> +- The default value is 10 seconds. + +ldap-server nss idle-timelimit <0 - 65535> +- The default value is 0 seconds. + +ldap-server nss retry <0 - 10> +- Default value is 0. + +ldap-server nss port <0 - 65535> +- Default value is 389. + +ldap-server nss scope [sub|one|base] +- Default value is "sub". + +ldap-server nss ldap-version [2|3] +- Default value is 3. + +ldap-server nss base + +ldap-server nss ssl [on|off|start_tls] +-Default value is "off" + +ldap-server nss binddn + +ldap-server nss bindpw + +ldap-server nss nss-base-passwd + +ldap-server nss nss-base-group + +ldap-server nss nss-base-shadow + +ldap-server nss nss-base-netgroup + +ldap-server nss nss-base-sudoers + +ldap-server nss nss-initgroups-ignoreusers + +======================================================================= + +LDAP server PAM commands: + +ldap-server pam timelimit <0 - 65535> +- The default value is 0 seconds. + +ldap-server pam bind-timelimit <0 - 65535> +- The default value is 10 seconds. + +ldap-server pam retry <0 - 10> +- Default value is 0. + +ldap-server pam port <0 - 65535> +- Default value is 389. + +ldap-server pam scope [sub|one|base] +- Default value is "sub". + +ldap-server pam ldap-version [2|3] +- Default value is 3. + +ldap-server pam base + +ldap-server pam ssl [on|off|start_tls] +-Default value is "off" + +ldap-server pam binddn + +ldap-server pam bindpw + +ldap-server pam pam-filter + +ldap-server pam pam-login-attribute + -Default value is "uid". + +ldap-server pam pam-group-dn + +ldap-server pam pam-member-attribute + +ldap-server pam nss-base-passwd + + +====================================================================== + +LDAP server sudo commands: + +ldap-server sudo timelimit <0 - 65535> +- The default value is 0 seconds. + +ldap-server sudo bind-timelimit <0 - 65535> +- The default value is 10 seconds. + +ldap-server sudo retry <0 - 10> +- Default value is 0. + +ldap-server sudo port <0 - 65535> +- Default value is 389. + +ldap-server sudo ldap-version [2|3] +- Default value is 3. + +ldap-server sudo base + +ldap-server sudo ssl [on|off|start_tls] +-Default value is "off" + +ldap-server sudo binddn + +ldap-server sudo bindpw + +ldap-server sudo sudoers-base + + +=============================================================================== + +LDAP server configuration commands: + +ldap-server host use-type [all|nss|sudo|pam] | port <1-665535> | priority <1 - 99> ssl [on|off|start_tls] + +- use-type [all|nss|sudo|pam] - Default value is "all" +- priority <1 - 99> - Default value is 1 +- ssl [on|off|start_tls] - Default value is "off" +- retry <1 - 10> - Default value is 0 + +=============================================================================== + +LDAP server MAP commands: + +ldap-server map attribute to + +Eg: ldap-server map attribute uid to "sAMAccountName" + ldap-server map attribute shadowLastChange to "pwdLastSet" + +ldap-server map objectclass to +ldap-server map default-attribute-value to +ldap-server map override-attribute-value to +``` + + +- Show commands. + +``` +sonic# show ldap-server +--------------------------------------------------------- +LDAP Global Configuration +--------------------------------------------------------- +binddn : dc=sji,dc=broadcom,dc=net +-------------------------------------------------------------------------------- +HOST USE-TYPE PRIORITY SSL RETRY +-------------------------------------------------------------------------------- +1.1.1.1 - 1 - - +sonic# + +sonic# show aaa +--------------------------------------------------------- +AAA Authentication Information +--------------------------------------------------------- +failthrough : True +login-method : ldap, local +sonic# + +``` + +## Click + +``` +$ show aaa +$ show ldap +``` + + +Similar to KLISH, but config, and show utilities. + +## Known Issues and Notes + +### Known Issues + +- The publicly available pre-built debian packages for libpam-ldap, libnss-ldap, +and sudo-ldap, do not have source-ip or Vrf support. The same applies for the openldap-client library. + +- authorization-sudoers exposes Linux/UNIX functionality, thus making it + apparent that SONiC is Linux/UNIX based. + +- The existing aaa authentication login-method KLISH cli, needs to be + future proofed for allowing for separate console and ssh login methods. + +### Implementation Notes + +#### Authentication Login: local,ldap ; Failthrough: False + +The LDAP user information is present in the *local* database (because with LDAP +authentication, *ldap* will be used as a *nss* source). Thus, *local* +authentication will be attempted for LDAP users, which will likely fail. +The recommendation is to enable *failthrough*. + +#### Authentication Login: ldap,local ; Failthrough: False + +For a local user that is not present in the LDAP, the authentication against +the *ldap* pam module will return a PAM_USER_UNKNOWN error, which is not +considered an authentication failure, thus authentication will take place +with the *local* (unix) pam module. + +#### Per LDAP Server Configuration ; Failthrough: True + +The OSS *pam_ldap* module can have only a single instance. Configuration +cannot be supported on a per individual server basis. An authentication failure +at a single LDAP server results in failure being returned for the entire +PAM stack. Please see [Linux-PAM](#Linux-PAM), under *config=* -- +"Configuring multiple instances of pam_ldap for the same service with +different configuration files is not supported, because the configuration +information is cached." + +### Implementation In a Future Release + +Some part of the design mentioned in the document is for Implementation In a +Future Release and have been marked as such with ? (IIFR) annotations. + +# Use Case + +## LDAP Authentication and LDAP Name Service + +LDAP provides the name services user, group information. +User's session is setup to create home directory locally, if it does not exist. +Users are authenticated through LDAP servers. + + +``` + + +------------+ +-----------+ +-----------------------+ + | | | | | | + | SSH Client |<-->| SONiC DUT |<----->| LDAP Name Service & | + | | | | | Authentication Server | + | | | | | | + +------------+ +-----------+ +-----------------------+ + +``` + +``` + +sonic(config)# ldap-server base "dc=example,dc=com" +sonic(config)# ldap-server ssl start_tls +sonic(config)# ldap-server 10.59.1.9 + +; Use defaults for locating the bases of passwd, shadow, and group entries. + +sonic(config)# aaa authentication failthrough enable +sonic(config)# aaa authentication login default group ldap local + +; Use defaults for name-service based on login authentication + +``` + + +## TACACS+ Authentication, LDAP Authorization, and LDAP Name Service + +LDAP provides the name service user, group information. +User's session is setup to create home directory locally, if it does not exist. +Users are authenticated through TACACS+ servers. +Users are authorized through LDAP servers for logon authorization only. +User's click interface is sudoers authorized through LDAP servers. (i.e + the rules to check whether the user is allowed to execute the sudo + command (config, show, sonic-clear) are obtained from LDAP servers. + This does not apply to KLISH interface, since the authorization for + that comes from the role assigned to the user). + +``` + + +----------------------------+ + | | + | LDAP Name Service, and | + +-->| LDAP sudoers Authorization | + | | Servers | + | | | + +------------+ +-----------+ | +----------------------------+ + | | | | | + | | | | | +----------------------------+ + | | | | | | | + | SSH Client |<-->| SONiC DUT |<--+-->| TACACS+ Authentication | + | | | | | | Servers | + | | | | | +----------------------------+ + | | | | | + | | | | | + +------------+ +-----------+ | +----------------------------+ + | | | + +-->| LDAP Login Authorization | + | Servers | + +----------------------------+ + +``` + +``` + + +; LDAP Name Service parameters +sonic(config)# ldap-server base "dc=example,dc=com" +sonic(config)# aaa name-service passwd group ldap +sonic(config)# aaa name-service shadow group ldap +sonic(config)# aaa name-service group group ldap + +; Use this server for name-service and sudo-ldap authorization only, with one +; reconnect attempt +sonic(config)# ldap-server rsa.loc.example.net use-type nss,sudo ldap-retry 1 + + +; Logon Authorization from LDAP +; Use this server for login authorization only +sonic(config)# ldap-server 10.59.1.9 use-type pam + +sonic(config)# ldap-server pam-groupdn "cn=t1.loc.example.com,ou=hostaccess,dc=example,dc=com" +sonic(config)# ldap-server pam-member-attribute "uniqueMember" +sonic(config)# aaa authorization login default group ldap + +; sudo-ldap authorization +sonic(config)# ldap-server sudoers-base "dc=example,dc=com" +sonic(config)# aaa name-service sudoers group ldap + +; Authentication from TACACS+ +sonic(config)# tacacs-server 10.59.1.15 +sonic(config)# aaa authentication failthrough enable +sonic(config)# aaa authentication login default group tacacs+ local + +``` + +# Flow Diagrams + +Please see the diagrams in section [Design](#design) + +# Error Handling + +PAM and NSS modules return errors as per [Linux-PAM](#Linux-PAM), and +[NSS](#NSS-Reference) respectively. + +# Serviceability and Debug + +The LDAP PAM and NSS modules can be debugged using the "debug" option +of AAA. (i.e. config aaa debug enable) + +# Warm Boot Support + +N/A + +# Scalability + +A maximum of 8 LDAP servers can be configured. + +# Unit Test + +## Prerequisites + +``` + LDAP Server with User, Group information. + TACACS+/RADIUS Server with User passwords configure. +``` + +### LDAP data + +For sudo authorization (*sudoRole* objectclass), the LDAP server schema +needs to be updated from DUT file /usr/share/doc/sudo-ldap/schema.OpenLDAP. +Please see next section for some tips. + + +``` + +$ ldapsearch -x -b "dc=sji,dc=broadcom,dc=net" +# extended LDIF +# +# LDAPv3 +# base with scope subtree +# filter: (objectclass=*) +# requesting: ALL +# + +# sji.broadcom.net +dn: dc=sji,dc=broadcom,dc=net +objectClass: top +objectClass: dcObject +objectClass: organization +o: sji.broadcom.net +dc: sji + +# admin, sji.broadcom.net +dn: cn=admin,dc=sji,dc=broadcom,dc=net +objectClass: simpleSecurityObject +objectClass: organizationalRole +cn: admin +description: LDAP administrator + +# Group, sji.broadcom.net +dn: ou=Group,dc=sji,dc=broadcom,dc=net +objectClass: organizationalUnit +ou: Group + +# People, sji.broadcom.net +dn: ou=People,dc=sji,dc=broadcom,dc=net +objectClass: organizationalUnit +ou: People + +# Sudoers, sji.broadcom.net +dn: ou=Sudoers,dc=sji,dc=broadcom,dc=net +objectClass: organizationalUnit +ou: Sudoers + +# sudo, Group, sji.broadcom.net +dn: cn=sudo,ou=Group,dc=sji,dc=broadcom,dc=net +objectClass: top +objectClass: posixGroup +gidNumber: 27 +memberUid: arun +memberUid: ldapuser +cn: sudo + +# docker, Group, sji.broadcom.net +dn: cn=docker,ou=Group,dc=sji,dc=broadcom,dc=net +objectClass: top +objectClass: posixGroup +gidNumber: 999 +memberUid: arun +memberUid: ldapuser +memberUid: raduser1 +cn: docker + +# arun, People, sji.broadcom.net +dn: cn=arun,ou=People,dc=sji,dc=broadcom,dc=net +objectClass: top +objectClass: account +objectClass: posixAccount +objectClass: shadowAccount +cn: arun +uid: arun +uidNumber: 1200 +gidNumber: 1000 +homeDirectory: /home/arun +loginShell: /bin/bash + +# ldapuser, People, sji.broadcom.net +dn: cn=ldapuser,ou=People,dc=sji,dc=broadcom,dc=net +objectClass: top +objectClass: account +objectClass: posixAccount +objectClass: shadowAccount +cn: ldapuser +uid: ldapuser +uidNumber: 1201 +gidNumber: 1000 +homeDirectory: /home/ldapuser +loginShell: /bin/bash + +# raduser1, People, sji.broadcom.net +dn: cn=raduser1,ou=People,dc=sji,dc=broadcom,dc=net +objectClass: top +objectClass: account +objectClass: posixAccount +objectClass: shadowAccount +cn: raduser1 +uid: raduser1 +uidNumber: 1202 +gidNumber: 100 +homeDirectory: /home/raduser1 +loginShell: /bin/bash + +# role1, Sudoers, sji.broadcom.net +dn: cn=role1,ou=Sudoers,dc=sji,dc=broadcom,dc=net +objectClass: sudoRole +objectClass: top +cn: role1 +sudoUser: raduser1 +sudoHost: ALL +sudoCommand: ALL +sudoRunAsUser: ALL +sudoRunAsGroup: ALL + +# search result +search: 2 +result: 0 Success + +# numResponses: 12 +# numEntries: 11 +$ + +``` + +### LDAP sudo-ldap.schema addition + +For 2.4.42+dfsg-2ubuntu3.7 OpenLDAP server (slapd), procedure like the +below was used. Your mileage may vary. + +``` + +$ scp admin@10.59.143.65:/usr/share/doc/sudo-ldap/schema.OpenLDAP /etc/ldap/schema/sudo-ldap.schema +admin@10.59.143.65's password: +schema.OpenLDAP 100% 2410 2.4KB/s 00:00 +$ + +$ touch /tmp/ldif/conf +$ echo 'include /etc/ldap/schema/sudo-ldap.schema' > /tmp/ldif/conf +$ slaptest -f /tmp/ldif/conf -F /tmp/ldif +config file testing succeeded +$ + +$ vi /tmp/ldif/cn\=config/cn\=schema/cn\=\{0}sudo-ldap.ldif + +[ Remove the header and trailer lines so now it looks something like this ] + +$ cat /tmp/ldif/cn\=config/cn\=schema/cn\=\{0}sudo-ldap.ldif +dn: cn=sudo-ldap,cn=schema,cn=config +objectClass: olcSchemaConfig +cn: sudo-ldap +olcAttributeTypes: {0}( 1.3.6.1.4.1.15953.9.1.1 NAME 'sudoUser' DESC 'User(s + ) who may run sudo' EQUALITY caseExactIA5Match SUBSTR caseExactIA5Substrin + gsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 ) +olcAttributeTypes: {1}( 1.3.6.1.4.1.15953.9.1.2 NAME 'sudoHost' DESC 'Host(s + ) who may run sudo' EQUALITY caseExactIA5Match SUBSTR caseExactIA5Substring + sMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 ) +olcAttributeTypes: {2}( 1.3.6.1.4.1.15953.9.1.3 NAME 'sudoCommand' DESC 'Com + mand(s) to be executed by sudo' EQUALITY caseExactIA5Match SYNTAX 1.3.6.1.4 + .1.1466.115.121.1.26 ) +olcAttributeTypes: {3}( 1.3.6.1.4.1.15953.9.1.4 NAME 'sudoRunAs' DESC 'User( + s) impersonated by sudo (deprecated)' EQUALITY caseExactIA5Match SYNTAX 1.3 + .6.1.4.1.1466.115.121.1.26 ) +olcAttributeTypes: {4}( 1.3.6.1.4.1.15953.9.1.5 NAME 'sudoOption' DESC 'Opti + ons(s) followed by sudo' EQUALITY caseExactIA5Match SYNTAX 1.3.6.1.4.1.1466 + .115.121.1.26 ) +olcAttributeTypes: {5}( 1.3.6.1.4.1.15953.9.1.6 NAME 'sudoRunAsUser' DESC 'U + ser(s) impersonated by sudo' EQUALITY caseExactIA5Match SYNTAX 1.3.6.1.4.1. + 1466.115.121.1.26 ) +olcAttributeTypes: {6}( 1.3.6.1.4.1.15953.9.1.7 NAME 'sudoRunAsGroup' DESC ' + Group(s) impersonated by sudo' EQUALITY caseExactIA5Match SYNTAX 1.3.6.1.4. + 1.1466.115.121.1.26 ) +olcAttributeTypes: {7}( 1.3.6.1.4.1.15953.9.1.8 NAME 'sudoNotBefore' DESC 'S + tart of time interval for which the entry is valid' EQUALITY generalizedTim + eMatch ORDERING generalizedTimeOrderingMatch SYNTAX 1.3.6.1.4.1.1466.115.12 + 1.1.24 ) +olcAttributeTypes: {8}( 1.3.6.1.4.1.15953.9.1.9 NAME 'sudoNotAfter' DESC 'En + d of time interval for which the entry is valid' EQUALITY generalizedTimeMa + tch ORDERING generalizedTimeOrderingMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1 + .24 ) +olcAttributeTypes: {9}( 1.3.6.1.4.1.15953.9.1.10 NAME 'sudoOrder' DESC 'an i + nteger to order the sudoRole entries' EQUALITY integerMatch ORDERING intege + rOrderingMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.27 ) +olcObjectClasses: {0}( 1.3.6.1.4.1.15953.9.2.1 NAME 'sudoRole' DESC 'Sudoer + Entries' SUP top STRUCTURAL MUST cn MAY ( sudoUser $ sudoHost $ sudoCommand + $ sudoRunAs $ sudoRunAsUser $ sudoRunAsGroup $ sudoOption $ sudoOrder $ su + doNotBefore $ sudoNotAfter $ description ) ) +$ + + +$ cp /tmp/ldif/cn\=config/cn\=schema/cn\=\{0}sudo-ldap.ldif /etc/ldap/schema/sudo-ldap.ldif + +$ ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/ldap/schema/sudo-ldap.ldifSASL/EXTERNAL authentication started +SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth +SASL SSF: 0 +adding new entry "cn=sudo-ldap,cn=schema,cn=config" + +$ + +``` + +### TACACS+ Server user + +``` +... +user = ldapuser { + default service = permit + chap = cleartext "ldapuser" + pap = cleartext "ldapuser" + service = exec { + priv-lvl=15 + } +} +... + +``` + +### RADIUS Server user + +``` +... +raduser1 Cleartext-Password := "password" + Management-Privilege-Level = 1 +... +ldapuser Cleartext-Password := ldapuser + Brocade-Auth-Role = admin, + Management-Privilege-Level = 15 +... + +``` + +## UT Authentication LDAP + +### Config DB: Before + +``` + +127.0.0.1:6379[4]> keys *LDAP* +(empty list or set) +127.0.0.1:6379[4]> keys *AAA* +(empty list or set) +127.0.0.1:6379[4]> + +``` + +### Click: show aaa, show ldap: Before + +``` + +admin@L10:~/show$ show aaa +AAA authentication login local (default) +AAA authentication failthrough False (default) +AAA nss passwd login (default) +AAA nss shadow login (default) +AAA nss group login (default) +AAA nss netgroup local (default) +AAA authorization login local (default) + +admin@L10:~/show$ + +admin@L10:~/show$ show ldap +LDAP global timelimit 0 (default) +LDAP global retry 0 (default) +LDAP global ldap_version 3 (default) +LDAP global scope sub (default) +LDAP global idle_timelimit 0 (default) +LDAP global ssl off (default) +LDAP global use_type all (default) +LDAP global port 389 (default) +LDAP global bind_timelimit 10 (default) + +admin@L10:~/show$ + +``` + +### Click CLI + +``` + +sudo config ldap add 10.59.143.229 +sudo config ldap base dc=sji,dc=broadcom,dc=net +sudo config aaa authentication login ldap local +sudo config aaa authentication failthrough enable + +``` + +### KLISH: show aaa, show ldap-server: Before + +``` +sonic# show aaa +--------------------------------------------------------- +AAA Authentication Information +--------------------------------------------------------- +failthrough : False +login-method : local +sonic# +sonic# show ldap-server +sonic# + +``` + +### KLISH CLI + +``` + +sonic(config)# ldap-server host 10.59.143.229 +sonic(config)# ldap-server base dc=sji,dc=broadcom,dc=net +sonic(config)# aaa authentication failthrough enable +sonic(config)# aaa authentication login default group ldap local + +``` + +### Config DB: After + +``` + +127.0.0.1:6379[4]> keys *LDAP* +1) "LDAP|global" +2) "LDAP_SERVER|10.59.143.229" +127.0.0.1:6379[4]> keys *AAA* +1) "AAA|authentication" +127.0.0.1:6379[4]> hgetall LDAP|global +1) "base" +2) "dc=sji,dc=broadcom,dc=net" +127.0.0.1:6379[4]> hgetall LDAP_SERVER|10.59.143.229 +1) "priority" +2) "1" +3) "retry" +4) "0" +5) "use_type" +6) "all" +127.0.0.1:6379[4]> hgetall AAA|authentication +1) "login" +2) "ldap,local" +3) "failthrough" +4) "True" +127.0.0.1:6379[4]> + +``` + +### Click: show aaa, show ldap: After + +``` + +admin@L10:~/show$ id ldapuser +uid=1201(ldapuser) gid=1000(admin) groups=1000(admin),27(sudo),999(docker) +admin@L10:~/show$ show ldap +LDAP global timelimit 0 (default) +LDAP global retry 0 (default) +LDAP global ldap_version 3 (default) +LDAP global scope sub (default) +LDAP global idle_timelimit 0 (default) +LDAP global ssl off (default) +LDAP global base dc=sji,dc=broadcom,dc=net +LDAP global use_type all (default) +LDAP global port 389 (default) +LDAP global bind_timelimit 10 (default) + +LDAP_SERVER address 10.59.143.229 + priority 1 + retry 0 + use_type all + +admin@L10:~/show$ show aaa +AAA authentication login ldap,local +AAA authentication failthrough True +AAA nss passwd login (default) +AAA nss shadow login (default) +AAA nss group login (default) +AAA nss netgroup local (default) +AAA authorization login local (default) + +admin@L10:~/show$ + +``` + +### KLISH: show aaa, show ldap-server: After + +``` +sonic# show aaa +--------------------------------------------------------- +AAA Authentication Information +--------------------------------------------------------- +failthrough : True +login-method : ldap, local +sonic# +sonic# show ldap-server +--------------------------------------------------------- +LDAP Global Configuration +--------------------------------------------------------- +binddn : dc=sji,dc=broadcom,dc=net +-------------------------------------------------------------------------------- +HOST USE-TYPE PRIORITY SSL RETRY +-------------------------------------------------------------------------------- +10.59.143.229 - 1 - - +sonic# + +``` + +### Verify ldapuser can login with LDAP password. + +``` + +$ ssh ldapuser@10.59.143.109 +ldapuser@10.59.143.109's password: +Creating directory '/home/ldapuser'. +Linux L10 4.9.0-11-2-amd64 #1 SMP Debian 4.9.189-3+deb9u2 (2019-11-11) x86_64 +... +ldapuser@L10:~$ + +``` + +## UT Authentication TACPLUS with Name Service LDAP + +### Click CLI + +``` + +sudo config tacacs add 10.59.143.229 +sudo config tacacs passkey testing123 +sudo config aaa authentication failthrough enable +sudo config aaa authentication login tacacs+ local + +sudo config ldap add 10.59.143.229 +sudo config ldap base dc=sji,dc=broadcom,dc=net +sudo config aaa nss passwd ldap +sudo config aaa nss shadow ldap +sudo config aaa nss group ldap + +``` + +### KLISH CLI + +``` + +sonic(config)# tacacs-server host 10.59.143.229 +sonic(config)# tacacs-server key testing123 +sonic(config)# aaa authentication failthrough enable +sonic(config)# aaa authentication login default group tacacs+ local + +sonic(config)# ldap-server host 10.59.143.229 +sonic(config)# ldap-server base dc=sji,dc=broadcom,dc=net +sonic(config)# aaa name-service passwd group ldap +sonic(config)# aaa name-service shadow group ldap +sonic(config)# aaa name-service group group ldap + +``` + +### Config DB: After + +``` + +127.0.0.1:6379[4]> keys *AAA* +1) "AAA|nss" +2) "AAA|authentication" +127.0.0.1:6379[4]> keys *TACPLUS* +1) "TACPLUS_SERVER|10.59.143.229" +2) "TACPLUS|global" +127.0.0.1:6379[4]> keys *LDAP* +1) "LDAP|global" +2) "LDAP_SERVER|10.59.143.229" +127.0.0.1:6379[4]> hgetall AAA|nss +1) "passwd" +2) "ldap" +3) "shadow" +4) "ldap" +5) "group" +6) "ldap" +127.0.0.1:6379[4]> hgetall AAA|authentication +1) "failthrough" +2) "True" +3) "login" +4) "tacacs+,local" +127.0.0.1:6379[4]> hgetall TACPLUS|global +1) "passkey" +2) "testing123" +127.0.0.1:6379[4]> hgetall TACPLUS_SERVER|10.59.143.229 +1) "priority" +2) "1" +3) "tcp_port" +4) "49" +127.0.0.1:6379[4]> hgetall LDAP|global +1) "base" +2) "dc=sji,dc=broadcom,dc=net" +127.0.0.1:6379[4]> hgetall LDAP_SERVER|10.59.143.229 +1) "priority" +2) "1" +3) "retry" +4) "0" +5) "use_type" +6) "all" +127.0.0.1:6379[4]> + +``` + +### Click: show aaa, show tacacs, show ldap: After + +``` + +admin@L10:~$ show aaa +AAA authentication login tacacs+,local +AAA authentication failthrough True +AAA nss group ldap +AAA nss passwd ldap +AAA nss shadow ldap +AAA nss netgroup local (default) +AAA authorization login local (default) + +admin@L10:~$ show tacacs +TACPLUS global auth_type pap (default) +TACPLUS global timeout 5 (default) +TACPLUS global passkey testing123 + +TACPLUS_SERVER address 10.59.143.229 + priority 1 + tcp_port 49 + +admin@L10:~$ show ldap +LDAP global timelimit 0 (default) +LDAP global retry 0 (default) +LDAP global ldap_version 3 (default) +LDAP global scope sub (default) +LDAP global idle_timelimit 0 (default) +LDAP global ssl off (default) +LDAP global base dc=sji,dc=broadcom,dc=net +LDAP global use_type all (default) +LDAP global port 389 (default) +LDAP global bind_timelimit 10 (default) + +LDAP_SERVER address 10.59.143.229 + priority 1 + retry 0 + use_type all + +admin@L10:~$ + +``` + +### KLISH: show aaa, show tacacs-server, show ldap-server: After + +``` +sonic# show aaa +--------------------------------------------------------- +AAA Authentication Information +--------------------------------------------------------- +failthrough : True +login-method : tacacs+, local +--------------------------------------------------------- +AAA Name-Service Information +--------------------------------------------------------- +passwd : ldap +shadow : ldap +group : ldap +sonic# +sonic# show tacacs-server global +--------------------------------------------------------- +TACACS Global Configuration +--------------------------------------------------------- +key : testing123 +sonic# +sonic# show tacacs-server host +-------------------------------------------------------------------------------- +HOST AUTH-TYPE KEY PORT PRIORITY TIMEOUT +-------------------------------------------------------------------------------- +10.59.143.229 pap 49 1 5 +sonic# +sonic# show ldap-server +--------------------------------------------------------- +LDAP Global Configuration +--------------------------------------------------------- +binddn : dc=sji,dc=broadcom,dc=net +-------------------------------------------------------------------------------- +HOST USE-TYPE PRIORITY SSL RETRY +-------------------------------------------------------------------------------- +10.59.143.229 - 1 - - +sonic# + +``` + +### Verify user can login using TACACS+ password with LDAP Name Service + +``` + +$ ssh ldapuser@10.59.143.109 +ldapuser@10.59.143.109's password: +Creating directory '/home/ldapuser'. +Linux L10 4.9.0-11-2-amd64 #1 SMP Debian 4.9.189-3+deb9u2 (2019-11-11) x86_64 +... +ldapuser@L10:~$ + +``` + +## UT Authentication RADIUS with Name Service LDAP + +### Click CLI + +``` + +sudo config radius add 10.59.143.229 +sudo config radius passkey sharedsecret +sudo config aaa authentication failthrough enable +sudo config aaa authentication login radius local + +sudo config ldap add 10.59.143.229 +sudo config ldap base dc=sji,dc=broadcom,dc=net +sudo config aaa nss passwd ldap +sudo config aaa nss shadow ldap +sudo config aaa nss group ldap + +``` + +### KLISH CLI + +``` + +sonic(config)# radius-server host 10.59.143.229 +sonic(config)# radius-server key sharedsecret +sonic(config)# aaa authentication failthrough enable +sonic(config)# aaa authentication login default group radius local + +sonic(config)# ldap-server host 10.59.143.229 +sonic(config)# ldap-server base dc=sji,dc=broadcom,dc=net +sonic(config)# aaa name-service passwd group ldap +sonic(config)# aaa name-service shadow group ldap +sonic(config)# aaa name-service group group ldap + +``` + +### Config DB: After + +``` + +127.0.0.1:6379[4]> keys *AAA* +1) "AAA|nss" +2) "AAA|authentication" +127.0.0.1:6379[4]> keys *RADIUS* +1) "RADIUS_SERVER|10.59.143.229" +2) "RADIUS|global" +127.0.0.1:6379[4]> keys *LDAP* +1) "LDAP|global" +2) "LDAP_SERVER|10.59.143.229" +127.0.0.1:6379[4]> hgetall AAA|nss +1) "passwd" +2) "ldap" +3) "shadow" +4) "ldap" +5) "group" +6) "ldap" +127.0.0.1:6379[4]> hgetall AAA|authentication +1) "failthrough" +2) "True" +3) "login" +4) "radius,local" +127.0.0.1:6379[4]> hgetall RADIUS|global +1) "passkey" +2) "sharedsecret" +127.0.0.1:6379[4]> hgetall RADIUS_SERVER|10.59.143.229 +1) "priority" +2) "1" +3) "auth_port" +4) "1812" +127.0.0.1:6379[4]> hgetall LDAP|global +1) "base" +2) "dc=sji,dc=broadcom,dc=net" +127.0.0.1:6379[4]> hgetall LDAP_SERVER|10.59.143.229 +1) "priority" +2) "1" +3) "retry" +4) "0" +5) "use_type" +6) "all" +127.0.0.1:6379[4]> + +``` + +### show aaa, show radius, show ldap: After + +``` + +admin@L10:~$ show aaa +AAA authentication login radius,local +AAA authentication failthrough True +AAA nss group ldap +AAA nss passwd ldap +AAA nss shadow ldap +AAA nss netgroup local (default) +AAA authorization login local (default) + +admin@L10:~$ show radius +RADIUS global auth_type pap (default) +RADIUS global retransmit 3 (default) +RADIUS global timeout 5 (default) +RADIUS global passkey sharedsecret + +RADIUS_SERVER address 10.59.143.229 + priority 1 + auth_port 1812 + +admin@L10:~$ + +admin@L10:~$ show ldap +LDAP global timelimit 0 (default) +LDAP global retry 0 (default) +LDAP global ldap_version 3 (default) +LDAP global scope sub (default) +LDAP global idle_timelimit 0 (default) +LDAP global ssl off (default) +LDAP global base dc=sji,dc=broadcom,dc=net +LDAP global use_type all (default) +LDAP global port 389 (default) +LDAP global bind_timelimit 10 (default) + +LDAP_SERVER address 10.59.143.229 + priority 1 + retry 0 + use_type all + +admin@L10:~$ + +``` + +### KLISH: show aaa, show radius-server, show ldap-server: After + +``` +sonic# show aaa +--------------------------------------------------------- +AAA Authentication Information +--------------------------------------------------------- +failthrough : True +login-method : radius, local +--------------------------------------------------------- +AAA Name-Service Information +--------------------------------------------------------- +passwd : ldap +shadow : ldap +group : ldap +sonic# +sonic# show radius-server +--------------------------------------------------------- +RADIUS Global Configuration +--------------------------------------------------------- +key : sharedsecret +-------------------------------------------------------------------------------- +HOST AUTH-TYPE KEY AUTH-PORT PRIORITY TIMEOUT RTSMT VRF +-------------------------------------------------------------------------------- +10.59.143.229 - - 1812 1 - - - +sonic# +sonic# show ldap-server +--------------------------------------------------------- +LDAP Global Configuration +--------------------------------------------------------- +binddn : dc=sji,dc=broadcom,dc=net +-------------------------------------------------------------------------------- +HOST USE-TYPE PRIORITY SSL RETRY +-------------------------------------------------------------------------------- +10.59.143.229 - 1 - - +sonic# + +``` + +### Verify user can login using RADIUS password with LDAP Name Service + +``` + +$ ssh ldapuser@10.59.143.109 +ldapuser@10.59.143.109's password: +Creating directory '/home/ldapuser'. +Linux L10 4.9.0-11-2-amd64 #1 SMP Debian 4.9.189-3+deb9u2 (2019-11-11) x86_64 +... +ldapuser@L10:~$ + +``` + +## UT Authentication RADIUS with Logon Authorization LDAP and Name Service LDAP + +### Click CLI + +``` + +sudo config radius add 10.59.143.229 +sudo config radius passkey sharedsecret +sudo config aaa authentication failthrough enable +sudo config aaa authentication login radius local + +sudo config ldap add 10.59.143.229 +sudo config ldap base dc=sji,dc=broadcom,dc=net +sudo config aaa nss passwd ldap +sudo config aaa nss shadow ldap +sudo config aaa nss group ldap + +sudo config ldap pam_groupdn cn=sudo,ou=Group,dc=sji,dc=broadcom,dc=net +sudo config ldap pam_member_attribute memberUid +sudo config aaa authorization login ldap + +``` + +### KLISH CLI + +``` + +sonic(config)# radius-server host 10.59.143.229 +sonic(config)# radius-server key sharedsecret +sonic(config)# aaa authentication failthrough enable +sonic(config)# aaa authentication login default group radius local + +sonic(config)# ldap-server host 10.59.143.229 +sonic(config)# ldap-server base dc=sji,dc=broadcom,dc=net +sonic(config)# aaa name-service passwd group ldap +sonic(config)# aaa name-service shadow group ldap +sonic(config)# aaa name-service group group ldap + +sonic(config)# ldap-server pam-groupdn cn=sudo,ou=Group,dc=sji,dc=broadcom,dc=net +sonic(config)# ldap-server pam-member-attribute memberUid +sonic(config)# aaa authorization login default group ldap + +``` + +### Config DB: After + +``` + +127.0.0.1:6379[4]> keys *AAA* +1) "AAA|authorization" +2) "AAA|nss" +3) "AAA|authentication" +127.0.0.1:6379[4]> keys *RADIUS* +1) "RADIUS_SERVER|10.59.143.229" +2) "RADIUS|global" +127.0.0.1:6379[4]> keys *LDAP* +1) "LDAP|global" +2) "LDAP_SERVER|10.59.143.229" +127.0.0.1:6379[4]> hgetall AAA|nss +1) "passwd" +2) "ldap" +3) "shadow" +4) "ldap" +5) "group" +6) "ldap" +127.0.0.1:6379[4]> hgetall AAA|authentication +1) "failthrough" +2) "True" +3) "login" +4) "radius,local" +127.0.0.1:6379[4]> hgetall AAA|authorization +1) "login" +2) "ldap" +127.0.0.1:6379[4]> hgetall RADIUS|global +1) "passkey" +2) "sharedsecret" +127.0.0.1:6379[4]> hgetall RADIUS_SERVER|10.59.143.229 +1) "priority" +2) "1" +3) "auth_port" +4) "1812" +127.0.0.1:6379[4]> hgetall LDAP|global +1) "base" +2) "dc=sji,dc=broadcom,dc=net" +3) "pam_groupdn" +4) "cn=sudo,ou=Group,dc=sji,dc=broadcom,dc=net" +5) "pam_member_attribute" +6) "memberUid" +127.0.0.1:6379[4]> + +``` + +### show aaa, show radius, show ldap: After + +``` + +admin@L10:~$ show aaa +AAA authentication login radius,local +AAA authentication failthrough True +AAA nss group ldap +AAA nss passwd ldap +AAA nss shadow ldap +AAA nss netgroup local (default) +AAA authorization login ldap + +admin@L10:~$ + +admin@L10:~$ show radius +RADIUS global auth_type pap (default) +RADIUS global retransmit 3 (default) +RADIUS global timeout 5 (default) +RADIUS global passkey sharedsecret + +RADIUS_SERVER address 10.59.143.229 + priority 1 + auth_port 1812 + +admin@L10:~$ + +admin@L10:~$ show ldap +LDAP global retry 0 (default) +LDAP global ssl off (default) +LDAP global idle_timelimit 0 (default) +LDAP global bind_timelimit 10 (default) +LDAP global timelimit 0 (default) +LDAP global pam_member_attribute memberUid +LDAP global pam_groupdn cn=sudo,ou=Group,dc=sji,dc=broadcom,dc=net +LDAP global ldap_version 3 (default) +LDAP global scope sub (default) +LDAP global base dc=sji,dc=broadcom,dc=net +LDAP global use_type all (default) +LDAP global port 389 (default) + +LDAP_SERVER address 10.59.143.229 + priority 1 + retry 0 + use_type all + +admin@L10:~$ + +``` + +### KLISH: show aaa, show radius-server, show ldap-server: After + +``` +sonic# show aaa +--------------------------------------------------------- +AAA Authentication Information +--------------------------------------------------------- +failthrough : True +login-method : radius, local +--------------------------------------------------------- +AAA Authorization Information +--------------------------------------------------------- +login : ldap +--------------------------------------------------------- +AAA Name-Service Information +--------------------------------------------------------- +passwd : ldap +shadow : ldap +group : ldap +sonic# +sonic# show radius-server +--------------------------------------------------------- +RADIUS Global Configuration +--------------------------------------------------------- +key : sharedsecret +-------------------------------------------------------------------------------- +HOST AUTH-TYPE KEY AUTH-PORT PRIORITY TIMEOUT RTSMT VRF +-------------------------------------------------------------------------------- +10.59.143.229 - - 1812 1 - - - +sonic# +sonic# show ldap-server +--------------------------------------------------------- +LDAP Global Configuration +--------------------------------------------------------- +binddn : dc=sji,dc=broadcom,dc=net +pam_member_attribute : memberUid +pam_groupdn : cn=sudo,ou=Group,dc=sji,dc=broadcom,dc=net +-------------------------------------------------------------------------------- +HOST USE-TYPE PRIORITY SSL RETRY +-------------------------------------------------------------------------------- +10.59.143.229 - 1 - - +sonic# + +``` + +### Verify user can login using RADIUS password with LDAP Name Service + +``` + +$ ssh ldapuser@10.59.143.109 +ldapuser@10.59.143.109's password: +Creating directory '/home/ldapuser'. +Linux L10 4.9.0-11-2-amd64 #1 SMP Debian 4.9.189-3+deb9u2 (2019-11-11) x86_64 +... +ldapuser@L10:~$ + +``` + + +## UT Authentication RADIUS with Name Service LDAP and sudo Authorization LDAP + +### Click CLI + +``` + +sudo config radius add 10.59.143.229 +sudo config radius passkey sharedsecret +sudo config aaa authentication failthrough enable +sudo config aaa authentication login radius local + +sudo config ldap add 10.59.143.229 +sudo config ldap base dc=sji,dc=broadcom,dc=net +sudo config aaa nss passwd ldap +sudo config aaa nss shadow ldap +sudo config aaa nss group ldap + +sudo config aaa nss sudoers ldap +sudo config ldap sudoers_base ou=Sudoers,dc=sji,dc=broadcom,dc=net + + +``` + +### KLISH CLI + +``` + +sonic(config)# radius-server host 10.59.143.229 +sonic(config)# radius-server key sharedsecret +sonic(config)# aaa authentication failthrough enable +sonic(config)# aaa authentication login default group radius local + +sonic(config)# ldap-server host 10.59.143.229 +sonic(config)# ldap-server base dc=sji,dc=broadcom,dc=net +sonic(config)# aaa name-service passwd group ldap +sonic(config)# aaa name-service shadow group ldap +sonic(config)# aaa name-service group group ldap + +sonic(config)# aaa name-service sudoers group ldap +sonic(config)# ldap-server sudoers-base ou=Sudoers,dc=sji,dc=broadcom,dc=net + +``` + +### Config DB: After + +``` + +127.0.0.1:6379[4]> keys *AAA* +1) "AAA|nss" +2) "AAA|authentication" +127.0.0.1:6379[4]> keys *RADIUS* +1) "RADIUS_SERVER|10.59.143.229" +2) "RADIUS|global" +127.0.0.1:6379[4]> keys *LDAP* +1) "LDAP|global" +2) "LDAP_SERVER|10.59.143.229" +127.0.0.1:6379[4]> hgetall AAA|nss +1) "passwd" +2) "ldap" +3) "shadow" +4) "ldap" +5) "group" +6) "ldap" +7) "sudoers" +8) "ldap" +127.0.0.1:6379[4]> hgetall AAA|authentication +1) "failthrough" +2) "True" +3) "login" +4) "radius,local" +127.0.0.1:6379[4]> hgetall RADIUS|global +1) "passkey" +2) "sharedsecret" +127.0.0.1:6379[4]> hgetall RADIUS_SERVER|10.59.143.229 +1) "priority" +2) "1" +3) "auth_port" +4) "1812" +127.0.0.1:6379[4]> hgetall LDAP|global +1) "base" +2) "dc=sji,dc=broadcom,dc=net" +3) "sudoers_base" +4) "ou=Sudoers,dc=sji,dc=broadcom,dc=net" +127.0.0.1:6379[4]> hgetall LDAP_SERVER|10.59.143.229 +1) "priority" +2) "1" +3) "retry" +4) "0" +5) "use_type" +6) "all" +127.0.0.1:6379[4]> + +``` + +### show aaa, show radius, show ldap: After + +``` + +admin@L10:~$ show aaa +AAA authentication login radius,local +AAA authentication failthrough True +AAA nss group ldap +AAA nss passwd ldap +AAA nss sudoers ldap +AAA nss netgroup local (default) +AAA nss shadow ldap +AAA authorization login local (default) + +admin@L10:~$ + +admin@L10:~$ show radius +RADIUS global auth_type pap (default) +RADIUS global retransmit 3 (default) +RADIUS global timeout 5 (default) +RADIUS global passkey sharedsecret + +RADIUS_SERVER address 10.59.143.229 + priority 1 + auth_port 1812 + +admin@L10:~$ + +admin@L10:~$ show ldap +LDAP global retry 0 (default) +LDAP global ssl off (default) +LDAP global sudoers_base ou=Sudoers,dc=sji,dc=broadcom,dc=net +LDAP global idle_timelimit 0 (default) +LDAP global bind_timelimit 10 (default) +LDAP global timelimit 0 (default) +LDAP global ldap_version 3 (default) +LDAP global scope sub (default) +LDAP global base dc=sji,dc=broadcom,dc=net +LDAP global use_type all (default) +LDAP global port 389 (default) + +LDAP_SERVER address 10.59.143.229 + priority 1 + retry 0 + use_type all + +admin@L10:~$ + +``` + +### KLISH: show aaa, show radius-server, show ldap-server: After + +``` +sonic# show aaa +--------------------------------------------------------- +AAA Authentication Information +--------------------------------------------------------- +failthrough : True +login-method : radius, local +--------------------------------------------------------- +AAA Name-Service Information +--------------------------------------------------------- +passwd : ldap +shadow : ldap +group : ldap +sudoers : ldap +sonic# +sonic# show radius-server +--------------------------------------------------------- +RADIUS Global Configuration +--------------------------------------------------------- +key : sharedsecret +-------------------------------------------------------------------------------- +HOST AUTH-TYPE KEY AUTH-PORT PRIORITY TIMEOUT RTSMT VRF +-------------------------------------------------------------------------------- +10.59.143.229 - - 1812 1 - - - +sonic# +sonic# show ldap-server +--------------------------------------------------------- +LDAP Global Configuration +--------------------------------------------------------- +binddn : dc=sji,dc=broadcom,dc=net +sudoers-base : ou=Sudoers,dc=sji,dc=broadcom,dc=net +-------------------------------------------------------------------------------- +HOST USE-TYPE PRIORITY SSL RETRY +-------------------------------------------------------------------------------- +10.59.143.229 - 1 - - +sonic# + +``` + +### Verify user can login using RADIUS password, LDAP Name Service, and sudo + +``` + +$ ssh raduser1@10.59.143.109 +raduser1@10.59.143.109's password: +Linux L10 4.9.0-11-2-amd64 #1 SMP Debian 4.9.189-3+deb9u2 (2019-11-11) x86_64 +... +raduser1@L10:~$ id +uid=1202(raduser1) gid=100(users) groups=100(users),999(docker) +raduser1@L10:~$ sudo -i +[sudo] password for raduser1: +root@L10:~# id +uid=0(root) gid=0(root) groups=0(root) +root@L10:~# exit +logout +raduser1@L10:~$ + +``` + +## UT Authentication Non Global NSS PAM SUDO options + +### Click CLI + +``` + +sudo config radius add 10.59.143.229 +sudo config radius passkey sharedsecret +sudo config aaa authentication failthrough enable +sudo config aaa authentication login radius local + +sudo config ldap add 10.59.143.229 +sudo config ldap base dc=sji,dc=broadcom,dc=net + +sudo config ldap nss base dc=sji,dc=broadcom,dc=net +sudo config aaa nss passwd ldap +sudo config aaa nss shadow ldap +sudo config aaa nss group ldap + +sudo config aaa nss sudoers ldap +sudo config ldap sudo sudoers_base ou=Sudoers,dc=sji,dc=broadcom,dc=net + +sudo config ldap pam pam_groupdn cn=docker,ou=Group,dc=sji,dc=broadcom,dc=net +sudo config ldap pam pam_member_attribute memberUid +sudo config aaa authorization login ldap + +``` + +### KLISH CLI + +``` + +sonic(config)# radius-server host 10.59.143.229 +sonic(config)# radius-server key sharedsecret +sonic(config)# aaa authentication failthrough enable +sonic(config)# aaa authentication login default group radius local + +sonic(config)# ldap-server host 10.59.143.229 +sonic(config)# ldap-server base dc=sji,dc=broadcom,dc=net +sonic(config)# ldap-server nss base dc=sji,dc=broadcom,dc=net +sonic(config)# aaa name-service passwd group ldap +sonic(config)# aaa name-service shadow group ldap +sonic(config)# aaa name-service group group ldap + +sonic(config)# aaa name-service sudoers group ldap +sonic(config)# ldap-server sudo sudoers-base ou=Sudoers,dc=sji,dc=broadcom,dc=net + +sonic(config)# ldap-server pam pam-groupdn cn=docker,ou=Group,dc=sji,dc=broadcom,dc=net +sonic(config)# ldap-server pam pam-member-attribute memberUid +sonic(config)# aaa authorization login default group ldap + +``` + +### Config DB: After + +``` + +127.0.0.1:6379[4]> keys *AAA* +1) "AAA|authorization" +2) "AAA|nss" +3) "AAA|authentication" +127.0.0.1:6379[4]> keys *RADIUS* +1) "RADIUS_SERVER|10.59.143.229" +2) "RADIUS|global" +127.0.0.1:6379[4]> keys *LDAP* +1) "LDAP|global" +2) "LDAP|nss" +3) "LDAP|pam" +4) "LDAP|sudo" +5) "LDAP_SERVER|10.59.143.229" +127.0.0.1:6379[4]> hgetall AAA|nss +1) "passwd" +2) "ldap" +3) "shadow" +4) "ldap" +5) "group" +6) "ldap" +5) "sudoers" +6) "ldap" +127.0.0.1:6379[4]> hgetall AAA|authentication +1) "failthrough" +2) "True" +3) "login" +4) "radius,local" +127.0.0.1:6379[4]> hgetall AAA|authorization +1) "login" +2) "ldap" +127.0.0.1:6379[4]> hgetall RADIUS|global +1) "passkey" +2) "sharedsecret" +127.0.0.1:6379[4]> hgetall RADIUS_SERVER|10.59.143.229 +1) "priority" +2) "1" +3) "auth_port" +4) "1812" +127.0.0.1:6379[4]> hgetall LDAP|global +1) "base" +2) "dc=sji,dc=broadcom,dc=net" +127.0.0.1:6379[4]> hgetall LDAP|nss +1) "base" +2) "dc=sji,dc=broadcom,dc=net" +127.0.0.1:6379[4]> hgetall LDAP|pam +1) "pam_groupdn" +2) "cn=docker,ou=Group,dc=sji,dc=broadcom,dc=net" +3) "pam_member_attribute" +4) "memberUid" +127.0.0.1:6379[4]> hgetall LDAP|sudo +1) "sudoers_base" +2) "ou=Sudoers,dc=sji,dc=broadcom,dc=net" +127.0.0.1:6379[4]> + +``` + +### show aaa, show radius, show ldap: After + +``` + +admin@L10:~$ show aaa +AAA authentication login radius,local +AAA authentication failthrough True +AAA nss group ldap +AAA nss passwd ldap +AAA nss shadow ldap +AAA nss sudoers ldap +AAA nss netgroup local (default) +AAA authorization login ldap + +admin@L10:~$ + +admin@L10:~$ show radius +RADIUS global auth_type pap (default) +RADIUS global retransmit 3 (default) +RADIUS global timeout 5 (default) +RADIUS global passkey sharedsecret + +RADIUS_SERVER address 10.59.143.229 + priority 1 + auth_port 1812 + +admin@L10:~$ + +admin@L10:~$ show ldap +LDAP global retry 0 (default) +LDAP global ssl off (default) +LDAP global idle_timelimit 0 (default) +LDAP global bind_timelimit 10 (default) +LDAP global timelimit 0 (default) +LDAP global ldap_version 3 (default) +LDAP global scope sub (default) +LDAP global base dc=sji,dc=broadcom,dc=net +LDAP global use_type all (default) +LDAP global port 389 (default) +LDAP nss base dc=sji,dc=broadcom,dc=net +LDAP pam pam_member_attribute memberUid +LDAP pam pam_groupdn cn=docker,ou=Group,dc=sji,dc=broadcom,dc=net +LDAP sudo sudoers_base ou=Sudoers,dc=sji,dc=broadcom,dc=net + +LDAP_SERVER address 10.59.143.229 + priority 1 + retry 0 + use_type all + +admin@L10:~$ + +``` + +### KLISH: show aaa, show radius-server, show ldap-server: After + +``` +sonic# show aaa +--------------------------------------------------------- +AAA Authentication Information +--------------------------------------------------------- +failthrough : True +login-method : radius, local +--------------------------------------------------------- +AAA Authorization Information +--------------------------------------------------------- +login : ldap +--------------------------------------------------------- +AAA Name-Service Information +--------------------------------------------------------- +passwd : ldap +shadow : ldap +group : ldap +sudoers : ldap +sonic# +sonic# show radius-server +--------------------------------------------------------- +RADIUS Global Configuration +--------------------------------------------------------- +key : sharedsecret +-------------------------------------------------------------------------------- +HOST AUTH-TYPE KEY AUTH-PORT PRIORITY TIMEOUT RTSMT VRF +-------------------------------------------------------------------------------- +10.59.143.229 - - 1812 1 - - - +sonic# +sonic# show ldap-server +--------------------------------------------------------- +LDAP Global Configuration +--------------------------------------------------------- +binddn : dc=sji,dc=broadcom,dc=net +--------------------------------------------------------- +LDAP NSS Configuration +--------------------------------------------------------- +binddn : dc=sji,dc=broadcom,dc=net +--------------------------------------------------------- +LDAP PAM Configuration +--------------------------------------------------------- +pam_member_attribute : memberUid +pam_groupdn : cn=docker,ou=Group,dc=sji,dc=broadcom,dc=net +--------------------------------------------------------- +LDAP sudo Configuration +--------------------------------------------------------- +sudoers_base : ou=Sudoers,dc=sji,dc=broadcom,dc=net +-------------------------------------------------------------------------------- +HOST USE-TYPE PRIORITY SSL RETRY +-------------------------------------------------------------------------------- +10.59.143.229 - 1 - - +sonic# + +``` + +### Verify user can login using RADIUS password with LDAP Name Service LDAP logon authorization and sudo authorization + +``` + +$ ssh raduser1@10.59.143.109 +raduser1@10.59.143.109's password: +Linux L10 4.9.0-11-2-amd64 #1 SMP Debian 4.9.189-3+deb9u2 (2019-11-11) x86_64 +... +raduser1@L10:~$ id +uid=1202(raduser1) gid=100(users) groups=100(users),999(docker) +raduser1@L10:~$ sudo -i +[sudo] password for raduser1: +root@L10:~# id +uid=0(root) gid=0(root) groups=0(root) +root@L10:~# exit +logout +raduser1@L10:~$ + +``` + +## UT Authentication LDAP AD ? For Implementation In a Future Release (IIFR) + +### Click CLI + +``` + +sudo config ldap add 10.75.16.174 +sudo config ldap base 'OU=San Jose,OU=California,OU=US,OU=Users,OU=Accounts,DC=Broadcom,DC=net' +sudo config ldap binddn 'binddn CN=Search(035)Account,OU=San Jose,OU=California,OU=US,OU=Users,OU=Accounts,DC=Broadcom,DC=net' +sudo config ldap bindpw searchPW +sudo config ldap pam_login_attribute sAMAccountName +sudo config ldap pam_filter objectclass=User + +sudo config ldap map add attribute uid sAMAccountName shadowLastChange pwdLastSet uniqueMember member +sudo config ldap map add objectclass posixAccount user shadowAccount user posixGroup group + + +sudo config aaa authentication login ldap local +sudo config aaa authentication failthrough enable + +``` + +### Config DB: After + +``` + +127.0.0.1:6379[4]> keys *LDAP* +1) "LDAP_MAP|OBJECTCLASS|posixGroup" +2) "LDAP_MAP|ATTRIBUTE|shadowLastChange" +3) "LDAP_MAP|ATTRIBUTE|uid" +4) "LDAP_MAP|ATTRIBUTE|uniqueMember" +5) "LDAP_MAP|OBJECTCLASS|posixAccount" +6) "LDAP_MAP|OBJECTCLASS|shadowAccount" +7) "LDAP_SERVER|10.75.16.174" +8) "LDAP|global" +127.0.0.1:6379[4]> keys *AAA* +1) "AAA|authentication" +127.0.0.1:6379[4]> hgetall LDAP_SERVER|10.75.16.174 +1) "priority" +2) "1" +3) "retry" +4) "0" +5) "use_type" +6) "all" +127.0.0.1:6379[4]> hgetall LDAP|global + 1) "pam_login_attribute" + 2) "sAMAccountName" + 3) "pam_filter" + 4) "objectclass=User" + 5) "binddn" + 6) "binddn CN=Search(035)Account,OU=San Jose,OU=California,OU=US,OU=Users,OU=Accounts,DC=Broadcom,DC=net" + 7) "bindpw" + 8) "searchPW" + 9) "base" +10) "OU=San Jose,OU=California,OU=US,OU=Users,OU=Accounts,DC=Broadcom,DC=net" +127.0.0.1:6379[4]> hgetall LDAP_MAP|OBJECTCLASS|posixGroup +1) "to" +2) "group" +127.0.0.1:6379[4]> hgetall LDAP_MAP|ATTRIBUTE|shadowLastChange +1) "to" +2) "pwdLastSet" +127.0.0.1:6379[4]> hgetall LDAP_MAP|ATTRIBUTE|uid +1) "to" +2) "sAMAccountName" +127.0.0.1:6379[4]> hgetall LDAP_MAP|ATTRIBUTE|uniqueMember +1) "to" +2) "member" +127.0.0.1:6379[4]> hgetall LDAP_MAP|OBJECTCLASS|posixAccount +1) "to" +2) "user" +127.0.0.1:6379[4]> hgetall LDAP_MAP|OBJECTCLASS|shadowAccount +1) "to" +2) "user" +127.0.0.1:6379[4]> + +``` + +### show aaa, show ldap: After + +``` + +admin@L10:~$ id aduser +uid=95431(aduser) gid=95431 groups=95431 +admin@L10:~$ +admin@L10:~$ show aaa +AAA authentication login ldap,local +AAA authentication failthrough True +AAA nss passwd login (default) +AAA nss shadow login (default) +AAA nss group login (default) +AAA nss netgroup local (default) +AAA authorization login local (default) + +admin@L10:~$ show ldap +LDAP global binddn CN=Search(035)Account,OU=San Jose,OU=California,OU=US,OU=Users,OU=Accounts,DC=Broadcom,DC=net +LDAP global retry 0 (default) +LDAP global ssl off (default) +LDAP global idle_timelimit 0 (default) +LDAP global bind_timelimit 10 (default) +LDAP global timelimit 0 (default) +LDAP global base OU=San Jose,OU=California,OU=US,OU=Users,OU=Accounts,DC=Broadcom,DC=net +LDAP global bindpw searchPW +LDAP global ldap_version 3 (default) +LDAP global pam_filter objectclass=User +LDAP global scope sub (default) +LDAP global pam_login_attribute sAMAccountName +LDAP global use_type all (default) +LDAP global port 389 (default) + +LDAP_MAP_ATTRIBUTE + uniqueMember member + shadowLastChange pwdLastSet + uid sAMAccountName + +LDAP_MAP_OBJECTCLASS + shadowAccount user + posixGroup group + posixAccount user + +LDAP_SERVER address 10.75.16.174 + priority 1 + retry 0 + use_type all + +admin@L10:~$ + +``` + +### Verify AD user can login with LDAP password. + +``` +$ ssh aduser@10.59.143.205 +aduser@10.59.143.205's password: +Creating directory '/home/aduser'. +Linux L10 4.9.0-11-2-amd64 #1 SMP Debian 4.9.189-3+deb9u2 (2019-11-11) x86_64 +... +aduser@L10:~$ + +``` + +# References + +## nss_ldap + +https://manpages.debian.org/testing/libnss-ldap/libnss-ldap.conf.5.en.html + +## pam_ldap + +https://manpages.debian.org/testing/libpam-ldapd/pam_ldap.8.en.html + +## sudo_ldap + +https://www.sudo.ws/man/1.8.17/sudoers.ldap.man.html + +## Linux PAM + +http://man7.org/linux/man-pages/man3/pam.3.html + +## NSS Reference + +http://man7.org/linux/man-pages/man5/nsswitch.conf.5.html + +## LDAP client configuration file + +https://linux.die.net/man/5/ldap.conf + +## TACPLUS Authentication + +https://github.com/Azure/SONiC/blob/master/doc/aaa/TACACS%2B%20Authentication.md + +## RADIUS Management User Authentication + +https://github.com/Azure/SONiC/blob/c82c8b67c31a63b1b438a143e2ce1a92b2b580fb/doc/aaa/radius_authentication.md + + diff --git a/doc/audit-log/Audit_Log_HLD.md b/doc/audit-log/Audit_Log_HLD.md new file mode 100644 index 0000000000..13d1956c10 --- /dev/null +++ b/doc/audit-log/Audit_Log_HLD.md @@ -0,0 +1,393 @@ +# Feature Name +Audit Log. +# High Level Design Document +#### Rev 0.1 + +# Table of Contents + * [List of Tables](#list-of-tables) + * [Revision](#revision) + * [About This Manual](#about-this-manual) + * [Scope](#scope) + * [Definition/Abbreviation](#definitionabbreviation) + +# List of Tables +[Table 1: Abbreviations](#table-1-abbreviations) + +# Revision +| Rev | Date | Author | Change Description | +|:---:|:-----------:|:------------------:|-----------------------------------| +| 0.1 | 06/16/2020 | Srinadh Penugonda | Initial version | + +# About this Manual +This document provides general information about the Audit Log feature implementation in SONiC. +# Scope +This document describes the high level design of Audit Log feature. + +# 1 Feature Overview + +Audit log provides a way to monitor several security relevant information on the system with an insight on security health of the system. + +## 1.1 Requirements + +- The log messages will be stored as a separate file under /var/log on the switch itself and will follow similar format to syslog messages. No support for remote logging or remote destination. + +- The audit log should contain the following messages: + 1. Log messages corresponding to login and logout through SSH and Console. The message content & format is from open source modules and is fixed. + 2. Log messages corresponding to when user tries config & show command via KLISH, gNMI and REST. The command message will not have command hierarchy information. + + Audit log result for REST & gNMI will be in simple flat list; message will not be in key/value format. + +- Log rotate is used to enforce size as 10M for audit log and goes through four rotations before being removed. + +- Audit log files should be included in tech-support bundle. + +### 1.1.2 Configuration and Management Requirements +- There need to be a show command to display the contents of audit log. + 1. show audit-log all - lists all messages from audit.log and rotated, uncompressed audit logs - namely, audit.log.1. + The command will not attempt to display compressed and rotated audit log files. + + The *all* filter will fetch all audit log messages and potentially have performance implications and high response time. + + 2. show audit-log - lists recent 20 messages from audit.log + +- There need to be a clear command to clear contents of audit logs. + +## 1.2 Design Overview +### 1.2.1 Basic Approach +The feature implements the framework for audit log. As of now, the audit log comprises of messages corresponding to +* login and logout related events through SSH and console +* requests from north bound interfaces: CLI, REST and gNMI + +Syslog rules are used to filter out these messages and populate audit log. + +The login linux package is updated to trigger syslog message for login through console. + +Any subsequent requirements to add new audit messages need to add a new syslog rule or augment existing syslog rules +as mentioned in this [section](#311-syslog-rules) + +# 2 Functionality +## 2.1 Target Deployment Use Cases +Audit Log collects certain types of system activity to facilitate incident investigation. The system activities includes +login/logout events through ssh/console. The other correpsonds to requests from north bound interfaces - CLI, REST and gNMI. + +# 3 Design +## 3.1 Overview +Specified audit messages are stored in /var/log/audit.log. + +Syslog rules with the help of various syslog properties like *syslogtag*, *syslogfacility*, *programname* are used to filter +these messages and redirect them to /var/log/audit.log. + +SSHD triggers messages for login and logout through ssh. For login and logout through console, linux login package need to be +modified to trigger a syslog message. + +Log rotate would manage audit log with size restrictions of 1M. + +### 3.1.1 Syslog Rules +SSHD generates successful login/logout messages with *authpriv* facility. Messages corresponding to invalid credentials triggered +with *auth* facility. + +``` +if $syslogtag contains 'sshd' then { + if $syslogfacility-text == 'authpriv' then { + action(type="omfile" file="/var/log/audit.log") + } + if $syslogfacility-text == 'auth' then { + action(type="omfile" file="/var/log/audit.log") + } +} +``` +Management framework log messages are sent with *local4* facility. +``` +if $syslogfacility-text == 'local4' then { + action(type="omfile" file="/var/log/audit.log") +} +``` +Some of the console login/logout messages comes through *systemd*. +``` +if $programname == 'systemd' then { + action(type="omfile" file="/var/log/audit.log") +} +``` + +The login linux package triggers messages corresponding to successful and failed login attempts with *programname* as **login**. +``` +if $programname == 'login' then { + action(type="omfile" file="/var/log/audit.log") +} +``` + +### 3.1.2 Linux login package +This package is modified to trigger a syslog message after user logs in. The build system is updated to build login target. + +### 3.1.3 Commands +'show audit-log' displays contents of audit.log and audit.log.1. +By default the command displays a brief snapshot of audit log by displaying around latest twenty messages. +With *all* option, all of the audit.log and audit.log.1 is displayed. + +Through REST, the path is /restconf/operations/sonic-auditlog:get-auditlog +For gNMI, the path is sonic-auditlog:get-auditlog + +``` +sonic# show audit-log + Display all of audit log + | Pipe through a command + +``` +Through REST, the path is /restconf/operations/sonic-auditlog:clear-auditlog +For gNMI, the path is /sonic-auditlog:clear-auditlog + +``` +sonic# clear audit-log + + +``` + +### 3.1.4 Log rotate +Log rotate is configured to enforce size limitation of 10M for audit log. There will be four rotations. Rsyslog will be restated +after each rotation. + +### 3.1.5 show tech-support +The existing show tech-support already packages all the files under /var/log into tech-support package. + +Currently, the following log files from /var/log are packaged with tech support: +audit.log, auth.log, cron.log, daemon.log, messages, syslog, stpd.log, teamd.log, telemetry.log, udldd.log, user.log and zebra.log + +## 3.2 User Interface +### 3.2.1 Data Models +``` +module sonic-auditlog { + namespace "http://github.com/Azure/sonic-auditlog"; + prefix auditlog; + yang-version 1.1; + + organization + "SONiC"; + + contact + "SONiC"; + + description + "SONiC yang for RPC based audit log operations."; + + revision 2020-05-29 { + description + "Initial revision."; + } + + rpc get-auditlog { + description "RPC for getting audit log."; + + input { + leaf content-type { + type string { + pattern "all"; + } + description "Indicates if user wants to get all or latest twenty lines of audit log"; + } + } + + output { + list audit-content { + leaf content { + type string; + description "Audit message"; + } + } + } + } + + rpc clear-auditlog { + description "RPC for clearing audit log."; + } +} + +``` +### 3.2.2 CLI +#### 3.2.2.1 Clear Commands +Clearing audit log is implemented as RPC and is achieved with the help of HostQuery module - clearaudit.py. +On receiving the request from container, this module will remove audit.log and restarts rsyslog. + +``` +sonic# clear audit-log +sonic# +``` + +#### 3.2.2.2 Show Commands +To display contents of audit log, show command is implemented and it is RPC. +To achieve better performance, audit log is mounted onto mgmt-framework container. +The command by default displays a brief snapshot of audit log by showing last twenty lines. +The command has an 'all' option to display all of the audit.log and audit.log.1. + +``` +sonic# show audit-log + Display all of audit log + | Pipe through a command + +``` + +### 3.2.3 REST API Support +#### 3.2.3.1 Show Audit Log +Path: /restconf/operations/sonic-auditlog:get-auditlog + +Input: "all" or none + +#### 3.2.3.2 Clear Audit Log +Path: /restconf/operations/sonic-auditlog:clear-auditlog + +### 3.2.4 GNMI API Support +#### 3.2.4.1 Show Audit Log +Path: /sonic-auditlog:get-auditlog + +Input: "all" or none + +#### 3.2.4.2 Clear Audit Log +Path: /sonic-auditlog:clear-auditlog + +# 4 Serviceability and Debug + +- **Successful login** + +Provides user name, IP from where user is logging in from. + +*SSH* + +Jun 2 22:47:08.619590 sonic INFO sshd[13990]: Accepted password for **admin** from **10.14.8.140** port 49074 ssh2 +Jun 2 22:47:08.711691 sonic INFO sshd[13990]: pam_unix(sshd:session): **session opened** for user **admin** by (uid=0) + +The first message displays that authentication was successful for the user 'admin'. The IP of the host from where host is trying to login is displayed along with its port. + +Timestamp at which the login occurred is displayed. + +The subsequent log indicates that session has been opened for user 'admin'. + +*Console* + +Jun 2 22:48:47.939333 sonic INFO login[30983]: Accepted password for **admin** on terminal=**'/dev/ttyS0'** +Jun 2 22:48:48.056522 sonic INFO login[30983]: pam_unix(login:session): **session opened** for user **admin** by LOGIN(uid=0) + +- **Successful logout** + +Provides user name, IP from where user is logging out from. + +*SSH* + +Jun 2 22:49:33.855434 sonic INFO sshd[14073]: Received disconnect from **10.14.8.140** port 49074:11: **disconnected by user** +Jun 2 22:49:33.966591 sonic INFO sshd[13990]: pam_unix(sshd:session): **session closed** for user **admin** + +The first message indicates that user has disconnected the session. The IP and port of the host from which user disconnected the session is displayed as well. + +The second message indicates that session for user 'admin' has been closed. + +Timestamp at which the login occurred is displayed. + +*Console* + +Jun 2 22:50:47.510380 sonic INFO login[30983]: pam_unix(login:session): **session closed** for user **admin** +Jun 2 22:50:47.665222 sonic INFO systemd[1]: Stopped Serial Getty on **ttyS0** + +- **Login with invalid username** + +Provides invalid user name and IP address of the host invalid user is trying to log in from. + +*SSH* + +Jun 2 22:51:53.619712 sonic INFO sshd[31688]: **Invalid user** **adminxxx** from **10.14.8.140** port 49090 + +The message indicates that there was an attempt to connect over ssh with an invalid user name. + +The user name is displayed along with the host IP and port from which user tried to login. + +*Console* + +Jun 2 22:52:49.080189 sonic NOTICE login[27921]: pam_unix(login:auth): **authentication failure**; logname=LOGIN uid=0 euid=0 **tty=/dev/ttyS0** ruser= rhost= +Jun 2 22:52:52.055801 sonic NOTICE login[27921]: FAILED LOGIN (1) on '/dev/ttyS0' FOR 'UNKNOWN', Authentication failure + +- **Login with invalid password** + +Informs that specified user trying to log in with an invalid password and informs IP of the host user is logging in from. + +*SSH* + +Jun 2 22:53:39.571670 sonic NOTICE sshd[6296]: pam_unix(sshd:auth): **authentication failure**; logname= uid=0 euid=0 tty=ssh ruser= rhost=**10.14.8.140** user=**admin** +Jun 2 22:53:41.163938 sonic INFO sshd[6296]: Failed password for admin from 10.14.8.140 port 49094 ssh2 + +The message indicates that user 'admin' entered wrong password that resulted in authentication failure. + +The host IP and port from which user tried the login is displayed in the message. + +*Console* + +Jun 2 22:54:54.938982 sonic NOTICE login[6927]: pam_unix(login:auth): **authentication failure**; logname=LOGIN uid=0 euid=0 **tty=/dev/ttyS0** ruser= rhost= user=**admin** +Jun 2 22:54:57.568058 sonic NOTICE login[6927]: FAILED LOGIN (1) on '/dev/ttyS0' FOR 'admin', Authentication failure + +- **Session Timeout** + +*SSH* +Jun 10 19:26:32.887528 sonic INFO sshd[24578]: **Timeout, client not responding.** + +Jun 10 19:26:33.025597 sonic INFO sshd[24481]: pam_unix(sshd:session): **session closed** for user **admin** + +*Console* + +Jun 10 20:02:33.904878 sonic INFO systemd[1]: **Stopped Serial Getty** on **ttyS0**. + +- **Set Request** + +Informs type of request, URI, user name, command string and status of command execution. + +*CLI* + +Jun 2 22:57:09.060819 sonic INFO mgmt-framework#clish: User "**admin**" command "**tacacs-server key mykey**" status - **success** + +The message displays the command string; its status and name of the user that executed the command. + +*REST/gNMI* + +Jun 12 19:33:40.728039 sonic INFO mgmt-framework#/usr/sbin/rest_server[711]: [REST-5] User "**admin@10.14.125.28**:55937" request "**PATCH** **/restconf/data/openconfig-system:system/aaa/server-groups/server-group=TACACS/config/openconfig-system-ext:secret-key**" status - **204** + +- **Delete Request** + +Informs type of request, URI, user name, command string and status of command execution. + +*CLI* + +May 27 21:19:32.141471 sonic INFO mgmt-framework#clish: User "**admin**" command "**no tacacs-server timeout**" status - **success** + +The message displays the command string; its status and name of the user that executed the command. + +*REST/gNMI* + +Jun 12 19:35:03.326971 sonic INFO mgmt-framework#/usr/sbin/rest_server[711]: [REST-6] User "**admin@10.14.125.28**:55937" request "**DELETE /restconf/data/openconfig-system:system/aaa/server-groups/server-group=TACACS/config/openconfig-system-ext:secret-key**" status - **204** + +- **Get Request** + +Informs type of request, URI, user name, command string and status of command execution. + +*CLI* + +Jun 2 22:55:55.171404 sonic INFO mgmt-framework#clish: User "**admin**" command "**show tacacs-server global**" status - **success** + +The message displays the command string; its status and name of the user that executed the command. + +*REST/gNMI* + +Jun 12 19:36:04.059130 sonic INFO mgmt-framework#/usr/sbin/rest_server[711]: [REST-7] User "**admin@10.14.125.28**:55937" request "**GET /restconf/data/openconfig-system:system/aaa/server-groups/server-group=TACACS/config/openconfig-system-ext:secret-key**" status - **200** + +# 5 Unit Test +- show audit-log : Should display around last twenty audit messages +- show audit-log all : Should display audit.log and audit.log.1 (if it exists) +- clear audit-log : clears audit-log +- Login through ssh, verify that show audit-log displays messages corresponding to login +- Logout through ssh, verify that show audit-log displays messages corresponding to logout +- Login through console, verify that show audit-log displays messages corresponding to login +- Logout through console, verify that show audit-log displays messages corresponding to logout +- Login with invalid password through ssh, verify that show audit-log displays messages corresponding to login +- Login with invalid password through console, verify that show audit-log displays messages corresponding to login +- Login with invalid username through ssh, verify that show audit-log displays messages corresponding to login +- Login with invalid username through console, verify that show audit-log displays messages corresponding to login +- timeout for an existing ssh session, verify that show audit-log displays messages corresponding to session clearing out +- timeout for an existing console session, verify that show audit-log displays messages corresponding to session clearing out +- set request through CLI, REST, gNMI, verify show audit-log displays user, command string (for CLI) / path (for REST/gNMI) and status +- get request through CLI, REST, gNMI, verify show audit-log displays user, command string (for CLI) / path (for REST/gNMI) and status +- delete request through CLI, REST, gNMI, verify show audit-log displays user, command string (for CLI) / path (for REST/gNMI) and status +- Verify that show tech-support has audit log packaged diff --git a/doc/ifa-hld.md b/doc/ifa-hld.md new file mode 100644 index 0000000000..b4e97db320 --- /dev/null +++ b/doc/ifa-hld.md @@ -0,0 +1,284 @@ +# Feature Name +Inband Flow Analyzer. +# High Level Design Document +#### Rev 0.1 + +# Table of Contents + * [List of Tables](#list-of-tables) + * [Revision](#revision) + * [About This Manual](#about-this-manual) + * [Scope](#scope) + * [Definition/Abbreviation](#definitionabbreviation) + +# List of Tables +[Table 1: Abbreviations](#table-1-abbreviations) + +# Revision +| Rev | Date | Author | Change Description | +|:---:|:-----------:|:------------------:|-----------------------------------| +| 0.1 | 10/30/2019 | Srinadh Penugonda| Initial version | + +# About this Manual +This document provides general information about the Inband Flow Analyzer feature implementation in SONiC. +# Scope +This document describes the north bound interface and unit tests for Inband Flow Analyzer feature. + +# Definition/Abbreviation + +### Table 1: Abbreviations +| **Term** | **Meaning** | +|--------------------------|-------------------------------------| +| IFA | Inband Flow Analyzer | +| TAM | Telemetry and Monitoring | +# 1 Feature Overview + +Inband Flow Analyzer is a flexible packet and flow monitoring inband telemtry solution. The feature allows configuration of IFA sessions that provide Inband telemetry over sampled traffic to collectors. + +It provides mechanism to monitor and analyze when packets enter/exit the network, the path packets and flows take through the network, the rate at which packets arrive at each hop and how log packets spend at each hop etc., Out of band management technhiques can not measure such details. + +## 1.1 Requirements + +### 1.1.1 Functional Requirements + +Provide management framework support to existing SONiC capabilities with respect to IFA. + +1.1 IFA feature is accomplished by configuring IFA session on various nodes that act as ingress, +intermediate and egress devices. Device role is per flow in a node and a single node can act as ingress +device for one flow and intermediate device for another flow. + +2.0.0.1 TAM device identifier to uniquely identify a device in network and insert the same in INT header. +2.0.0.2 ACL configuration to identify a flow and sample packets from that flow to insert IFA headers. +2.0.0.3 TAM collector configuration that can be attached to IFA flow on egress device to forward telemetry +data. +3.0 UI commands available to configure TAM device identifier, TAM collector and IFA configuration. +3.1 UI commands available to show TAM device identifier, TAM collector, IFA configuration +3.2 UI commands available to clear IFA configuration +4.0 The maximum number of IFA 􀃖ows are platform dependent. +4.1 Only one collector can be con􀃕gured in a device. +4.5 Some platforms may require provisioning to enable IFA. 'ifa -config -enable' command to be issued to +provision such platforms for IFA functionality. 'ifa -config -disable' command can be issued to disable +provisioning of IFA on such platforms. + +### 1.1.2 Configuration and Management Requirements +1. CLI configuration/show support +2. REST API support +3. gNMI support + +## 1.2 Design Overview +### 1.2.1 Basic Approach + +As there is no opeconfig/ietf yang file exists for this feature, it is decided to go with sonic yang. + +### 1.2.2 Container +The changes are in sonic-management-framework container. + +There will be additional files added. +1. XML file for the CLI +2. Python script to handle CLI request (actioner) +3. Jinja template to render CLI output (renderer) +4. YANG models + sonic-ifa.yang + +### 1.2.3 SAI Overview +N/A + +# 2 Functionality +## 2.1 Target Deployment Use Cases +Whenever operator want to track packet latency/congestion metrics. +## 2.2 Functional Description +The management UI provides an user friendly interface to configure Inband Flow Analyzer. +# 3 Design +## 3.1 Overview +The packet action field in acl rule indicates the type of IFA device: ingress and egress. 'int_insert' makes the device as ingress IAF device and 'int_delete' as egress IFA device. Ingress IAF device makes a sample of a flow and tags them for analysis and data collection. Egress device is responsible for terminating IFA flow by summarizing the telemtry data of the entire path and sending it to the collector. + +The device identifer uniquely identifies the device in the network and inserts the ID in the IFA header. + +Collectors receive the telemetry data from egress devices. + +Flow configuration will contain sampling rate at which rate traffic will be sampled. + +IFA feature can be enabled or disabled. +## 3.6 User Interface +### 3.6.1 Data Models + +https://github.com/project-arlo/sonic-mgmt-framework/blob/master/models/yang/sonic/sonic-ifa.yang + +module: sonic-ifa + +--rw sonic-ifa + +--rw TAM_INT_IFA_FEATURE_TABLE + | +--rw TAM_INT_IFA_FEATURE_TABLE_LIST* [name] + | +--rw name enumeration + | +--rw enable? boolean + +--rw TAM_INT_IFA_FLOW_TABLE + +--rw TAM_INT_IFA_FLOW_TABLE_LIST* [name] + +--rw name string + +--rw acl-table-name string + +--rw acl-rule-name string + +--rw sampling-rate? uint16 + +--rw collector-name? string + +https://github.com/project-arlo/sonic-mgmt-framework/blob/master/models/yang/sonic/sonic-tam.yang + +module: sonic-tam + +--rw sonic-tam + +--rw TAM_DEVICE_TABLE + | +--rw TAM_DEVICE_TABLE_LIST* [name] + | +--rw name enumeration + | +--rw deviceid? uint16 + +--rw TAM_COLLECTOR_TABLE + +--rw TAM_COLLECTOR_TABLE_LIST* [name] + +--rw name string + +--rw ipaddress-type? enumeration + +--rw ipaddress? inet:ip-address + +--rw port? inet:port-number + +### 3.6.2 CLI +#### 3.6.2.1 Configuration Commands +1. Command : confg tam device-id +Attribute : +The command is used to configure TAM device identifier. + +2. Command : config tam collector +Attribute(s) : {collector-name} ip-type ip-addr
port +The command is used to configure TAM collector and IFA report will be forwarded to the collector. + +3. Command : config tam int-ifa feature +Attribute : +The command is used to enable or disable the IFA feature. + +4. Command : config tam int-ifa flow +Attribute(s) : acl-rule acl-table { sampling-rate collector } +The command is used to specify flow criteria to match against incoming flow and tag with IFA data. When sampling rate is specified, one packet will be sampled out of its value. When collector is specified, IFA report will be forwarded to it. + +5. Command : config-tam no device-id +The command is used to clear user configured device identifier. Default device identifier is used. + +6. Command : config-tam no collector +Attribute : +The command is used to delete previously configured collector information. + +7. Command : config-tam-int-ifa no flow +Attribute : +The command is used to delete previously configure flow information. + +#### 3.6.2.2 Show Commands +1. Command : show tam device +The command is used to show TAM device identifier. + +2. Command : show tam collector +Attribute : { | all } +The command is used to show TAM collector information. + +3. Command : show tam int-ifa flow +Attribute(s) : { | all } +The command is used to display configured IFA flow information. + +4. Command : show tam int-ifa statistics +Attribute(s) : { | all } +The command is used to display statisitcs of IFA flow. + +5. Command : show tam int-ifa status +The command is used to display status of TAM. + +#### 3.6.2.3 Debug Commands +N/A + +### 3.6.3 REST API Support + +1. Get Device Information +sonic-tam:sonic-tam/TAM_DEVICE_TABLE + +2. Get Collector information +sonic-tam:sonic-tam/TAM_COLLECTOR_TABLE + +3. Get particular collector information +sonic-tam:sonic-tam/TAM_COLLECTOR_TABLE/TAM_COLLECTOR_TABLE_LIST={name} + +4. Get IFA feature information +sonic-ifa:sonic-ifa/TAM_INT_IFA_FEATURE_TABLE + +5. Get IFA flow information +sonic-ifa:sonic-ifa/TAM_INT_IFA_FLOW_TABLE + +6. Get particular IFA flow information +sonic-ifa:sonic-ifa/TAM_INT_IFA_FLOW_TABLE/TAM_INT_IFA_FLOW_TABLE_LIST={name} + +7. Set device identifier +sonic-tam:sonic-tam/TAM_DEVICE_TABLE/TAM_DEVICE_TABLE_LIST={device}/deviceid +{ + "sonic-tam:deviceid": 0 +} + +8. Set TAM collector +sonic-tam:sonic-tam/TAM_COLLECTOR_TABLE/TAM_COLLECTOR_TABLE_LIST +{ + "sonic-tam:TAM_COLLECTOR_TABLE_LIST": [ + { + "name": "string", + "ipaddress-type": "ipv4", + "v4addr": "string", + "v6addr": "string", + "port": 0 + } + ] +} + +9. Set TAM INT IFA feature +sonic-ifa:sonic-ifa/TAM_INT_IFA_FEATURE_TABLE/TAM_INT_IFA_FEATURE_TABLE_LIST={feature}/enable +{ + "sonic-ifa:enable": true +} + +10. Set TAM INT IFA flow +sonic-ifa:sonic-ifa/TAM_INT_IFA_FLOW_TABLE/TAM_INT_IFA_FLOW_TABLE_LIST={name} +{ + "sonic-ifa:TAM_INT_IFA_FLOW_TABLE_LIST": [ + { + "name": "string", + "acl-table-name": "string", + "acl-rule-name": "string", + "sampling-rate": 0, + "collector": "string" + } + ] +} + +11. Delete TAM device identifier +sonic-tam:sonic-tam/TAM_DEVICE_TABLE/TAM_DEVICE_TABLE_LIST={device}/deviceid + +12. Delete TAM collector +sonic-tam:sonic-tam/TAM_COLLECTOR_TABLE/TAM_COLLECTOR_TABLE_LIST={name} + +13. Delete IFA flow +sonic-ifa:sonic-ifa/TAM_INT_IFA_FLOW_TABLE/TAM_INT_IFA_FLOW_TABLE_LIST={name} + + +# 4 Flow Diagrams +N/A +# 5 Error Handling +N/A +# 6 Serviceability and Debug +TBD + +# 7 Warm Boot Support +N/A +# 8 Scalability +N/A +# 9 Unit Test +| Test Name | Test Description | +| :------ | :----- | +| Create TAM DEvice Identifier | Verify device-id is configured. Verify a value with more than five digits will be rejected | +| Delete TAM device Identifier | Verify when device-id is deleted, it defaults to default value of 0 | +| Create TAM collector | Verify TAM collector is configured ( ip address validation is does by application ) | +| Delete TAM collector | Verify TAM collector can be deleted | +| Enable TAM INT IFA Feature | Verify user can enable/disable IFA feature | +| Create IFA Flow | Verify IFA flow is configured. Verify configuration fails when user uses invalid acl table/rule. Verify configuration fails when user uses invalid collector name | +| Delete IFA flow | Verify IFA flow is deleted | +| Show TAM Device | Verify configured device identifier is displayed with the show command | +| Show TAM collector | Verify all collectors are displayed with 'all' keyword. Verify specified collector is displayed with the name. Verify command fails when an invalid flow name is used to display | +| Show TAM INT IFA flow | Verify all flow information is displayed when used with 'all'. Verify particular flow is displayed when name is supplied. Verify command fails when an non-existent flow name is given | +| show tam int ifa supported | Verify feature status is correctly displayed | + + +# 10 Internal Design Information +N/A diff --git a/doc/in-band-mgmt/SONiC_in_band_mgmt_via_mgmt_Vrf_HLD.md b/doc/in-band-mgmt/SONiC_in_band_mgmt_via_mgmt_Vrf_HLD.md new file mode 100644 index 0000000000..287d3be254 --- /dev/null +++ b/doc/in-band-mgmt/SONiC_in_band_mgmt_via_mgmt_Vrf_HLD.md @@ -0,0 +1,57 @@ + +# Introduction + +The scope of this document is to provide the requirements and a high-level design proposal for in-band management via mgmt VRF. + +# Requirements + +The following are the high level requirements for the in-band management via mgmt VRF. + +1. Bind/Unbind L3 interface (e.g Phy/Port-channel/VLAN..etc) into mgmt VRF +2. Create/Delete mgmt VRF in HW to trap in-band management traffic to CPU +3. Add/Delete IP configured on the L3 interface into HW for ip2Me action even if the L3 interface is part of mgmt VRF. + +# Design Proposal + +The design is intended to have a generic approach for in-band management via mgmt VRF feature. A user can set an attribute "in_band_mgmt_enabled" to the config_db for MGMT_VRF_CONFIG table entry. The default value if not specified would be "false" + +The schema change for in-band management is as below: + +``` +MGMT_VRF_CONFIG|vrf_global + "mgmtVrfEnabled" : "true" + "in_band_mgmt_enabled": "true" + +``` +``` +; Defines management VRF table schema + +key = MGMT_VRF_CONFIG|vrf_global ; Same as existing +; field +mgmtVrfEnabled = "true"/"false" ; Same as existing +in_band_mgmt_enabled = "true" / "false" ; Default "false" (Optional attribute), this field is active only when mgmtVrfEnabled is set to true. + +``` +``` +key = VLAN_INTERFACE|{{intf_name}} ; Any L3 interface table entry e.g INTERFACE, PORTCHANNEL_INTERFACE..etc. +; field +"vrf_name" = "mgmt" ; Existing field but accepts mgmt VRF name + +``` +# Flows + +The following diagrams capture the kernel and SAI configuration flows. + +## Mgmt VRF configuration flow + +![](in_band_mgmt_vrf_config_flow.png) + +## L3 inteface to mgmt VRF bind flow + +![](in_band_mgmt_vrf_intf_config_flow.png) + +# Additional Notes +1. The user has to decide whether eth0 and L3 interface (e.g mgmt VLAN) can co-exist in the mgmt VRF, if yes, the corresponding configurations have to be taken care. +2. The user has to take care of configuring ACL to provide higher priority to mgmt traffic trapped from mgmt VRF, to avoid any potential drop in the NPU because of the data traffic. +3. User has to decide whether to use same data port for data & mgmt traffic or only for mgmt traffic based on the use-case. +4. The user has to make sure STP configurations are done such a way no impact to mgmt traffic via mgmt VLAN. diff --git a/doc/in-band-mgmt/in_band_mgmt_vrf_config_flow.png b/doc/in-band-mgmt/in_band_mgmt_vrf_config_flow.png new file mode 100644 index 0000000000..d0442d5105 Binary files /dev/null and b/doc/in-band-mgmt/in_band_mgmt_vrf_config_flow.png differ diff --git a/doc/in-band-mgmt/in_band_mgmt_vrf_intf_config_flow.png b/doc/in-band-mgmt/in_band_mgmt_vrf_intf_config_flow.png new file mode 100644 index 0000000000..cda56e31a4 Binary files /dev/null and b/doc/in-band-mgmt/in_band_mgmt_vrf_intf_config_flow.png differ diff --git a/doc/mgmt/Developer Guide.md b/doc/mgmt/Developer Guide.md new file mode 100644 index 0000000000..0ad53bf7dd --- /dev/null +++ b/doc/mgmt/Developer Guide.md @@ -0,0 +1,2192 @@ +# SONiC Management Framework Developer Guide + +## Rev 0.9 + +## Table of Contents + +* [List of Tables](#list-of-tables) +* [Revision](#revision) +* [About this Manual](#about-this-manual) +* [Scope](#scope) +* [Definition/Abbreviation](#definitionabbreviation) +* [References](#references) +* [Prerequisites](#prerequisite) +* [1 Architecture](#1-architecture) + * [1.1 Overview](#11-overview) + * [1.2 Repositories](#12-repositories) + * [1.3 Containers](#13-containers) +* [2 Developer Workflow](#2-developer-workflow) + * [2.1 ABNF Schema](#21-abnf-schema) + * [2.2 SONiC YANGs](#22-sonic-yang) + * [2.3 OpenConfig YANGs](#23-openconfig-yang) + * [2.3.1 Directory structure](#231-directory-structure) + * [2.3.2 Extension YANGs](#232-extension-yangs) + * [2.3.3 Extension YANG Style Guide](#233-extension-yang-style-guide) + * [2.3.4 Backward Compatibility](#234-backward-compatibility) + * [2.3.5 Versioning](#235-versioning) + * [2.3.6 Tools](#236-tools) + * [2.4 Config Translation App](#24-config-translation-app) + * [2.4.1 Translation Approaches](#241-translation-approaches) + * [2.4.2 YGOT](#242-ygot) + * [2.4.3 DB Access APIs](#243-db-access-apis) + * [2.4.4 Logging](#244-logging) + * [2.4.5 Error Response](#245-error-response) + * [2.4.6 D-Bus messaging](#246-d-bus-messaging) + * [2.5 Transformer](#25-transformer) + * [2.5.1 Annotation File](#251-annotation-file) + * [2.5.2 Annotations](#252-annotate-yang-paths) + * [2.5.2.1 sonic-ext:table-name](#2521-sonic-exttable-name-table_name) + * [2.5.2.2 sonic-ext:field-name](#2522-sonic-extfield-name-field_name) + * [2.5.2.3 sonic-ext:key-delimiter](#2523-sonic-extkey-delimiter-delim) + * [2.5.2.4 sonic-ext:key-name](#2524-sonic-extkey-name-value) + * [2.5.2.5 sonic-ext:key-transformer](#2525-sonic-extkey-transformer-function) + * [2.5.2.6 sonic-ext:field-transformer](#2526-sonic-extfield-transformer-function) + * [2.5.2.7 sonic-ext:table-transformer](#2527-sonic-exttable-transformer-function) + * [2.5.2.8 sonic-ext:subtree-transformer](#2528-sonic-extsubtree-transformer-function) + * [2.5.2.9 sonic-ext:db-name](#2529-sonic-extdb-name-name) + * [2.5.2.10 sonic-ext:post-transformer](#25210-sonic-extpost-transformer-function) + * [2.5.2.11 sonic-ext:get-validate](#25211-sonic-extget-validate-function) + * [2.5.2.12 sonic-ext:rpc-callback](#25212-sonic-extrpc-callback-function) + * [2.5.3 Guidelines for Choosing Annotation Type](#253-guidelines-for-choosing-annotation-type) + * [2.5.4 Manifest file](#254-manifest-file) + * [2.5.5 Callback functions](#255-callback-functions) + * [2.5.6 Usage Examples](#256-usage-examples) + * [2.6 App Module](#26-app-module) + * [2.7 Config Validation Library](#27-config-validation-library) + * [2.7.1 CVL Schema](#271-cvl-schema) + * [2.7.2 Custom Validation](#272-custom-validation) + * [2.7.3 Platform Validation](#273-platform-validation) + * [2.7.4 CVL Debug Logs](#274-cvl-debug-logs) + * [2.8 Non-DB Data](#28-non-db-data) + * [2.8.1 Host Modules](#281-host-modules) + * [2.8.2 FRR Integration](#282-frr-integration) + * [2.9 KLISH CLI](#29-klish-cli) + * [2.9.1 CLI components](#291-cli-components) + * [2.9.2 CLI development steps](#292-cli-development-steps) + * [2.9.3 Enhancements to Klish](#293-enhancements-to-klish) + * [2.9.4 Preprocess XML files](#294-preprocess-xml-files) + * [2.9.5 CLI directory structure](#295-cli-directory-structure) + * [2.9.6 Generic REST Client](#296-generic-rest-client-for-actioner-scripts) + * [2.9.7 Tool for JSON navigation in jinja templates](#297-tool-for-json-navigation-in-jinja-templates) + * [2.10 REST Server](#210-rest-server) + * [2.11 gNMI](#211-gnmi) + * [2.12 Compilation](#212-compilation) + * [2.12.1 sonic-mgmt-common](#2121-sonic-mgmt-common) + * [2.12.2 sonic-mgmt-framework](#2122-sonic-mgmt-framework) + * [2.12.2 sonic-telemetry](#2122-sonic-telemetry) + * [2.13 Unit Testing](#213-unit-testing) + * [2.13.1 REST API testing](#2131-rest-api-testing) + * [2.13.2 Local REST Server and CLI](#2132-local-rest-server-and-cli) + * [2.13.3 gNMI Unit Testing](#2133-gnmi-unit-testing) + * [2.13.3 Spytest](#2133-spytest) +* [3 Appendix](#3-appendix) + * [3.1 Code Generators](#31-code-generators) + * [3.2 Debugging](#32-debugging) + * [3.2.1 Debugging performance issues](#321-debugging-performance-issues) + +## List of Tables + +[Table 1: Abbreviations](#table-1-abbreviations) + +## Revision + +| Rev | Date | Author | Change Description | +|:---:|:-----------:|:-----------------------:|----------------------------------------------------------------------| +| 0.1 | 09/12/2019 | Anand Kumar Subramanian | Initial version | +| 0.2 | 09/16/2019 | Prabhu Sreenivasan | Added references, prerequisites and updated section 2.1.2 SONiC YANG | +| 0.3 | 09/16/2019 | Partha Dutta | Updated SONiC YANG Guideline link | +| 0.4 | 09/18/2019 | Anand Kumar Subramanian | Updated transformer section | +| 0.5 | 09/20/2019 | Mohammed Faraaz C | Updated REST Unit testing | +| 0.6 | 09/20/2019 | Prabhu Sreenivasan | Updated reference links and yang path | +| 0.7 | 09/23/2019 | Partha Dutta | Updated SONiC YANG, CVL, gNMI section | +| 0.8 | 06/18/2019 | Kwangsuk Kim | Updated CLI section | +| 0.9 | 06/22/2020 | Sachin Hola | Incorporated Phase-II changes, with new repo structure. | +| 1.0 | 10/09/2020 | Anand Kumar Subramanian | Debugging performance issues | + +## About this Manual + +This document provides developer guidelines for Management framework feature development in SONiC. + +## Scope + +This document describes the steps the feature developers need to follow to develop a CLI, REST and gNMI for a given feature using the Management framework. + +## Definition/Abbreviation + +### Table 1: Abbreviations + +| **Term** | **Meaning** | +|--------------------------|-------------------------------------| +| CVL | Config Validation Library | +| NBI | North Bound Interface | +| ABNF | Augmented Backus-Naur Form | +| YANG | Yet Another Next Generation | +| JSON | Java Script Object Notation | +| XML | eXtensible Markup Language | +| gNMI | gRPC Network Management Interface | +| YGOT | YANG Go Tools | +| SONiC YANG | YANG file which describes Redis DB schema for a given feature | +| RPC | Remote Procedure Call | + +## References + +| Document | Link | +|:--------:|:-------| +| SONiC YANG model guideline | https://github.com/Azure/SONiC/blob/master/doc/mgmt/SONiC_YANG_Model_Guidelines.md | +| SONiC Management Framework HLD | https://github.com/Azure/SONiC/pull/436 | +| RFC 7950 | August 2016 | https://tools.ietf.org/html/rfc7950 | + +## Prerequisite + +* Knowledge of YANG syntax +* GO Programming language +* SONiC Management Framework Architecture + +## 1 Architecture + +### 1.1 Overview + +![Management Framework Architecture diagram](https://github.com/Azure/SONiC/blob/34cac1aabdc865fc41cbe064a2ab2442645524b1/doc/mgmt/images/Mgmt_Frmk_Arch.jpg) + +In the above architecture diagram developer needs to write all the elements in green. Following are blocks that the developer has to write in order to get CLI, REST and gNMI support for the new feature being developed. + +1) YANG Models + +2) ABNF schema and SONiC YANG + +3) Transformer - includes YANG annotations and custom translation + +4) KLISH XML including Actioner and Renderer scripts + +### 1.2 Repositories + +Management framework and application code are distributed across 3 repositories. + +1) **sonic-mgmt-common** + + - Translib, CVL and transformer infrastructure code + - SONiC YANGs + - OpenConfig, IETF YANGs and their extension YANGs + - Trabsformer annotations and custom transformer functions + - CVL custom validation functions + - Host modules (application logic that runs on switch host) + +2) **sonic-mgmt-framework** + + - YANG to OpenAPI converter and OpenAPI codegen tools + - REST Server + - KLISH CLI core and SONiC specific patches + - CLI models (KLISH XML) + - CLI actioner scripts + - CLI renderer templates + +3) **sonic-telemetry** + + - Telemetry server + +All YANG data models and their translation logic goes into sonic-mgmt-common repository. +sonic-mgmt-framework adds REST and CLI interfaces for these YANG data models. +sonic-telemetry adds gNMI interface. + +### 1.3 Containers + +There will be 2 docker containers providing management services. + +1) **mgmt-framework** -- provides REST Server and CLI +2) **telemetry** -- Telemetry service + +## 2 Developer Workflow + +Following are the steps to be followed by the application developer. + +### 2.1 ABNF Schema + +First step is to define the redis DB schema for the application/feature. +Usual SONiC practice is to express the database schema in ABNF. +An existing DB table can be modified or new tables may be introduced. + +If you are defining new tables, try to keep it in line with the OpenConfig YANG model +for the feature (explained in next steps). Usually a redis table maps to YANG 'list' node. +Try to avoid mapping one redis table to multiple YANG lists. +This will make translation logic complex and requires custom translation code. + +### 2.2 SONiC YANG + +SONiC YANGs express the redis db schema in YANG format. +These YANGs should strictly adhere to the [SONiC YANG Model Guidelines](https://github.com/Azure/SONiC/blob/master/doc/mgmt/SONiC_YANG_Model_Guidelines.md) published by SONiC Community. + +SONiC YANG models **MUST** be defined for every DB schema consumed by the management applications. +CVL, transformer and other framework components use these YANGs to understand the DB schema. + +SONiC YANGs are maintained in `sonic-mgmt-common/models/yang/sonic` directory. + +### 2.3 OpenConfig YANG + +Every feature should support corresponding OpenConfig YANG model. +An OpenConfig compliant YANG model should be defined if there was no standard OpenConfig YANG. +Extension YANGs should be written to add additional data nodes over standard YANG and +to remove unsupported nodes. + +Few existing features implemented IETF YANGs. However going forward only OpenConfig YANGs +and OpenConfig compliant extension YANGs should be used. + +#### 2.3.1 Directory structure + +All YANGs are maintained under `sonic-mgmt-common/models/yang` directory. + +``` +models +└── yang : Standard OpenConfig and IETF YANGs + ├── annotations : Transformer annotations (section 2.5.2) + ├── common : Dependencies for standard YANGs + ├── extensions : Extensions YANGs + ├── sonic : SONiC YANGs (section 2.2) + ├── testdata : Test YANGs - ignored + └── version.xml : YANG bundle version configuration file +``` + +Unmodified OpenConfig and IETF YANGs should be kept in `yang` directory. +Framework will automatically enable REST and gNMI interfaces for these YANGs. + +These YANGs refer few common YANG models, like ietf-interfaces.yang, for which we dont want +north bound interfaces. Such dependent YANGs must be kept in `yang/common` directory. +Framework will use these YANGs only to resolve compilation dependencies but not enable +REST and gNMI interaces for them. + +All custom YANGs should be kept in `yang/extensions` directory. + +#### 2.3.2 Extension YANGs + +**Standard OpenConfig and IETF YANGs must not be modified directly**. +All customizations must be applied through extension YANGs. +Even new OpenConfig compliant YANGs (when there is no standard YANG) will also +be treated as an extension to OpenConfig. + +Extension YANG should use YANG [augment](https://tools.ietf.org/html/rfc6020#section-4.2.8) +statements to add new data nodes to standard OpenConfig or IETF YANG data tree. + +YANG [deviation](https://tools.ietf.org/html/rfc6020#section-7.18.3) statement should +be used to mark an existing data node as "not-supported" or to modify it. + +Example: + +```yang +module openconfig-acl-ext { + namespace "http://github.com/Azure/oc-acl-ext"; + prefix oc-acl-ext; + + import openconfig-acl { prefix oc-acl; } + + /* Add an alt-name config to ACL */ + augment "/oc-acl:acl/acl-sets/acl-set/config" { + leaf alt-name { + type string; + } + } + + /* Mark ACL description not supported */ + deviation "/oc-acl:acl/acl-sets/acl-set/config/description" { + deviate not-supported; + } + + /* Restrict ACL name to max 127 characters */ + deviation "/oc-acl:acl/acl-sets/acl-set/config/name" { + deviate replace { + type string { + length "1..127"; + } + } + } +} +``` + +#### 2.3.3 Extension YANG Style Guide + +OpenConfig extension YANGs should adhere to [OpenConfig style guidelines](https://github.com/openconfig/public/blob/master/doc/openconfig_style_guide.md). + +Please review [current model hierarchy](http://www.openconfig.net/projects/models) +and IETF draft [draft-openconfig-netmod-opstate-01](https://tools.ietf.org/html/draft-openconfig-netmod-opstate-01) +on overall OpenConfig modeling approach. + +General guidelines for extension YANGs: + +- All extension YANGs should be named as **openconfig-{function}-ext.yang**. + +- **namespace** should be "http://openconfig.net/yang/{function}-ext" or "http://openconfig.net/yang/{function}/extension" + +- Namespace **prefix** should be "oc-{function}-ext". + +- Specify **organization**, **contact** and **oc-ext:origin** as "SONiC". + +- Add module **description** statement with new features covered in the extension yang; +like [this](https://github.com/openconfig/public/blob/master/release/models/policy/openconfig-routing-policy.yang#L24) + +- Include **revision** and **oc-ext:openconfig-version** statements. + - oc-ext:openconfig-version should be "0.1.0" for the initial version (when a new extension YANG is defined) + - Update the oc-ext:openconfig-version and add a revision statement for every enhancement to the mode. + An enhancement may be split into multiple commits within a release. + But all such related commits (within a release) should be grouped as one version change. + - Version updates should follow [sematic versioning](https://semver.org/#semantic-versioning-specification-semver) + guidelines. More details are at [section 2.3.5](#235-versioning) + - Revision statements should be reverse chronological order (latest should be at the top). + +- Do not use tab characters. Always expand tabs into 4 spaces. + +- Try to maintain 72 column width through-out the file (like RFC document) + +Naming conventions: + +- Use **lower case with dash separated words** for all YANG node names (container, list, leaf-list, leaf, grouping etc). + Example: "ip-address" + + Do not use upper case, underscores, camel case; like "IP_Address" or "ipaddress" or "ipAddress". + +- Use **upper case with underscore separated words** for enum and identity names. +Example: "IF_ETHERNET". + +- Try to avoid cryptic and abbreviated words, unless they are widely used feature names. + + Example: "arp", "mpls", "fbs" are okay. But avoid names like "addr", "ip-addr", "neigh". + +- Grouping names should make it easy to quickly understand the nature of the data within. +A suggested convention is ***xxx-yyy[-config|state|top]***, where xxx is the top-level module name +(without the openconfig prefix), yyy is a string which indicates the contents of the groupings. + + Example: "interface-common-state", "subinterfaces-config". + +- Use standard typedefs as much as possible. +Most of the common data types are already defined in one of these YANGs. + - openconfig-types.yang + - openconfig-inet-types.yang + - openconfig-yang-types.yang + - ietf-inet-types.yang + - ietf-yang-types.yang + + Also look for openconfig-{feature}-types.yang for the feature specific typedefs. + +Data tree guidelines: + +- TODO + +#### 2.3.4 Backward Compatibility + +All extensions to standard OpenConfig/IETF YANGs or incremental changes to the extension +YANGs can affect REST and gNMI clients. Strict adherence to +[YANG module update guidelines](https://tools.ietf.org/html/rfc6020#section-10) is a must. + +#### 2.3.5 Versioning + +Individual YANG models should be versioned through **revision** and **oc-ext:openconfig-version** statements. +In addition, a YANG bundle version (collective version number for all supported YANG modules) +is managed via `models/yang/version.xml` file. +YANG bundle version is used for API versioning feature (in future release). + +YANG bundle version uses sematic versioning format -- ie, **{major}.{minor}.{patch}** format. +It should be updated for every YANG update or once in a release. + +**Major version** should be incremented if YANG model is changed in a non backward compatible manner. +Such changes should be avoided. + +- Delete, rename or relocate data node +- Change list key attributes +- Change data type of a node to an incompatible type +- Change leafref target + +**Minor version** should be incremented if the YANG change modifies the API in a backward +compatible way. Patch version should be reset to 0. +Candidate YANG changes for this category are: + +- Add new YANG module +- Add new YANG data nodes +- Mark a YANG data node as deprecated +- Change data type of a node to a compatible type +- Add new enum or identity + +**Patch version** should incremented for cosmetic fixes that do not change YANG API. +Candidate YANG changes for this category are: + +- Change description, beautification. +- Expand pattern or range of a node to wider set. +- Change must expression to accept more cases. +- Error message or error tag changes. + +#### 2.3.6 Tools + +Framework provides multiple tools and code generators to work with the YANGs. +Details of each code generators is in [section 3.1](#31-code-generators). +All the tools are automatically run during sonic-mgmt-common build. +They can also be triggered manually to validate YANG changes. + +```bash +cd /sonic/sonic-buildimage/src/sonic-mgmt-common +make models +``` + +It validates all the YANGs under `yang` directory using [pyang tool](http://www.yang-central.org/twiki/pub/Main/YangTools/pyang.1.html) and generates human readable reports. + +##### 2.3.6.1 YANG Text Tree + +This is the standard pyang text tree output generated through "-f tree" option. +Two files are generated: + +1) `sonic-mgmt-common/models/yang/allyangs.tree` -- +Combined tree for the OpenConfig and IETF YANGs and their extension YANGs + +2) `sonic-mgmt-common/models/yang/sonic/allyangs.tree` -- +Combined tree for all SONiC YANGs + +Sample: + +```text +module: openconfig-lldp + +--rw lldp + +--rw config + | +--rw enabled? boolean + | +--rw hello-timer? uint64 + | +--rw suppress-tlv-advertisement* identityref + | +--rw system-name? string + | +--rw system-description? string + | +--rw oc-lldp-ext:multiplier? uint8 + | +--rw oc-lldp-ext:mode? lldp-ext-mode-type +``` + +##### 2.3.6.2 YANG HTML Tree + +This is also a standard pyang tree output generated through "-f jstree" option. +It is equivalent to the text tree but provides expand/collapse controls at each node. + +Two files are generated: +1) `sonic-mgmt-common/models/yang/allyangs_tree.html` +2) `sonic-mgmt-common/models/yang/sonic/allyangs_tree.html` + +##### 2.3.6.3 YANG Markdown Tree + +Markdown tree is similar to text tree but contains markdown annotations to highlight +the nodes added and removed though extension YANGs. +Newly added nodes (through augment statements) are highlighted in green colour. +Removed nodes (through "not-supported" deviation) are highlighted in red color. +This is equivalent of diff between standard YANG data tree and the final YANG tree +after applying all extension YANGs. + +Markdown tree file path: `sonic-mgmt-common/models/yang/yang_tree.md` + +Sample: + +```diff + module: openconfig-lldp + +--rw lldp + +--rw config + | +--rw enabled? boolean + | +--rw hello-timer? uint64 + | +--rw suppress-tlv-advertisement* identityref + | +--rw system-name? string + | +--rw system-description? string ++ | +--rw oc-lldp-ext:multiplier? uint8 ++ | +--rw oc-lldp-ext:mode? lldp-ext-mode-type +- | +--rw chassis-id? string +- | +--rw chassis-id-type? oc-lldp-types:chassis-id-type +``` + +##### 2.3.6.3 OpenConfig Style Checker + +TODO + + +### 2.4 Config Translation App + +Config Translation App translates data from northbound API schema to native +[ABNF schema](#21-abnf-schema) and vice versa. +Northbound API schema are defined by OpenConfig or IETF YANGs and ther extensions. + +#### 2.4.1 Translation Approaches + +There are two approaches to develop the translation app. + +1. [Transformer](#25-transformer) +2. [App Module](#26-app-module) + +Transformer is the preferred approach for develpoing the translation app. +It reduces the coding and maintenance effort by hiding the most of the YANG +hierarchy tranversal in the framework itself. +New features can be introduced into Translib without affecting existing translation apps. + +An App Module would receive the raw inputs passed by the clients and shloud code for every possible scenario. + +Type of translation app can be selected per YANG module. +New additions to an existing YANG module should use the current translation approach used by that YANG. +It is not possible to support part of the YANG tree through App Modules and part through Transformer. + +All new YANG modules should use transformer. + +#### 2.4.2 YGOT + +TODO + +#### 2.4.3 DB Access APIs + +The DB access layer implements a wrapper over the go-redis package enhancing the functionality as described in: +[Management Framework HLD, DB access layer](https://github.com/Azure/SONiC/blob/master/doc/mgmt/Management%20Framework.md#32264-db-access-layer) + +#### 2.4.4 Logging + +Translation apps should use Go "github.com/golang/glog" library for logging. + +Guidelines: +- Use `glog.Infof()` for logging info messges. Info is enabled by default. +- Use `glog.Errorf()` for logging error messages. +- Use `glog.V(1).Infof()` for debug messages. +Typically, payloads or DB data should be logged this way. +This style of logging is called verbose logging and they are not enabled by default. +Program should be started with a command line argument "-v" to enable verbosity levels. +- Use `glog.V(2).Infof()` for more verbose developer traces. +- Avoid using higer verbosity levels, fatal logs, `fmt.Printf()` and other logging libraries. + +Examples: + +```go +import ( + "github.com/golang/glog" +) + +// INFO log +glog.Infof("Processing ACL %s, %v", aclName, aclType) + +// ERROR log +glog.Errorf("GetEntry returned error %v", err) + +// DEBUG log +glog.V(1).Infof("Payload received = %v", data) + +// DEBUG log which requires lot of formatting. +if glog.V(1) { + s := doNiceFormatting(bigdata) + glog.Infof("data = %s", s) +} +``` + +Log verbosity level for REST and gNMI servers can be changed via CONFIG_DB entries. +Details are in [Management Framework HLD](https://github.com/project-arlo/SONiC/blob/master/doc/mgmt/Management%20Framework.md#322414-db-schema). + +#### 2.4.5 Error Response + +Translation apps should return Go error objects defined in +`sonic-mgmt-common/translib/tlerr/app_errors.go` file to report errors. +Translib infrastructure stops further processing of the request and returns the error +back to the service (REST or gNMI) which invoked the Translib APIs. +Protocol specific error response will be prepared by individual service. + +Application error types: + +| Error object | Error condition | Constructor function | +|--------------------|--------------------------------------|---------------------------| +| InvalidArgsError | Bad request, constraint error | `tlerr.InvalidArgs()` | +| NotFoundError | Target resource not found | `tlerr.NotFound()` | +| AlreadyExistsError | Dupicate create | `tlerr.AlreadyExists()` | +| NotSupportedError | Unspported operation | `tlerr.NotSupported()` | +| InternalError | Generic error during app execution | `tlerr.New()` | +| AuthorizationError | User is not authorized for the operation | | + +Exampels: + +```go +import ( + "github.com/Azure/sonic-mgmt-common/translib/tlerr" +) + +// InvalidArgsError +return tlerr.InvalidArgs("Priority value %d is out of range[1-65535]", priority) + +// NotFoundError +return tlerr.NotFound("ACL %s/%v not found", aclName, aclType) + +// AlreadyExistsError +return tlerr.AlreadyExists("Duplicate ACL %s/%v", aclName, aclType) + +// NotSupportedError +return tlerr.NotSupported("Cannot delete Ethernet interface") + +// NotSupportedError +return tlerr.New("FRR returned %d", status) +``` + +Avoid returning other error objects created through `errors.New()` or `fmt.Errorf()`. +Translib will propagate such errors but REST/gNMI service cannot map them to correct protocol error codes. + +Error types defined in `sonic-mgmt-common/translib/tlerr/tlerr.go` are reserved for Translib's internal use. + +#### 2.4.6 D-Bus messaging + +TODO + +### 2.5 Transformer + +Transformer provides a generic infrastructure for Translib to programmatically translate YANG to ABNF/Redis schema and vice versa, using YANG extensions to define translation hints along the YANG paths. At run time, the translation hints are mapped to an in-memory Transformer Spec that provides two-way mapping between YANG and ABNF/Redis schema for Transformer to perform data translation. + +In case that SONiC YANG modules are used by NBI applications, the Transformer performs 1:1 mapping between a YANG object and a SONiC DB object without a need to write special translation codes or any translation hints. + +If you use the openconfig YANGs for NBI applications, you need special handling to translate the data between YANG and ABNF schema. In such case, you have to annotate YANG extensions and write callbacks to perform translations where required. + +In either case, the default application [common-app.go](https://github.com/project-arlo/sonic-mgmt-framework/blob/transformer-phase1/src/translib/common_app.go) generically handles both set and get requests with the returned data from Transformer. + +Basic worklflow for developing a Translation App using transformer: + +1) Create an annotation file template (for new YANGs) +2) Fill transformer annotations for YANG paths in the annotation file. +3) Register YANG and annotation file names in the manifest file (for new YANGs) +4) Implement the transformer callback functions if any. + +#### 2.5.1 Annotation File + +Annotation file contains translation hints in YANG format. +It uses YANG deviation statements to attach transformer annotation to specific YANG paths. +However these deviations do not affect the data model. +It only attaches additional metdata to YANG paths for Transformer's use. + +All annotation files should be kept in `sonic-mgmt-common/models/yang/annotations` directory. + +A template annotation file can be generated using `sonic-mgmt-common/tools/xfmr/annotate.sh` script. +It takes one or more YANG file names and writes the annotation template file contents to stdout. +Output can be redirected to specific annotation file under `yang/annotations` directory. +Template file contains empty "deviate add" statements for all YANG paths from the specified YANG files. + +```bash +$ cd /sonic/sonic-buildimage/src/sonic-mgmt-common +$ tools/xfmr/annotate.sh +usage: tools/xfmr/annotate.sh YANG_FILE_NAME... + +$ tools/xfmr/annotate.sh openconfig-lldp.yang +module openconfig-lldp-annot { + + yang-version "1"; + + namespace "http://openconfig.net/yang/annotation/oc-lldp-annot"; + prefix "oc-lldp-annot"; + + import openconfig-extensions { prefix oc-ext; } + import openconfig-lldp-types { prefix oc-lldp-types; } + import openconfig-interfaces { prefix oc-if; } + import ietf-yang-types { prefix yang; } + import openconfig-lldp { prefix oc-lldp; } + + deviation /oc-lldp:lldp { + deviate add { + } + } + + deviation /oc-lldp:lldp/oc-lldp:state { + deviate add { + } + } + + deviation /oc-lldp:lldp/oc-lldp:state/oc-lldp:system-description { + deviate add { + } + } + ..... +``` + +#### 2.5.2 Annotate YANG Paths + +Transformer provides custom YANG statements to specify translation hints. +They are defined in `sonic-mgmt-common/models/yang/annotations/sonic-extensions.yang` file. +These custom YANG statements should be used to annotate specific paths in the annotation file. + +Example: + +```yang +import sonic-extensions { prefix sonic-ext; } + +deviation /oc-lldp:lldp/oc-lldp:config { + deviate add { + sonic-ext:table-name "LLDP"; + sonic-ext:key-transformer "lldp_global_key_xfmr"; + } +} + +deviation /oc-lldp:lldp/oc-lldp:config/oc-lldp:hello-timer { + deviate add { + sonic-ext:field-name "hello_time"; + } +} +``` + +Following annotation statements are supported. + +##### 2.5.2.1 sonic-ext:table-name {TABLE_NAME} + +Maps a YANG container or list to redis table name. + +The table-name is **inherited** to all descendant nodes unless another one is defined. + +##### 2.5.2.2 sonic-ext:field-name {FIELD_NAME} + +Maps a YANG leaf or leaf-list node to redis hash field name. + +##### 2.5.2.3 sonic-ext:key-delimiter {DELIM} + +Override the default delimiter "|" with a custom delimiter. +This is used to derive the redis table key by concatenating the table name and key components. + +##### 2.5.2.4 sonic-ext:key-name {VALUE} + +Specify the fixed redis key value for singleton tables. +Useful when top level YANG container is mapped to a redis table. + +Example: below annotation results is redis key "UDLD|GLOBAL". + +``` +deviate add { + sonic-ext:table-name "UDLD"; + sonic-ext:key-name "GLOBAL"; +} +``` + +##### 2.5.2.5 sonic-ext:key-transformer {FUNCTION} + +Overloads the default method to generate DB key(s). +Used when the key values in a YANG list are different from ones in DB table. + +A pair of Go callbacks should be implemented to support 2 way translation - YangToDB_xxx, DbToYang_xxx. +Function signatures: + +```go +/** + * KeyXfmrYangToDb type is defined to use for conversion of Yang key to DB Key + * + * Param: XfmrParams structure having Database info, YgotRoot, operation, Xpath + * Return: Database keys to access db entry, error + **/ +type KeyXfmrYangToDb func (inParams XfmrParams) (string, error) + +/** + * KeyXfmrDbToYang type is defined to use for conversion of DB key to Yang key + * + * Param: XfmrParams structure having Database info, operation, Database keys to access db entry + * Return: multi dimensional map to hold the yang key attributes of complete xpath, error + **/ +type KeyXfmrDbToYang func (inParams XfmrParams) (map[string]interface{}, error) +``` + +##### 2.5.2.6 sonic-ext:field-transformer {FUNCTION} + +Overloads default method for generating DB field value. +Used when the leaf/leaf-list values defined in a YANG list are different from the field values in DB. + +A pair of callbacks should be implemented to support 2 way translation - YangToDB_xxx, DbToYang_xxx. +Function signatures: + +```go +/** + * FieldXfmrYangToDb type is defined to use for conversion of yang Field to DB field + * + * Param: Database info, YgotRoot, operation, Xpath + * Return: multi dimensional map to hold the DB data, error + **/ +type FieldXfmrYangToDb func (inParams XfmrParams) (map[string]string, error) + +/** + * FieldXfmrDbtoYang type is defined to use for conversion of DB field to Yang field + * + * Param: XfmrParams structure having Database info, operation, + * DB data in multidimensional map, output param YgotRoot + * Return: error + **/ +type FieldXfmrDbtoYang func (inParams XfmrParams) (map[string]interface{}, error) +``` + +##### 2.5.2.7 sonic-ext:table-transformer {FUNCTION} + +Map a YANG container/list to TABLE name(s). +Used to dynamically map a YANG list/container to table names based on URI and payload. + +The table-transformer is **inherited** to all descendant nodes until another one is defined +or table-name “NONE” is defined. + +Table transformer function signature: + +```go +/** + * TableXfmrFunc type is defined to use for table transformer function for + * dynamic derviation of redis table. + * + * Param: XfmrParams structure having database pointers, current db, operation, + * DB data in multidimensional map, YgotRoot, uri + * Return: List of table names, error + **/ +type TableXfmrFunc func (inParams XfmrParams) ([]string, error) +``` + +##### 2.5.2.8 sonic-ext:subtree-transformer {FUNCTION} + +Overloading default translation method for the current subtree. +Allows the callback function to take **full control of translation**. +Note that, if any other extensions are annotated to the nodes on the subtree, they are not effective. + +The subtree-transformer is **inherited** to all descendant nodes unless another one is defined. +i.e. the scope of subtree-transformer callback is limited to the current and descendant nodes +along the YANG path until a new subtree transformer is annotated. + +A pair of callbacks should be implemented to support 2 way translation - YangToDB_xxx, DbToYang_xxx. +Function signatures: + +```go +/** + * SubTreeXfmrYangToDb type is defined to use for handling the yang subtree to DB + * + * Param: XfmrParams structure having Database info, YgotRoot, operation, Xpath + * Return: multi dimensional map to hold the DB data, error + **/ +type SubTreeXfmrYangToDb func (inParams XfmrParams) (map[string]map[string]db.Value, error) + +/** + * SubTreeXfmrDbToYang type is defined to use for handling the DB to Yang subtree + * + * Param : XfmrParams structure having Database pointers, current db, operation, + * DB data in multidimensional map, output param YgotRoot, uri + * Return : error + **/ +type SubTreeXfmrDbToYang func (inParams XfmrParams) (error) +``` + +##### 2.5.2.9 sonic-ext:db-name {NAME} + +Specify DB name to access data from. +Valid values are: +- CONFIG_DB (default value) +- APPL_DB +- ASIC_DB +- COUNTERS_DB +- FLEX_COUNTER_DB +- STATE_DB + +Used for GET operation to non CONFIG_DB, applicable only to SONiC YANG + +The db-name is **inherited** to all descendant nodes unless another one. +It must be defined with the **table-name** annotation. + +##### 2.5.2.10 sonic-ext:post-transformer {FUNCTION} + +A special hook to update the mapped DB data after all translation is done, but before writing to DB. +Analogous to the postponed YangToDB subtree callback that is invoked at the very end by the Transformer. + +This will be useful to add or update additional data to to be written to DB. +Example: adding the default ACL rule during ACL creation. + +Post-transformer can be annotated only to the top-level container(s) within each YANG module. + +Port-transformer callback function signature: + +```go +/** + * PostXfmrFunc type is defined to use for handling any default handling operations + * required as part of the CREATE. + * + * Param: XfmrParams structure having database pointers, current db, operation, + * DB data in multidimensional map, YgotRoot, uri + * Return: Multi dimensional map to hold the DB data Map (tblName, key and Fields), error + **/ +type PostXfmrFunc func (inParams XfmrParams) (map[string]map[string]db.Value, error) +``` + +##### 2.5.2.11 sonic-ext:get-validate {FUNCTION} + +A special hook to validate YANG nodes, to populate data read from database. + +Allows developers to instruct the Transformer to choose a YANG node among multiple nodes +while constructing the response payload. +Typically used to check the "when" condition. + +Callback function signature: + +```go +/** + * ValidateCallpoint is used to validate a YANG node during data translation + * back to YANG as a response to GET. + * + * Param : XfmrParams structure having Database pointers, current db, operation, + * DB data in multidimensional map, output param YgotRoot, uri + * Return : bool + **/ +type ValidateCallpoint func (inParams XfmrParams) (bool) +``` + +##### 2.5.2.12 sonic-ext:rpc-callback {FUNCTION} + +Handler function for YANG custom RPCs and actions. +Transformer does not process input or output payloads. + +Function signature: + +```go +/** + * RpcCallpoint is used to invoke a callback for action + * + * Param : []byte input payload, dbi indices + * Return : []byte output payload, error + **/ +type RpcCallpoint func (body []byte, dbs [db.MaxDB]*db.DB) ([]byte, error) +``` + +#### 2.5.3 Guidelines for Choosing Annotation Type + +1) If the translation is simple mapping between YANG container/list and TABLE, consider using the extensions - table-name, field-name, optionally key-delimiter + +2) If the translation requires a complex translation with your codes, consider the following transformer extensions - key-transformer, field-transformer, subtree-transformer to take a control during translation. Note that multiple subtree-transformers can be annotated along YANG path to divide the scope + +3) If multiple tables are mapped to a YANG list, e.g. openconfig-interface.yang, use the table-transformer to dynamically choose tables based on URI/payload + +4) In Get operation access to non CONFIG_DB, you can use the db-name extension + +5) In Get operation, you can annotate the subtree-transformer on the node to implement your own data access and translation with DbToYang_xxx function + +6) In case of mapping a container to TABLE/KEY you can use the key-name/key-transfomer along with the table-name/table-transformer extension + +#### 2.5.4 Manifest file + +Manifest file `sonic-mgmt-common/config/transformer/models_list` contains all the YANG +file names that require transformer based translation. + +Each line is treated as one file name; lines stating with `#` are ignored. +Add all the standard OpenConfig, IETF YANG names along with their extension YANGs and annotation file here. +SONiC YANG file names should not be added here. + +Example: + +``` +openconfig-lldp.yang +openconfig-lldp-ext.yang +openconfig-lldp-annot.yang +openconfig-interfaces.yang +openconfig-if-ip.yang +openconfig-if-ethernet.yang +openconfig-if-ethernet-ext.yang +openconfig-if-ethernet-ext2.yang +openconfig-interfaces-annot.yang +``` + +#### 2.5.5 Callback functions + +Transformer callback functions should be implemented in Go source files in +`sonic-mgmt-common/translib/transformer` directory. +It is suggested to have feature and sub-feature specific source files for easy browsing. + +Go function for each of the callback names (defined of annotation file) should be registered +throigh `XlateFuncBind` API call during transformer module initialization. +This can be done in the `init()` function of every Go source file where the callback functions are implemented. + +Example: + +```go +func init () { + XlateFuncBind("intf_table_xfmr", intf_table_xfmr) + XlateFuncBind("YangToDb_intf_name_xfmr", YangToDb_intf_name_xfmr) + XlateFuncBind("DbToYang_intf_name_xfmr", DbToYang_intf_name_xfmr) +} +``` + +Note that [key-transformer](#2525-sonic-extkey-transformer-function), +[field-transformer](#2526-sonic-extfield-transformer-function) and +[subtree-transformer](#2528-sonic-extsubtree-transformer-function) +requires a pair of callbacks - with `YangToDb_` and `DbToYang_` prefix added to +the function name specified in annotation file. + +Only one callback is required for +[table-transformer](#2527-sonic-exttable-transformer-function), +[post-transformer](#25210-sonic-extpost-transformer-function), +[get-validate](#25211-sonic-extget-validate-function) and +[rpc-callback](#25212-sonic-extrpc-callback-function). + +All the callback functions, except rpc-callback, receive context information +through a `XfmrParams` object. +The rpc-callback callback function receives raw bytes sent by the client. + +```go +// RedisDbMap is map[DB_NUM]map[TABLE]map[KEY]db.Value +type RedisDbMap = map[db.DBNum]map[string]map[string]db.Value + +// XfmrParams contains context, data passed to the callback functions. +type XfmrParams struct { + d *db.DB + dbs [db.MaxDB]*db.DB + curDb db.DBNum + + // requestUri holds the resource URI specified in client app. + // It can be pointing to parent or child node of the node at which + // this callback is registered. -- (RO) + requestUri string + + // uri contains path to the node where a callback gets invoked. + // Functions should use this value to identify target resource. -- (RO) + uri string + + // ygRoot is the ygot struct initialized by translib for the requestUri. -- (RW) + ygRoot *ygot.GoStruct + + // oper indicates the current operation requested by translib. -- (RO) + // 1=GET, 2=CREATE, 3=REPLACE, 4=UPDATE, 5=DELETE + oper int + + // key contains the DB key of current node. -- (RO) + key string + + // dbDataMap is the internal map referenced by callback. -- (RO) + dbDataMap *map[db.DBNum]map[string]map[string]db.Value + + // subOpDataMap is a map[OPCODE]map[DB_NUM]map[TABLE]map[KEY]db.Value -- (RW) + // It should be filled by YangToDb_xxx callbacks to record sub-operations. + subOpDataMap map[int]*RedisDbMap + + // param is the ygot GoStruct initialized for payload -- (RO) + param interface{} + + // txCache is a cache to pass data/state between callbacks. -- (RW) + // Scope of this limited to one translib transaction. + txCache *sync.Map + + skipOrdTblChk *bool +} +``` + +These structures and callback function templates are defined in +`sonic-mgmt-common/translib/transformer/xfmr_interface.go` file. + +#### 2.5.6 Usage Examples + +TBD + +### 2.6 App Module + +Instead of using the transformer, developers can write the complete App module that aids in the conversion of the incoming request to the SONiC ABNF format and vice versa. Following are the steps to be performed. + +1. Define a structure to hold all the incoming information as well as the translation information + +Example: + + type AclApp struct { + pathInfo *PathInfo + ygotRoot *ygot.GoStruct + ygotTarget *interface{} + + aclTs *db.TableSpec + ruleTs *db.TableSpec + + aclTableMap map[string]db.Value + ruleTableMap map[string]map[string]db.Value + } + +2. App modules will implement an init function which registers itself with the translib. In addition it should also add the YANG models it supports here to serve the capabilities API of gNMI. + Example: + func init () { + log.Info("Init called for ACL module") + err := register("/openconfig-acl:acl", + &appInfo{appType: reflect.TypeOf(AclApp{}), + ygotRootType: reflect.TypeOf(ocbinds.OpenconfigAcl_Acl{}), + isNative: false, + tablesToWatch: []*db.TableSpec{&db.TableSpec{Name: ACL_TABLE}, &db.TableSpec{Name: RULE_TABLE}}}) + + if err != nil { + log.Fatal("Register ACL App module with App Interface failed with error=", err) + } + + err = appinterface.AddModel(&gnmi.ModelData{Name:"openconfig-acl", + Organization:"OpenConfig working group", + Version:"1.0.2"}) + if err != nil { + log.Fatal("Adding model data to appinterface failed with error=", err) + } + } + +App Modules will implement the following interface functions that will enable conversion from YGOT struct to ABNF format. + + initialize(data appData) + translateCreate(d *db.DB) ([]db.WatchKeys, error) + translateUpdate(d *db.DB) ([]db.WatchKeys, error) + translateReplace(d *db.DB) ([]db.WatchKeys, error) + translateDelete(d *db.DB) ([]db.WatchKeys, error) + translateGet(dbs [db.MaxDB]*db.DB) error + translateSubscribe(dbs [db.MaxDB]*db.DB, path string) (*notificationOpts, *notificationInfo, error) + processCreate(d *db.DB) (SetResponse, error) + processUpdate(d *db.DB) (SetResponse, error) + processReplace(d *db.DB) (SetResponse, error) + processDelete(d *db.DB) (SetResponse, error) + processGet(dbs [db.MaxDB]*db.DB) (GetResponse, error) + +initialize – Populate the app structure that we created in step 1 with the incoming appData which contains path, payload, YGOT root structure, YGOT target structure. + +translate(CRUD) – Convert the information in the YGOT root and target structure to the corresponding ABNF key value pair and store it in the structure created in step 1 + +translateGet – Convert the incoming path to corresponding ABNF keys to be got from the redis DB using the DB access layer APIs similar to the python SWSSSDK. + +translateSubscribe – Convert the incoming path in the argument to corresponding ABNF keys as part of the notificationInfo structure. In addition to this subscribe parameters like type of subscription supported, SAMPLE interval can also be specified as part of the notificationOpts structure. + +process(CRUD) – Write the converted data from translate(CRUD) function into the Redis DB using the DB access layer APIs + + +### 2.7 Config Validation Library + +Config Validation Library (CVL) performs the automatic syntactic and semantic validation +of DB writes based on the YANG constraints defined in SONiC YANG. +These constraints include data type, ranges, patterns, "leafref", "must" and "when" statements. +Constraints written for read-only state nodes (non CONFIG_DB tables), RPC and notification nodes are ignored. +Complex validations that cannot be expressed via YANG constraints can be implemented via +custom validation functions written in Go. + +All DB writes performed via Translib DB Acceess APIs would automatically trigger CVL validations. + +#### 2.7.1 CVL Schema + +CVL uses YIN schema derived from SONiC YANGs for validation. +CVL schema is automatically generated during sonic-mgmt-common build. +Any changes to SONiC YANG requires sonic-mgmt-common compilation for it to reflect in CVL. +Details of the schema generator are in [Appendix 3.1.3](#313-cvl-schema-generator). + +#### 2.7.2 Custom Validation + +When to use custom validation? +* Complext validations, which can't be defined using YANG must/when expression. +* YANG based must/when consraints are taking lot of time + +Custom validation functions can be registered to a DB table or field definition in SONiC YANGs. +CVL provides a YANG extension statement **sonic-ext:custom-validation** for this. + +```yang +import sonic-extension { prefix sonic-ext; } + +.... +list ACL_RULE_LIST { + leaf IP_TYPE { + sonic-ext:custom-validation ValidateAclRuleIPAddress; + ... + } +.... +``` + +A Go function with same name should be defined in a Go source file under `sonic-mgmt-common/cvl/custom_validation` directory. +Use feature specific source file for better code organization - like "custom_validation_{feature}.go". +Function name should be unique across all SONiC YANGs. +So it is suggested to include table name and field name in the function name. + +Signature for custom validation handler function: + +```go +func (t *CustomValidation) (vc *CustValidationCtxt) CVLErrorInfo + +// CustValidationCtxt is the validation context passed to custom validation function +type CustValidationCtxt struct { + ReqData []CVLEditConfigData //All request data + CurCfg *CVLEditConfigData //Current request data for which validation should be done + YNodeName string //YANG node name + YNodeVal string //YANG node value, leaf-list will have "," separated value + YCur *xmlquery.Node //YANG data tree + SessCache *CustValidationCache //Session cache, can be used for storing data, persistent in session + RClient *redis.Client //Redis client +} +``` + +#### 2.7.3 Platform Validation + +CVL allows platform specific validation in following ways: + +* Add YANG 'deviation' files (e.g. sonic-acl-deviation.yang for ACL YANG) per platform and place it in 'models/yang/sonic/platform' folder for static platform validation. These files are automatically picked up during build time, processed and applied at run time. + +``` +models/yang/sonic/platform/ +├── accton_as5712 +│ └── sonic-acl-deviation.yang +└── quanta_ix8 + └── sonic-acl-deviation.yang +``` + +Note : All platform specific deviation files are packaged in a single image file. At runtime, based on detected platform from provisioned “DEVICE_METADATA:platform” field, deviation files are applied. + +* Implement platform specific custom validation through CVL custom validation support for dynamic platform validation. + +#### 2.7.4 CVL Debug Logs + +For enabling CVL debug log please refer to "Debugging Info" section in `sonic-mgmt-common/cvl/README.md`. + + +### 2.8 Non-DB Data + +#### 2.8.1 Host Modules + +TODO + +#### 2.8.2 FRR Integration + +TODO + + +### 2.9 KLISH CLI + +Open source Klish is integrated to sonic-mgmt-framework to provide the command line interface tool to perform network operations more efficiently in SONiC. Klish will provide the core functionality of command parsing, syntax validation, command help and command auto-completion. + +Open Source [klish](http://libcode.org/projects/klish/.) is used here. + +#### 2.9.1 CLI components + +1. CLI Parser engine: Open source Klish + +2. XML files: +XML files, defined by developer, that describe the CLI command structure. +Klish uses XML based command tree inputs to build the parser command tree. +All CLI commands to be supported are specified in XML format in module specific XML file. +XML files can be defined with macros and entity references, preprocessed by scripts to generate the expanded XML files. + +3. Actioner: Script that will transform CLI commands to form the corresponding REST requests and invoke the REST client API. + +4. Renderer: Script that will receive the JSON response from REST client API and use the jinja2 template file to render(display) the CLI output in the desired format. + +#### 2.9.2 CLI development steps + +Following are the steps to add a new CLI command. Please refer to https://github.com/sipwise/klish/blob/master/doc/klish.md for detail. + +1. Create a CLI XML .xml file that defines the CLI command structure. This file defines the following + * CLI command format + * Parameters that the command requires + * Help string to be displayed for the command and parameters + * Datatype of the parameters. + * View name for which the command needs to be available. Eg: configure-view(config mode) or enable-view(exec mode) + +Example: + +```xml + + + + + + + if test "${access-list-name}" = ""; then + python $SONIC_CLI_ROOT/sonic-cli.py get_acl_acl_sets show_access_list.j2 + else + python $SONIC_CLI_ROOT/sonic-cli.py get_acl_set_acl_entries ${access-list-name} ACL_IPV4 show_access_list.j2 + fi + + +``` + +2. Write/Update an actioner script: The actioner script prepares the message body having the JSON request and invokes the REST client API. The actioner script is invoked by the klish and the input parameters are passed to it from the XML file. +Actioner can be defined with the tag in the XML file. + + There are three different methods available to implement the Actioner: sub-shell, clish_restcl and clish_pyobj. Sub-shell is spawned by Klish to run the script defined in tag. Both clish_pyobj and clish_restcl methods are part of Klish built-in fucntions and invoked by Klish. The built-in fucntions can be used for commands that would reduce time taken to execute a command by eliminating the sub-shell interpreter overhead. + + * Spawn a sub-shell to run the scripts defined in a command's tag. The shebang can be specified for the script execution. By default the "/bin/sh" is used. To customize shebang the 'shebang' field of the ACTION tag is used. + + The sub-shell runs the Python script sonic_cli_.py + + sonic_cli_.py [parameters . . .] + The sonic_cli_.py has a dispatch function to call a REST client method with parameters passed from user input. + + **Example**: + Refer the tag in the above example command. + The actioner scripts are placed in the following location: + sonic-mgmt-framework/CLI/actioner/ + One actioner script will be written per module. + Eg: sonic_cli_if.py can be used to handle the interface cases. + + * Invoke the built-in function, clish_restcl, to use libcurl to make REST client call + This builtin uses libcurl to connect to the REST server Format of ACTION tag argument: + + oper= url= body={..} + where oper can be PUT/POST/PATCH/DELETE and body is optional. Note that oper GET is not supported as we currently don't handle rendering using jinja templates. + + **Example**: + + oper=PATCH url=/restconf/data/openconfig-interfaces:interfaces/interface=Vlan${vlan-id}/config body={"openconfig-interfaces:config": {"name": "Vlan${vlan-id}"}} + + * Invoke the built-in function, clish_pyobj, to use the embedding Python to make REST client call + This builtin uses embedded python library. + Format of ACTION tag argument is similar to the one of sub-shell. + For 'show' commands, the jinja2 template is passed to the Python script to apply template to redender CLI output. + The ${__full_line} variable is also required to support Pipe, e.g sonic# show vlan | no-more. + Note that the "if-else" statement defined in the sub-shell can be moved to Python script. + + **Example**: + sonic_cli_vlan get_sonic_vlan_sonic_vlan Vlan${id} show_vlan.j2 ${__full_line} + + **Example**: + + Below example shows that the clish_pyobj can be used to set a dynamic variable "supported_breakout_modes" to check the breakout capability for a given port. + Once the result is returned from the Python fucntion, the variable keeps the result and pass to like below. +``` + + sonic_cli_breakout.py capability + + + +``` +3. Write/Update Renderer scripts and templates. +The JSON response from the swagger client API is received by the actioner and passes the response to the renderer script. +The renderer script will invoke the jinja2 template with the JSON response. The template will parse the JSON response and generate the CLI output. +Please use [JSON tool](#297-tool-for-json-navigation-in-jinja-templates) to efficiently access the JSON data. +Refer files in the below path for an example of usage + + **Renderer path**: + sonic-mgmt-framework/CLI/renderer + **Renderer script**: + scripts/render_cli.py + **Renderer template**: + templates/show_access_list.j2 + + Every show command response can have a separate template file. + +#### 2.9.3 Enhancements to Klish + +Additional enhancements can be done to open source klish as below. Enhancements may include defining a new data types for CLI command parameters, define new macros that can be referenced for CLI commands structure that have repetitive definitions and defining platform specific entity values. + +1. PTYPES + New parameter types (PTYPES) can be defined and used in the CLI XML files. + * PTYPE represents a parameter type, a syntactical template which parameters reference. + * sonic-clish.xsd defines the tag and the attributes associated with the PTYPE tag. + * Whenever a new attribute or tag introduced the xsd rules should also be updated. + * sonic-clish.xsd is avilable at sonic-mgmt-framework/CLI/clitree/scripts/ + * The klish source supports certain primitive PTYPEs. + * New user defined PTYPEs can be added to the SONIC project and can be defined in sonic_types.xml file. + * sonic_types.xml is available in sonic-mgmt-framework/CLI/clitree/cli-xml/sonic_types.xml + + **Example**: STRING_32 + + ```xml + + ``` + +2. MACROS + * Macros can be introduced by defining them in _macro.xml + * Macros are used where repetitive command snippets (or command parameter snippets) with minor variations are needed in XML files. + * It is possible to create a macro definition in a _macro.xml file and use the macro in the .xml file. + * For cases where variations in the values of these macro options are needed, the XML attributes can be passed as arguments to macro and substituted as values inside the macro definition. + * As part of parser tree preprocessing during Make, the referenced macro is expanded with the macro definitions and then the parser file is exported to the target XML files in CLI/target/command-tree to be consumed by Klish parser. + * One macro file can be written per module. + * The macro files are placed at sonic-mgmt-framework/CLI/clitree/macro/ + * Macros can also be nested with reference to another macro definition. + + **Example**: + + ```xml + + + + + + + ``` + + * The previous macro “IPV4-SRC-OPTIONS“ is used by the macro below "IPV4-ACL". + + ```xml + + + + + + + + + + + + + + + + + ``` + +3. Entities + + * Entities can be defined and referenced in the XML files. + * One common use case for entities is to specify platform specific parameters and parameter value limits in XML files. + * To define an entity based parameter value range limit, three tasks must be done: + + 1. Define the feature value as entity in mgmt_clish_feature_master.xsd + **Example**: + + ```xsl + + + This contains the allowed feature names for any platform specific customizable feature names. + If you are adding a new platform customizable feature, add a record in the enumerated list here. + + + + + + + + + + This contains the allowed feature-value names for any platform specific customizable feature-value + strings. If you are adding a new platform customizable entity, add a record in the enumerated list here. + + + + + + + ``` + + 2. Set the attribute 'value' to the right limit on respective platform configuration file that can be defined statically (used at build time). + Refer to a dummy platform (platform_dummy.xml) configuration file. + The platform_dummy.xml file can be located at sonic-mgmt-framework/CLI/clitree/scripts/. + + **Example**: + + MAX_MTU + + 3. Use the ENTITY just like any other XML ENTITY in the CLI parser XML file. + +4. Extended support with ext_help and regexp_select + The ext_help is similar to the pattern used in select method and used with regexp_select method. The value of the ext_pattern attribute is used for auto completion in regexp_select method. + + **Example**: + Interfaces can be defined with regexp_select method and ext_pattern to support abbreviated interface naming, i.e eth20, e20, Ether20 etc. Here, we are left the help part as empty, and define the help string in the place where we use this PTYPE. +``` + + + +``` + +#### 2.9.4 Preprocess XML files + +* The preprocessing scripts are invoked at compile time and no action is required to add a new CLI command. + +* This section gives information about what is done during preprocessing stage. + + * The preprocessing scripts preprocess the raw CLI XML files and generate a target XML file that can be consumed by the klish open source parser. + * The inputs to the preprocessing scripts are the raw CLI XML files, macro files and other utility files like platform specifics. + * The cli-xml files are validated as part of compilation. + * The 'xmllint' binary is used to validate all the processed XML files (i.e. after macro substitution and pipe processing) against the detailed schema kept at sonic-clish.xsd + * The following preprocessing scripts are introduced: + + * **klish_ins_def_cmd.py**: This script is used to append the "exit" and "end" commands to the views of the Klish XML files + * **klish_insert_pipe.py**: This script extends every show and get COMMAND with pipe option + * **klish_platform_features_process.sh**: Validate all platform XML files. Generate the entity.xml files. + * **klish_replace_macro.py**: This script does macro replacement on the XML files which are used by klish to define CLI structure. + +#### 2.9.5 CLI directory structure + +All CLI infrastructure, CLI definitions and their handlers are maintained +under `sonic-mgmt-framework/CLI` directory. + +``` +CLI +├── actioner /* CLI actioner scripts */ +├── clicfg /* Entity substitution */ +├── clitree +│   ├── cli-xml /* CLI model XMLs */ +│   ├── macro /* Macro definitions */ +│   └── scripts /* Preprocessor scripts */ +├── klish /* KLISH patches and launcher scripts */ +│   └── patches +└── renderer + ├── scripts /* Renderer scripts */ + └── templates /* Renderer jinja templates */ +``` + +Below directories are used to collect XML files and utility scripts to generate the target XML files for CLI command tree build-out. + +**clitree/cli-xml** - contains unprocessed/raw CLI XML files defined by developer + +**clitree/macro** - contains macro definitions used/referenced in CLI XML files defined by developer + +**clitree/scripts** - contains utility scripts to process raw CLI XML files defined by developer into processed CLI XMLs to be consumed by Klish Parser. + +**clitree/Makefile** - rules to preprocess the raw CLI XMLs and validate the processed output against the DTD in sonic-clish.xsd + +**clicfg** - files for platform specific entity substitution. + +**renderer** - templates and scripts to be used in rendering the show commands output. + +After compilation the processed CLI XMLs can be found in sonic-mgmt-framework/build/cli/command-tree + +#### 2.9.6 Generic REST Client for Actioner Scripts + +Actioner scripts should use the generic REST client for makeing to communicate to REST Server. +Generic client code is avalable at `sonic-mgmt-framework/CLI/actioner/cli_client.py`. +This is tailor made to connect REST Server on local switch and handle the CLI actioner usecases. + +Typical usage: + +Create a CLI client object. +Client object can be created as a global variable in every actioner script. +Server sonnection and session are managed internally. + +```python +import cli_client as cc + +api = cc.ApiClient() +``` + +Create a path object. It accepts parameterized path template and parameter values. +Path template is similar to the template used by Swagger UI. +Parameter values will be URL-encoded and substituted in the path template. + +```python +path = cc.Path('/restconf/data/openconfig-acl:acl/acl-sets/acl-set={name},{type}/acl-entries', + name='MyNewAccessList', type='ACL_IPV4') +``` + +Invoke REST API.. `ApiClient` object supports get, post, put, patch and delete operations. +All these operations send a REST request and return a response object wrapping the REST response data. + +```python +response = api.get(path, response_type=None) +``` + +An optional argument known as response_type is applicable only for GET and POST operations. +It instructs the REST Client infrastructure to return the response from REST server in the form of +JSON string instead of JSON (python dictionary). +If the API does not specify value for response_type, the default is JSON. +The supported values for response_type are string and json. + +**The use case for using response_type** + +If the response from the rest server contains a non-hierarchical JSON string and if it's quite big enough, +then the recommendation is to operate on the JSON string itself instead of converting it to python dictionary. +This will help the actioner consume less memory. It was observed that loading into dictionary takes considerably +large memory. + +Check API status through `response.ok()` function, which returns true if API was success (HTTP 2xx status). +`ok()` function returns false if server returned an error. +Use `response.error_message()` function to get the displayable error message. + +```python +if response.ok() { + respJson = response.content + # invoke renderer to display respJson +} else { + print response.error_message() + return -1 +} +``` + +Examples of other REST API calls. + +```python +# construct your request data json +jsonDict = {} +jsonDict["acl-set"]=[{ "name":the_acl_name, .... }] + +# POST request with default response_type i.e. JSON +response = api.post(path, data=jsonDict) + +# POST request with string as response_type +response = api.post(path, data=jsonDict, response_type='string') + +# PUT reuest +reponse = api.put(path, data=jsonDict) + +# PATCH request +response = api.patch(path, data=jsonDict) + +# GET request with default response_type i.e. JSON +response = api.get(path) + +# GET request with string as response_type +response = api.get(path, response_type='string') + +# DELETE request +response = api.delete(path) +``` + +Above example used `response.error_message()` function to get ser displayable error message from the REST response. +REST server always returns error information in standard RESTCONF error format. +The `error_message()` function looks for the **error-message** attribute to get user displayable message. +If there was no **error-message** attribute, a generic message will be returned based on **error-tag** value. +Actioners can override this logic through customiz error message formatter function. +The custom formtter function receives HTTP state code and RESTCONF error json; and should return a string. +However default error formatter is sufficient for most of the cases. + +```python +if not response.ok(): + print response.error_message(formatter_func=my_new_error_message_formatter) + +# ..... + +def my_new_error_message_formatter(status_code, error_entry): + if err_entry['error-type'] == 'protocol': + return "System error.. Please reboot!!" + return "Application error.. Please retry." +``` + +#### 2.9.7 Tool for JSON navigation in jinja templates + +Developers can use this tool to retrieve the JSON data efficiently and check for the existence of nodes. +The tool uses the path in the GNMI style for querying and checking the nodes. +Following are the APIs exposed by the tool + +#### get(json_data, path) + params + json_data: JSON data in the form of dictionary + path: The GNMI style path, examples below + return value + On success returns the queried nodes (format/type will be as present in JSON data) + on Failure returns None + +##### Example: + +```code +(Pdb) get(data, "/acl-sets/acl-set[name=MyACL1][type=ACL_IPV4]/config/") +{'name': 'MyACL1', 'type': 'ACL_IPV4', 'description': 'Description for MyACL1'} + +``` + +#### get_str(json_data, path) + params + json_data: JSON data in the form of dictionary + path: The GNMI style path, examples below + return value + On success returns the queried nodes (format/type will be strigified) + on Failure returns None + +##### Example: + +```code +(Pdb) get_str(data, "/acl-sets/acl-set[name=MyACL1][type=ACL_IPV4]/config/") +"{'name': 'MyACL1', 'type': 'ACL_IPV4', 'description': 'Description for MyACL1'}" + +``` + +#### contains(json_data, path) + params + json_data: JSON data in the form of dictionary + path: The GNMI style path, examples below + return value + On success returns True + on Failure returns False + +##### Example: + +```code +(Pdb) contains(data, "/acl-sets/acl-set[name=MyACL1][type=ACL_IPV4]/config/") +True +(Pdb) contains(data, "/acl-sets/acl-set[name=MyACL1][type=ACL_IPV4]/config/not_present") +False + +``` + +#### Sample usage in the renderer + +```text +{{json_tools.get(json_output, "/acl-sets/acl-set[name=MyACL1][type=ACL_IPV4]")}} +{{json_tools.get(json_output, "/acl-sets/acl-set[name=MyACL1]")}} +{{json_tools.get(json_output, "/acl-sets/acl-set[name=MyACL1][type=ACL_IPV4]/config/type")}} +{{json_tools.get_str(json_output, "/acl-sets/acl-set[name=MyACL1][type=ACL_IPV4]/config/type")}} +{{json_tools.contains(json_output, "/acl-sets/acl-set[name=MyACL1][type=ACL_IPV4]/config/type_not")}} +{{json_tools.contains(json_output, "/acl-sets/acl-set[name=MyACL1][type=ACL_IPV4]/config/type")}} + +``` + + +### 2.10 REST Server + +sonic-mgmt-framework repository contains REST Server infrastructure code. +It provides [RESTCONF](https://tools.ietf.org/html/rfc8040) APIs for the YANGs defined in sonic-mgmt-common. +These APIs are serviced by Translib; see [HLD](https://github.com/Azure/SONiC/blob/master/doc/mgmt/Management%20Framework.md#3224-REST-server) for details. + +YANGs are automatically plugged into REST Server infrastructure during sonic-mgmt-framework compilation. +Details are in [section 3.1.5](#315-swagger-server-generator). + +### 2.11 gNMI + +gNMI interfaces are automatically supported for all YANGs definied under `sonic-mgmt-common/models/yang` directiry. +This includes OpenConfig & IETF YANGs with their extensions and SONiC YANGs. + +gNMI and telemetry infrastructure code is maintained in **sonic-telemetry** repository. + +### 2.12 Compilation + +#### 2.12.1 sonic-mgmt-common + +sonic-mgmt-common repository compiles into **sonic-mgmt-common_1.0.0_{arch}.deb** and +**sonic-mgmt-common-codegen_1.0.0_{arch}.deb** packages. + +The sonic-mgmt-common_1.0.0_{arch}.deb contains YANGs, YANG derivatives and configuration files +required by Translib, transformer and CVL. +These are installed in **mgmt-framework** and **telemetry** containers. + +sonic-mgmt-common-codegen_1.0.0_{arch}.deb contains YGOT bindings and other generated source files +which are required for sonic-mgmt-framework and sonic-telemetry compilation. +When package cache mode is enabled, the build infrastructure will not compile sonic-mgmt-common +files if there are no changes in it since last compilation. +Generated source files are restored by installing sonic-mgmt-common-codegen_1.0.0_{arch}.deb in +the build container. + +#### 2.12.2 sonic-mgmt-framework + +Code from sonic-mgmt-framework repository compiles into **sonic-mgmt-framework_1.0-01_{arch}.deb** +and a docker image **docker-sonic-mgmt-framework.gz**. +REST Server and CLI code are packaged in sonic-mgmt-framework_1.0-01_xyz.deb. +This and sonic-mgmt-common_1.0.0_{arch}.deb are included in the docker image. + +To rebuld the docker image both these debs should be cleaned. + +```bash +# Only if you have changed something in sonic-mgmt-common +find target -name "sonic-mgmt-common*" -delete -print + +# Always do it - for both mgmt-common and mgmt-framework changes +find target -name "*sonic-mgmt-framework*" -delete -print + +make target/docker-sonic-mgmt-framework.gz +``` + +To load new docker image on the switch: + +```bash +admin@sonic:~$ sudo sonic_installer upgrade_docker mgmt-framework docker-sonic-mgmt-framework.gz +``` + +#### 2.12.2 sonic-telemetry + +TODO + +### 2.13 Unit Testing + +#### 2.13.1 REST API testing + +REST server provides a test UI (Swagger UI) will display the operations defined in OpenAPI spec along with options to test it inline. To launch the Swagger UI, open “https://SWITCH_IP/ui” from the browser. Browser may complain that the certificate is not trusted. This is expected since the REST Server uses a temporary self signed certificate by default. Ignore the warning and proceed, this loads a home page as shown below, which contains links for OpenAPIs both derived from yang models and also manually written OpenAPI specs. + +![Swagger UI Home Page](images/homepage.jpeg) +Upon clicking of any one of the link, The Swagger UI correspoding to that openAPI spec will be opened as shown below. + +![OpenAPI Page](images/openapis.jpeg) + +Expand any section to view description, parameters and schema/sample data.. + +Use “Try it out” button test the API from the Swagger UI itself. It opens a form to enter URI parameters and body data. Enter values and hit “Execute” button. REST API will be invoked and results are shown. + +#### 2.13.2 Local REST Server and CLI + +TODO + +#### 2.13.3 gNMI Unit Testing + +gNMI unit testing can be performed using open-source gNMI tools. 'telemetry' docker already contains such tools e.g. gnmi_get, gnmi_set, gnmi_cli etc. You can directly invoke these tools from linux shell of a switch as below : + +``` +docker exec -it telemetry gnmi_get -xpath /openconfig-interfaces:interfaces/interface[name=Ethernet0] -target_addr 127.0.0.1:8080 -insecure + +``` + +* Use 'gnmi_get' tool to perform a get request against the gNMI target in switch. For sample get request and response, refer to https://github.com/project-arlo/sonic-telemetry/blob/master/test/acl_get.test and https://github.com/project-arlo/sonic-telemetry/blob/master/test/acl_get.result respectively. +* Use 'gnmi_set' tool to perform a set request against the gNMI target in switch. For sample set request and response, refer to https://github.com/project-arlo/sonic-telemetry/blob/master/test/acl_set.test and https://github.com/project-arlo/sonic-telemetry/blob/master/test/acl_set.result respectively. +* Use 'gnmi_cli' to get the capabilities of gNMI target. For example : + +``` +docker exec -it telemetry gnmi_cli -capabilities -insecure -logtostderr -address 127.0.0.1:8080 +``` +* Use 'gnmi_cli' to perform subscription request. For sample subscribe request and response, refer to https://github.com/project-arlo/sonic-telemetry/blob/master/test/ethernet_oper_status_subscribe.test and https://github.com/project-arlo/sonic-telemetry/blob/master/test/ethernet_oper_status_subscribe.result respectively. + +#### 2.13.3 Spytest + +TODO + + +## 3 Appendix + +### 3.1 Code Generators + +#### 3.1.1 YANG Tree Generator + +YANG tree generators are run during compilation of `sonic-mgmt-common/models/yang` directory. + +Following trees are generated: +1) Standard [Text tree](#2361-yang-text-tree) using `pyang -f tree` option. +2) Standard [HTML tree](#2362-yang-html-tree) using `pyang -f jstree` option. +3) Custom [Markdown tree](#2363-yang-markdown-tree) which highlights nodes added/removed by extension YANGs. +Uses a custom pyang plugin `sonic-mgmt-common/tools/pyang/pyang_plugins/doctree.py`. + +All tree generators are automatically run during sonic-mgmt-common compilation. +It can be manually trigged through following commands: + +```bash +cd /sonic/sonic-buildimage/src/sonic-mgmt-common/models/yang +make # generates all trees +make allyangs.tree # generates only text tree +make allyangs_tree.html # generates only html tree +make yang_tree.md # generates only md tree +``` + +Requires: +- Pyang 2.1.1 +- Python 2.7.14 or later (or Python 3) + +#### 3.1.2 YGOT Binding Generator + +This tool generates YGOT binding code for YANG files under `sonic-mgmt-common/models/yang` directory. +YGOT bindings are written to `sonic-mgmt-common/translib/ocbinds/ocbinds.go` file. + +It is run automatically during compilation of sonic-mgmt-common repository. +It can be manually triggered to test YANG changes through following steps. + +```bash +cd /sonic/sonic-buildimage/src/sonic-mgmt-common +make go-deps +cd -C translib ocbinds/ocbinds.go +``` + +Requires: +- Go 1.13.8 or later +- customized ygot (patched during sonic-mgmt-common compilation) + +#### 3.1.3 CVL Schema Generator + +This tool generates CVL YIN schema from the SONiC YANGs. +It removes all read-only state nodes, RPC and notification nodes using a +pyang plugin `sonic-mgmt-common/tools/pyang/pyang_plugins/yin_cvl.py`. + +Schema files will be generated under `sonic-mgmt-common/build/cvl/schema` directory. + +CVL Schema generator is automatically run during sonic-mgmt-common compilation. +It can be manually trigged through following commands: + +```bash +cd /sonic/sonic-buildimage/src/sonic-mgmt-common/cvl +make schema +``` + +Requires: +- Pyang 2.1.1 +- Python 2.7.14 or later (or Python 3) + +#### 3.1.4 OpenAPI Generator + +This tool generates OpenAPI specs (aka swagger spec) from YANGs. +It is implemented as a pyang plugin `sonic-mgmt-framework/tools/pyang/pyang_plugins/openapi.py`. + +YANG file to YAML file mapping is not one to one; i.e, there will not be a `.yaml` for every `.yang` file. +OpenAPI generator first constructs a normalized YANG tree using all the YANG files defined under +`sonic-mgmt-common/models/yang`, `sonic-mgmt-common/models/yang/extensions` +and `sonic-mgmt-common/models/yang/sonic` directories. +YANGs from `sonic-mgmt-common/models/yang/common` are used to resolve compilation dependencies only. +YAML files will be generated for each top level YANG module in normalized tree. + +Eg, below 3 YANG files will result in only one OpenAPI spec - main.yaml. + +```text ++---------------------------+-------------------------------+-------------------------------+ +| yang/main.yang | yang/common/sub1.yang | yang/extensions/sub2.yang | ++---------------------------+-------------------------------+-------------------------------+ +| module main { | module sub1 { | module sub2 { | +| container feature1 { | grouping newstuffs {...} | grouping morestuffs {...} | +| } | augment /main:feature1 { | augment /main:feature1 { | +| } | uses newstuffs; | uses morestuffs; | +| | } | } | +| | } | } | ++---------------------------+-------------------------------+-------------------------------+ +``` + +Generated OpenAPI specs will be written to `sonic-mgmt-framework/build/yaml` directory. + +Note that transformer annotations YANGs are not considered - they are not data models. + +OpenAPI Generator is automatically run during sonic-mgmt-framework compilation. +It can be manually trigged through following commands: + +```bash +cd /sonic/sonic-buildimage/src/sonic-mgmt-framework/models +make -f yang_to_openapi.mk +``` + +Requires: +- Pyang 2.1.1 +- mmh3 2.5.1 (python library) +- Python 2.7.14 or later (or Python 3) + +#### 3.1.5 Swagger Server Generator + +This tool generates Swagger Go server handler code from the OpenAPIs specs. +It consumes YANG generated OpenAPI specs from `sonic-mgmt-framework/build/yaml` directory +and manually written OpenAPI specs from `sonic-mgmt-framework/models/openapi` directory, if any. + +Generated Go source files will be available in `sonic-mgmt-framework/build/rest_server/dist/openapi` directory. +No manual edits are required in the generated server handler functions. +Customized mustache templates are used to generate the server handlers with REST Server specific logic. +Mustache templates are in `sonic-mgmt-framework/tools/swagger_codegen/go-server` directory. + +Swagger server generator is automatically run during sonic-mgmt-framework compilation. +It can be manually trigged through following commands (assumes OpenAPI generator is already run): + +```bash +cd /sonic/sonic-buildimage/src/sonic-mgmt-framework/models +make -f openapi_codegen.mk go-server +``` + +Requires: +- Java 1.8 +- Python 3 +- jinja2 2.10.3 + +#### 3.1.6 Swagger Client SDK Generator + +This tool generates swagger client SDK code in python from the OpenAPI specs. +Python client SDK was originally intended to be used by KLISH CLI actioners, +but now CLI actioners are using a generic REST client. +Hence this code generator is disabled by default. +It can be enabled for specific OpenAPIs specs by listing them in `sonic-mgmt-framework/models/codegen.config` file. + +```makefile +# Generated OpenAPI specs from sonic-mgmt-framework/build/yaml directory +PY_YANGAPI_CLIENTS += openconfig-interfaces +PY_YANGAPI_CLIENTS += openconfig-platform +PY_YANGAPI_CLIENTS += openconfig-system + +# OpenAPI specs from sonic-mgmt-framework/models/openapi directory +PY_OPENAPI_CLIENTS += my-new-apis +``` + +Generated client SDK code will be available in `sonic-mgmt-framework/build/openapi_client_py` directory. + +Note: Swagger generates lot of client SDK code but only fraction of it will be exercised by KISH CLI. +Framework provides a generic REST client for CLI use which is more lighter and faster than swagger client SDK. +**Recommendation for CLIs is to use this generic REST client and not Swagger Client SDK**. + +Swagger client SDK generator is automatically run during sonic-mgmt-framework compilation +if some OpenAPI specs are whitelisted in `sonic-mgmt-framework/models/codegen.config`. +It can be manually run through following commands (assumes OpenAPI generator is already run): + +```bash +cd /sonic/sonic-buildimage/src/sonic-mgmt-framework/models +make -f openapi_codegen.mk py-client # generates for all whitelited specs + +# To force generation for a specific spec +PY_YANGAPI_CLIENTS=openconfig-interfaces make -f openapi_codegen.mk py-client +``` + +Requires: +- Java 1.8 + +#### 3.1.7 Swagger UI Generator + +This tool generates Sweagger UI for the REST Server as expaned in [section 2.13.1](#2131-rest-api-testing). +It is automatically run along with [Swagger Server Generator](#315-swagger-server-generator). + +#### 3.1.8 RESTCONF Documentation Generator + +RESTCONF documentation is generated automatically as part of build during the phase when OpenAPIs(YAML) for YANG modules are generated. +No extra step is needed from developers. RESTCONF document generator is a submodule in OpenAPI Generator and it relies on YANG's description +statements for documentation text. + +Developers are requested to have a description statements in YANG models for all Data nodes such as container/list/leaf/leaf-list/rpc. +For any data node which does not have description statement, the generated document will have blank text i.e. empty sections + +The generated document will be in a github complaint markdown format (.md) +The document will be generated for each top level YANG module in normalized tree. For more information on how generator works please refer [section 3.1.4](#314-OpenAPI-Generator) + +The generated documents will be written to `sonic-mgmt-framework/build/restconf_md` directory. + +#### 3.1.9 CLI Documentation Generator + +CLI Document generator tool will generate the documentation using KLISH CLI model XMLs. +Command name, mode, syntax and parameter description are already available in model XML. +Additional tags will be introduced for developers to specify command description, usage info and examples for each COMMAND. + +- DOCGEN tag to group all document generation related tags - it can include one DESCRIPTION, one USAGE and one or more EXAMPLE, ALTCMD and FEATURE tags. +- DESCRIPTION tag to provide a detailed description of the command. +- USAGE tag to specify usage guidelines - when to use, preconditions, suggested next actions, etc. +- EXAMPLE tags to specify examples and sample output. +- ALTCMD tags to specify alternative commands in other CLI framework such as VTYSH, CLICK and in fact other alternate commands in KLISH itself (if available). + +All these documentation tags will be optional. They are used only for document generation and not to render the commands. + +Document generator tool will consume all model XMLs from src/CLI/clitree/cli-xml directory to generate CLI document in markdown syntax. +Below table summarizes how each section of document template will be derived from above defined XML tags. + +```text ++----------------+-----------------+-------------------+---------------------------------------------------------------------------+ +| Section. | XML TAG | Attribute | Comments | ++----------------+-----------------+-------------------+---------------------------------------------------------------------------+ +| Command name | COMMAND | | | +|----------------|-----------------|-------------------|---------------------------------------------------------------------------+ +| Description | DESCRIPTION | | Use “help” value of COMMAND tag if DESCRIPTION tag is not specified. | +|----------------|-----------------|-------------------|---------------------------------------------------------------------------| +| Syntax | PARAM | name | Lists all possible combinations of parameters | +|----------------|-----------------|-------------------|---------------------------------------------------------------------------| +| Parameters | PARAM | name, help, ptype | | +|----------------|-----------------|-------------------|---------------------------------------------------------------------------| +| Modes | VIEW | name | Parent command name will be shown here. The VIEW “name” attribute provides| +| | | | internal name of the mode. Tool will lookup the parent command by matching| +| | | | the VIEW “name” value with other COMMAND tag’s “view” attribute value | +|----------------|-----------------|-------------------|---------------------------------------------------------------------------| +| Usage | | | | +| guidelines | USAGE | | Will be skipped if developer did not specify USAGE tag | +|----------------|-----------------|-------------------|---------------------------------------------------------------------------| +| Examples | EXAMPLE | | Will be skipped if developer did not specify EXAMPLE tags | +|----------------|-----------------|-------------------|---------------------------------------------------------------------------| +| Features | FEATURE | | Will be skipped if developer did not specify FEATURE tags | +|----------------|-----------------|-------------------|---------------------------------------------------------------------------| +| ALTCMDS | ALTCMD | type | Will be skipped if developer did not specify ALTCMD tags | ++----------------+-----------------|-------------------|---------------------------------------------------------------------------| + +``` +CLI document generator is automatically run during sonic-mgmt-framework compilation. +It can be manually trigged through following commands: + +```bash +cd /sonic/sonic-buildimage/src/sonic-mgmt-framework +make clidocgen (to generate the document) +make clidocgen-clean (to clean the document) +``` +The document will be written to `sonic-mgmt-framework/build/cli/command-tree/industry_standard_cli_reference_guide.md` directory. + +#### 3.1.9.1 Sample CLI Model + +```text + + + + + + + ... + + + Detailed command description + + + Usage information + + + Example1 body + + + Example2 body + + + ZTP + ZTP-cli + + config ztp -y enable + + + .... + + + + + + + +``` + +#### 3.1.9.2 Sample Generated Document + +```text +### command-tokens + +Detailed command description + +#### Syntax + +command-tokens param1 +command-tokens param2 + +#### Parameters + +Name | Description | Data type +-------|------------------------|-------- +Param1 | Help string for param1 | UINT +Param2 | Help string for param2 | STRING + +#### Mode + +Parent command - derived from view-name + +#### Usage Guidelines + +Usage information + +#### Examples + +Example1 description + +Example1 body + +Example2 description + +Example2 body + +### Alternate Commands + +#### click +config ztp -y enable + +#### vtysh +... + +### Features this CLI belongs to +* ZTP +* ZTP-cli + +``` +### 3.2 Debugging + +#### 3.2.1 Debugging performance issues + +For debugging performance issues we need to collect the profiling numbers which the REST query or the KLISH CLI is executed. Following is the procedure to collect the same. + +1. Restart the rest-server (command: "systemctl restart mgmt-framework") as root user (This is to ensure the old data is not collected as part of profiling) +2. From the client, execute only the REST API query or KLISH CLI for which you need to collect the profiling data +3. Go to the mgmt-framework docker (command: docker exec -it mgmt-framework bash) +4. Send the "SIGUSR1" signal to the rest process (command: Kill -10 `pidof rest_server`) +5. Copy the /var/log/rest_server/cpu.pprof.1 and /usr/sbin/rest_server file to your VDI +6. To generate the report in a text file (command: go tool pprof --txt ./rest_server ./cpu.pprof.1 > report.txt) +7. You would need to install the "graphviz" package to generate the pdf report (command: sudo apt-get install -y graphviz) +8. To generate the report in pdf format (command: go tool pprof --pdf ./rest_server ./ cpu.pprof.1 > report.pdf) + +The generated pdf will give the profiling information of all the calls in the REST server during the REST query or KLISH CLI. From this data we will come to know which function in the REST server is consuming more time during the execution. diff --git a/doc/mgmt/Docker to Host communication.md b/doc/mgmt/Docker to Host communication.md new file mode 100644 index 0000000000..85b19d0231 --- /dev/null +++ b/doc/mgmt/Docker to Host communication.md @@ -0,0 +1,204 @@ +# Feature Name +Docker to Host communication + +# High Level Design Document +#### Rev 0.2 + +# Table of Contents + * [List of Tables](#list-of-tables) + * [Revision](#revision) + * [About This Manual](#about-this-manual) + * [Scope](#scope) + * [Definition/Abbreviation](#definitionabbreviation) + +# List of Tables +[Table 1: Abbreviations](#table-1-abbreviations) + +# Revision +| Rev | Date | Author | Change Description | +|:---:|:-----------:|:------------------:|-----------------------------------| +| 0.1 | 10/28/2019 | Nirenjan Krishnan | Initial version | +|:---:|:-----------:|:------------------:|-----------------------------------| +| 0.2 | 12/08/2019 | Mike Lazar | Add details about architecture | +|:---:|:-----------:|:------------------:|-----------------------------------| +| 0.3 | 12/16/2019 | Mike Lazar | Add security and logging info | + + +# About this Manual +This document provides general information about the Docker to Host +communication feature in SONiC. + +# Scope +This document describes the high level design of Docker to Host communication. +This describes the infrastructure provided by the feature, and example usage, +however, it does not describe the individual host-specific features. + +# Definition/Abbreviation + +### Table 1: Abbreviations +| **Term** | **Meaning** | +|--------------------------|---------------------------------------------------| +| D-Bus | Desktop Bus: https://en.wikipedia.org/wiki/D-Bus | + +# 1 Feature Overview + +The management framework runs within a Docker container, and performs actions +translating the user CLIs or REST requests to actions. Most of these actions +perform some operation on the Redis database, but some of them require +operations to be done on the host, i.e., outside the container. This document +describes the host server, and translib API that are used to communicate between +the Docker container and the host. + + +## 1.1 Requirements + +### 1.1.1 Functional Requirements + +* The SONiC Management Framework and Telemetry containers must be able to issue + requests to the host, and return the responses from the host. +* The individual applications that need access to the host must be able to + create a host module and easily issue requests and get responses back from the + host. +* The host communication API shall be available in Translib, and shall provide + both synchronous and asynchronous communication methods. +* It shall be possible to configure the identity of the Linux user accounts who have access to a D-Bus socket. +* It shall be possible to configure containers in such a way that only certain containers (e.g. SONiC Mgmt.) + have access to the D-Bus socket. + +### 1.1.2 Configuration and Management Requirements + +N/A + +### 1.1.3 Scalability Requirements + +N/A + +### 1.1.4 Warm Boot Requirements + +N/A + +## 1.2 Design Overview +### 1.2.1 Basic Approach + +The code will extend the existing Translib modules to provide a D-Bus based +query API to issue requests to the host. The host service will be a Python based +application which listens on known D-Bus endpoints.https://en.wikipedia.org/wiki/D-Bus + +The individual app modules can extend the host service by providing a small +Python snippet that will register against their application endpoint. + +### 1.2.2 Container + +SONiC Management Framework, gNMI Telemetry containers + +### 1.2.3 SAI Overview + +N/A + +# 2 Functionality +## 2.1 Target Deployment Use Cases + +All deployments + +## 2.2 Functional Description + +This feature enables management applications to issue +requests to the host to perform actions such as: +* image install / upgrade +* ZTP enable/disable +* initiate reboot and warm reboot using existing scripts +* create show-tech tar file using existing show-tech script +* config save/reload using existing scripts + +# 3 Design +## 3.1 Overview + +The feature extends the SONiC management framework to add a D-Bus service on the +host. This service will register against a known endpoint, and will service +requests to the endpoint. Application modules will add snippets to the host +service, which will automatically register their endpoints, and the app module +in the container can use the APIs provided in Translib to send the request to +the host, and either wait for the response (if the request was synchronous), or +receive a channel and wait for the request to return the response on the +channel (asynchronous request). + +The architecture of a D-Bus host service in a SONiC environment is illustrated in the diagram below: +![](images/docker-to-host-services-architecture.jpg) + +Note. The Linux D-Bus implementation uses Unix domain sockets for client to D-Bus service communications. +All containers that use D-Bus services will bind mount +(-v /var/run/dbus:/var/run/dbus:rw) the host directory where D-Bus service sockets are created. +This ensures that only the desired containers access the D-Bus host services. + +D-Bus provides a reliable communication channel between client (SONiC management container) and service (native host OS) – all actions are acknowledged and can provide return values. It should be noted that acknowledgements are important for operations such as “image upgrade” or “config-save”. In addition, D-Bus methods can return values of many types – not just ACKs. For instance, they can return strings, useful to return the output of a command. + +### 3.1.1 Security of D-Bus Communications +In addition to standard Linux security mechanisms for file/Unix socket access rights (read/write), D-Bus provides a separate security layer, using the D-Bus service configuration files. +This allows finer grain access control to D-Bus objects and methods - D-Bus can restrict access only to certain Linux users. + +### 3.1.2 Command Logging + +It is possible to track and log the user name and the command that the user has requested. +The log record is created in the system log. + +## 3.2 DB Changes +### 3.2.1 CONFIG DB +N/A +### 3.2.2 APP DB +N/A +### 3.2.3 STATE DB +N/A +### 3.2.4 ASIC DB +N/A +### 3.2.5 COUNTER DB +N/A + +## 3.3 Switch State Service Design +### 3.3.1 Orchestration Agent +N/A +### 3.3.2 Other Process +N/A +## 3.4 SyncD +N/A +## 3.5 SAI +N/A + +## 3.6 User Interface +### 3.6.1 Data Models +N/A +### 3.6.2 CLI +#### 3.6.2.1 Configuration Commands +N/A +#### 3.6.2.2 Show Commands +N/A +#### 3.6.2.3 Debug Commands +N/A +#### 3.6.2.4 IS-CLI Compliance +N/A +### 3.6.3 REST API Support +N/A + +# 4 Flow Diagrams + +![](images/docker-to-host-service.svg) + +# 5 Error Handling + +The `hostQuery` and `hostQueryAsync` APIs return a standard Go `error` object, +which can be used to handle any errors that are returned by the D-Bus +infrastructure. + +# 6 Serviceability and Debug +N/A + +# 7 Warm Boot Support +N/A + +# 8 Scalability +N/A + +# 9 Unit Test +List unit test cases added for this feature including warm boot. + +# 10 Internal Design Information +N/A diff --git a/doc/mgmt/Management Framework.md b/doc/mgmt/Management Framework.md new file mode 100644 index 0000000000..d9e64563c8 --- /dev/null +++ b/doc/mgmt/Management Framework.md @@ -0,0 +1,2702 @@ +# SONiC Management Framework + +## High level design document + +### Rev 0.17 + +## Table of Contents + +* [List of Tables](#list-of-tables) +* [Revision](#revision) +* [About this Manual](#about-this-manual) +* [Scope](#scope) +* [Definition/Abbreviation](#definitionabbreviation) +* [Table 1: Abbreviations](#table-1-abbreviations) +* [1 Feature Overview](#1-feature-overview) + * [1.1 Requirements](#11-requirements) + * [1.2 Design Overview](#12-design-overview) + * [1.2.1 Basic Approach](#121-basic-approach) + * [1.2.2 Container](#122-container) +* [2 Functionality](#2-functionality) + * [2.1 Target Deployment Use Cases](#21-target-deployment-use-cases) +* [3 Design](#3-design) + * [3.1 Overview](#31-overview) + * [3.1.1 Build time flow](#311-build-time-flow) + * [3.1.2 Run time flow](#312-run-time-flow) + * [3.1.2.1 CLI](#3121-cli) + * [3.1.2.2 REST](#3122-REST) + * [3.1.2.3 gNMI](#3123-gnmi) + * [3.2 SONiC Management Framework Components](#32-sonic-management-framework-components) + * [3.2.1 Build time components](#321-build-time-components) + * [3.2.1.1 Yang to OpenAPI converter](#3211-yang-to-openapi-converter) + * [3.2.1.1.1 Overview](#32111-overview) + * [3.2.1.1.2 Supported HTTP verbs](#32112-supported-http-verbs) + * [3.2.1.1.3 Supported Data Nodes](#32113-supported-data-nodes) + * [3.2.1.1.4 Data Type Details](#32114-data-type-details) + * [3.2.1.1.5 Special Handling](#32115-special-handling) + * [3.2.1.1.6 Future enhancements](#32115-future-enhancements) + * [3.2.1.2 OpenAPI generator](#3212-OpenAPI-generator) + * [3.2.1.3 YGOT generator](#3213-YGOT-generator) + * [3.2.1.4 pyang compiler](#3214-pyang-compiler) + * [3.2.2 Run time components](#322-run-time-components) + * [3.2.2.1 CLI](#3221-cli) + * [3.2.2.2 REST Client SDK](#3222-REST-client-sdk) + * [3.2.2.3 gNMI Client](#3223-gnmi-client) + * [3.2.2.4 REST server](#3224-REST-server) + * [3.2.2.4.1 Transport options](#32241-transport-options) + * [3.2.2.4.2 Translib linking](#32242-Translib-linking) + * [3.2.2.4.3 Media Types](#32243-media-types) + * [3.2.2.4.4 Payload Validations](#32244-payload-validations) + * [3.2.2.4.5 Concurrency](#32245-concurrency) + * [3.2.2.4.6 API Versioning](#32246-api-versioning) + * [3.2.2.4.7 RESTCONF Entity-tag](#32247-restconf-entity-tag) + * [3.2.2.4.8 RESTCONF Discovery](#32248-restconf-discovery) + * [3.2.2.4.9 RESTCONF Query Parameters](#32249-restconf-query-parameters) + * [3.2.2.4.10 RESTCONF Operations](#322410-restconf-operations) + * [3.2.2.4.11 RESTCONF Notifications](#322411-restconf-notifications) + * [3.2.2.4.12 Authentication](#322412-authentication) + * [3.2.2.4.13 Error Response](#322413-error-response) + * [3.2.2.4.14 DB Schema](#322414-db-schema) + * [3.2.2.4.15 API Documentation](#322415-api-documentation) + * [3.2.2.5 gNMI server](#3225-gnmi-server) + * [3.2.2.5.1 Files changed and added](#32251-files-changed-and-added) + * [3.2.2.5.2 Sample Requests](#32252-sample-requests) + * [3.2.2.5.3 Authentication](#32253-authentication) + * [3.2.2.5.4 Error Response](#32254-error-response) + * [3.2.2.5.5 DB Schema](#32255-db-schema) + * [3.2.2.5.6 Transport options](#32256-transport-options) + * [3.2.2.5.7 Translib linking](#32257-Translib-linking) + * [3.2.2.5.8 Encodings](#32258-encodings) + * [3.2.2.5.9 Payload Validations](#32259-payload-validations) + * [3.2.2.5.10 Concurrency](#322510-concurrency) + * [3.2.2.5.11 API Versioning](#322511-api-versioning) + * [3.2.2.5.12 gNOI RPCs](#322512-gnoi-rpcs) + + * [3.2.2.6 Translib](#3226-Translib) + * [3.2.2.6.1 App Interface](#32261-app-interface) + * [3.2.2.6.2 Translib Request Handler](#32262-Translib-request-handler) + * [3.2.2.6.3 YGOT request binder](#32263-YGOT-request-binder) + * [3.2.2.6.4 YANG Model Versioning](#32264-yang-model-versioning) + * [3.2.2.6.5 DB access layer](#32265-db-access-layer) + * [3.2.2.7 Transformer](#3227-transformer) + * [3.2.2.7.1 Components](#32271-components) + * [3.2.2.7.2 Design](#32272-design) + * [3.2.2.7.3 Process](#32273-process) + * [3.2.2.7.4 Common App](#32274-common-app) + * [3.2.2.7.5 YANG Extensions](#32275-yang-extensions) + * [3.2.2.7.6 Public Functions](#32276-public-functions) + * [3.2.2.7.7 Overloaded Modules](#32277-overloaded-modules) + * [3.2.2.7.8 Utilities](#32278-utilities) + * [3.2.2.8 Config Validation Library (CVL)](#3228-config-validation-library-cvl) + * [3.2.2.8.1 Architecture](#32281-architecture) + * [3.2.2.8.2 Validation types](#32282-validation-types) + * [3.2.2.8.3 CVL APIs](#32283-cvl-apis) + * [3.2.2.9 Redis DB](#3229-redis-db) + * [3.2.2.10 Non DB data provider](#32210-non-db-data-provider) +* [4 Flow Diagrams](#4-flow-diagrams) + * [4.1 REST SET flow](#41-REST-set-flow) + * [4.2 REST GET flow](#42-REST-get-flow) + * [4.3 Translib Initialization flow](#43-Translib-initialization-flow) + * [4.4 gNMI flow](#44-gNMI-flow) + * [4.5 CVL flow](#45-CVL-flow) +* [5 Developer Work flow](#5-Developer-Work-flow) + * [5.1 Developer work flow for custom (SONiC/CVL) YANG](#51-Developer-work-flow-for-custom-SONiCCVL-YANG) + * [5.1.1 Define Config Validation YANG schema](#511-Define-Config-Validation-YANG-schema) + * [5.1.2 Generation of REST server stubs and Client SDKs for YANG based APIs](#512-Generation-of-REST-server-stubs-and-Client-SDKs-for-YANG-based-APIs) + * [5.1.3 Config Translation App (Go language)](#513-Config-Translation-App-Go-language) + * [5.1.4 IS CLI](#514-IS-CLI) + * [5.1.5 gNMI](#515-gNMI) + * [5.2 Developer work flow for standard (OpenConfig/IETF) YANG](#52-Developer-work-flow-for-standard-OpenConfigIETF-YANG) + * [5.2.1 Identify the standard YANG module for the feature for northbound APIs](#521-Identify-the-standard-YANG-module-for-the-feature-for-northbound-APIs) + * [5.2.2 Define the Redis schema for the new feature. (not applicable for legacy/existing feature)](#522-Define-the-Redis-schema-for-the-new-feature-not-applicable-for-legacyexisting-feature) + * [5.2.3 Define Config Validation YANG schema](#523-Define-Config-Validation-YANG-schema) + * [5.2.4 Generation of REST server stubs and Client SDKs for YANG based APIs](#524-Generation-of-REST-server-stubs-and-Client-SDKs-for-YANG-based-APIs) + * [5.2.5 Config Translation App (Go language)](#525-Config-Translation-App-Go-language) + * [5.2.6 IS CLI](#526-IS-CLI) + * [5.2.7 gNMI](#527-gNMI) +* [6 Error Handling](#6-error-handling) +* [7 Serviceability and Debug](#7-serviceability-and-debug) +* [8 Warm Boot Support](#8-warm-boot-support) +* [9 Scalability](#9-scalability) +* [10 Unit Test](#10-unit-test) +* [11 Appendix A](#11-appendix-a) +* [12 Appendix B](#11-appendix-b) + + +## List of Tables + +[Table 1: Abbreviations](#table-1-abbreviations) + +## Revision + +| Rev | Date | Author | Change Description | +|:---:|:-----------:|:-----------------------:|-----------------------------------| +| 0.1 | 06/13/2019 | Anand Kumar Subramanian | Initial version | +| 0.2 | 07/05/2019 | Prabhu Sreenivasan | Added gNMI, CLI content from DELL | +| 0.3 | 08/05/2019 | Senthil Kumar Ganesan | Updated gNMI content | +| 0.4 | 08/07/2019 | Arun Barboza | Clarifications on Table CAS | +| 0.5 | 08/07/2019 | Anand Kumar Subramanian | Translib Subscribe support | +| 0.6 | 08/08/2019 | Kwangsuk Kim | Updated Developer Workflow and CLI sections | +| 0.7 | 08/09/2019 | Partha Dutta | Updated Basic Approach under Design Overview | +| 0.8 | 08/15/2019 | Anand Kumar Subramanian | Addressed review comments | +| 0.9 | 08/19/2019 | Partha Dutta | Addressed review comments related to CVL | +| 0.10 | 09/25/2019 | Kwangsuk Kim | Updated Transformer section | +| 0.11 | 09/30/2019 | Partha Dutta | Updated as per SONiC YANG guideline | +| 0.12 | 10/19/2019 | Senthil Kumar Ganesan | Added Appendix B | +| 0.13 | 11/27/2019 | Anand Kumar Subramanian | Added new APIs in translib | +| 0.14 | 12/03/2019 | Sachin Holla | RESTCONF yang library and other enhancements | +| 0.15 | 12/19/2019 | Partha Dutta | Added new CVL API, platform and custom validation details | +| 0.16 | 04/08/2020 | Sachin Holla | API versioning enhancement | +| 0.17 | 04/08/2020 | Mohammed Faraaz | OpenAPI 3.0 enhancements | +| 0.18 | 04/09/2020 | Kwangsuk Kim | Updated CLI and Transformer enhancement | + +## About this Manual + +This document provides general information about the Management framework feature implementation in SONiC. + +## Scope + +This document describes the high level design of Management framework feature. + +## Definition/Abbreviation + +### Table 1: Abbreviations + +| **Term** | **Meaning** | +|--------------------------|-------------------------------------| +| CVL | Config Validation Library | +| NBI | North Bound Interface | +| ABNF | Augmented Backus-Naur Form | +| YANG | Yet Another Next Generation | +| JSON | Java Script Object Notation | +| XML | eXtensible Markup Language | +| gNMI | gRPC Network Management Interface | +| YGOT | YANG Go Tools | + +## 1 Feature Overview + +Management framework is a SONiC application which is responsible for providing various common North Bound Interfaces (NBIs) for the purposes of managing configuration and status on SONiC switches. The application manages coordination of NBI’s to provide a coherent way to validate, apply and show configuration. + +### 1.1 Requirements + +* Must provide support for: + + 1. Standard [YANG](https://tools.ietf.org/html/rfc7950) models (e.g. OpenConfig, IETF, IEEE) + 2. Custom YANG models ([SONiC YANG](https://github.com/Azure/SONiC/blob/master/doc/mgmt/SONiC_YANG_Model_Guidelines.md)) + 3. Industry-standard CLI / Cisco like CLI + +* Must provide support for [OpenAPI spec](https://OpenAPI.io/specification/) to generate REST server side code +* Must provide support for NBIs such as: + + 1. CLI + 2. gNMI + 3. REST/RESTCONF + +* Must support the following security features: + + 1. Certificate-based authentication + 2. User/password based authentication + 3. Role based authorization + +* Ease of use for developer workflow + + 1. Specify data model and auto-generate as much as possible from there + +* Must support Validation and Error Handling - data model, platform capability/scale, dynamic resources +* SNMP integration in SONiC is left for future study + +### 1.2 Design Overview + +Management framework makes use of the translation library (Translib) written in golang to convert the data models exposed to the management clients into the Redis ABNF schema format. Supported management servers can make use of the Translib to convert the incoming payload to SONiC ABNF schema and vice versa depending on the incoming request. Translib will cater to the needs of REST and gNMI servers. Later the Translib can be enhanced to support other management servers if needed. This framework will support both standard and custom YANG models for communication with the corresponding management servers. Management framework will also take care of maintaining data consistency, when writes are performed from two different management servers at the same time. Management framework will provide a mechanism to authenticate and authorize any incoming requests. Management framework will also take care of validating the requests before writing them into the Redis DB. Config Validation Library is used for syntactic and semantic validation of ABNF JSON based on YANG derived from Redis ABNF schema. + +#### 1.2.1 Basic Approach + +* Management framework takes comprehensive approach catering: + * Standard based YANG Models and custom YANG + * Open API spec + * Industry standard CLI + * Config Validation +* REST server, gNMI server, App module and Translib - all in Go +* Translation by using the Translib Library and application specific modules +* Marshalling and unmarshalling using YGOT +* Redis updated using CAS(Check-and-Set) trans. (No locking, No rollback) +* Config Validation by using YANG model from ABNF schema +* CLI with Klish framework + +#### 1.2.2 Container + +The management framework is designed to run in a single container named “sonic-mgmt-framework”. The container includes the REST server linked with Translib, and CLI process. +The gNMI support requires the gNMI server which is provided as a part of sonic-telemetry container. We would like to rename this container as the sonic-gnmi container as now it can perform configurations as well through the gNMI server. +Will introduce a new container sonic-mgmt-common to host the common code that is used both in the mgmt-framework and sonic-telemetry container. This new repo will compile into static libraries that will be used in the other two repos. This way sonic-telemetry repo can be compiled without the mgmt-framework being present in the code base. + +## 2 Functionality + +### 2.1 Target Deployment Use Cases + +1. Industry Standard CLI which will use REST client to talk to the corresponding servers to send and receive data. +2. REST client through which the user can perform POST, PUT, PATCH, DELETE, GET operations on the supported YANG paths. +3. gNMI client with support for capabilities, get, set, and subscribe based on the supported YANG models. + +## 3 Design + +### 3.1 Overview + +The SONiC management framework comprises two workflows: + +1. Build time flow +2. Run time flow + +as show in the architecture diagram below. + +![Management Framework Architecture diagram](images/Mgmt_Frmk_Arch.jpg) + +#### 3.1.1 Build time flow + +The Developer starts by defining the desired management objects and the access APIs to provide for the target application. This can be done in one of the two ways: - +1) A YANG data model +2) An OpenAPI spec + +This can be an independent choice on an application by application basis. However note that using YANG allows for richer data modelling, and therefore superior data validation. + +1. In case of YANG, if the developer chooses standard YANG model (Openconfig, IETF etc.), a separate SONiC YANG model has to be written based on Redis ABNF schema for validating Redis configuration and transformer hints should be written in a deviation file for standard YANG model to Redis DB coversion and vice versa (refer to [3.2.2.7 Transformer](#3227-transformer) for details). However, if custom SONiC YANG model is written based on guidelines, CVL YANG is automatically derived from it and the same is used for validation purpose and there is no need of writing any deviation file for transformer hints. Based on the given YANG model as input, the pyang compiler generates the corresponding OpenAPI spec which is in turn given to the OpenAPI generator to generate the REST client SDK and REST server stubs in golang. The YANG data model is also provided to the [YGOT](https://github.com/openconfig/YGOT) generator to create the YGOT bindings. These are used on the interface between Translib and the selected App module. Specifically, Translib populates the binding structures based upon the incoming server payload, and the App module processes the structure accordingly. Additionally, a YANG annotation file must also be provided, for data models that do not map directly to the SONiC YANG structure. The requests in this case will be populated into the YGOT structures and passed to App module for conversion. The App module uses the YANG annotations to help convert and map YANG objects to DB objects and vice-versa. + +2. In case of OpenAPI spec, it is directly given to the [OpenAPI](https://OpenAPI.io) generator to generate the REST client SDK and REST server stubs in golang. In this case the REST server takes care of validating the incoming request to be OpenAPI compliant before giving the same to Translib. There is no YANG, and therefore no YGOT bindings are generated or processed, and so the Translib infra will invoke the App module functions with the path and the raw JSON for App modules to convert. For configuration validation purpose, SONiC YANG model has to be written based on Redis ABNF schema. + +#### 3.1.2 Run time flow + +##### 3.1.2.1 CLI + +1. CLI uses the KLISH framework to provide a CLI shell. The CLI request is converted to a corresponding REST client request using c, and is sent to the REST server. +2. The OpenAPI generated REST server handles all the REST requests from CLI and invokes a common handler for all the create, update, replace, delete and get operations along with path and payload. This common handler converts all the requests into Translib arguments and invokes the corresponding Translib provided APIs. +3. Translib API then calls the corresponding handler defined in Common App module, subsequently invokes Transformer functions with YGOT structured request to perform model to model transformation, i.e. Openconfig Model to SONiC model, to generate the request DB map. +4. Common App handlers pass the request DB map to DB Access module to access REDIS Database, process the request DB map. The response to GET operation is passed to Transformer to perform reverse transformation to Openconfig Model, and wired out to CLI as JSON response. +5. Further processing of CLI commands will be handled by Translib componenets that will be discussed in more detail in the later sections. + +##### 3.1.2.2 REST + +1. REST client will use the OpenAPI generated client SDK to send the request to the REST server. +2. From then on the flow is similar to the one seen in the CLI. + +##### 3.1.2.3 gNMI + +GNMI service defines a gRPC-based protocol for the modification and retrieval of configuration from a target device, as well as the control and generation of telemetry streams from a target device to a data collection system. Refer [GNMI spec](https://github.com/openconfig/reference/blob/master/rpc/gnmi/gnmi-specification.md) + +![GNMI Service High level diagram (Proposed)](images/GNMI_Server.png) + +1. Existing SONiC telemetry framework has been extended to support the new GNMI services. +2. All 4 GNMI services are supported: Get, Set, Capabilities and Subscribe. +3. A new transl data client is added to process the incoming YANG-based request (either standard or proprietary) +4. The new transl data client relies on Translib infra provided to translate, get, set of the YANG objects. + +More details on the GNMI server, Client and workflow provided later in the document. + +### 3.2 SONiC Management Framework Components + +Management framework components can be classified into + +1. Build time components +2. Run time components + +#### 3.2.1 Build time components + +Following are the build time components of the management framework + +1. Pyang compiler (for YANG to OpenAPI conversion) +2. OpenAPI generator +3. YGOT generator +4. pyang compiler (for YANG to YIN conversion) + +##### 3.2.1.1 YANG to OpenAPI converter + +##### 3.2.1.1.1 Overview + +Open source Python-based YANG parser called pyang is used for YANG parsing and building a Python object dictionary. A custom plugin is developed to translate this Python object dictionary into an OpenAPI spec. The OpenAPI specs are [3.0](https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.3.md#schemaObject) complaint. + +URI format and payload is RESTCONF complaint and is based on the [RFC8040](https://tools.ietf.org/html/rfc8040). The Request and Response body is in JSON format in this release. + +##### 3.2.1.1.2 Supported HTTP verbs + +Following table lists the HTTP methods generated for different types of YANG nodes. + + YANG node type | HTTP methods +------------------------|----------------- +Configuration data | POST, PUT, PATCH, DELETE, GET, HEAD +Non configuration data | GET, HEAD +YANG RPC | POST + +##### 3.2.1.1.3 Supported Data Nodes + +For each of the below-listed Data keywords nodes in the YANG model, the OpenAPI (path) will be generated + +* Container +* List +* Leaf +* Leaf-list +* Rpc + +##### 3.2.1.1.4 Data Type Details + + YANG Type | OpenAPI Type +--------------|------------------ +int8 | Integer +int16 | Integer +int32 | Integer +int64 | Integer +uint8 | Integer +uint16 | Integer +uint32 | Integer +uint64 | Integer +decimal64 | Number +String | string +Enum | Enum +Identityref | String (Future can be Enum) +long | Integer +Boolean | Boolean +Binary | String with Format as Binary +bits | integer + +* All list keys will be made mandatory in the payload and URI +* YANG mandatory statements will be mapped to the required statement in OpenAPI +* Default values, Enums are mapped to Default and Enums statements of OpenAPI +* Since OpenAPI spec does not provide a way to define an unsigned types, a custom vendor extension x-yang-type is introduced which will contain the actual yang data type of a leaf/leaf-list. This vendor extension will be part of type object. + +Example below + +``` + /restconf/data/ietf-snmp:snmp/tlstm/cert-to-name={id}: + put: + tags: + - ietf-snmp + parameters: + - name: id + in: path + required: true + schema: + format: int32 + maximum: 4294967295 + minimum: 0 + type: integer + **x-yang-type: uint32** +``` +* As seen in above example all Integer types will have maximum and minimum fields. If the range statement is defined on the YANG node, then the value for maximum and minimum is derived from the range statement otherwise default range specified by YANG will be considered. And also a custom vendor extension called x-range will include a copy of range statement's value. If a type has one more range statements (may be indirectly using typedefs) all the range statements will be included as part of x-range extension separated by '|' character. + +Exampe below +``` + - format: int32 + **maximum: 2147483647** + **minimum: 1** + type: integer + **x-range: 1..2147483647 | 4..10** + x-yang-type: int32 +``` +* A string data types will have a vendor extension x-pattern whose value will be a regex. This field will only be present when the YANG node has it. Although OpenAPI 3.0 has a pattern field under schema object, but it is not used because the regex required for OpenAPI 3.0 is a W3C complaint regex but Openconfig models follow POSIX style regex. If a type has one more regex statements (may be indirectly using typedefs) all the regex statements will be included as part of x-pattern extension, they will be ANDed together. + +Example below +``` + - maxLength: 18446744073709551615 + minLength: 0 + type: string + **x-pattern: (([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])(%[\p{N}\p{L}]+)?** + x-yang-type: string +``` + +* As seen in above example all String types will have maxLength and minLength fields. If the length statement is defined on the YANG node, then the value for maxLength and minLength is derived from the length statement otherwise default length specified by YANG will be considered. And also a custom vendor extension called x-length will include a copy of length statement's value. If a type has one more length statements (may be indirectly using typedefs) all the length statements will be included as part of x-length extension separated by '|' character. + +* An Union data type is supported through oneOf directive. oneOf is a JSON schema directive supported by openAPI 3.0. It takes an array of objects and at any given time only one of item can be used. + +Example below +``` + - name: security-model + in: path + required: true + schema: + oneOf: + - enum: + - v1 + - v2c + - usm + - tsm + maxLength: 18446744073709551615 + minLength: 0 + type: string + x-yang-type: string + - format: int32 + maximum: 2147483647 + minimum: 1 + type: integer + x-range: 1..2147483647 + x-yang-type: int32 +``` + +##### 3.2.1.1.5 Special handling + +* A list and leaf-list will have maxItems and minItems properties set if a YANG node defines max-elements and min-elements statements. + +Example below +``` + security-model: + type: array + items: + oneOf: + - enum: + - v1 + - v2c + - usm + - tsm + maxLength: 18446744073709551615 + minLength: 0 + type: string + x-yang-type: string + - format: int32 + maximum: 2147483647 + minimum: -2147483648 + type: integer + x-range: 1..2147483647 + x-yang-type: int32 + minItems: 1 +``` +* A Choice-case statement is supported using oneOf directive. + +Example below +``` + ietf-snmp:auth: + type: object + properties: {} + **oneOf:** + - type: object + properties: + md5: + type: object + properties: + key: + maxLength: 18446744073709551615 + minLength: 0 + type: string + x-pattern: ([0-9a-fA-F]{2}(:[0-9a-fA-F]{2})*)? + x-yang-type: string + required: + - key + - type: object + properties: + sha: + type: object + properties: + key: + maxLength: 18446744073709551615 + minLength: 0 + type: string + x-pattern: ([0-9a-fA-F]{2}(:[0-9a-fA-F]{2})*)? + x-yang-type: string + required: + - key +``` + +##### 3.2.1.1.6 Future enhancements + +* Support for additional YANG actions and notifications(if required). +* Support for RESTCONF query parameters such as depth, filter, etc +* Convert Yang identities in to enums + +##### 3.2.1.2 OpenAPI generator + +OpenAPI-codegen tool (github.com/OpenAPITools/openapi-generator) is used to generate REST server and client code from the OpenAPI definitions. It consumes the OpenAPI definitions generated from YANG files and any other manually written OpenAPI definition files. + +REST server is generated in Go language. Customized OpenAPI-codegen templates are used to make each server stub invoke a common request handler function. The common request handler will invoke Translib APIs to service the request. + +REST client is generated in Python language. Client applications can generate the REST client in any language using standard OpenAPI-codegen tool. + +##### 3.2.1.3 YGOT generator + +YGOT generator generates Go binding structures for the management YANG. The generated Go binding structures are consumed by the Translib to validate the incoming payload and help in conversion of Redis data to management YANG specific JSON output payload. + +##### 3.2.1.4 pyang compiler + +Open source pyang tool is used to compile YANG models and generate output file in required format. For example SONiC YANG model is compiled and YIN schema file is generated for validation purpose. Similarly, after compiling YANG model, OpenAPI spec file is generated for generating REST client SDK and server code using OpenAPI-codegen. + +#### 3.2.2 Run time components + +Following are the run time components in the management framework + +1. CLI +2. REST Client SDK +3. REST server +4. gNMI Client +5. gNMI server +6. Translib +7. Config Validation Library (CVL) +8. Redis DB +9. Non DB data provider + +##### 3.2.2.1 CLI + +Open source Klish is integrated into sonic-mgmt-framework container to provide the command line interface tool to perform more efficient network operations more efficiently in SONiC. Klish will provide the core functionality of command parsing, syntax validation, command help and command auto-completion. The following diagram shows how the CLI commands are built, processed, and executed. + +![CLI components Interaction Diagram](images/cli_interactions.jpg) + +1. CLI command input from user +2. Klish invokes the actioner defined in command tag +3. Actioner invokes either sub shell or the built-in methods to make a REST client API call using C based libcurl or Python Requests package to the REST server. +4. Receive response from OpenAPI clientAPI and pass it to renderer scripts. +5. Renderer scripts processes the JSON response from REST Client and optionally formats the output using a Jinja template +6. CLI output is rendered to the terminal. + +###### 3.2.2.1.1 CLI components + +CLI consists of the following components. + +* **Open source Klish** - CLI parser framework to support Command Line Interface Shell +* **XML files** to define CLI command line options and actions + * Klish uses XML to define CLI commands to build the command tree. Klish provides modular CLI tree configurations to build command trees from multiple XML files. XML elements can be defined with macros and entity references, which are then preprocessed by utility scripts to generate the expanded XML files that are ultimately used by Klish. +* **Actioner** - Python scripts or built-in methods defined as a command `action`, to form the request body and invoke the OpenAPI client API +* **Renderer** - Python scripts defined with Jinja templates. Receives the JSON response from Swagger API and use the jinja2 template file to render and format CLI output. +* **Preprocess scripts** - Validates XML files and applies some processing from a developer-friendly form into a "raw" form that is compatible with Klish. + +###### 3.2.2.1.2 Preprocessing XML files + +Multiple scripts are executed at build time to preprocess special XML tags/attributes - MACRO substitution, adding pipe processing, etc. - and generate target XML files in a form that is consumable by Klish. + +The XML files are also validated as part of compilation. `xmllint` is used to validate all the processed XML files after macro substitution and pipe processing against the detailed schema defined in `sonic-clish.xsd`. Once the XML files are fully validated and preprocessed, the target XML files are generated in the folder `${workspace}/sonic-mgmt-framework/build/cli/target/command-tree`. + +The following preprocessing scripts are introduced: +* `klish_ins_def_cmd.py` - append the "exit" and "end" commands to the views of the Klish XML files +* `klish_insert_pipe.py` - extend every show and get COMMAND with pipe option +* `klish_platform_features_process.sh` - validate all platform XML files. Generate the entity.XML files. +* `klish_replace_macro.py` – perform macro substitution on the Klish XML files + +###### 3.2.2.1.3 MACROs + +There are some CLI commands that can have the same set of options, where the set of XML tags would need to be repeated in all those CLI commands. Macros can be used to avoid such repetitions and to keep the options in one place, so that it is possible to make a reference to a macro in multiple command definitions. There are cases where we may need to use variations in the values of these macro options. In such cases, it is also possible to pass those XML attributes as an argument to macros and substitute those values inside the macro definition. The macro definition is referred as the ```` and ```` tags. `klish_replace_macro.py` is used to process macros at the compile time to expand the references to the target XML files. +The macro definition files are located in the folder `${workspace}/sonic-mgmt-framework/src/CLI/clitree/macro`. + +Example: + +Before macro substitution: + +```XML + + + + + ... + +``` + +After macro substitution: + +```XML + + + + + + + + + ... + +``` + +###### 3.2.2.1.4 ENTITY +XML files can include an ENTITY that refers to a predefined value. Entity is typically used to define platform specific values and processed by `klish_platform_features_process.sh` to prepend the ENTITY values to the target XML files. By default, there is a default file called `platform_dummy.XML` that defines a platform default ENTITY list. Note that the platform specific is not supported yet. + +Example: `platform_dummy.XML` + +```XML + + START_PORT_ID + MAX_PORT_IDma + START_SUB_PORT_ID + MAX_SUB_PORT_ID + MAX_MTU + +``` + +The ENTITY name can be referenced in command definitions. For example, PTYPE for RANGE_MTU, used for interface commands: + +```XML + +``` + +###### 3.2.2.1.5 Actioner + +The Actioner is used to convert CLI commands to the corresponding OpenAPI client requests on the target resources. It is defined with the `` tag in the XML file. There are three different methods available in the Actioner: sub-shell, builtin clish_restcl and clish_pyobj. CLI builtin functions "clish_pyobj" and "clish_restcl" can be applied to reduce time take to execute a command by eliminating the sub-shell interpreter overhead. + +1) Spawn a sub-shell to interpret Python scripts +Klish spawns a sub-shell to interpret the Python scripts defined in a command's `` tag. +The sub-shell runs the wrapper script `sonic_cli_.py` +``` + sonic_cli_.py [parameters . . .] +``` +The `sonic_cli_.py` has a dispatch function to call a OpenAPI client method with parameters passed from user input. + +Example: +```XML + + + + + + if test "${direction-switch}" = "in"; then + Python $SONIC_CLI_ROOT/target/sonic-cli.py post_list_base_interfaces_interface ${access-list-name} ACL_IPV4 ${iface} ingress + else + Python $SONIC_CLI_ROOT/target/sonic-cli.py post_list_base_interfaces_interface ${access-list-name} ACL_IPV4 ${iface} egress + fi + + + ... +``` + +2) Invoke the built-in function, clish_restcl, to use libcurl to make REST client call +This builtin uses libcurl to connect to the REST server +Format of ACTION tag argument: +oper= url= body={..} +oper can be PUT/POST/PATCH/DELETE +body is optional. +oper GET is not supported as we currently don't handle rendering using jinja templates. +Example: +``` +oper=PATCH url=/restconf/data/openconfig-interfaces:interfaces/interface=Vlan${vlan-id}/config body={"openconfig-interfaces:config": {"name": "Vlan${vlan-id}"}} +``` + +3) Invoke the built-in function, clish_pyobj, to use embedding Python to make REST client call +This builtin uses embedded python approach. + +Format of ACTION tag argument: + +( is the actioner script without the extension. The name cannot include "-") + +Example: +``` +sonic_cli_if patch_openconfig_interfaces_interfaces_interface_config Vlan${vlan-id} +``` + +###### 3.2.2.1.6 Renderer scripts + +The actioner script receives the JSON output from the OpenAPI client API and invokes the renderer script. The renderer script will send the JSON response to the jinja2 template file to parse the response and generate the CLI output. + +Example: "show acl" + +``` +{% set acl_sets = acl_out['openconfig_aclacl']['acl_sets']['acl_set'] %} + {% for acl_set in acl_sets %} + Name: {{ acl_set['state']['description'] }} + {% endfor %} +``` + +###### 3.2.2.1.7 Workflow (to add a new CLI) + +The following steps are to be followed when a new CLI is to be added. +1. Create an XML file that defines CLI command and parameters that the command requires. +2. Define the CLI help string to be displayed and datatype for the parameters. New parameter types (PTYPES), macros, and entities can be defined and used in the XML files. Valid XML tags are defined in the `sonic-clish.xsd` file. +3. Add the actioner to `` tag to run the wrapper script with the OpenAPI client method name and parameters +4. Add the code to the wrapper script to construct the payload in `generate_body()` and handle the response +5. For ‘show’ commands, create a Jinja template to format the output + + +##### 3.2.2.2 REST Client SDK + +Framework provides OpenAPI-codegen generated Python client SDK. Developers can generate client SDK code in other programming languages from the OpenAPI definitions as required. + +Client applications can use OpenAPI generated client SDK or any other REST client tool to communicate with the REST server. + +##### 3.2.2.3 gNMI Client + +SONiC Telemetry service provides the gNMI server, while the client must be provided by the user. gNMI is typically used as a programmatic interface and therefore it is typically called directly from programming language environments using their respective gRPC libraries (https://github.com/grpc/grpc). For testing and scripting purposes, several CLI programs are provided as well. + +GNMI clients developed by google and modified by JipanYANG.(github.com/jipanYANG/gnxi/gnmi_get, github.com/jipanYANG/gnxi/gnmi_set) +was taken and further modified to support new features. gnmi_cli from openconfig (https://github.com/openconfig/gnmi/tree/master/cmd/gnmi_cli) is also taken for testing subscribe and capabilities operations. Finally, a new client was developed gnoi_client, for test gNOI RPC operations. + +Note: Although the gRPC protocol allows for many encodings to be used, our usage is restricted to JSON_IETF encoding. + +Supported RPC Operations: +------------------------- +- Get: Get one or more paths and have value(s) returned in a GetResponse. +- Set: Update, replace or delete objects + + Update: List of one or more objects to update + + Replace: List of one or objects to replace existing objects, any unspecified fields wil be defaulted. + + Delete: List of one or more object paths to delete +- Capabilities: Return gNMI version and list of supported models and model versions. + + A gNMI Extension field has been added as well to return the SONiC model "bundle version". +- Subscribe: + + Subscribe to paths using either streaming or poll, or once based subscription, with either full current state or updated values only. + * Once: Get single subscription message. + * Poll: Get one subscription message for each poll request from the client. + * Stream: Get one subscription message for each object update (called ON_CHANGE mode), or at each sample interval if using sample mode. target_defined uses the values pre-configured for that particular object. Not all paths support ON_CHANGE mode due to performance considerations, while all paths will support sample mode and therefore target defined as well since it will default to sample if ON_CHANGE is not supported. + + +Example Client Operations: +-------------------------- +Using opensource clients, these are example client operations. The .json test payload files are available here: https://github.com/project-arlo/sonic-mgmt-common/tree/master/src/Translib/test + +Get: +---- +`./gnmi_get -xpath /openconfig-acl:acl/interfaces -target_addr 127.0.0.1:8080 -alsologtostderr -insecure true -pretty` + +Set: +---- +Replace: +-------- + `./gnmi_set -replace /openconfig-acl:acl/:@./test/01_create_MyACL1_MyACL2.json -target_addr 127.0.0.1:8080 -alsologtostderr -insecure true -pretty` +Delete: +------- + `./gnmi_set -delete /openconfig-acl:acl/ -target_addr 127.0.0.1:8080 -insecure` + +Subscribe: +---------- +Streaming sample based: +----------------------- +`./gnmi_cli -insecure -logtostderr -address 127.0.0.1:8080 -query_type s -streaming_sample_interval 3 -streaming_type SAMPLE -q /openconfig-acl:acl/ -v 0 -target YANG` + +Poll based: +----------- +`./gnmi_cli -insecure -logtostderr -address 127.0.0.1:8080 -query_type p -polling_interval 1s -count 5 -q /openconfig-acl:acl/ -v 0 -target YANG` + +Once based: +----------- +`./gnmi_cli -insecure -logtostderr -address 127.0.0.1:8080 -query_type o -q /openconfig-acl:acl/ -v 0 -target YANG` + + +gNOI: +-------------- +gNOI (gRPC Network Operations Interface) extends the gNMI server, adding new custom RPC's to execute management functions on the switch. + +gNOI Clear Neighbor: +------------------- +``` +gnoi_client -module Sonic -rpc clearNeighbors -jsonin '{"sonic-neighbor:input": {"force": true, "ip": "4.4.4.1"}}' -insecure +``` + +##### 3.2.2.4 REST Server + +The management REST server is a HTTP server implemented in Go language. +It supports following operations: + +* RESTCONF APIs for YANG data +* REST APIs for manual OpenAPI definitions + +###### 3.2.2.4.1 Transport options + +REST server supports only HTTPS transport and listens on default port 443. +Server port can be changed through an entry in ConfigDB REST_SERVER table. + +By default a temporary self signed certificate is used as TLS server certificate. +It can be overridden by specifiying a valid TLS private key and certificate file +paths through the REST_SERVER table. + +REST_SERVER table schema is described in [DB Schema](#322414-db-schema) section. + +###### 3.2.2.4.2 Translib linking + +REST server will statically link with Translib. For each REST request, the server +invokes Translib API which then invokes appropriate App module to process the request. +Below is the mapping of HTTP operations to Translib APIs: + + HTTP Method | Translib API | Request data | Response data +-------------|------------------|---------------|--------------- + GET | translib.Get | path | status, payload + POST | translib.Create | path, payload | status + POST (for YANG RPC) | translib.Action | path, payload | status, payload + PATCH | translib.Update | path, payload | status + PUT | translib.Replace | path, payload | status + DELETE | translib.Delete | path | status + HEAD | translib.Get | path | status, (payload ignored) + OPTIONS | - | - | - + +More details about Translib APIs are in section [3.2.2.6](#3_2_2_6-Translib). + +REST OPTIONS requests do not invoke any Translib API. They return list of supported HTTP methods +for requested path in the "Allow" response header. If PATCH method was supported, the response +will also include an "Accept-Patch" header with the value "application/yang-data+json". + +###### 3.2.2.4.3 Media Types + +YANG defined RESTCONF APIs support **application/yang-data+json** media type. +Request and response payloads follow [RFC7951](https://tools.ietf.org/html/rfc7951) +defined encoding rules. Media type **application/yang-data+xml** is not supported +in first release. + +OpenAPI defined REST APIs can use any media type depending on App module +implementation. However content type negotiation is not supported by REST server. +A REST API should not be designed to consume or produce multiuple content types. +OpenAPI definition for each REST API should have maximun one cone media type in +its "consumes" and "produces" statements. + +###### 3.2.2.4.4 Payload Validations + +REST server does not validate request payload for YANG defined RESTCONF APIs. +Payload will be validated automatically in lower layers when it gets loaded +into YGOT bindings. + +For OpenAPI defined REST APIs the REST server will provide limited payload +validation. JSON request payloads (content type **application/json**) will be +validated against the schema defined in OpenAPI. Response data and non-JSON +request data are not validated by the REST server - this is left to the App module. + +###### 3.2.2.4.5 Concurrency + +REST server will accept concurrent requests. Translib provides appropriate locking mechanism - parallel reads and sequential writes. + +###### 3.2.2.4.6 API Versioning + +REST server will allow clients to specify API version through a custom HTTP header **Accept-Version**. + + Accept-Version: 1.0.3 + +Version text should be in **MAJOR.MINOR.PATCH** format. REST server will extract version text from +the request header and pass it to the Translib API as **ClientVersion** argument. +Section [3.2.2.6.4.2](#322642-version-checks) explains Translib version checking logic. + +Version checks are bypassed if Accept-Version header is not present in the request. + +For YANG defined RESTCONF APIs, the server's version can be discovered through the standard YANG +module library API "GET /restconf/data/ietf-yang-library:modules-state/module-set-id". +Client should not pass Accept-Version header for this API! This is safe since module-set-id +is a standard API defined by [RFC7895](https://tools.ietf.org/html/rfc7895) for this very purpose. + +API Versioning is not supported for manual OpenAPI definitions in this release. +Accept-Version header value will be ignored even if specified. + +###### 3.2.2.4.7 RESTCONF Entity-tag + +REST server does not maintain entity-tag and last-modified timestamp for configuration data nodes. +GET and HEAD responses does not include "ETag" and "Last-Modified" headers. + +[RFC7232](https://tools.ietf.org/html/rfc7232) style HTTP conditional requests is also not supported. +REST server ignores "If-Match", "If-Modified-Since" like conditional request headers. + +###### 3.2.2.4.8 RESTCONF Discovery + +REST server supports following APIs for clients to discover various RESTCONF protocol features +as described in [RFC8040](https://tools.ietf.org/html/rfc8040). + +Method | Path | Purpose +-------|------------------------------------------------|--------------------------- +GET | /.well-known/host-meta | RESTCONF root path +GET | /restconf/yang-library-version | YANG library version +GET | /restconf/data/ietf-yang-library:modules-state | RFC7895 YANG module library +GET | /restconf/data/ietf-restconf-monitoring:restconf-state/capabilities | RESTCONF Capabilities +GET | /models/yang/{filename} | YANG download + +###### 3.2.2.4.8.1 RESTCONF root + +Server supports RESTCONF root path discovery through "GET /.well-known/host-meta" API +as described in [RFC8040, section 3.1](https://tools.ietf.org/html/rfc8040#page-18). +RESTCONF root path is "/restconf". + +###### 3.2.2.4.8.2 Yang library version + +REST server supports "GET /restconf/yang-library-version" API to advertise YANG library version. +Response will indicate version "2016-06-21" as described in [RFC8040, section 3.3.3](https://tools.ietf.org/html/rfc8040#section-3.3.3). +This advertises that the server supports [RFC7895](https://tools.ietf.org/html/rfc7895) compliant +YANG library operations. + +###### 3.2.2.4.8.3 YANG module library + +REST server allows clients to discover and download all YANG modules supported by the server +via "GET /restconf/data/ietf-yang-library:modules-state" API. Response data includes YANG module +information as per [RFC7895](https://tools.ietf.org/html/rfc7895) requirements. + +REST server allows clients to download the YANG files via "GET /models/yang/{filename}" API. +YANG module library response includes full download URL for every YANG module entry. + +Note - RFC7895 has been recently obsoleted by [RFC8525](https://tools.ietf.org/html/rfc8525). +It supports YANG library with multi data stores and data store schemas. However these features +are not available in SONiC. Hence REST server does not implement RFC8525 YANG library APIs (for now). + +###### 3.2.2.4.8.4 RESTCONF capabilities + +REST server supports "GET /restconf/data/ietf-restconf-monitoring:restconf-state/capabilities" API to +advertise its capabilities as described in [RFC8040, section 9.1](https://tools.ietf.org/html/rfc8040#section-9.1). +REsponse includes below mentioned capability information. + + urn:ietf:params:restconf:capability:defaults:1.0?basic-mode=report-all + +###### 3.2.2.4.9 RESTCONF Query Parameters + +RESTCONF Query Parameters will be supported in future release. All query parameters will be ignored by REST server in this release. + +###### 3.2.2.4.10 RESTCONF Operations + +REST server supports invoking YANG RPCs through "POST /restconf/operations/{rpc_name}" API +as described in [RFC8040, section 4.4.2](https://tools.ietf.org/html/rfc8040#section-4.4.2). +Operations modeled through YANG v1.1 action statement are not supported in this release. + +YGOT binding objects are not available for YANG RPC input and output data model. +Hence payload validation is not performed by REST server or Translib for these APIs. +App modules will receive raw JSON data. + +###### 3.2.2.4.11 RESTCONF Notifications + +RESTCONF Notification are not supported by framework. Clients can use gNMI for monitoring and notifications. + +###### 3.2.2.4.12 Authentication + +REST server supporsts following authentication modes. + +* HTTP Basic authentication (username/password authentication) +* HTTP Bearer token authentication with JSON Web Token (JWT) format. +* TLS Certificate authentication +* Any combination of above 3 modes +* No authentication + +Details are in [SONiC RBAC HLD](https://github.com/project-arlo/SONiC/blob/master/doc/aaa/SONiC%20RBAC%20HLD.md). + +By default HTTP Baisc and bearer token authentication modes are enabled. +It can be overridden through ConfigDB [REST_SERVER table](#322414-db-schema) entry. + +###### 3.2.2.4.13 Error Response + +REST server sends back HTTP client error (4xx) or server error (5xx) status when request processing +fails. Response status and payload will be as per RESTCONF specifications - [RCF8040, section7](https://tools.ietf.org/html/rfc8040#page-73). +Error response data will be a JSON with below structure. Response Content-Type will be +"application/yang-data+json". + + +---- errors + +---- error* + +---- error-type "protocol" or "application" + +---- error-tag string + +---- error-app-tag? string + +---- error-path? xpath + +---- error-message? string + +Note: REST server will not populate error-app-tag and error-path fields in this release. It can be +enhanced in a future release. A sample error response: + + { + "ietf-restconf:errors" : { + "error" : [ + { + "error-type" : "application", + "error-tag" : "invalid-value", + "error-message" : "VLAN 100 not found" + } + ] + } + } + +**error-type** can be either "protocol" or "application", indicating the origin of the error. +RESTCONF defines two more error-type enums "transport" and "rpc"; they are not used by REST server. + +**error-tag** indicates nature of error as described in [RFC8040, section 7](https://tools.ietf.org/html/rfc8040#page-74). + +**error-message** field carries a human friendly error message that can be displayed to the end +user. This is an optional field; system error do not include error-message, or have generic +messages like "Internal error". App module developer should use human friendly messages while +returning application errors. In case of CVL constraint violation the REST server will pick +the error message from the YANG "error-message" statement of CVL schema YANG. + +Table below lists possible error conditions with response status and data returned by REST server. + +Method | Error condition | Status | error-type | error-tag | error-message +--------|--------------------------|--------|-------------|------------------|---------------------- +*any* | Incorrect request data | 400 | protocol | invalid-value | +*write* | Bad content-type | 415 | protocol | invalid-value | Unsupported content-type +*write* | OpenAPI schema validation fails | 400 | protocol| invalid-value | Content not as per schema +*write* | YGOT schema validation fails | 400 | protocol| invalid-value | *YGOT returned message* +*any* | Invalid user credentials | 401 | protocol | access-denied | Authentication failed +*write* | User is not an admin | 403 | protocol | access-denied | Authorization failed +*write* | Translib commit failure | 409 | protocol | in-use | +*any* | Bad Accept-Version value | 400 | protocol | invalid-value | Invalid Accept-Version +*any* | Version ckeck fails | 400 | protocol | operation-not-supported | Unsupported client version *X.Y.Z* +*any* | Unknown HTTP server failure | 500 | protocol | operation-failed | Internal error +*any* | Not supported by App module | 405 | application | operation-not-supported | *App module returned message* +*any* | Incorrect payload | 400 | application | invalid-value | *App module returned message* +*any* | Resource not found | 404 | application | invalid-value | *App module returned message* +POST | Resource exists | 409 | application | resource-denied | *App module returned message* +*any* | Unknown error in Translib | 500 | application | operation-failed | Internal error +*any* | Unknown App module failure | 500 | application | operation-failed | *App module returned message* +*any* | CVL constraint failure | 500 | application | invalid-value | *error-message defined in SONiC YANG* + + +###### 3.2.2.4.14 DB Schema + +A new table "REST_SERVER" is introduced in ConfigDB for maintaining REST server configurations. + + key = REST_SERVER:default ; REST server configurations. + ;field = value + port = 1*5DIGIT ; server port - defaults to 443 + client_auth = "none"/"password"/"jwt"/"cert" + ; Client authentication mode. + ; none: No authentication, all clients + ; are allowed. Should be used only + ; for debugging. + ; password: HTTP Basic authentication. + ; jwt : HTTP Bearer Token authentication with + ; JSON Web Token format. + ; cert: Certificate based authentication. + ; Requires REST_SERVER['certs']['ca_crt'] configuration. + ; Any combination of "password", "jwt" and "cert" modes can be + ; enabled by specifying a comma separated values. + ; Eg: "password,jwt" enables both password and jwt modes. + log_level = DIGIT ; Verbosity for glog.V logs + + + key = REST_SERVER:certs ; Server certificate configurations + ;field = value + server_crt = STRING ; Path to TLS certificate file + server_key = STRING ; Path to TLS private key file + ca_crt = STRING ; Path to the CA certificate to be used for + ; client certificate validation. + +###### 3.2.2.4.15 API Documentation + +REST server will provide [OpenAPI UI](https://github.com/OpenAPI-api/OpenAPI-ui) based online +documentation and test UI for all REST APIs it supports. Documentation can be accessed by launching +URL **https://REST_SERVER_IP/ui** in a browser. This page will list all supported OpenAPI +definition files (both YANG generated and manual) along with link to open OpenAPI UI for them. + + +##### 3.2.2.5 gNMI server + +1. gNMI server is part of the telemetry process that supports telemtry as well as gNMI. +2. The gRPC server opens a TCP port and allows only valid mutually authenticated TLS connections, which requires valid Client, server and CA Certificates be installed as well a properly configured DNS. Multiple simultaneous connections are allowed to gNMI server. +3. The gNMI Agent uses the db client, as well as the non-db client to access and modify data directly in the Redis DB. +4. The Translib client is used to provide alternative models of access such as Openconfig models as opposed to the native Redis schema, as long as the Translib supports these models. Translib offers bidirectional translation between the native Redis model and the desired north bound model, as well as notifications/updates on these model objects to support telemetry and asynchronous updates, alarms and events. Translib should also provide information about what models it supports so that information can be returned in gNMI Capabilities response. +5. The gNMI server defines the four RPC functions as required by the gNMI Specification: Get, Set, Capabilities and Subscribe. +6. Since the db, non-db and Translib clients offer the functionality to support these functions, gNMI only has to translate the paths and object payloads into the correct parameters for the client calls and package the results back into the response gNMI objects to return to the gNMI Client, which is a straightforward operation, since no additional processing of the data is expected to be done in the gNMI server itself. When new models are added to Translib, no additional work should be required to support them in gNMI server. +7. All operations in a Set request are processed in a single transaction that will either succeed or fail as one operation. The translib client supports a Bulk operation in order to achieve the transactional behavior of multiple set operations. The order in which multiple set operations occures in a single set request is defined in the [gNMI Specification](https://github.com/openconfig/reference/blob/master/rpc/gnmi/gnmi-specification.md#34-modifying-state). Care must be taking when doing bulk operations that the objects being operated on are created before performing further operations on them. Otherwise an error may be returned. +8. Subscribe operations: Once, Poll and Stream require that the gRPC connection remain open until the subscription is completed. This means many connections must be supported. Subscribe offers several options, such as only sending object updates (not the whole object) which requires support form the db clients. Subscribe also allows for periodic sampling defined by the client. This must be handled in the gNMI agent itself. This requires a timer for each subscribe connection of this type in order to periodically poll the db client and return the result in a Subscribe Response. These timers should be destroyed when the subscription gRPC connection is closed. + +###### 3.2.2.5.1 Files changed and added: + + |-- gnmi_server + | |-- client_subscribe.go + | |-- server.go ------------------- MODIFIED (Handles creation of transl_data_client for GET/SET/CAPABILITY) + | |-- server_test.go + |-- sonic_data_client + | |-- db_client.go ---------------- MODIFIED (Common interface Stub code for new functions as all data clients implement common interface functions) + | |-- non_db_client.go ------------ MODIFIED (Common interface Stub code for new functions as all data clients implement common interface functions) + | |-- transl_data_client.go ------- ADDED (Specific processing for GET/SET/CAPABILITY for transl data clients) + | |-- trie.go + | |-- virtual_db.go + | + |-- transl_utils -------------------- ADDED + |-- transl_utils.go ------------- ADDED (Layer for invoking Translib APIs) + +###### 3.2.2.5.2 Sample Requests + +go run gnmi_get.go -xpath /openconfig-acl:acl/acl-sets/acl-set[name=MyACL4][type=ACL_IPV4]/acl-entries/acl-entry[sequence-id=1] -target_addr 10.130.84.34:8081 -alsologtostderr -insecure true -pretty + +go run gnmi_set.go -replace /openconfig-acl:acl/acl-sets/acl-set[name=MyACL4][type=ACL_IPV4]/acl-entries/acl-entry=2/actions/config:@openconfig.JSON -target_addr 10.130.84.34:8081 -alsologtostderr -insecure true -pretty + +go run gnmi_capabilities.go -target_addr 10.130.84.34:8081 -alsologtostderr -insecure true -pretty + +###### 3.2.2.5.3 Authentication + +gNMI Server supports four modes of authentication: + + - Basic Authentication - Requires passing of username and password in the gRPC metadata via the username and password keys. + - JSON Web Tokens (JWT) - Requires initial authentication via either basic or certificate authentication using the gNOI Authenticate RPC, afterwhich a token is recieved that can be used with future requests to avoid further authentication. The JWT token is sent in the metadata of the gRPC requests with the access_token key. + - Certificate - A valid client certificate is used with the username embedded in the certificate CN field. This allows the requests to be authenticaed against the CA certificate and the username can be used for authorization. + +By default Basic and JWT authentication modes are enabled. Details are in [SONiC RBAC HLD](https://github.com/project-arlo/SONiC/blob/master/doc/aaa/SONiC%20RBAC%20HLD.md). + +###### 3.2.2.5.4 Error Responses + +gNMI Server sends back gRPC error codes along with error messages in the gRPC response messages. + +A full list of possible response codes can be found [here](https://godoc.org/google.golang.org/grpc/codes) + +###### 3.2.2.5.5 DB Schema + +gNMI Server already has a ConfigDB table "TELEMETRY|gnmi" but some keys have been added or modified. + +``` +key = TELEMETRY|gnmi ; gNMI Server configurations +port = 1*5DIGIT ; server port - defaults to 8080 +client_auth = "none" / "password" / "jwt" / "cert" + ; Client authentication mode. + ; none: No authentication, all clients + ; are allowed. Should be used only + ; for debugging. + ; password: HTTP Basic authentication. + ; jwt : HTTP Bearer Token authentication with + ; JSON Web Token format. + ; cert: Certificate based authentication. + ; Requires ca_crt configuration. + ; Any combination of "password", "jwt" and "cert" modes can be + ; enabled by specifying a comma separated values. + ; Eg: "password,jwt" enables both password and jwt modes. +jwt_refresh = 1-jwt_valid + ; The number of seconds before the JWT ; token expires that you can refresh the ; token. - defaults to 900 seconds. +jwt_valid = 1-2^64 + ; Number of seconds the JWT tokens are ; valid. - defaults to 3600 + +``` + +###### 3.2.2.5.6 Transport options + +gNMI server supports only TCP transport with TLS enabled and listens on default port 8080. +Server port can be changed through an entry in ConfigDB TELEMETRY table. + +By default a temporary self signed certificate is used as TLS server certificate. +It can be overridden by specifiying a valid TLS private key and certificate file paths through the TELEMETRY table. + +###### 3.2.2.5.7 Translib linking + +gNMI server will statically link with Translib. For each gNMI request, the server +invokes Translib API which then invokes appropriate App module to process the request. +Below is the mapping of gRPC operations to Translib APIs: + + gRPC Method | Translib API | Request data | Response data +-------------|-------------------|---------------|--------------- + GET | translib.Get | path | status, payload + SET/UPDATE | translib.Update | path, payload | status + SET/REPLACE | translib.Replace | path, payload | status + SET/DELETE | translib.Delete | path | status + (ANY OC-RPC)| translib.Action | RPC,payload | status, payload + SUBSCRIBE | translib.Subscribe| path | status, payload + CAPABILITIES| translib.GetModels| | models, versions, encodings + + +More details about Translib APIs are in section [3.2.2.6](#3_2_2_6-Translib). + + +###### 3.2.2.5.8 Encodings + +gNMI Currently only supports JSON_IETF type encoding in the same was as REST server since translib nativly uses this format. + +###### 3.2.2.5.9 Payload Validations + +gNMI server does not validate request payload. Payload will be validated automatically in lower layers when it gets loaded +into YGOT bindings. + +###### 3.2.2.5.10 Concurrency + +gNMI server will accept concurrent requests. Translib provides appropriate locking mechanism - parallel reads and sequential writes. + +###### 3.2.2.5.11 API Versioning + +gNMI server will allow clients to specify API version through a custom gNMI Extension field BundleVersion with extentsion id 700. + + BundleVersion: 1.0.3 + +Version text should be in **MAJOR.MINOR.PATCH** format. gNMI server will extract version text from +the gNMI request protobuf object and pass it to the Translib API as **ClientVersion** argument. +Section [3.2.2.6.4.2](#322642-version-checks) explains Translib version checking logic. + +Version checks are bypassed if BundleVersion header is not present in the request. + +For YANG defined APIs, the server's version can be discovered through the standard Capabilities gNMI RPC. The capabilities response will also contain an extension field SupportedBundleVersions with extension id 701. + +The gNMI protobuf extension fields are defined as follows in the sonic.proto file in the telemetry repository: + + message BundleVersion { + string version = 1; + } + message SupportedBundleVersions { + string bundle_version = 1; + string base_version = 2; + } + + + +###### 3.2.2.5.12 gNOI RPCs + +Along with gNMI RPCS (GET,SET,SUBSCRIBE,CAPABILITIES), custom RPCs are exposed on the [gNOI](https://github.com/openconfig/gnoi) server on the same port as gNMI server. + +gNOI Allows the gNMI server to add custom RPC operations. gNOI Provides a number of standardized RPCs, however currently on the the system/Time RPC is available from those RPCs. + +OpenConfig and Sonic customer RPCs are also available via gNOI via the sonic_gnoi.proto file in the telemetry repository. This file provides the protobuf definition for calling these RPCs as well as the format of their arguments and their responses. + +Also, the JWT authentication method uses two custom RPCs called Authenticate and Refresh to manage the JWT tokens. These RPCs are available in the sonic_gnoi_jwt.proto file in the telemetry repository. + +For gNOI client usage examples see [3.2.2.3](#3223-gnmi-client) + + +##### 3.2.2.6 Translib + +Translib is a library that adapts management server requests to SONiC data providers and vice versa. Translib exposes the following APIs for the management servers to consume. + + func Create(req SetRequest) (SetResponse, error) + func Update(req SetRequest) (SetResponse, error) + func Replace(req SetRequest) (SetResponse, error) + func Delete(req SetRequest) (SetResponse, error) + func Get(req GetRequest) (GetResponse, error) + func Action(req ActionRequest) (ActionResponse, error) + func Bulk(req BulkRequest) (BulkResponse, error) + func Subscribe(req SubscribeRequest) ([]*IsSubscribeResponse, error) + func IsSubscribeSupported(req IsSubscribeRequest) ([]*IsSubscribeResponse, error) + func GetModels() ([]ModelData, error) + + Translib Structures: + type ErrSource int + + const ( + ProtoErr ErrSource = iota + AppErr + ) + + type SetRequest struct { + Path string + Payload []byte + User string + ClientVersion Version + } + + type SetResponse struct { + ErrSrc ErrSource + Err error + } + + type GetRequest struct { + Path string + User string + ClientVersion Version + } + + type GetResponse struct { + Payload []byte + ErrSrc ErrSource + } + + type ActionRequest struct { + Path string + Payload []byte + User string + ClientVersion Version + } + + type ActionResponse struct { + Payload []byte + ErrSrc ErrSource + } + + type BulkRequest struct { + DeleteRequest []SetRequest + ReplaceRequest []SetRequest + UpdateRequest []SetRequest + CreateRequest []SetRequest + User string + ClientVersion Version + } + + type BulkResponse struct { + DeleteResponse []SetResponse + ReplaceResponse []SetResponse + UpdateResponse []SetResponse + CreateResponse []SetResponse + } + + type SubscribeRequest struct { + Paths []string + Q *queue.PriorityQueue + Stop chan struct{} + User string + ClientVersion Version + } + + type SubscribeResponse struct { + Path string + Payload []byte + Timestamp int64 + SyncComplete bool + IsTerminated bool + } + + type NotificationType int + + const ( + Sample NotificationType = iota + OnChange + ) + + type IsSubscribeRequest struct { + Paths []string + User string + } + + type IsSubscribeResponse struct { + Path string + IsOnChangeSupported bool + MinInterval int + Err error + PreferredType NotificationType + } + + type ModelData struct { + Name string + Org string + Ver string + } + + type Version struct { + Major uint32 + Minor uint32 + Patch uint32 + } + +Translib has the following sub modules to help in the translation of data + +1. App Interface +2. Translib Request Handlers +3. YGOT request binder +4. DB access layer +5. App Modules +6. Transformer + +###### 3.2.2.6.1 App Interface + +App Interface helps in identifying the App module responsible for servicing the incoming request. It provides the following APIs for the App modules to register themselves with the App interface during the initialization of the app modules. + + func Register(path string, appInfo *AppInfo) error + This method can be used by any App module to register itself with the Translib infra. + Input Parameters: + path - base path of the model that this App module services + appInfo - This contains the reflect types of the App module structure that needs to be instantiated for each request, corresponding YGOT structure reflect type to instantiate the corresponding YGOT structure and boolean indicating if this is native App module to differentiate between OpenAPI spec servicing App module and the YANG serving App module. + Returns: + error - error string + func AddModel(model *gnmi.ModelData) error + This method can be used to register the models that the App module supports with the Translib infra. + Input Parameters: + model - Filled ModelData structure containing the Name, Organisation and version of the model that is being supported. + Returns: + error - error string + + App Interface Structures: + //Structure containing App module information + type AppInfo struct { + AppType reflect.Type + YGOTRootType reflect.Type + IsNative bool + tablesToWatch []*db.TableSpec + } + + Example Usages: + func init () { + log.Info("Init called for ACL module") + err := register("/openconfig-acl:acl", + &appInfo{appType: reflect.TypeOf(AclApp{}), + ygotRootType: reflect.TypeOf(ocbinds.OpenconfigAcl_Acl{}), + isNative: false, + tablesToWatch: []*db.TableSpec{&db.TableSpec{Name: ACL_TABLE}, &db.TableSpec{Name: RULE_TABLE}}}) + + if err != nil { + log.Fatal("Register ACL App module with App Interface failed with error=", err) + } + + err = appinterface.AddModel(&gnmi.ModelData{Name:"openconfig-acl", + Organization:"OpenConfig working group", + Version:"1.0.2"}) + if err != nil { + log.Fatal("Adding model data to appinterface failed with error=", err) + } + } + + type AclApp struct { + path string + YGOTRoot *YGOT.GoStruct + YGOTTarget *interface{} + } + +Translib request handlers use the App interface to get all the App module information depending on the incoming path as part of the requests. + +###### 3.2.2.6.2 Translib Request Handler + +These are the handlers for the APIs exposed by the Translib. Whenever a request lands in the request handler, the handler uses the App interface to get the App module that can process the request based on the incoming path. It then uses the YGOT binder module, if needed, to convert the incoming path and payload from the request into YGOT structures. The filled YGOT structures are given to the App Modules for processing. The Translib also interacts with the DB access layer to start, commit and abort a transaction. + +###### 3.2.2.6.3 YGOT request binder + +The YGOT request binder module uses the YGOT tools to perform the unmarshalling and validation. YGOT (YANG Go Tools) is an open source tool and it has collection of Go utilities which are used to + + 1. Generate a set of Go structures for bindings for the given YANG modules at build time + 2. Unmarshall the given request into the Go structure objects. These objects follows the same hierarchical structure defined in the YANG model, and it's simply a data instance tree of the given request, but represented using the generated Go structures + 3. Validate the contents of the Go structures against the YANG model (e.g., validating range and regular expression constraints). + 4. Render the Go structure objects to an output format - such as JSON. + +This RequestBinder module exposes the below mentioned APIs which will be used to unmarshall the request into Go structure objects, and validate the request + + func getRequestBinder(uri *string, payload *[]byte, opcode int, appRootNodeType *reflect.Type) *requestBinder + This method is used to create the requestBinder object which keeps the given request information such as uri, payload, App module root type, and unmarshall the same into object bindings + Input parameters: + uri - path of the target object in the request. + payload - payload content of given the request and the type is byte array + opcode - type of the operation (CREATE, DELETE, UPDATE, REPLACE) of the given request, and the type is enum + appRootNodeType - pointer to the reflect.Type object of the App module root node's YGOT structure object + Returns: + requestBinder - pointer to the requestBinder object instance + + func (binder *requestBinder) unMarshall() (*YGOT.GoStruct, *interface{}, error) + This method is be used to unmarshall the request into Go structure objects, and validates the request against YANG model schema + Returns: + YGOT.GoStruct - root Go structure object of type Device. + interface{} - pointer to the interface type of the Go structure object instance of the given target path + error - error object to describe the error if the unmarshalling fails, otherwise nil + +Utilities methods: +These utilities methods provides below mentioned common operations on the YGOT structure which are needed by the App module + + func getParentNode(targetUri *string, deviceObj *ocbinds.Device) (*interface{}, *YANG.Entry, error) + This method is used to get parent object of the given target object's uri path + Input parameters: + targetUri - path of the target URI + deviceObj - pointer to the base root object Device + Returns + interface{} - pointer to the parent object of the given target object's URI path + YANG.Entry - pointer to the YANG schema of the parent object + error - error object to describe the error if this methods fails to return the parent object, otherwise nil + + func getNodeName(targetUri *string, deviceObj *ocbinds.Device) (string, error) + This method is used to get the YANG node name of the given target object's uri path. + Input parameters: + targetUri - path of the target URI + deviceObj - pointer to the base root object Device + Returns: + string - YANG node name of the given target object + error - error object to describe the error if this methods fails to return the parent object, otherwise nil + + func getObjectFieldName(targetUri *string, deviceObj *ocbinds.Device, YGOTTarget *interface{}) (string, error) + This method is used to get the go structure object field name of the given target object. + Input parameters: + targetUri - path of the target URI + deviceObj - pointer to the base root object Device + YGOTTarget - pointer to the interface type of the target object. + Returns: + string - object field name of the given target object + error - error object to describe the error if this methods fails to perform the desired operation, otherwise nil + +###### 3.2.2.6.4 YANG Model Versioning + +###### 3.2.2.6.4.1 Version Management + +Translib maintains a "YANG bundle version" which is the collective version number for all the YANG +modules deployed in the Management Framework Service. This version is maintained in a configuration +file. It uses **MAJOR.MINOR.PATCH** syntax as per [Sematic Versioning](https://semver.org) specification. +Developer will have to update the bundle version number when he makes any YANG change. + +**Major version** is fixed to 1 in this release. +It will be incremented in future releases when any YANG model is changed in a non backward +compatible manner. Following are the candidates: + +* Delete, rename or relocate data node +* Change list key attributes +* Change data type of a node to an incompatible type +* Change leafref target + +**Minor version** is incremented if the YANG change modifies the API in a backward +compatible way. Patch version is reset to 0. +Candidate YANG changes for this category are: + +* Add new YANG module +* Add new YANG data nodes +* Mark a YANG data node as deprecated +* Change data type of a node to a compatible type +* Add new enum or identity + +**Patch version** is incremented for cosmetic fixes that do not change YANG API. +Candidate YANG changes for this category are: + +* Change description, beautification. +* Expand pattern or range of a node to wider set. +* Change must expression to accept more cases. +* Error message or error tag changes. + +###### 3.2.2.6.4.2 Version Checks + +Version check ensures that translib processes the requests only from compatible clients. +All translib APIs accept an optional **ClientVersion** argument. +Clients can pass the YANG bundle version that they are capable of handling through this argument. +If specified, request will be processed only if below criteria is satisfied. + + Base_Version <= ClientVersion <= YANG_Bundle_Version + +Base Version is **1.0.0** for current release. +It will change in a future release when major version is incremented. + +Version check is bypassed if client does not send its version number in the request. +However some requests may succeed and some may fail due to input or schema mismatch. + +###### 3.2.2.6.5 DB access layer + +The DB access layer implements a wrapper over the [go-redis](https://github.com/go-redis/redis) package +enhancing the functionality in the following ways: + + * Provide a sonic-py-swsssdk like API in Go + * Enable support for concurrent access via Redis CAS (Check-And-Set) + transactions. + * Invoke the CVL for validation before write operations to the Redis DB + +The APIs are broadly classified into the following areas: + + * Initialization/Close: NewDB(), DeleteDB() + * Read : GetEntry(), GetKeys(), GetKeysPattern(), GetTable() + * Write : SetEntry(), CreateEntry(), ModEntry(), DeleteEntry() + DeleteEntryField() + * Transactions : StartTx(), CommitTx(), AbortTx(), AppendWatchTx() + * Map : GetMap(), GetMapAll() + * Subscriptions : SubscribeDB(), UnsubscribeDB() + * Publish : Publish() + +Detail Method Signature: + Please refer to the code for the detailed method signatures. + +Concurrent Access via Redis CAS transactions: + + Upto 4 levels of concurrent write access support. + + 1. Table based watch keys (Recommended): + At App module registration, the set of Tables that are to be managed by + the module are provided. External (i.e. non-Management-Framework) + applications may choose to watch and set these same table keys to detect + and prevent concurrent write access. The format of the table key is + "CONFIG_DB_UPDATED_". (Eg: CONFIG_DB_UPDATED_ACL_TABLE) + + 2. Row based watch keys: + For every transaction, the App module provides a list of keys that it + would need exclusive access to for the transaction to succeed. Hence, + this is more complex for the app modules to implement. The external + applications need not be aware of table based keys. However, the + concurrent modification of yet to be created keys (i.e. keys which are + not in the DB, but might be created by a concurrent application) may not + be detected. + + 3. A combination of 1. and 2.: + More complex, but easier concurrent write access detection. + + 4. None: + For applications not needing concurrent write access protections. + + +DB access layer, Redis, CVL Interaction: + + DB access | PySWSSSDK API | RedisDB Call at | CVL Call at + | | at CommitTx | invocation + ----------------|------------------|-------------------|-------------------- + SetEntry(k,v) | set_entry(k,v) | HMSET(fields in v)|If HGETALL=no entry + | | HDEL(fields !in v | ValidateEditConfig + | | but in | (OP_CREATE) + | | previous HGETALL)| + | | |Else + | | | ValidateEditConfig( + | | | {OP_UPDATE, + | | | DEL_FIELDS}) + ----------------|------------------|-------------------|-------------------- + CreateEntry(k,v)| none | HMSET(fields in v)| ValidateEditConfig( + | | | OP_CREATE) + ----------------|------------------|-------------------|-------------------- + ModEntry(k,v) | mod_entry(k,v) | HMSET(fields in v)| ValidateEditConfig( + | | | OP_UPDATE) + ----------------|------------------|-------------------|-------------------- + DeleteEntry(k,v)|set,mod_entry(k,0)| DEL | ValidateEditConfig( + | | | OP_DELETE) + ----------------|------------------|-------------------|-------------------- + DeleteEntryField| none | HDEL(fields) | ValidateEditConfig( + (k,v) | | | DEL_FIELDS) + +Notes: + 1. SetEntry(), CreateEntry(), ModEntry() with empty fields in v, will map to + DeleteEntry(). [ There may be an option to disable this in the future. ] + + +##### 3.2.2.7 Transformer + +Transformer provides the underlying infrastructure for developers to translate data from YANG to ABNF/Redis schema and vice versa, using YANG extensions annotated to YANG paths to provide translation methods. At run time, the YANG extensions are mapped to an in-memory Transformer Spec that provides two-way mapping between YANG and ABNF/Redis schema for Transformer to perform data translation while processing SET/GET operations. + +In case that SONiC YANG modules are used by NBI applications, the Transformer performs 1:1 mapping between a YANG object and a SONiC DB object without a need to write special translation codes. If the openconfig YANGs are used by NBI applications, you may need special handling to translate data between YANG and ABNF schema. In such case, you can annotate YANG extensions and write callbacks to perform translations where required. + +For special handling, a developer needs to provide: +1. An annotation file to define YANG extensions on YANG paths where translation required +e.g. In openconfig-acl.yang, ACL FORWARDING_ACTION "ACCEPT" mapped to "FORWARD", ACL sequence id ‘1’ mapped to ‘RULE_1’ in Redis DB etc. +2. Transformer callbacks to perform translation + +###### 3.2.2.7.1 Components + +Transformer consists of the following components and data: +* **Transformer Spec:** a collection of translation hints +* **Spec Builder:** loads YANG and annotation files to dynamically build YANG schema tree and Transformer Spec. +* **Transformer Core:** perform main transformer tasks, i.e. encode/decode YGOT, traverse the payload, lookup Transformer spec, call Transformer methods, construct the results, error reporting etc. +* **Built-in Default Transformer method:** perform static translation +* **Overloaded Transformer methods:** callback functions invoked by Transformer core to perform complex translation with developer supplied translation logic +* **Ouput framer:** aggregate the translated pieces returned from default and overloaded methods to construct the output payload +* **Method overloader:** dynamically lookup and invoke the overloaded transformer methods during data translation +* **YANG schema tree:** provides the Transformer with the schema information that can be accessed by Transformer to get node information, like default values, parent/descendant nodes, etc. + +![Transformer Components](images/transformer_components_v1.png) + +###### 3.2.2.7.2 Design + +Requests from Northbound Interfaces (NBI) are processed by Translib public APIs - Create, Replace, Update, Delete, (CRUD) and Get - that call a specific method on the common app module. The common app calls Transformer APIs to translate the request, then use the translated data to proceed to DB/CVL layer to set or get data in Redis DB. The **common app** as a default application module generically handles both SET (CRUD) and GET, and Subscribe requests with Transformer. Note that a specific app module can be registered to the Translib to handle the requests if needed. + +![Transformer Design](images/transformer_design.PNG) + +At Transformer init, it loads YANG modules pertaining to the applications. Transformer parses YANG modules with the extensions to dynamically build an in-memory schema tree and transformer spec. + +Below structure is defined for the transformer spec: + +```YANG +type yangXpathInfo struct { + yangDataType string + tableName *string + xfmrTbl *string + childTable []string + dbEntry *yang.Entry + yangEntry *yang.Entry + keyXpath map[int]*[]string + delim string + fieldName string + xfmrFunc string + xfmrField string + xfmrPost string + validateFunc string + rpcFunc string + xfmrKey string + keyName *string + dbIndex db.DBNum + keyLevel int + isKey bool + defVal string + hasChildSubTree bool +} +``` + +When a request lands at the common app in the form of a YGOT structure from the Translib request handler, the request is passed to Transformer that decodes the YGOT structure to read the request payload and look up the spec to get translation hints. Note that the Transformer cannot be used for OpenAPI spec. The Transformer Spec is structured with a two-way mapping to allow Transformer to map YANG-based data to ABNF data and vice-versa via reverse lookup. The reverse mapping is used to populate the data read from Redis DB to YANG structure for the response to get operation. + +Transformer has a built-in default transformer method to perform static, simple translation from YANG to ABNF or vice versa. It performs simple mapping - e.g. a direct name/value mapping, table/key/field name - which can be customized by a YANG extension. + +Additionally, for more complex translations of non-ABNF YANG models, i.e. OpenConfig models, Transformer also allows developers to overload the default method by specifying a callback fucntion in YANG extensions, to perform translations with developer-supplied translation codes as callback functions. Transformer dynamically invokes those functions instead of using default method. Each transformer callback must be defined to support two-way translation, i.e, YangToDb_ and DbToYang_, which are invoked by Transformer core. + +###### 3.2.2.7.3 Process + +CRUD requests (configuration) are processed via the following steps: + +1. App module calls transformer, passing it a YGOT populated Go structure to translate YANG to ABNF +2. App module calls CVL API to get all depenent table list to watch tables, and get the ordered table list +3. Transformer allocates buffer with 3-dimensional map: `[table-name][key-values][attributes]` +4. Transformer decodes YGOT structure and traverses the incoming request to get the YANG node name +5. Transformer looks up the Transformer Spec to check if a translation hint exists for the given path +6. If no spec or hint is found, the name and value are copied as-is +7. If a hint is found, check the hint to perform the action, either simple data translation or invoke external callbacks +8. Repeat steps 4 through 7 until traversal is completed +9. Invoke any annotated post-Transformer functions +10. Transformer aggregates the results to returns to App module +11. App module proceeds to update DB to ensure DB update in the order learnt from step 2 + +GET requests are processed via the following steps: +1. App module asks the transformer to translate the URL to the keyspec to the query target + ```YANG + type KeySpec struct { + DbNum db.DBNum + Ts db.TableSpec + Key db.Key + Child []KeySpec + IgnoreParentKey bool + } + ``` +2. Transformer proceeds to traverse the DB with the keyspec to get the results +3. Transformer translate the results from ABNF to YANG, with default transformer method or callbacks +4. Transformer aggregate the translated results, return to the App module to unmarshall the JSON payload + +###### 3.2.2.7.4 Common App + +The Common App is a default app that handles the GET/SET/Subscribe requests for SONiC or OpenConfig YANG modules unless an app module is registered to Translib. + +Here is a diagram to show how the common app supports SET(CRUD)/GET requests. + +![sequence diagram_for_set](images/crud_v1.png) + +![sequence_diagram_for_get](images/get_v1.png) + +If a request is associated with multiple tables, the common app module processes the DB updates in the table order learned from CVL layer. +e.g. in case that the sonic-acl.yang used by NBI and the payload with CREATE operation has a data including both ACL_TABLE and ACL_RULE, the common app updates the CONFIG-DB to create an ACL TABLE instance, followed by ACL_RULE entry creation in this order. DELETE operates in the reverse order, i.e. ACL_RULE followed by ACL_TABLE. + +###### 3.2.2.7.5 YANG Extensions + +The translation hints are defined as YANG extensions to support simple table/field name mapping or more complex data translation by external callbacks. + +---------- + +1. `sonic-ext:table-name [string]`: +Map a YANG container/list to TABLE name, processed by the default transformer method. Argument is a table name statically mapped to the given YANG container or list node. +The table-name is inherited to all descendant nodes unless another one is defined. + +2. `sonic-ext:field-name [string]`: +Map a YANG leafy - leaf or leaf-list - node to FIELD name, processed by the default transformer method + +3. `sonic-ext:key-delimiter [string]`: +Override the default key delimiters used in Redis DB, processed by the default transformer method. +Default delimiters are used by Transformer unless the extension is defined - CONFIG_DB: "|", APPL_DB: ":", ASIC_DB: "|", COUNTERS_DB: ":", FLEX_COUNTER_DB: "|", STATE_DB: "|" + +4. `sonic-ext:key-name [string]`: +Fixed key name, used for YANG container mapped to TABLE with a fixed key, processed by the default transformer method. Used to define a fixed key, mainly for container mapped to TABLE key +e.g. Redis can have a hash “STP|GLOBAL” +```YANG +container global + sonic-ext:table-name “STP” + sonic-ext:key-name “GLOBAL” +``` +5. `sonic-ext:key-transformer [function]`: +Overloading default method with a callback to generate DB keys(s), used when the key values in a YANG list are different from ones in DB TABLE. +A pair of callbacks should be implemented to support 2 way translation - **YangToDB***function*, **DbToYang***function* + +6. `sonic-ext:field-transformer [function]`: +Overloading default method with a callback to generate FIELD value, used when the leaf/leaf-list values defined in a YANG list are different from the field values in DB. +A pair of callbacks should be implemented to support 2 way translation - **YangToDB***function*, **DbToYang***function* + +7. `sonic-ext:subtree-transformer [function]`: +Overloading default method with a callback for the current subtree, allows the sub-tree transformer to take full control of translation. Note that, if any other extensions, e.g. table-name etc., are annotated to the nodes on the subtree, they are not effective. +The subtree-transformer is inherited to all descendant nodes unless another one is defined, i.e. the scope of subtree-transformer callback is limited to the current and descendant nodes along the YANG path until a new subtree transformer is annotated. +A pair of callbacks should be implemented to support 2 way translation - **YangToDB***function*, **DbToYang***function* + +8. `sonic-ext:db-name [string]`: +DB name to access data – “APPL_DB”, “ASIC_DB”, “COUNTERS_DB”, “CONFIG_DB”, “FLEX_COUNTER_DB”, “STATE_DB”. The default db-name is CONFIG_DB, Used for GET operation to non CONFIG_DB, applicable only to SONiC YANG. Processed by Transformer core to traverse database. +The db-name is inherited to all descendant nodes unless another one. Must be defined with the table-name + +9. `sonic-ext:post-transformer [function]`: +A special hook to update the DB requests right before passing to common-app, analogous to the postponed YangToDB subtree callback that is invoked at the very end by the Transformer. +Used to add/update additional data to the maps returned from Transformer before passing to common-app, e.g. add a default acl rule +Note that the post-transformer can be annotated only to the top-level container(s) within each module, and called once for the given node during translation + +10. `sonic-ext:table-transformer [function]`: +Dynamically map a YANG container/list to TABLE name(s), allows the table-transformer to map a YANG list/container to table names. +Used to dynamically map a YANG list/container to table names based on URI and payload. +The table-transformer is inherited to all descendant nodes unless another one is defined + +11. `sonic-ext:get-validate [function]`: +A special hook to validate YANG nodes, to populate data read from database, allows developers to instruct Transformer to choose a YANG node among multiple nodes, while constructing the response payload. +Typically used to check the “when” condition to validate YANG node among multiple nodes to choose only valid nodes from sibling nodes. + +---------- + + +Note that the key-transformer, field-transformer and subtree-transformer have a pair of callbacks associated with 2 way translation using a prefix - **YangToDB***function*, **DbToYang***function*. It is not mandatory to implement both functions. E.g. if you need a translation for GET operation only, you can implement only **DbToYang***function*. + +The template annotation file can be generated and used by the developers to define extensions to the yang paths as needed to translate data between YANG and ABNF format. Refer to the 3.2.2.7.8 Utilities. + +Here is the general guide you can check to find which extensions can be annotated in implementing your model. +```YANG +1) If the translation is simple mapping between YANG container/list and TABLE, consider using the extensions - table-name, field-name, optionally key-delimiter +2) If the translation requires a complex translation with your codes, consider the following transformer extensions - key-transformer, field-transformer, subtree-transformer to take a control during translation. Note that multiple subtree-transformers can be annotated along YANG path to divide the scope +3) If multiple tables are mapped to a YANG list, e.g. openconfig-interface.yang, use the table-transformer to dynamically choose tables based on URI/payload +4) In Get operation access to non CONFIG_DB, you can use the db-name extension +5) In Get operation, you can annotate the subtree-transformer on the node to implement your own data access and translation with DbToYangxxx function +6) In case of mapping a container to TABLE/KET, you can use the key-name along with the table-name extension +``` + +###### 3.2.2.7.6 Public Functions + +`XlateToDb()` and `GetAndXlateFromDb` are used by the common app to request translations. + +```go +func XlateToDb(path string, opcode int, d *db.DB, yg *ygot.GoStruct, yt *interface{}) (map[string]map[string]db.Value, error) {} + +func GetAndXlateFromDB(xpath string, uri *ygot.GoStruct, dbs [db.MaxDB]*db.DB) ([]byte, error) {} +``` + +###### 3.2.2.7.7 Overloaded Methods + +The function prototypes for external transformer callbacks are defined in the following- + +```go +type XfmrParams struct { + d *db.DB + dbs [db.MaxDB]*db.DB + curDb db.DBNum + ygRoot *ygot.GoStruct + uri string + requestUri string //original uri using which a curl/NBI request is made + oper int + key string + dbDataMap *map[db.DBNum]map[string]map[string]db.Value + subOpDataMap map[int]*RedisDbMap // used to add an in-flight data with a sub-op + param interface{} + txCache *sync.Map + skipOrdTblChk *bool +} + +/** + * KeyXfmrYangToDb type is defined to use for conversion of Yang key to DB Key + * Transformer function definition. + * Param: XfmrParams structure having Database info, YgotRoot, operation, Xpath + * Return: Database keys to access db entry, error + **/ +type KeyXfmrYangToDb func (inParams XfmrParams) (string, error) +/** + * KeyXfmrDbToYang type is defined to use for conversion of DB key to Yang key + * Transformer function definition. + * Param: XfmrParams structure having Database info, operation, Database keys to access db entry + * Return: multi dimensional map to hold the yang key attributes of complete xpath, error + **/ +type KeyXfmrDbToYang func (inParams XfmrParams) (map[string]interface{}, error) + +/** + * FieldXfmrYangToDb type is defined to use for conversion of yang Field to DB field + * Transformer function definition. + * Param: Database info, YgotRoot, operation, Xpath + * Return: multi dimensional map to hold the DB data, error + **/ +type FieldXfmrYangToDb func (inParams XfmrParams) (map[string]string, error) +/** + * FieldXfmrDbtoYang type is defined to use for conversion of DB field to Yang field + * Transformer function definition. + * Param: XfmrParams structure having Database info, operation, DB data in multidimensional map, output param YgotRoot + * Return: error + **/ +type FieldXfmrDbtoYang func (inParams XfmrParams) (map[string]interface{}, error) + +/** + * SubTreeXfmrYangToDb type is defined to use for handling the yang subtree to DB + * Transformer function definition. + * Param: XfmrParams structure having Database info, YgotRoot, operation, Xpath + * Return: multi dimensional map to hold the DB data, error + **/ +type SubTreeXfmrYangToDb func (inParams XfmrParams) (map[string]map[string]db.Value, error) +/** + * SubTreeXfmrDbToYang type is defined to use for handling the DB to Yang subtree + * Transformer function definition. + * Param : XfmrParams structure having Database pointers, current db, operation, DB data in multidimensional map, output param YgotRoot, uri + * Return : error + **/ +type SubTreeXfmrDbToYang func (inParams XfmrParams) (error) +/** + * ValidateCallpoint is used to validate a YANG node during data translation back to YANG as a response to GET + * Param : XfmrParams structure having Database pointers, current db, operation, DB data in multidimensional map, output param YgotRoot, uri + * Return : bool + **/ +type ValidateCallpoint func (inParams XfmrParams) (bool) +/** + * RpcCallpoint is used to invoke a callback for action + * Param : []byte input payload, dbi indices + * Return : []byte output payload, error + **/ +type RpcCallpoint func (body []byte, dbs [db.MaxDB]*db.DB) ([]byte, error) +/** + * PostXfmrFunc type is defined to use for handling any default handling operations required as part of the CREATE + * Transformer function definition. + * Param: XfmrParams structure having database pointers, current db, operation, DB data in multidimensional map, YgotRoot, uri + * Return: multi dimensional map to hold the DB data, error + **/ +type PostXfmrFunc func (inParams XfmrParams) (map[string]map[string]db.Value, error) +/** + * TableXfmrFunc type is defined to use for table transformer function for dynamic derviation of redis table. + * Param: XfmrParams structure having database pointers, current db, operation, DB data in multidimensional map, YgotRoot, uri + * Return: List of table names, error + **/ +type TableXfmrFunc func (inParams XfmrParams) ([]string, error) + +``` + +###### 3.2.2.7.8 Utilities + +The goyang package is extended to generate the template annotation file for any input yang files. A new output format type "annotate" can be used to generate the template annotation file.The goyang usage is as below: + +``` +Usage: goyang [-?] [--format FORMAT] [--ignore-circdep] [--path DIR[,DIR...]] [--trace TRACEFILE] [FORMAT OPTIONS] [SOURCE] [...] + -?, --help display help + --format=FORMAT + format to display: annotate, tree, types + --ignore-circdep + ignore circular dependencies between submodules + --path=DIR[,DIR...] + comma separated list of directories to add to search path + --trace=TRACEFILE + write trace into to TRACEFILE + +Formats: + annotate - generate template file for yang annotations + + tree - display in a tree format + + types - display found types + --types_debug display debug information + --types_verbose + include base information +``` +The $(SONIC_MGMT_FRAMEWORK)/gopkgs/bin is added to the PATH to run the goyang binary. + +For example: + +``` +goyang --format=annotate --path=/path/to/yang/models openconfig-acl.yang > openconfig-acl-annot.yang + +Sample output: +module openconfig-acl-annot { + + yang-version "1" + + namespace "http://openconfig.net/yang/annotation"; + prefix "oc-acl-annot" + + import openconfig-packet-match { prefix oc-pkt-match } + import openconfig-interfaces { prefix oc-if } + import openconfig-yang-types { prefix oc-yang } + import openconfig-extensions { prefix oc-ext } + + deviation oc-acl:openconfig-acl { + deviate add { + } + } + deviation oc-acl:openconfig-acl/oc-acl:acl { + deviate add { + } + } + deviation oc-acl:openconfig-acl/oc-acl:acl/oc-acl:state { + deviate add { + } + } + deviation oc-acl:openconfig-acl/oc-acl:acl/oc-acl:state/oc-acl:counter-capability { + deviate add { + } + } + deviation oc-acl:openconfig-acl/oc-acl:acl/oc-acl:acl-sets { + deviate add { + } + } + deviation oc-acl:openconfig-acl/oc-acl:acl/oc-acl:acl-sets/oc-acl:acl-set { + deviate add { + } + } + deviation oc-acl:openconfig-acl/oc-acl:acl/oc-acl:acl-sets/oc-acl:acl-set/oc-acl:type { + deviate add { + } + } +... +... + deviation oc-acl:openconfig-acl/oc-acl:acl/oc-acl:config { + deviate add { + } + } +} +``` + + +##### 3.2.2.8 Config Validation Library (CVL) + +Config Validation Library (CVL) is an independent library to validate ABNF schema based SONiC (Redis) configuration. This library can be used by component like [Cfg-gen](https://github.com/Azure/sonic-buildimage/blob/master/src/sonic-config-engine/sonic-cfggen), Translib, [ZTP](https://github.com/Azure/SONiC/blob/master/doc/ztp/ztp.md) etc. to validate SONiC configuration data before it is written to Redis DB. Ideally, any component importing config_db.json file into Redis DB can invoke CVL API to validate the configuration. + +CVL uses SONiC YANG models written based on ABNF schema along with various constraints. These native YANG models are simple and have a very close mapping to the associated ABNF schema. Custom YANG extensions (annotations) are used for custom validation purpose. Specific YANG extensions (rather metadata) are used to translate ABNF data to YANG data. In the context of CVL these YANG models are called CVL YANG models and generated from SONiC YANG during build time. Opensource [libyang](https://github.com/CESNET/libyang) library is used to perform YANG data validation. + +SONiC YANG can be used as Northbound YANG for management interface by adding other data definitions such as state data (read only data), RPC or Notification as needed.Such YANG models are called SONiC NBI YANG models. Since CVL validates configuration data only, these data definition statements are ignored by CVL. During build time CVL YANG is actually generated from SONiC NBI YANG models with the help of CVL specific pyang plugin. + +CVL supports multiple user-defined Redis database instances based on the [Multi DB instance HLD](https://github.com/Azure/SONiC/blob/master/doc/database/multi_database_instances.md) specification. During CVL initialization time, the DB configuration file is read from the predefined location and details are stored in internal data structure. CVL implements internal API for connecting to a Redis DB instance using the details stored in its internal data structure. + +###### 3.2.2.8.1 Architecture + +![CVL architecture](images/CVL_Arch.jpg) + +1. During build time, developer writes SONiC YANG schema based on ABNF schema following [SONiC YANG Guidelines](https://github.com/Azure/SONiC/blob/master/doc/mgmt/SONiC_YANG_Model_Guidelines.md) and adds metadata and constraints as needed. Custom YANG extensions are defined for this purpose. +2. The YANG models are compiled using Pyang compiler. CVL specific YIN schema files are derived from SONiC YANG with the help of CVL specific pyang plugin. Finally, generated YIN files are packaged in the build. +3. During boot up/initialization sequence, YIN schemas generated from SONiC YANG models are parsed and schema tree is build using libyang API. +4. Application calls CVL APIs to validate the configuration. In case of Translib library DB Access layer calls appropriate CVL APIs to validate the configuration. +5. ABNF JSON goes through a translator and YANG data is generated. Metadata embedded in the YANG schema are used to help this translation process. +6. Then YANG data is fed to libyang for performing syntax validation first. If error occurs, CVL returns appropriate error code and details to application without proceeding further. +7. If syntax validation is successful, CVL uses dependent data from translated YANG data or if needed, fetches the dependent data from Redis DB. Dependent data refers to the data needed to validate constraint expressed in YANG syntax such as 'leafref', 'must', 'when' etc. +8. Finally translated YANG data and dependent data are merged and fed to libyang for performing semantics validation. If error occurs, CVL returns appropriate error code and details to application, else success is returned. +9. Platform validation is specific syntax and semantics validation only performed with the help of dynamic platform data as input. + +###### 3.2.2.8.2 Validation types + +Config Validator does Syntactic, Semantic validation and Platform Validation as per YIN schema with metadata. + +###### 3.2.2.8.2.1 Syntactic Validation + +Following are some of the syntactic validations supported by the config validation library + +* Basic data type +* Enum +* Ranges +* Pattern matching +* Check for mandatory field +* Check for default field +* Check for number of keys are their types +* Check for table size etc. + +###### 3.2.2.8.2.2 Semantic Validation + +* Check for key reference existence in other table +* Check any conditions between fields within same table +* Check any conditions between fields across different table + +###### 3.2.2.8.2.3 Platform specific validation + +There can be two types of platform constraint validation + +###### 3.2.2.8.2.3.1 Static Platform Constraint Validation + +* Platform constraints (range, enum, ‘must’/’when’ expression etc.) are expressed in SONiC YANG deviation model for each feature. + +Example of 'deviation' : + +``` +module sonic-acl-deviation { + ...... + ...... + deviation /sacl:sonic-acl/sacl:ACL_TABLE/sacl:ACL_TABLE_LIST { + deviate add { + max-elements 3; + } + } + + deviation /sacl:sonic-acl/sacl:ACL_RULE/sacl:ACL_RULE_LIST { + deviate add { + max-elements 768; + } + } +} +``` + +* All deviation models are compiled along with corresponding CVL YANG model and are placed in platform specific schema folder. At runtime based on detected platform from provisioned “DEVICE_METADATA:platform” field, deviation files are applied. + +* Here is the sample folder structure for platform specific deviation files. +``` + models/yang/sonic/platform/ + |-- accton_as5712 + | |--- sonic-acl-deviation.yang + |-- quanta_ix8 + |--- sonic-acl-deviation.yang +``` + +###### 3.2.2.8.2.3.2 Dynamic Platform Constraint Validation + +###### 3.2.2.8.2.3.2.1 Platform data is available in Redis DB table. + +* Based on Redis DB, platform specific YANG data defintion can be added in SONiC YANG models. Constraints like ‘must’ or ‘when’ are used in feature YANG by cross-referencing platform YANG models. + +###### 3.2.2.8.2.3.2.2 Platform data is available through APIs + +* If constraints cannot be expressed using YANG syntax or platform data is available through feature/component API (APIs exposed by a feature to query platform specific constants, resource limitation etc.), custom validation needs to be hooked up in SONiC YANG model through custom YANG extension. +* Feature developer implements the custom validation functions with functional validation code. The validation function may call feature/component API and fetch required parameter for checking constraints. +* Based on YANG extension syntax, CVL will call the appropriate custom validation function along with YANG instance data to be validated. Below is the custom validation context structure definition. + +``` + //Custom validation context passed to custom validation function + type CustValidationCtxt struct { + ReqData []CVLEditConfigData //All request data + CurCfg *CVLEditConfigData //Current request data for which validation should be done + YNodeName string //YANG node name + YNodeVal string //YANG node value, leaf-list will have "," separated value + YCur *xmlquery.Node //YANG data tree + RClient *redis.Client //Redis client + } +``` + +###### 3.2.2.8.3 CVL APIs + +``` + //Structure for key and data in API + type CVLEditConfigData struct { + VType CVLValidateType //Validation type + VOp CVLOperation //Operation type + Key string //Key format : "PORT|Ethernet4" + Data map[string]string //Value : {"alias": "40GE0/28", "mtu" : 9100, "admin_status": down} + } + + /* CVL Error Structure. */ + type CVLErrorInfo struct { + TableName string /* Table having error */ + ErrCode CVLRetCode /* Error Code describing type of error. */ + Keys []string /* Keys of the Table having error. */ + Value string /* Field Value throwing error */ + Field string /* Field Name throwing error . */ + Msg string /* Detailed error message. */ + ConstraintErrMsg string /* Constraint error message. */ + } + + /* Structure for dependent entry to be deleted */ + type CVLDepDataForDelete struct { + RefKey string /* Ref Key which is getting deleted */ + Entry map[string]map[string]string /* Entry or field which should be deleted as a result */ + } + + /* Maintain time stats for call to ValidateEditConfig(). */ + type ValidationTimeStats struct { + Hits uint /* Total number of times ValidateEditConfig() called */ + Time time.Duration /* Total time spent in ValidateEditConfig() */ + Peak time.Duration /* Highest time spent in ValidateEditConfig() */ + } + + /* Validation type */ + type CVLValidateType uint + const ( + VALIDATE_NONE CVLValidateType = iota //Data is used as dependent data + VALIDATE_SYNTAX //Syntax is checked and data is used as dependent data + VALIDATE_SEMANTICS //Semantics is checked + VALIDATE_ALL //Syntax and Semantics are checked + ) + + /* Validation operation */ + type CVLOperation uint + const ( + OP_NONE CVLOperation = 0 //Used to just validate the config without any operation + OP_CREATE = 1 << 0//For Create operation + OP_UPDATE = 1 << 1//For Update operation + OP_DELETE = 1 << 2//For Delete operation + ) + + /* Error code */ + type CVLRetCode int + const ( + CVL_SUCCESS CVLRetCode = iota + CVL_SYNTAX_ERROR /* Generic syntax error */ + CVL_SEMANTIC_ERROR /* Generic semantic error */ + CVL_ERROR /* Generic error */ + CVL_SYNTAX_MISSING_FIELD /* Missing field */ + CVL_SYNTAX_INVALID_FIELD /* Invalid Field */ + CVL_SYNTAX_INVALID_INPUT_DATA /*Invalid Input Data */ + CVL_SYNTAX_MULTIPLE_INSTANCE /* Multiple Field Instances */ + CVL_SYNTAX_DUPLICATE /* Duplicate Fields */ + CVL_SYNTAX_ENUM_INVALID /* Invalid enum value */ + CVL_SYNTAX_ENUM_INVALID_NAME /* Invalid enum name */ + CVL_SYNTAX_ENUM_WHITESPACE /* Enum name with leading/trailing whitespaces */ + CVL_SYNTAX_OUT_OF_RANGE /* Value out of range/length/pattern (data) */ + CVL_SYNTAX_MINIMUM_INVALID /* min-elements constraint not honored */ + CVL_SYNTAX_MAXIMUM_INVALID /* max-elements constraint not honored */ + CVL_SEMANTIC_DEPENDENT_DATA_MISSING /* Dependent Data is missing */ + CVL_SEMANTIC_MANDATORY_DATA_MISSING /* Mandatory Data is missing */ + CVL_SEMANTIC_KEY_ALREADY_EXIST /* Key already existing. */ + CVL_SEMANTIC_KEY_NOT_EXIST /* Key is missing. */ + CVL_SEMANTIC_KEY_DUPLICATE /* Duplicate key. */ + CVL_SEMANTIC_KEY_INVALID /* Invaid key */ + CVL_NOT_IMPLEMENTED /* Not implemented */ + CVL_INTERNAL_UNKNOWN /*Internal unknown error */ + CVL_FAILURE /* Generic failure */ + ) +``` + +1. func Initialize() CVLRetCode + - Initialize the library only once, subsequent calls does not affect once library is already initialized . This automatically called when if ‘cvl’ package is imported. + +2. func Finish() + - Clean up the library resources. This should ideally be called when no more validation is needed or process is about to exit. + +3. func (c *CVL) ValidateConfig(jsonData string) CVLRetCode + - Just validates json buffer containing multiple row instances of the same table, data instance from different tables. All dependency are provided in the payload. This is useful for bulk data validation. + +4. func (c *CVL) ValidateEditConfig(cfgData []CVLEditConfigData) (cvlErr CVLErrorInfo, ret CVLRetCode) + - Validates the JSON data for create/update/delete operation. Syntax or Semantics Validation can be done separately or together. Related data should be given as dependent data for validation to be successful. + +5. func (c *CVL) ValidateKeys(key []string) CVLRetCode + - Just validates the key and checks if it exists in the DB. It checks whether the key value is following schema format. Key should have table name as prefix. + +6. func (c *CVL) ValidateFields(key string, field string, value string) CVLRetCode + - Just validates the field:value pair in table. Key should have table name as prefix. + +7. func (c *CVL) SortDepTables(inTableList []string) ([]string, CVLRetCode) + - Sort the list of given tables as per their dependency imposed by leafref. + +8. func (c *CVL) GetOrderedTables(yangModule string) ([]string, CVLRetCode) + - Get the sorted list of tables in a given YANG module based on leafref relation. + +9. func (c *CVL) GetDepTables(yangModule string, tableName string) ([]string, CVLRetCode) + - Get the list of dependent tables for a given table in a YANG module. + +10. func (c *CVL) GetDepDataForDelete(redisKey string) ([]CVLDepDataForDelete) + - Get the dependent data (Redis keys) to be deleted or modified for a given entry getting deleted. + +11. func GetValidationTimeStats() (ValidationTimeStats) + - Provides statistics details of time spent in ValidateEditConfig(). + +12. func ClearValidationTimeStats() + - Clears statistics details of time spent in ValidateEditConfig(). + +##### 3.2.2.9 Redis DB + +Please see [3.2.2.6.5 DB access layer](#32265-db-access-layer) + +##### 3.2.2.10 Non DB data provider + +Currently, it is up to each App module to perform the proprietary access +mechanism for the app specific configuration. + +## 4 Flow Diagrams + +### 4.1 REST SET flow + +![REST SET flow](images/write.jpg) + +1. REST client can send any of the write commands such as POST, PUT, PATCH or DELETE and it will be handled by the REST Gateway. +2. All handlers in the REST gateway will invoke a command request handler. +3. Authentication and authorization of the commands are done here. +4. Request handler invokes one of the write APIs exposed by the Translib. +5. Translib infra populates the YGOT structure with the payload of the request and performs a syntactic validation +6. Translib acquires the write lock (mutex lock) to avoid another write happening from the same process at the same time. +7. Translib infra gets the App module corresponding to the incoming URI. +8. Translib infra calls the initialize function of the App module with the YGOT structures, path and payload. +9. App module caches the incoming data into the app structure. +10. App module calls Transformer function to translate the request from cached YGOT structures into Redis ABNF format. It also gets all the keys that will be affected as part of this request. +11. App module returns the list of keys that it wants to keep a watch on along with the status. +12. Translib infra invokes the start transaction request exposed by the DB access layer. +13. DB access layer performs a WATCH of all the keys in the Redis DB. If any of these keys are modified externally then the EXEC call in step 26 will fail. +14. Status being returned from Redis. +15. Status being returned from DB access layer. +16. Translib then invokes the processWrite API on the App module. +17. App modules perform writes of the translated data to the DB access layer. +18. DB access layer validates the writes using CVL and then caches them. +19. Status being returned from DB access layer. +20. Status being returned from App module. +21. Translib infra invokes the commit transaction on the DB access layer. +22. DB access layer first invokes MULTI request on the Redis DB indicating there are multiple writes coming in, so commit everything together. All writes succeed or nothing succeeds. +23. Status returned from Redis. +24. Pipeline of all the cached writes are executed from the DB access layer. +25. Status retuned from Redis. +26. EXEC call is made to the Redis DB. Here if the call fails, it indicates that one of the keys that we watched has changed and none of the writes will go into the Redis DB. +27. Status returned from Redis DB. +28. Status returned from DB access layer. +29. Write lock acquired in Step 6 is released. +30. Status returned from the Translib infra. +31. REST Status returned from the Request handler. +32. REST response is sent by the REST gateway to the REST client. + +### 4.2 REST GET flow + +![REST GET flow](images/read.jpg) + +1. REST GET request from the REST client is sent to the REST Gateway. +2. REST Gateway invokes a common request handler. +3. Authentication of the incoming request is performed. +4. Request handler calls the Translib exposed GET API with the uri of the request. +5. Translib infra gets the App module corresponding to the incoming uri. +6. Translib infra calls the initialize function of the App module with the YGOT structures and path. App module caches them. +7. Status retuned from App module. +8. App module queries Transformer to translate the path to the Redis keys that need to be queried. +9. Status returned from App module. +10. Translib infra calls the processGet function on the App module +11. App modules calls read APIs exposed by the DB access layer to read data from the Redis DB. +12. Data is read from the Redis DB is returned to the App module +13. App module fills the YGOT structure with the data from the Redis DB and validated the filled YGOT structure for the syntax. +14. App module converts the YGOT structures to JSON format. +15. IETF JSON payload is returned to the Translib infra. +16. IETF JSON payload is returned to the request handler. +17. Response is returned to REST gateway. +18. REST response is returned to the REST client from the REST gateway. + +### 4.3 Translib Initialization flow + +![Translib Initialization flow](images/Init.jpg) + +1. App module 1 `init` is invoked +2. App module 1 calls `Register` function exposed by Translib infra to register itself with the Translib. +3. App module 2 `init` is invoked +4. App module 2 calls `Register` function exposed by Translib infra to register itself with the Translib. +5. App module N `init` is invoked +6. App module N calls `Register` function exposed by Translib infra to register itself with the Translib. + +This way multiple app modules initialize with the Translib infra during boot up. + +### 4.4 gNMI flow + +![GNMI flow](images/GNMI_flow.jpg) + +1. GNMI requests land in their respective GET/SET handlers which then redirect the requests to corresponding data clients. +2. If user does not provide target field then by default the request lands to the transl_data_client. +3. Next, the transl_data_client provides higher level abstraction along with collating the responses for multiple paths. +4. Transl Utils layer invokes Translib API's which in turn invoke App-Module API's and data is retrieved and modified in/from Redis Db/non-DB as required. + +### 4.5 CVL flow + +![CVL flow](images/CVL_flow.jpg) + +Above is the sequence diagram explaining the CVL steps. Note that interaction between DB Access layer and Redis including transactions is not shown here for brevity. + +1. REST/GNMI invokes one of the write APIs exposed by the Translib. +2. Translib infra populates the YGOT structure with the payload of the request and performs a syntactic validation. +3. Translib acquires the write lock (mutex lock) to avoid another write happening from the same process at the same time. +4. Translib infra gets the App module corresponding to the incoming uri. +5. Translib infra calls the initialize function of the App module with the YGOT structures, path and payload. +6. App module calls Transformer to translate the request from cached YGOT structure into Redis ABNF format. It also gets all the keys that will be affected as part of this request. +7. App modules returns the list of keys that it wants to keep a watch on along with the status. +8. Translib infra invokes the start transaction request exposed by the DB access layer. +9. Status being returned from DB access layer. +10. Translib then invokes the processWrite API on the App module. +11. App modules perform writes of the translated data to the DB access layer. +12. DB access layer calls validateWrite for CREATE/UPDATE/DELETE operation. It is called with keys and Redis/ABNF payload. +13. validateSyntax() feeds Redis data to translator internally which produces YANG XML. This is fed to libyang for validating the syntax. +14. If it is successful, control goes to next step, else error is returned to DB access layer. The next step is to ensure that keys are present in Redis DB for Update/Delete operation. But keys should not be present for Create operation. +15. Status is returned after checking keys. +16. CVL gets dependent data from incoming Redis payload. For example if ACL_TABLE and ACL_RULE is getting created in a single request. +17. Otherwise dependent should be present in Redis DB, query is sent to Redis to fetch it. +18. Redis returns response to the query. +19. Finally request data and dependent is merged and validateSemantics() is called. +20. If above step is successful, success is returned or else failure is returned with error details. +21. DB Access layer forwards the status response to App mpdule. +22. App module forwards the status response to Translib infra. +23. Translib infra invokes the commit transaction on the DB access layer. +24. Status is returned from DB access layer after performing commit operation. +25. Write lock acquired in Step 3 is released. +26. Final response is returned from the Translib infra to REST/GNMI. + +## 5 Developer Work flow +Developer work flow differs for standard YANG (IETF/OpenConfig) vs proprietary YANG used for a feature. When a standards-based YANG model is chosen for a new feature, the associated Redis DB design should take the design of this model into account - the closer the mapping between these, then the less translation logic is required in the Management path. This simplifies the work flow as translation intelligence can be avoided as both Redis schema and NB YANG schema are aligned. + +Where the YANG does not map directly to the Redis DB, the management framework provides mechanisms to represent complex translations via developer written custom functions. + +SONiC YANG should always be developed for a new feature, and should be purely based on the schema of config objects in DB. + +For the case where developer prefers to write a non-standard YANG model for a new or existing SONIC feature, the YANG should be written aligned to Redis schema such that the same YANG can be used for both northbound and CVL. This simplifies the developer work flow (explained in section 5.1) + + +### 5.1 Developer work flow for non Standards-based SONiC YANG + + Typical steps for writing a non-standard YANG for management framework are given below. +- Write the SONiC YANG based upon the Redis DB design. +- Add 'must','when' expressions to capture the dependency between config objects. +- Add required non-config, notification and RPC objects to the YANG. +- Add meta data for transformer. + +#### 5.1.1 Define SONiC YANG schema +Redis schema needs to be expressed in SONiC proprietary YANG model with all data types and constraints. Appropriate custom YANG extensions need to be used for expressing metadata. The YANG model is used by Config Validation Library(CVL) to provide automatic syntactic and semantic validation and Translib for Northbound management interface. + +Custom validation code needs to be written if some of the constraints cannot be expressed in YANG syntax. + +Please refer to [SONiC YANG Guidelines](https://github.com/Azure/SONiC/blob/master/doc/mgmt/SONiC_YANG_Model_Guidelines.md) for detiailed guidelines on writing SONiC YANG. + +#### 5.1.2 Generation of REST server stubs and Client SDKs for YANG based APIs + +* Place the main YANG modules under sonic-mgmt-common/models/yang directory. + * By placing YANG module in this directory, on the next build, OpenAPI YAML (OpenAPI spec) is generated for the YANG. + * If there is YANG which is augmenting the main YANG module, this augmenting YANG should also be placed in sonic-mgmt-common/models/yang directory itself. +* Place all dependent YANG modules which are imported into the main YANG module such as submodules or YANGs which define typedefs, etc under sonic-mgmt-common/models/yang/common directory. + * By placing YANG module in this directory, OpenAPI YAML (OpenAPI spec) is not generated for the YANG modules, but the YANGs placed under sonic-mgmt-common/models/yang can utilize or refer to types, and other YANG constraints from the YANG modules present in this directory. + * Example: ietf-inet-types.yang which mainly has typedefs used by other YANG models and generally we won't prefer having a YAML for this YANG, this type of YANG files can be placed under sonic-mgmt-common/models/yang/common. +* Generation of REST server stubs and client SDKs will automatically happen when make command is executed as part of the build. + +#### 5.1.3 Config Translation App (Go language) +Config Translation App consists of two parts - Transformer and App module. They translate the data in Northbound API schema to the native Redis schema (refer to section 5.1.1) and vice versa. All Northbound API services like REST, GNMI, NETCONF invoke this App to read and write data through Translib API calls. + +Key features: + +* Go language. +* YANG to Redis and vice-versa data translation is handled by Transformer. The data translation happens based on the YANG model developed as per section 5.1.1. +* The processing of data is taken care by App module + * App consumes/produces YANG data through [YGOT](https://github.com/openconfig/YGOT) structures. + * Framework provides Go language APIs for Redis DB access. APIs are similar to existing Python APIs defined in sonic-py-swsssdk repo. + * The Transformer converts between these formats. + + * For read operation + * App module receives the YANG path to read in a Go struct + * App module uses the Transformer to get a DB entry(s) reference from this path + * App module reads the Redis entry(s) + * App module uses the Transformer to convert the returned DB object(s) to a Go struct + * App module returns this to TransLib + * For write operations + * App module receives the target YANG path and data in a Go struct + * App module uses the Transformer to translates the Go struct into appropriate Redis calls + * DB Access layer takes care of DB transactions - write everything or none + +* REST server provides a test UI for quick UT of App modules. This UI lists all REST APIs for a YANG and provides option to try them out. +* Pytest automation integration can make use of direct REST calls or CLI (which also makes use of REST APIs internally). Framework generates REST client SDK to facilitate direct REST API calls. + +#### 5.1.4 IS CLI +IS CLI is achieved using KLISH framework. + +* CLI tree is expressed in the XML file with node data types and hierarchy along with different modes. +* Actioner handler needs to be hooked up in XML for corresponding CLI syntax. Actioner handler should be developed to call client SDK APIs (i.e one action handler might need to call multiple client SDK APIs.) +* Show command output formatting is achieved using [Jinja](http://jinja.pocoo.org/) templates. So, the developer needs to check if an existing template can be used or new template needs to be written. + +#### 5.1.5 gNMI +There is no specific steps required for gNMI. + + +### 5.2 Developer work flow for standard-based (e.g. OpenConfig/IETF) YANG + +#### 5.2.1 Identify the standard YANG module for the feature for northbound APIs. +The Developer starts by selecting the standard YANG to be the basis for the Management data model. For SONiC, OpenConfig is the preferred source. However, in the absence of a suitable model there, other standards can be considered (e.g. IETF, IEEE, OEM de-facto standards). + +The SONiC feature implementation may not exactly match the chosen model - there may be parts of the YANG that are not implemented by (or have limitations in) the feature, or the Developer may wish to expose feature objects that are not covered by the YANG. In such cases, the Developer should extend the model accordingly. The preferred method is to write a deviation file and then use modification statements there (e.g. deviate, augment) accordingly. + +#### 5.2.2 Define the Redis schema for the new feature. (not applicable for legacy/existing feature) +The Redis DB design should take the YANG design into account, and try to stay as close to it as possible. This simplifies the translation processes within the Management implementation. Where this is not possible or appropriate, custom translation code must be provided. + + +#### 5.2.3 Define SONiC YANG schema +Redis schema needs to be expressed in SONiC YANG model with all data types and constraints. Appropriate custom YANG extensions need to be used for expressing this metadata. The YANG model is used by Config Validation Library(CVL)to provide automatic syntactic and semantic validation. + +Custom validation code needs to be written if some of the constraints cannot be expressed in YANG syntax. + +Please refer to [SONiC YANG Guidelines](https://github.com/Azure/SONiC/blob/master/doc/mgmt/SONiC_YANG_Model_Guidelines.md) for detiailed guidelines on writing SONiC YANG. + +#### 5.2.4 Generation of REST server stubs and Client SDKs for YANG based APIs + +* Place the main YANG modules under sonic-mgmt-common/models/yang directory. + * By placing YANG module in this directory, YAML (OpenAPI spec) is generated for the YANG. + * If there is YANG which is augmenting the main YANG module, this augmenting YANG should also be placed in sonic-mgmt-common/models/yang directory itself. +* Place all dependent YANG modules such as submodules or YANGs which define typedefs, etc under sonic-mgmt-common/models/yang/common directory. + * By placing YANG module in this directory, YAML (OpenAPI spec) is not generated for the YANG modules, but the YANGs placed under sonic-mgmt-common/models/yang can utilize or refer to types, and other YANG constraints from the YANG modules present in this directory. + * Example: ietf-inet-types.yang which mainly has typedefs used by other YANG models and generally we won't prefer having a YAML for this YANG, this type of YANG files can be placed under sonic-mgmt-common/models/yang/common. +* Generation of REST-server stubs and client SDKs will automatically happen when make command is executed as part of the build. + + +#### 5.2.5 Config Translation App (Go language) +Config Translation App consists of two parts - Transformer and App module. They translate the data in Northbound API schema (defined in step#1) to the native Redis schema (defined in step#2) and vice versa. All Northbound API services like REST, GNMI, NETCONF invoke this App to read and write data. + +Key features: + +* Go language. +* YANG to Redis and vice-versa data translation is handled by Transformer. In order to facilitate data translation, the developer needs to provide just the YANG file for the data model + * YANG file for the data model + * Optionally, a YANG annotation file (refer to [3.2.2.7.8 Utilities](#32278-utilities)) to define translation hints to map YANG objects to DB objects. These translation hints are external callbacks for performing complex translation whereas simple translations are handled by Transformer's built-in methods. The annotation file is also placed in `sonic-mgmt-common/models/yang` + * Code to define the translation callbacks, in `sonic-mgmt-common/src/translib/transformer` +* The processing of data is taken care by App module + * App consumes/produces YANG data through [YGOT](https://github.com/openconfig/YGOT) structures. + * Framework provides Go language APIs for Redis DB access. APIs are similar to existing Python APIs defined in sonic-py-swsssdk repo. + * For read operation + * App module receives the YANG path to read in a Go struct + * App module uses the Transformer to get a DB entry(s) reference from this path + * App module reads the Redis entry(s) + * App module uses the Transformer to convert the returned DB object(s) to a Go struct + * App module returns this to TransLib + * For write operations + * App module receives the target YANG path and data as YGOT tree + * App module translates the YGOT tree data into appropriate Redis calls using reference from Transformer + * App module also handles additional complex logic like transaction ordering or checking for dependencies + * Translation Framework takes care of DB transactions - write everything or none +* REST server provides a test UI for quick UT of translation app. This UI lists all REST APIs for a YANG and provide option to try them out. REST server invokes Translation Apps. +* Spytest automation integration can make use of direct REST calls or CLI (which also makes use of REST internally - step#5). Framework generates REST client SDK to facilitate direct REST calls. + + +#### 5.2.6 IS CLI +IS CLI is achieved using KLISH framework and steps are same as in SONiC YANG model. Please refer to section [5.1.4 IS CLI](#514-IS-CLI). + +#### 5.2.7 gNMI +There is no specific steps required for gNMI. + + +## 6 Error Handling + +Validation is done at both north bound interface and against database schema. Appropriate error code is returned for invalid configuration. +All application errors are logged into syslog. + +## 7 Serviceability and Debug + +1. Detailed syslog messages to help trace a failure. +2. Debug commands will be added when debug framework becomes available. +3. CPU profiling enable/disable with SIGUR1 signal. + +## 8 Warm Boot Support + +Management Framework does not disrupt data plane traffic during warmboot. No special handling required for warmboot. + +## 9 Scalability + +Manageability framework will be scalable to handle huge payloads conforming to the standard/SONiC yang models. + +## 10 Unit Test + +#### GNMI +1. Verify that gnmi_get is working at Toplevel module +2. Verify thet gnmi_get is working for each ACL Table +3. Verify gnmi_get working for each ACL Rule: +4. Verify that gnmi_get is working for all ACL interfaces +5. Verify that gnmi_get is working for each ACL interface name +6. Verify that gnmi_get fails for non-existent ACL name and type +7. Verify that TopLevel node can be deleted +8. Verify that a particular ACL Table can be deleted +9. Verify that ACL rule can be deleted +10. Verify that ACL table can be created +11. Verify that ACL rule can be created +12. Verify that ACL binding can be created +13. Verify that creating rule on non existent ACL gives error +14. Verify that giving invalid interface number is payload gives error. +15. Verify that GNMI capabalities is returning correctly. + +#### Request Binder (YGOT) +1. create a YGOT object binding for the uri ends with container +2. create a YGOT object binding for the uri ends with leaf +3. create a YGOT object binding for the uri ends with list +4. create a YGOT object binding for the uri ends with leaf-list +5. create a YGOT object binding for the uri which has keys +6. create a YGOT object binding for the uri which has keys and ends with list with keys +7. validate the uri which has the correct number of keys +8. validate the uri which has the invalid node name +9. validate the uri which has the invalid key value +10. validate the uri which has the incorrect number of keys +11. validate the uri which has the invalid leaf value +12. validate the payload which has the incorrect number of keys +13. validate the payload which has the invalid node name +14. validate the payload which has the invalid leaf value +15. validate the uri and the payload with the "CREATE" operation +16. validate the uri and the payload with the "UPDATE" operation +17. validate the uri and the payload with the "DELETE" operation +18. validate the uri and the payload with the "REPLACE" operation +19. validate the getNodeName method for LIST node +20. validate the getNodeName method for leaf node +21. validate the getNodeName method for leaf-list node +22. validate the getParentNode method for LIST node +23. validate the getParentNode method for leaf node +24. validate the getParentNode method for leaf-list node +25. validate the getObjectFieldName method for LIST node +26. validate the getObjectFieldName method for leaf node +27. validate the getObjectFieldName method for leaf-list node + +#### DB access layer +1. Create, and close a DB connection. (NewDB(), DeleteDB()) +2. Get an entry (GetEntry()) +3. Set an entry without Transaction (SetEntry()) +4. Delete an entry without Transaction (DeleteEntry()) +5. Get a Table (GetTable()) +6. Set an entry with Transaction (StartTx(), SetEntry(), CommitTx()) +7. Delete an entry with Transaction (StartTx(), DeleteEntry(), CommitTx()) +8. Abort Transaction. (StartTx(), DeleteEntry(), AbortTx()) +9. Get multiple keys (GetKeys()) +10. Delete multiple keys (DeleteKeys()) +11. Delete Table (DeleteTable()) +12. Set an entry with Transaction using WatchKeys Check-And-Set(CAS) +13. Set an entry with Transaction using Table CAS +14. Set an entry with Transaction using WatchKeys, and Table CAS +15. Set an entry with Transaction with empty WatchKeys, and Table CAS +16. Negative Test(NT): Fail a Transaction using WatchKeys CAS +17. NT: Fail a Transaction using Table CAS +18. NT: Abort an Transaction with empty WatchKeys/Table CAS +19. NT: Check V logs, Error logs +20. NT: GetEntry() EntryNotExist. + +#### ACL app (via REST) +1. Verify that if no ACL and Rules configured, top level GET request should return empty response +2. Verify that bulk request for ACLs, multiple Rules within each ACLs and interface bindings are getting created with POST request at top level +3. Verify that all ACLs and Rules and interface bindings are shown with top level GET request +5. Verify that GET returns all Rules for single ACL +6. Verify that GET returns Rules details for single Rule +7. Verify that GET returns all interfaces at top level ACL-interfaces +8. Verify that GET returns one interface binding +9. Verify that single or multiple new Rule(s) can be added to existing ACL using POST/PATCH request +10. Verify that single or mutiple new ACLs can be added using POST/PATCH request +11. Verify that single or multiple new interface bindings can be added to existing ACL using POST/PATCH request +12. Verify that single Rule is deleted from an ACL with DELETE request +13. Verify that single ACL along with all its Rules and bindings are deleted with DELETE request +14. Verify that single interface binding is deleted with DELETE request +15. Verify that all ACLs and Rules and interface bindings are deleted with top level DELETE request +16. Verify that CVL throws error is ACL is created with name and type same as existing ACL with POST request +17. Verify that CVL throws error is RULE is created with SeqId, ACL name and type same as existing Rule with POST request +18. Verify that GET returns error for non exising ACL or Rule +19. Verify that CVL returns errors on creating rule under non existent ACL using POST request +20. Verify that CVL returns error on giving invalid interface number in payload during binding creation + +#### CVL +1. Check if CVL validation passes when data is given as JSON file +2. Check if CVL Validation passes for Tables with repeated keys like QUEUE,WRED_PROFILE and SCHEDULER +3. Check if CVL throws error when bad schema is passed +4. Check if debug trace level is changed as per updated conf file on receiving SIGUSR2 +5. Check must constraint for DELETE throws failure if condition fails, (if acl is a bound to port, deleting the acl rule throws error due to must constraint) +6. Check if CVL Validation passes when data has cascaded leafref dependency (Vlan Member->Vlan->Port) +7. Check if Proper Error Tag is returned when must condition is not satisfied +8. Check if CVL Validation passes if Redis is loaded with dependent data for UPDATE operation. +9. Check is CVL Error is returned when any mandatory node is not provided. +10. Check if CVL validation passes when global cache is updated for PORT Table for "must" expressions. +11. Check if CVL is able to validate JSON data given in JSON file for VLAN , ACL models +12. Check if CVL initialization is successful +13. Check if CVL is able to validate JSON data given in string format for CABLE LENGTH +14. Check if CVL failure is returned if input JSON data has incorrect key +15. Check if CVL is returning CVL_SUCCESS for Create operation if Dependent Data is present in Redis +16. Check if CVL is returning CVL_FAILURE for Create operation with invalid field for CABLE_LENGTH . +17. Check is CVL Error is returned for any invalid field in leaf +18. Check is Valid CVL_SUCCESS is returned for Valid field for ACL_TABLE when data is given in Test Structure +19. Check is Valid CVL_SUCCESS is returned for Valid field for ACL_RULE where Dependent data is provided in same session +20. Check if CVL is returning CVL_FAILURE for Create operation with invalid Enum vaue +21. Check if CVL validation fails when incorrect IP address prefix is provided. +22. Check is CVL validation fails when incorrect IP address is provided. +23. Check is CVL validation fails when out of bound are provided. +24. Check is CVL validation fails when invalid IP protocol +25. Check is CVL validation fails when out of range values are provided. +26. Check if CVL validation fails when incorrect key name is provided . +27. Check if CVL validation passes is any allowed special character is list name. +28. Check if CVL validation fails when key names contains junk characters. +29. Check if CVL validation fails when additional extra node is provided +30. Check is CVL validation passes when JSON data is given as buffer for DEVICE METEADATA +31. Check if CVL validation fails when key name does not contain separators. +32. Check if CVL validation fails when one of the keys is missing for Create operation +33. Check if CVL validation fails when there are no keys between separators for Create operation +34. Check if CVL validation fails when missing dependent data is provided for Update operation in same transaction +35. Check if CVL validation fails when missing dependent data is provided for Create operation in same transaction. +36. Check if CVL validation fails when there are no keys between separators for DELETE operation +37. Check if CVL validation fails when there are no keys between separators for UPDATE operation +38. Check if CVL validation fails when invalid key separators are provided for Delete operation +39. Check if CVL validation fails if UPDATE operation is given with invalid enum value +40. Check if CVL validation fails if UPDATE operation is given with invalid key containing missing keys +41. Check if CVL validation passes with dependent data present in same transaction for DELETE operation. +42. Check if CVL validation fails if DELETE operation is given with missing key for DELETE operation. +43. Check if CVL validation fails if UPDATE operation is given with missing key +44. Check if CVL validation fails when an existing key is provided in CREATE operation +45. Check if CVL validation passes for INTERFACE table +46. Check if CVL validation fails when configuration not satisfying must constraint is provided +47. Check if CVL validation passes when Redis has valid dependent data for UPDATE operation +48. Check if CVL validation fails when two different sequences are passed(Create and Update is same transaction) +49. Check if CVL validation fails for UPDATE operation when Redis does not have dependent data. +50. Check if CVL validation passes with valid dependent data given for CREATE operation. +51. Check if CVL validation fails when user tries to delete non existent key +52. Check if CVL Validation passes if Cache contains dependent data populated in same sessions but separate transaction. +53. Check if CVL Validation passes if Cache data dependent data that is populated across sessions +54. Check if CVL Validation fails when incorrect dependent Data is provided for CREATE operation +55. Check if CVL validation passes when valid dependent data is provided for CREATE operation +56. Check if Proper Error Tag is returned when must condition is not satisfied in "range" +57. Check if Proper Error Tag is returned when must condition is not satisfied in "length" +58. Check if Proper Error Tag is returned when must condition is not satisfied in "pattern" +59. Check if DELETE fails when ACL Table is tried to Rule or when DELETE tries to delete TABLE with non-empty leafref +60. Check if validation fails when non-existent dependent data is provided. +61. Check if CVL validation fails when DELETE tries to delete leafref of another table(delete ACL table referenced by ACL rule) +62. Check if CVL Validation fails when unrelated chained dependent data is given. +63. Check if CVL Validation fails when VLAN range is out of bound and proper error message is returned +64. Check if Logs are printed as per configuration in log configuration file. +65. Check if DELETE operation is performed on single field +66. Check if CVL validation passes when valid dependent data is provided using a JSON file. +67. Check if CVL validation is passed when when delete is performed on Table and then connected leafref +68. Check if CVL validation is passes when JSON data can be given in file format +69. Check if CVL Finish operation is successful +70. Check if CVL validation passes when Entry can be deleted and created in same transaction +71. Check if CVL validation passes when two UPDATE operation are given +72. Check if CVL validation passes when 'leafref' points to a key in another table having multiple keys. +73. Check if CVL validation passes when 'leafref' points to non-key in another table. +74. Check if CVL validation passes when 'leafref' points to a key which is drived in predicate from another table in cascaded fashion. +75. Check if CVL validation passes when 'must' condition involves checking with fields having default value which are not provided in request a data. +76. Check if CVL validation passes when 'must' condition has predicate field/key value derived from another table using another predicate in cascaded fashion. +77. Check if CVL validation passes for 'when' condition present within a leaf/leaf-list. +78. Check if CVL validation passes for 'when' condition present in choice/case node. +79. Check if CVL validation passes if 'max-elements' is present in a YANG list. +80. Check if CVL can sort the list of given tables as per their dependency imposed by leafref. +81. Check if CVL can return sorted list of tables in a given YANG module based on leafref relation. +82. Check if CVL can return the list of dependent tables for a given table in a YANG module. +83. Check if CVL can return the dependent data(Redis keys) to be deleted or modified for a given entry getting deleted. + +## 11 Appendix A + +Following are the list of Open source tools used in Management framework + +1. [Gorilla/mux](https://github.com/gorilla/mux) +2. [Go datastructures](https://github.com/Workiva/go-datastructures/tree/master/queue) +3. [OpenAPI](https://OpenAPI.io) +4. [gNMI client](https://github.com/jipanyang/gnxi) +5. [goyang](https://github.com/openconfig/goyang) +6. [YGOT](https://github.com/openconfig/ygot/ygot) +7. [GO Redis](https://github.com/go-redis/redis) +8. [Logging](https://github.com/google/glog) +9. [Profiling](https://github.com/pkg/profile) +10. [Validation](https://gopkg.in/go-playground/validator.v9) +11. [JSON query](https://github.com/antchfx/jsonquery) +12. [XML query](https://github.com/antchfx/xmlquery) +13. [Sorting](https://github.com/facette/natsort) +14. [pyangbind](https://github.com/robshakir/pyangbind) +15. [libYang](https://github.com/CESNET/libyang) + + +## 12 Appendix B + +Following are the list of Open source libraries used in telemetry container. +Always refer to the [Makefile](https://github.com/Azure/sonic-telemetry/blob/master/Makefile) for the sonic-telemetry container for current package list. + + +1. [GRPC](https://google.golang.org/grpc) +2. [GNMI](https://github.com/openconfig/gnmi/proto/gnmi) +3. [Protobuf](https://github.com/golang/protobuf/proto) +4. [goyang](https://github.com/openconfig/goyang) +5. [GO Redis](https://github.com/go-redis/redis) +6. [YGOT](https://github.com/openconfig/ygot/ygot) +7. [Logging](https://github.com/google/glog) +8. [GO Context](https://golang.org/x/net/context) +9. [Credentials](https://google.golang.org/grpc/credentials) +10. [Validation](https://gopkg.in/go-playground/validator.v9) +11. [GNXI utils](https://github.com/google/gnxi/utils) +12. [Gorilla/mux](https://github.com/gorilla/mux) +13. [jipanyang/xpath](https://github.com/jipanyang/gnxi/utils/xpath) +14. [c9s/procinfo](https://github.com/c9s/goprocinfo/linux) +15. [Workkiva/Queue](https://github.com/Workiva/go-datastructures/queue) +16. [jipanyang/gnmi client](https://github.com/jipanyang/gnmi/client/gnmi) +17. [xeipuuv/gojsonschema](https://github.com/xeipuuv/gojsonschema) diff --git a/doc/mgmt/REST_DOC_GEN_HLD.md b/doc/mgmt/REST_DOC_GEN_HLD.md new file mode 100644 index 0000000000..dd43fbc2a6 --- /dev/null +++ b/doc/mgmt/REST_DOC_GEN_HLD.md @@ -0,0 +1,98 @@ +# The RESTCONF API document generator design +# High Level Design Document +### Rev 0.1 + +# Table of Contents + * [Revision](#revision) + * [About this Manual](#about-this-manual) + * [Scope](#scope) + * [Definitions/Abbreviation](#definitionsabbreviation) + * [Introduction](#introduction) + + * [1 Overview](#1-overview) + * [1.1 Design Diagram](#11-system-chart) + * [1.2 Modules description](#12-modules-description) + * [1.2.1 Pyang](#121-pyang) + * [1.2.2 OpenAPI plugin](#122-openapi-plugin) + * [1.2.3 Markdown formatter](#123-markdown-formatter) + * [2 Requirements](#2-requirements) + * [3 Document Generation Approach](#3-document-generation-approach) + * [3.1 Auto Generated document](#31-auto-generated-document) + * [3.2 Manually written template document](#32-manually-written-template-document) + * [3.3 Final Document](#33-final-document) + +###### Revision +| Rev | Date | Author | Change Description | +|:---:|:-----------:|:------------------:|-----------------------------------| +| 0.1 | 01/08/2020 | Mohammed Faraaz | Initial version | + +# About this Manual +This document provides general information about the design of the [RESTCONF 8040](https://tools.ietf.org/html/rfc8040) API document generator and the REST document syntax. + +# Scope +This document describes only the high level design of the [RESTCONF 8040](https://tools.ietf.org/html/rfc8040) API document related components. It is intending to provide high level details about Pyang and its associated OpenAPI plugin. + +# Introduction +The Existing OpenAPI plugin for Pyang is enhanced to generate a document for RESTCONF API, it uses the same Utils and tools used by OpenAPI Plugin. The generated document will be in markdown(.md) format. + +# Definitions/Abbreviation + +| Definitions/Abbreviation | Description | +|--------------------------|--------------------------------------------| +| REST | Representational state transfer | +| API | Application Programmable Interface | + +# 1 Overview + +## 1.1 Design + +Following diagram describes a RESTCONF API document generator design + +![Design](https://docs.google.com/drawings/d/1mIejWXE6kqnxcN933_VMDKgtIeyobT0sw-w0DErkRHQ/edit?folder=0AM8ib1ydVn2cUk9PVA) + +Note: +Temporary using G-link above will change it actual file name after review. + +## 1.2 Modules description + +### 1.2.1 Pyang + +Pyang is an open source Yang Parser written in Python, It parses the yang modules and provides yang data in a programmable object (python objects) format, this data is passed on to the OpenAPI plugin for OpenAPI spec and RESTCONF document generation. + +### 1.2.2 OpenAPI Plugin + +This is an extension plugin written on top of Pyang parser, it walks over the generated Yang data (present in a form of Python object) and generates a yang tree along with required metadata for the generation of OpenAPI spec (swagger spec 2.0) and a RESTCONF document for each yang module in a markdown (.md) format. + +### 1.2.3 Markdown Formatter + +It is an extension/module inside OpenAPI plugin, it walks over the yang tree produced and uses the same Utils and metadata generated by OpenAPI generator and generates a REST document in a markdown(.md) Format. + +# 2 Requirements + +* Tool shall generate RESTCONF compatible URIs and payload. +* Tool shall output the Request and Response payload for the URIs in a JSON format. +* Tool shall fetch the Description from the yang and describe the URIs in a generated document +* Tool shall list and describe the Parameters which will be part of URIs +* Tool shall generate a RESTCONF API document in a markdown(.md) format. +* Tool shall as part of build and can be turned off using an option in Makefile. +* Tool shall generate documents for both Standard and Sonic yang models. + +# 3 Document Generation Approach + +## 3.1 Auto Generated document + +The RESTCONF document generation tool will generate one markdown (.md) file per Yang module. +These auto-generated documents will be placed under sonic-mgmt-framework/build/restconf_md directory. +The auto-generated documents will have following sections +* Configuration APIs - Describes Configuration URIs i.e. config:true Yang statement. +* Operational-state APIs - Describes Operational-state URIs i.e. config:false Yang statement. +* Operations APIs - Describes RPCs defined in yang model. + +## 3.2 Manually written template document + +The base document known as RESTCONF_documentation_index.template is a manually written jinja based markdown document which contains a static sections describing RESTCONF protocol itself, authentication, supported media types, RESTCONF capabilities, Response messages, protocol and transport requirements. This is a checked-in file. + +## 3.3 Final Document + +The Final RESTCONF document called RESTCONF_documentation_index.md is auto-generated during build time which contains contents of static file described in above [3.2](#32-manually-written-template-document) and the hyperlinks to the RESTCONF document for individual Yang models generated in section [3.1](#31-auto-Generated-document). + diff --git a/doc/mgmt/SONiC Management Framework Show Techsupport HLD.md b/doc/mgmt/SONiC Management Framework Show Techsupport HLD.md new file mode 100644 index 0000000000..83d2aa0323 --- /dev/null +++ b/doc/mgmt/SONiC Management Framework Show Techsupport HLD.md @@ -0,0 +1,298 @@ +# Show techsupport +Diagnostic information aggregated presentation +# High Level Design Document +#### Rev 0.1 + +# Table of Contents + * [List of Tables](#list-of-tables) + * [Revision](#revision) + * [About This Manual](#about-this-manual) + * [Scope](#scope) + * [Definition/Abbreviation](#definitionabbreviation) + +# List of Tables + +# Revision +| Rev | Date | Author | Change Description | +|:---:|:-----------:|:------------------:|-----------------------------------| +| 0.1 | 10/06/2019 | Kerry Meyer | Initial version | + +# About this Manual +This document provides general information about presentation of aggregated diagnostic information for the SONiC subsystem via the Management Framework infrastructure. +# Scope +This document describes the high level design for the "show techsupport" command implementation under the control of the Management Framework infrastructure. It is intended to cover the general approach and method for providing a flexible collection of diagnostic information items and a mechanism for adding new items to the list of information to be provided. It also considers the basic mechanisms to be used for the various types of information to be aggregated. It does not address specific details for collection of all supported classes of information. + +# Definition/Abbreviation + +# 1 Feature Overview + +Provide Management Framework functionality to process the "show techsupport" command: + - Create an aggregated file containing the information items needed for analysis and diagnosis of problems occurring during switch operations. + - Support reduction of aggregated log file information via an optional "--since" parameter specifying the desired logging start time. + +NOTE: The underlying feature for which this Management Framework feature provides "front end" client interfaces is unchanged by the addition of these interfaces. (The "since " option available through these interfaces, however, is restricted to the IETF/YANG date/time format.) Please refer to the following document for a description of the "show techsupport" base feature: + +https://github.com/Azure/sonic-utilities/blob/master/doc/Command-Reference.md#troubleshooting-commands + +## 1.1 Requirements + +### 1.1.1 Functional Requirements + +Provide a Management Framework based interface for the "show tech-support" command. + +### 1.1.2 Configuration and Management Requirements +Provide the ability to invoke the command via the following client interfaces: + + - Management Framework CLI (same syntax as the existing Click-based + API except for tighter restriction of the "DateTime" format to + conform with the Yang/IETF DateTime standard) + - REST API + - gNOI + +(See Section 3 for additional details.) + +### 1.1.3 Scalability Requirements + +Time and storage space constraints: The large number of information items collected and the potentially large size of some of the items (e.g. interface information display in a large system) present an exposure to the risk of long processing times and significant demands on disk storage space. The Management Framework interface invokes the same command used for the Click-based interface. It adds no significant additional overhead or processing time. The storage space requirements are unchanged. +### 1.1.4 Warm Boot Requirements +N/A +## 1.2 Design Overview +### 1.2.1 Basic Approach +This feature will be implemented using the Management Framework infrastructure supplemented with customized access mechanisms for handling "non-DB" data items. + +### 1.2.2 Container +The user interface (front end) portion of this feature is implemented within the Management Framework container. + +### 1.2.3 SAI Overview +N/A (non-hardware feature) + +# 2 Functionality +## 2.1 Target Deployment Use Cases +This feature provides a quick and simple mechanism for network administrators or other personnel with no detailed knowledge of switch internal details to gather an extensive set of information items by using a single command. These items provide critical information to help development and sustaining engineering teams in the analysis and debugging of problems encountered in deployment and test environments. +## 2.2 Functional Description +The set of items to be gathered for a given software release is defined by the development team. It is specified in a way that enables run-time access to the desired set of information items to be collected. The definition of the set of information items to be collected includes specification of the access function to be used for each item in the list to gather the information, format it as needed, and pack it into the output file. The location of the resulting output file is provided to the requesting client at the completion of command execution. + +The output file name has the following form: + +` +/var/dump/sonic_dump_sonic_YYYYMMDD_HHMMSS.tar.gz +` + +Example: + +`/var/dump/sonic_dump_sonic_20191118_221625.tar.gz +` +See section 3.6.2.2 for an explanation of the output file name format. + +To view the contents of the file, the user must copy it to a local file in the client file system. If the file is to be extracted within the directory to which it is copied, the directory should have at least 50 MB of available space. To extract the file inside of the directory to which it has been copied while displaying a list of output files, the following command can be used: + +` +tar xvzf filename.tar.gz +` +The files are extracted to a directory tree, organized based on the type of information contained in the files. Example file categories for which sub-directories are provided in the output file tree include: + +- log files ("log" directory ) +- Linux configuration files ("etc" directory) +- generic application "dump" output ("dump" directory) +- network hardware driver information ("sai" directory) +- detailed information on various processes ("proc" directory). + +To extract the file contents to an alternate location, the following form of the "tar" command can be used: + +` + tar xvzf filename.tar.gz -C /path/to/destination/directory +` +Some of the larger "extracted" files will be compressed in gzip format. This includes log files and core files and also includes other files containing a large amount of output (e.g. a dump of all BGP tables). These files have a ".gz" file type. They can be extracted using: + +` +gunzip +` + + +# 3 Design +## 3.1 Overview +The "show techsupport" command causes invocation of an RPC sent from the management framework to a process in the host to cause collection of a list of flexibly defined sets of diagnostic information (information "items"). The collected list of items is stored in a compressed "tar" file with a unique name. The command output provides the location of the resulting compressed tar file. + +The "since" option can be used, if desired, to restrict the time scope for log files and core files to be collected. This option is passed to the host process for use during invocation of the applicable information gathering sub-functions. + +## 3.2 DB Changes +N/A +### 3.2.1 CONFIG DB +### 3.2.2 APP DB +### 3.2.3 STATE DB +### 3.2.4 ASIC DB +### 3.2.5 COUNTER DB + +## 3.3 Switch State Service Design +N/A +### 3.3.1 Orchestration Agent +### 3.3.2 Other Process +The "show techsupport" feature requires RPC support in a process running within the host context. The host process handling the RPC is responsible for dispatching "show techsupport" requests from the management framework container to trigger allocation of an output file, gathering and packing of the required information into the output file, and sending a response to the management framework RPC agent to specify the name and path of the output file. + +## 3.4 SyncD +N/A + +## 3.5 SAI +N/A + +## 3.6 User Interface +### 3.6.1 Data Models +The following Sonic Yang model is used for implementation of this feature: + +```module: sonic-show-techsupport + + rpcs: + +---x sonic-show-techsupport-info + +---w input + | +---w date? yang:date-and-time + +--ro output + +--ro output-filename? string +``` + + + +### 3.6.2 CLI +#### 3.6.2.1 Configuration Commands +N/A +#### 3.6.2.2 Show Commands + +Command syntax summary: + +` +show techsupport [since ] +` + +Command Description: + +Gather information for troubleshooting. Display the name of a file containing the resulting group of collected information items in a compressed "tar" file. + + +Syntax Description: + +| Keyword | Description | +|:--------------|:----------- | +| since | This option uses a text string containing the desired starting Date/Time for collected log files and core files. The format of the Date/Time in the string is defined by the Yang/IETF date-and-time specification (REF http://www.netconfcentral.org/modules/ietf-yang-types, based on http://www.ietf.org/rfc/rfc6020.txt). If "since is specified, this value is passed to the host process for use during invocation of the applicable log/core file gathering sub-functions.| + +Command Mode: User EXEC + +Output format example and summary: + +``` +Example: + +Output stored in: /var/dump/sonic_dump_sonic_20191008_082312.tar.gz + +-------------------------------------------------- + +Output file name sub-fields are defined a follows: + +- YYYY = Year +- MM = Month (numeric) +- DD = Day of the Month +- HH = hour of the current time (based on execution of the Linux "**date**" command) at the start of command execution +- MM = minute of the current time (based on execution of the Linux "**date**" command) at the start of command execution +- SS = second of the current time (based on execution of the Linux "**date**" command) at the start of command execution +``` + +Command execution example (basic command): + +``` +sonic# show techsupport + +Output stored in: /var/dump/sonic_dump_sonic_20191008_082312.tar.gz + +``` +Command execution Example (using the "since" keyword/subcommand): + +``` +sonic# show tech-support + since Collect logs and core files since a specified date/time + | Pipe through a command + + +sonic# show tech-support since + String date/time in the format: + + "YYYY-MM-DDTHH:MM:SS[.ddd...]Z" or + "YYYY-MM-DDTHH:MM:SS[.ddd...]+hh:mm" or + "YYYY-MM-DDTHH:MM:SS[.ddd...]-hh:mm" Where: + + YYYY = year, MM = month, DD = day, + T (required before time), + HH = hours, MM = minutes, SS = seconds, + .ddd... = decimal fraction of a second (e.g. ".323") + Z indicates zero offset from local time + +/- hh:mm indicates hour:minute offset from local time + +sonic# show tech-support since 2019-11-27T22:02:00Z +Output stored in: /var/dump/sonic_dump_sonic_20191127_220334.tar.gz +``` +Command execution example invocation via REST API: + +``` +REST request via CURL: + +curl -X POST "https://10.11.68.13/restconf/operations/sonic-show-techsupport:sonic-show-techsupport-info" -H "accept: application/yang-data+json" -H "Content-Type: application/yang-data+json" -d "{ \"sonic-show-techsupport:input\": { \"date\": \"2019-11-27T22:02:00.314+03:08\" }}" + +Request URL: + +https://10.11.68.13/restconf/operations/sonic-show-techsupport:sonic-show-techsupport-info + +Response Body: + +{ + "sonic-show-techsupport:output": { + "output-filename": "/var/dump/sonic_dump_sonic_20191128_013141.tar.gz" + } +} +``` + +Command execution example invocation via gNOI API: + +``` +root@sonic:/usr/sbin# ./gnoi_client -module Sonic -rpc showtechsupport -jsonin "{\"input\":{\"date\":\"2019-11-27T22:02:00Z\"}}" -insecure +Sonic ShowTechsupport +{"sonic-show-techsupport:output":{"output-filename":"/var/dump/sonic_dump_sonic_20191202_194856.tar.gz"}} +``` + +NOTE: See section 3.6.1 for a description of the limitations of the current implementation. A supplementary capability to transfer the tech support file and other diagnostic information files to the client via the Management Framework interface is highly desirable for a future release. + +#### 3.6.2.3 Debug Commands +N/A +#### 3.6.2.4 IS-CLI Compliance +The current implementation differs from the IS-CLI. + + Instead of dumping the technical support information to the output buffer, this implementation dumps the information to a compressed "tar" file and sends the name of the file to the output buffer. This implementation matches the current SONiC implementation. A supplementary capability to enable transfer of the specified file to the client is highly desirable for full functionality of this command when using a REST API or gNMI/gNOI client interface. Without this capability, it is necessary to open a shell on the host and use the SONiC host CLI interface to transfer the file. + +### 3.6.3 REST API Support +REST API support is provided. The REST API corresponds to the SONiC Yang model described in section 3.6.1. + +# 4 Flow Diagrams + +# 5 Error Handling +N/A + +# 6 Serviceability and Debug +Any errors encountered during execution of the "show tech-support" command that prevent retrieval or saving of information are reported in the command output at completion of the operation. + +# 7 Warm Boot Support +N/A + +# 8 Scalability +Refer to section 1.1.3 + +# 9 Unit Test + +| Case | Trigger | Result | +|:-----------|:--------|:-------| +| Basic command execution | Execute the "show techsupport" command with no parameters. | Confirm that the command is accepted without errors and a "result" file name is returned. Confirm that the result file contains the expected set of items. (Examine/expand the contents of the file to ensure that the top level directory tree is correct and that the number of sub-files within the tar file is correct.)| +"since" option (postive test case) | Execute the command with the "--since" TEXT option with a valid date string specifying a time near the end of one of the unfiltered output from the first test.| Same as the "Basic command execution" case. Additionally, confirm that the expected time filtering has occurred by examining one of the affected sub-files.| +"since" option (negative test case #1)|Execute the command with the "--since" TEXT option with an invalid date string.|Verify that an error is returned.| +"since" option (negative test case #2)|Execute the command with the "--since" TEXT option with no date string.|Verify that an error is returned.|Execute the command with the "--since" option with no date string.| Verify that an error is returned.| + + + + + +# 10 Internal Design Information +N/A diff --git a/doc/mgmt/SONiC-Management-Framework-Image-mgmt-HLD.md b/doc/mgmt/SONiC-Management-Framework-Image-mgmt-HLD.md new file mode 100644 index 0000000000..b2bfff5f0e --- /dev/null +++ b/doc/mgmt/SONiC-Management-Framework-Image-mgmt-HLD.md @@ -0,0 +1,250 @@ +# Image Management +Software upgrade and image installation support for SONiC +# High Level Design Document +#### Rev 0.1 + +# Table of Contents + * [List of Tables](#list-of-tables) + * [Revision](#revision) + * [About This Manual](#about-this-manual) + * [Scope](#scope) + * [Definition/Abbreviation](#definitionabbreviation) + +# List of Tables +[Table 1: Abbreviations](#table-1-abbreviations) + +# Revision +| Rev | Date | Author | Change Description | +| :--: | :--------: | :---------------: | ------------------ | +| 0.1 | 09/13/2019 | Arunsundar Kannan | Initial version | + +# About this Manual +This document provides general information about the Image management and installation feature implementation in SONiC. +# Scope +Covers northbound interface for the Image Management feature, as well as unit test cases. + +# Definition/Abbreviation + +### Table 1: Abbreviations +| **Term** | **Meaning** | +| ---------------- | ------------------------------ | +| Image Management | Image installation, validation | + +# 1 Feature Overview + +Provide management framework capabilities to handle: + +- Image Installation +- List available images +- Removal of an available image +- Set default image for next boot + + +## 1.1 Requirements + +### 1.1.1 Functional Requirements + +Provide management framework support to existing SONiC capabilities with respect to Image Management + +### 1.1.2 Configuration and Management Requirements +- CLI configuration and show commands +- REST API support +- gNMI Support + +### 1.1.3 Scalability Requirements +N/A +### 1.1.4 Warm Boot Requirements +N/A +## 1.2 Design Overview +### 1.2.1 Basic Approach +Implement Image Management support using translib in sonic-mgmt-framework. +### 1.2.2 Container +Management Container + +### 1.2.3 SAI Overview +N/A + +# 2 Functionality +## 2.1 Target Deployment Use Cases + +**image install** + +This command is used to install a new image on the alternate image partition. This command takes a path to an installable SONiC image or URL and installs the image. + +**show image-list** + +This command displays information about currently installed images. It displays a list of installed images, currently running image and image set to be loaded in next reboot. + +**image set-default** + +This command is be used to change the image which can be loaded by default in all the subsequent reboots. + +**image remove** + +This command is used to remove the unused SONiC image from the disk. Note that it's *not* allowed to remove currently running image. + +## 2.2 Functional Description +After recieving the request from the client, via an RPC, the REST server will transfer the control to processAction method in the app module(inside trasformer). This method will parse the target uri path and will branch to the corresponding function. These functions will call the python scripts in the host to perform image management related actions, like install, remove ..etc. The response from the output of the script is propagated back to processAction method and is converted to json. The json message is sent back to the client. + +# 3 Design +## 3.1 Overview +Enhancing the management framework backend code and transformer methods to add support for Image management + +## 3.2 DB Changes +State DB + +### 3.2.1 CONFIG DB + +N/A + +### 3.2.2 APP DB + +N/A + +### 3.2.3 STATE DB +The State DB is populated with details regarding the currently used image, image to be in the next boot and the list of available images. + + +### 3.2.4 ASIC DB + +N/A + +### 3.2.5 COUNTER DB + +N/A + +## 3.3 Switch State Service Design +### 3.3.1 Orchestration Agent + +N/A + +### 3.3.2 Other Process +N/A + +## 3.4 SyncD +N/A + +## 3.5 SAI +N/A + +## 3.6 User Interface +### 3.6.1 Data Models + +``` +module: sonic-image-management + +--rw sonic-image-management + +--rw IMAGE_GLOBAL + | +--rw IMAGE_GLOBAL_LIST* [img_key] + | +--rw img_key enumeration + | +--rw current? string + | +--rw next-boot? string + +--rw IMAGE_TABLE + +--rw IMAGE_TABLE_LIST* [image] + +--rw image string + + rpcs: + +---x image-install + | +---w input + | | +---w imagename? filename-uri-type + | +--ro output + | +--ro status? int32 + | +--ro status-detail? string + +---x image-remove + | +---w input + | | +---w imagename? string + | +--ro output + | +--ro status? int32 + | +--ro status-detail? string + +---x image-default + +---w input + | +---w imagename? string + +--ro output + +--ro status? int32 + +--ro status-detail? string + +``` + +### 3.6.2 CLI +#### 3.6.2.1 Configuration Commands + +**image install** + +``` +sonic# image install https://sonic-jenkins.westus.cloudapp.azure.com/job/xxxx/job/buildimage-xxxx-all/xxx/artifact/target/sonic-xxxx.bin + +Done +``` + +**image set-default** + +``` +sonic# image set-default SONiC-OS-HEAD.XXXX +``` + +**image remove** + +``` +sonic# image remove SONiC-OS-HEAD.YYYY + +Image removed +``` + + + +#### 3.6.2.2 Show Commands + +**show image-list** + +``` +sonic# show image-list +Current: SONiC-OS-HEAD.XXXX +Next: SONiC-OS-HEAD.XXXX +Available: +SONiC-OS-HEAD.XXXX +SONiC-OS-HEAD.YYYY +``` + + +#### 3.6.2.3 Debug Commands + +N/A + +#### 3.6.2.4 IS-CLI Compliance + +N/A + +### 3.6.3 REST API Support +* get_sonic_image_management_sonic_image_management +* rpc_sonic_image_management_image_install +* rpc_sonic_image_management_image_remove +* rpc_sonic_image_management_image_default + +# 4 Flow Diagrams +N/A + +# 5 Error Handling + +TBD + +# 6 Serviceability and Debug + +TBD + +# 7 Warm Boot Support + +TBD + +# 8 Scalability +N/A + +# 9 Unit Test +List unit test cases added for this feature including warm boot. + +| Test Name | Test Description | +| :------ | :----- | +| Image install | Image installed successfully and an entry is added in grub.cfg | +| Image remove | Image is removed and the corresponding entry is removed from grub.cfg | +| Image set default | Image is set as zeroth entry(entry for the default image) in grub.cfg | +| Show image list | Image list shows all the entries present in grub.cfg | +# 10 Internal Design Information + diff --git a/doc/mgmt/SONiC_AliasFeature_HLD.md b/doc/mgmt/SONiC_AliasFeature_HLD.md new file mode 100644 index 0000000000..7436c2af7c --- /dev/null +++ b/doc/mgmt/SONiC_AliasFeature_HLD.md @@ -0,0 +1,411 @@ +# Interface-Naming Feature + +Implement alternative interface naming convention for ethernet interfaces via CLI/REST/gNMI in SONiC management framework. + +# High Level Design Document + +#### Rev 0.1 + +# Table of Contents + +- [List of Tables](#list-of-tables) +- [Revision](#revision) +- [About This Manual](#about-this-manual) +- [Scope](#scope) +- [Definition/Abbreviation](#definitionabbreviation) + +# List of Tables + +[Table 1: Abbreviations](#table-1-abbreviations) + +# Revision + +| Rev | Date | Author | Change Description | +| ---- | ---------- | ------------ | ------------------ | +| 0.1 | 05/01/2020 | Justine Jose | Initial draft | +| 0.2 | 05/29/2020 | Justine Jose | v1.0 | | +| | | | | + +# About this Manual + +This document provides general information about interface-naming related syntax & overview for ethernet interfaces in break out & non-breakout mode. + +# Scope + +Covers general design for supporting interface-naming feature. Scope of interface-naming feature is restricted to SONiC management framework. SONiC CLICK based support is unaffected. Setttig interface-naming to standard in config_db.json is mentioned in the ZTP section. + +# Definition/Abbreviation + +### Table 1: Abbreviations + +# 1 Feature Overview + +The document covers various interfaces for interface-naming feature using SONiC management framework. Currently in SONiC, the ethernet interfaces are shown as a flat number. This flat interface numbering does not give mapping information about actual ports that are mapped to the interface number. By setting interface-naming to standard, user has better usability & mapping for actual port numbers. + +## 1.1 Requirements + +### 1.1.1 Functional Requirements + +Provide ability to have an alternative configuration for Ethernet Interface using SONiC management framework. + +### 1.1.2 Configuration and Management Requirements + +- Two ways to set the `interface-naming`. They are classified as `standard` and `native`. +- Implement interface-naming related config & show commands. +- REST & gNMI set/get support for interface-naming. + +### 1.1.3 Scalability Requirements + +### 1.1.4 Warm Boot Requirements + +## 1.2 Design Overview + +### 1.2.1 Basic Approach + +- By default system will boot in native interface-naming. Native interface-naming is where interfaces are displayed as Ethernet 1, Ethernet 4, Ethernet 8 etc. This also ensures that backwards compatibility is maintained. +- Once interface naming is set to standard, all subsequent CLI, REST & gNMI interfaces will use the alternative name for ethernet interface configuration & retrieval. +- Existing SSH sessions:- All the existing sessions where change in interface-naming mode is required will be notified to restart their existing sonic-cli sessions. This will be supported through `wall`command. + +### 1.2.2 Container + +### 1.2.3 SAI Overview + +# 2 Functionality + +## 2.1 Target Deployment Use Cases + +## 2.2 Functional Description + +# 3 Design + +## 3.1 Overview + +- For dynamically using the right string for ethernet interfaces (i.e Ethernet 4 in native mode & Eth 1/4 in standard mode), XML file has to be modified to use the correct PTYPE. `PHY_INTERFACE` has to be used as PTYPE for Ethernet Interface. +- When interface-naming is set to standard, ethernet interfaces will be represented as Eth[slot/port/breakout-port], where slot will start from 1 and port/breakout-port numbers will start from 1. Slot will always likely to be 1 in fixed pizza-box format, but may take different values in chassis format. This name format is fixed and will not have references to interface speed. For e.g Eth1/1 (in non breakout mode), Eth1/1/1 (in breakout mode). +- The standard interface name {e.g Eth1/1) is picked from platform.json file. All platforms need to be updated with this format for port name. +- For Ethernet interfaces, (i.e Ethernet 4 in native interface-naming & Eth 1/4 in standard interface-naming), XML file has to be modified to use the correct PTYPE. `PHY_INTERFACE` has to be used as PTYPE for Ethernet interface. +Example: +``` + +``` + +- Annotations to be added for interface-naming + + What App-owners need to do? + + *1. Add “value-transformer” annotation for the leaves & keys that need special handling, in the sonic-yang annot file.* + *2. Write value-transformer callback api* + + Actions that Xfmr-infra will do: + *1. Process dbDataMap & invoke value-transformer, if present.* + + + **Use case:** + +“value-transformer” (callback_xfmr) annotated for a leaf(leaf-A) in a sonic-yang(assume sonic-A.yang), will be invoked for that leaf(leaf-A). + +If leaf-A in sonic-A.yang is referred from multiple other yang-leaves(assume leaf-B in sonic-B.yang & leaf-C in sonic-C.yang has leaf-ref to leaf-A in sonic-A.yang), then the “callback_xfmr” will be inherited & invoked for these leaves(leaf-B & C) as well, without any explicit annotation(i.e. no “value-transformer” annotation is needed for leaf-B or leaf-C). + + User can override inherited value-transformer, by annotating a new value-xfmr at that level. i.e. If an explicit value-transformer(“callback_NEW_xfmr”) is annotated for leaf-B in sonic-B.yang(which has leaf-ref to leaf-A), then “callback_NEW_xfmr” will be invoked for leaf-B. + + + + E.g. For interface-naming feature, “/sonic-port/PORT/PORT_LIST/ifname” is annotated with value-transformer ("**alias_value_xfmr**"). This api("**alias_value_xfmr**") will be invoked for `/sonic-port/PORT/PORT_LIST/ifname`, as well as for all the yang-leaves, which has leaf-ref to this (like `/sonic-acl/ACL_TABLE/ACL_TABLE_LIST/ports`, `/sonic-udld/UDLD_PORT/UDLD_PORT_LIST/ifname`, `/sonic-mirror-session/MIRROR_SESSION/MIRROR_SESSION_LIST/dst_port` etc.). + + **Sample YANG annotation**: + +[sonic-port-annot.yang](https://github.com/project-arlo/sonic-mgmt-framework/blob/master/models/yang/annotations/sonic-port-annot.yang) + ``` + deviation /prt:sonic-port/prt:PORT/prt:PORT_LIST/prt:ifname { + deviate add { + sonic-ext:value-transformer "alias_value_xfmr"; + } + } + ``` + + **API signature:** + + ​ `func alias_value_xfmr(inParams XfmrDbParams) (string, error)` + + + + **XfmrDbParams contents** +``` + type XfmrDbParams struct { + db db.DBNum + oper int + tableName string + key string + field string + fieldYangType yang.TypeKind + value string + } +``` + + **Below is the data flow** + + ***CRU operation (OC & Sonic)***: + +| **Existing implementation** | **New implementation** | +| ------------------------------------------------------------ | :----------------------------------------------------------- | +| For any CRU request that reaches Xfmr, xfmr-infra will process payload & invoke overloaded functions (i.e fieldxfmr, subtreexfmr…), if present. It then creates dbDataMap and performs DB operations on dbDataMap | For any CRU that reaches Xfmr, Xfmr-infra will process payload & invoke any overloaded functions if present (i.e fieldxfmr, subtreexfmr…). It then creates dbDataMap & invokes value-transformer( when present), process dbDataMap and create a new processed-dbDataMap· DB operations are now performed on processed-dbDataMap. | + + + + ***GET operation(OC & Sonic)*** + +| **Existing implementation** | **New implementation** | +| ------------------------------------------------------------ | ------------------------------------------------------------ | +| For GET request that reaches Xfmr, Xfmr-infra will read data from db and create dbDataMap. It then invokes overloaded functions(fieldxfmr, subtreexfmr…) with dbDataMap. Fills the ygot tree with data from dbDataMap & merges it with ygot-tree from subtree-xfmr and sends response back. | For GET request that reaches Xfmr, xfmr-infra will read data from db and create dbDataMap. First value-xfmr (if present) is invoked, dbDataMap is processed and a new processed-dbDataMap is created. Overloaded functions(fieldxfmr, subtreexfmr…) with processed-dbDataMap will be invoked next. Fills the ygot tree with data from dbDataMap & merges it with ygot-tree from subtree-xfmr and sends response back. | + + **Sub-tree Transformer**: In cases where applications use subtree transformer, then need to use an API that will return the standard interface-naming string. The API will internally figure out if interface-naming is set to standard or native. + + Following are the APIs used: + These APIs are available in `translib/utils` package. + + **GetNativeNameFromUIName** + `func GetNativeNameFromUIName(ifName *string) *string` + + Retrieves native interface name from user input name, if interface-naming is set to standard. User input name can be standard or native. API will provide native interface name if the user input is standard interface name, otherwise it will return the native interface name passed to the API. If interface-naming is native, API will return the input string passed. + + **GetUINameFromNativeName** + `func GetUINameFromNativeName(ifName *string) *string` + + Retrieves standard interface name from user input name, if interface-naming is set to standard. User input name can be standard or native. API will provide standard interface name if the user input is native interface name, otherwise it will return the standard interface name passed to the API. If interface-naming is native, API will return the input string passed. + +## 3.2 DB Changes +DB references to ports are unchanged, native name will be used. + +### 3.2.1 CONFIG DB +Interface-naming configuration is stored in the `DEVICE_METADATA` table as an entry `intf_naming_mode` with value as `standard` or `native`. +``` +127.0.0.1:6379[4]> hgetall DEVICE_METADATA|localhost + 1) "hwsku" + 2) "Force10-S6000" + 3) "type" + 4) "LeafRouter" + 5) "hostname" + 6) "sonic" + 7) "platform" + 8) "x86_64-dell_s6000_s1220-r0" + 9) "mac" +10) "90:b1:1c:f4:ab:da" +11) "intf_naming_mode" +12) "standard" +``` + +#### 3.2.1.1 + +### 3.2.2 APP DB + +### 3.2.3 STATE DB + +### 3.2.4 ASIC DB + +### 3.2.5 COUNTER DB + +## 3.3 Switch State Service Design + +### 3.3.1 Orchestration Agent + +### 3.3.2 Other Process + +## 3.4 SyncD + +## 3.5 SAI + +## 3.6 User Interface + +### 3.6.1 Data Models +YANG model used for interface-naming handling +[sonic-device-metadata.yang](https://github.com/project-arlo/sonic-mgmt-framework/blob/master/models/yang/sonic/sonic-device-metadata.yang) + +Supported YANG objects and attributes are highlighted in green: +```diff +module: sonic-device-metadata + +--rw sonic-device-metadata + +--rw DEVICE_METADATA + +--rw DEVICE_METADATA_LIST* [name] + +--rw name string ++ +--rw intf_naming_mode? enumeration +``` +*Note*: intf_naming_mode takes enum value `native` or `standard`. + +### 3.6.2 CLI + +#### 3.6.2.1 Configuration Commands + +#### Interface naming configuration + +`[no] interface-naming standard` +*Note*: Interface naming can be set to standard or native using the above command in `CONFIG` mode. Once the mode is set, please logout and login to enter the right interface-naming session. +``` +sonic(config)# interface-naming standard +sonic(config)# no interface-naming standard +``` +#### Config commands when interface naming is standard +`E1/2`, `e1/2`, `E 1/2`, `e 1/2`, `Et 1/2`, `et1/2`, `Eth 1/2`, `Eth1/2` options are supported to get into standard interface naming. The parser converts all to `Eth1/2`. Native interface name is considered to be an invalid output. + +``` +sonic(config)# interface e1/2 +sonic(conf-if-Eth1/2)# +``` +``` +sonic(conf-if-Eth1/2)# ip address 2.2.2.2/24 +``` + +#### 3.6.2.2 Show Commands + +#### Display interface naming + +`show interface-naming` +``` +sonic(config)# interface-naming standard +sonic# show interface-naming +Interface naming is set to standard +``` +``` +sonic(config)# no interface-naming standard +sonic# show interface-naming +Interface naming is set to native +``` +#### Show commands when interface-naming is standard +``` +show vlan +Q: A - Access (Untagged), T - Tagged +NUM Status Q Ports +2 Inactive A Eth1/3 + T Eth1/5/2 + T Eth1/10 +``` +``` +sonic# show interface status +------------------------------------------------------------------------------------------------------------------- +Name Description Admin Oper Speed MTU Alternate Name +------------------------------------------------------------------------------------------------------------------- +Eth1/1 - down down 40000 9100 Ethernet0 +Eth1/2 - down down 40000 9100 Ethernet4 +... +... +``` +#### Show commands when interface-naming is native +``` +show vlan +Q: A - Access (Untagged), T - Tagged +NUM Status Q Ports +2 Inactive A Ethernet8 + T Ethernet18 + T Ethernet36 +``` +``` +sonic# show interface status +------------------------------------------------------------------------------------------------------------------- +Name Description Admin Oper Speed MTU Alternate Name +------------------------------------------------------------------------------------------------------------------- +Ethernet0 - down down 40000 9100 Eth1/1 +Ethernet4 - down down 40000 9100 Eth1/2 +... +... +``` + + +#### 3.6.2.3 Debug Commands + +#### 3.6.2.4 IS-CLI Compliance + +Cisco has no equivalent command. Port naming is always fixed. + +The following table maps SONiC CLI commands to corresponding IS-CLI commands. The compliance column identifies how the command comply to the IS-CLI syntax: + +- **IS-CLI drop-in replace** \u2013 meaning that it follows exactly the format of a pre-existing IS-CLI command. +- **IS-CLI-like** \u2013 meaning that the exact format of the IS-CLI command could not be followed, but the command is similar to other commands for IS-CLI (e.g. IS-CLI may not offer the exact option, but the command can be positioned is a similar manner as others for the related feature). +- **SONiC** - meaning that no IS-CLI-like command could be found, so the command is derived specifically for SONiC. + +| CLI Command | Compliance | IS-CLI Command (if applicable) | Link to the web site identifying the IS-CLI command (if applicable) | +| ----------- | ---------- | ------------------------------ | ------------------------------------------------------------ | +| | | | | + +### 3.6.3 REST API Support + +#### 3.6.3.1 + +##### Following REST operations will be supported + +**PATCH, PUT, DELETE and GET** + +- `​/sonic-device-metadata:sonic-device-metadata​/DEVICE_METADATA​/DEVICE_METADATA_LIST={name}​/intf_naming_mode` + +**REST query when interface-naming is set to Standard** +``` +curl -X GET "https:///restconf/data/openconfig-interfaces:interfaces/interface=Eth1%2F2" -H "accept: application/yang-data+json" -k -u "admin:admin" +``` +*Note: '/' part of interface name has to be replaced by %2F irrespective of whether the port is in breakout mode or not.* + + +# 4 Flow Diagrams + +# 5 Error Handling + +# 6 ZTP + +Setting interface-naming to standard in case of ZTP can be done by adding `intf_naming_mode` to DEVICE_METADATA table in the `config_db.json` file. Interface naming is set to native by default. +Example: +``` +DEVICE_METADATA": { + "localhost": { + "hostname": "sonic", + "hwsku": "Force10-S6000", + "mac": "90:b1:1c:f4:ab:da", + "platform": "x86_64-dell_s6000_s1220-r0", + "type": "LeafRouter", + "intf_naming_mode": "standard" + } + } +``` +# 7 Logging +SONiC backend uses native name irrespective of whether interface-naming is set to standard or native. + +# 8 Serviceability and Debug + +# 9 Warm Boot Support + +# 10 Scalability + +# 11 Unit Test and Automation + +The following test cases will be tested using CLI/REST/gNMI management interfaces. +#### Configuration and Show via CLI + +| Test Name | Test Description | +| :------ | :----- | +| Native interface-naming verification | Verify interface-naming is set to native | +| Set interface-naming to standard | Verify whether interface-naming is set to standard using show command | +| Standard interface-naming verification | Standard interface-naming can be verified using config and show commands involving physical interface whether standard interface-naming has taken effect | +| Set interface-naming to native | Verify whether interface-naming is set to native using show command | +| Native interface-naming verification | Native interface-naming can be verified using config and show commands involving physical interface whether native interface-naming has taken effect | +| Save and reload test | Save the config and reload the box, make sure that the system comes up with native interface-naming | + + +#### Configuration via gNMI + +Same as CLI configuration test, but using gNMI SET request + +#### Get configuration via gNMI + +Same as CLI show test, but using gNMI GET request, verify the JSON response. + +#### Configuration via REST + +Same as CLI configuration test, but using REST request + +#### Get configuration via REST + +Same as CLI configuration test, but using REST request + +#### Automation + +Spytest cases will be implemented for new CLI and APIs. diff --git a/doc/mgmt/SONiC_DNS_Support.md b/doc/mgmt/SONiC_DNS_Support.md new file mode 100644 index 0000000000..37e6b3d99b --- /dev/null +++ b/doc/mgmt/SONiC_DNS_Support.md @@ -0,0 +1,433 @@ +# Feature Name +Domain Name System (DNS) support + +# High Level Design Document + +#### Rev 0.1 + +# Table of Contents + * [List of Tables](#list-of-tables) + * [Revision](#revision) + * [About This Manual](#about-this-manual) + * [Scope](#scope) + * [Definition/Abbreviation](#definitionabbreviation) + +# List of Tables +[Table 1: Abbreviations](#table-1-abbreviations) + +# Revision +| Rev | Date | Author | Change Description | +|:---:|:-----------:|:------------------:|-----------------------------------| +| 0.1 | 05/05 | Venkatesan Mahalingam | Initial version | +| 0.2 | 05/11 | Venkatesan Mahalingam | Added source interface support and addressed comments | + + +# About this Manual + +This document introduces the support of DNS in SONiC. + +# Scope + +This document covers the following, +1) Click commands to configure DNS server & Source interface +2) Click command to show DNS hosts +3) New tables in config DB to handle DNS configs +4) DNS table configuration and show based on OpenConfig YANG model +5) KLISH commands to configure DNS server & Source interface +6) KLISH commands to show DNS hosts +7) Backend support to add DNS configs in "/etc/resolv.conf" file & iptable rules to change the source-IP of the DNS query packets +8) Unit Testcases + +# Definition/Abbreviation + +### Table 1: Abbreviations +| **Term** | **Meaning** | +|--------------------------|-------------------------------------| +| DNS | Domain Name System | + + + +# 1 Feature Overview + +DNS stands for Domain Name System. It is used to translate human readable domain names into machine readable IP addresses. + +With this feature, users will be able to configure DNS servers and source interface using SONiC Click commands and north bound interface (KLISH/REST/gNMI) provided by management framework module using OpenConfig models. + +Also, backend support to handle new config DB table events to populate nameservers in "/etc/resolv.conf" and add iptable rules to change the source IP (present on the source interface) of the DNS query that is being sent to the DNS server. + +## 1.1 Requirements + +### 1.1.1 Front end configuration and get capabilities + +#### 1.1.1.1 add/delete DNS server +This requirement is to add/delete DNS name server information in the Redis ConfigDB (DNS_SERVER) using Click and mgmt-framework. +The DNS server can be IPv4/IPv6 ipaddress and multiple DNS name servers can be configured. + +#### 1.1.1.2 add/delete DNS source interface +This requirement is to add/delete the global DNS source interface information in the Redis ConfigDB (DNS) using Click and mgmt-framework. + +Only one DNS source interface can be configured in the global DNS table. A new source interface will override the existing DNS source interface. + +#### 1.1.1.4 add/delete VRF name +No special handling is required in "/etc/resolv.conf" file to work on the management VRF (for this release). + +#### 1.1.1.5 Get DNS hosts +This displays the output of the DNS source interface and nameservers. + +### 1.1.2 Backend mechanisms to support configuration and get + +#### 1.1.2.1 add/delete DNS server +The creates or deletes a DNS server entry in the Redis ConfigDB. + +``` + "DNS_SERVER|10.11.0.1": { + "type": "hash", + "value": { + "NULL": "NULL" + } + }, + "DNS_SERVER|2001:aa:aa::a": { + "type": "hash", + "value": { + "NULL": "NULL" + } + }, + } +``` + +A change in the DNS_SERVER entry triggers hostcfgd to start the DNS configuration script, which in turn writes each DNS server to the resolv.conf and then restart the dns-config service. + +#### 1.1.2.2 add/delete DNS source interface + +This creates or deletes a global DNS source interface entry in the Redis ConfigDB. + +``` + "DNS|global": { + "type": "hash", + "value": { + "src_intf": "Loopback0" + } + } +``` + +A change in this entry triggers hostcfgd to add iptables SNAT rule incase of IPv4 address and ip6tables SNAT rule in IPv6 source address. + +Only one global DNS source interface is allowed. + +#### 1.1.2.4 get DNS hosts + +Transformer function issues get on DNS tables, parses the response and maps the outputs to the OpenConfig DNS states incase of mgmt-framework interface. + +Incase of Click commands, show commands directly fetches the information from DNS & DNS_SERVER tables and display to the user. + +### 1.1.3 Functional Requirements + +Provide management framework support to +- configure DNS name server +- configure DNS source interface + +### 1.1.4 Configuration and Management Requirements +- Click configuration and show commands +- CLI style configuration and show commands +- REST API support +- gNMI Support + +Details described in Section 3. + +### 1.1.6 Scalability Requirements + +### 1.1.7 Warm Boot Requirements + +## 1.2 Design Overview + +### 1.2.1 Basic Approach +- Implement Click commands using SONiC utilities. +- Implement DNS support using transformer in sonic-mgmt-framework. + +### 1.2.2 Container +The front end code change will be done in management-framework container including: +- XML file for the CLI +- Python script to handle CLI request (actioner) +- Jinja template to render CLI output (renderer) +- OpenConfig YANG model for DNS openconfig-system.yang +- SONiC DNS model for DNS based on Redis DB schema +- transformer functions to + * convert OpenConfig YANG model to SONiC YANG model for DNS related configurations + +### 1.2.3 SAI Overview + +# 2 Functionality + +## 2.1 Target Deployment Use Cases +Manage/configure DNS configurations via gNMI, REST and CLI interfaces + +## 2.2 Functional Description +Provide CLI, gNMI and REST support for DNS configurations. + +## 2.3 Backend change to support new configurations +Provide change in management framework and hostcfgd modules. + +# 3 Design + +## 3.1 Overview + +Enhancing the management framework backend code and transformer methods to add support for DNS. + +## 3.2 DB Changes + +### 3.2.1 CONFIG DB +``` +; DNS configuration attributes global to the system. Only one instance of the table exists in the system. +; Key +global_key = “global” ; DNS global configuration +; Attributes +src_intf = ifname ; source interface address for the outgoing DNS packets + +``` + +``` +; DNS name server configuration in the system. +; Key +server_key = IPAddress ; DNS server’s address +; Attributes +No attributes are introduced as part of this design. +``` + +### 3.2.2 APP DB + +### 3.2.3 STATE DB + +### 3.2.4 ASIC DB + +### 3.2.5 COUNTER DB + +## 3.3 Switch State Service Design + +### 3.3.1 Orchestration Agent + +### 3.3.2 Other Process + +## 3.4 SyncD + +## 3.5 SAI + +## 3.6 User Interface + +### 3.6.1 Data Models + +YANG models needed for DNS handling in the management framework: +1. **openconfig-system.yang** + +2. **sonic-system-dns.yang** + +Supported yang objects and attributes: +```diff + +module: openconfig-system + +--rw system + + +--rw dns + | +--rw config + | | +--rw search* oc-inet:domain-name + | | +--rw oc-sys-ext:source-intf? -> /oc-if:interfaces/interface/name + | +--ro state + | | +--ro search* oc-inet:domain-name + | | +--ro oc-sys-ext:source-intf? -> /oc-if:interfaces/interface/name + | +--rw servers + | | +--rw server* [address] + | | +--rw address -> ../config/address + | | +--rw config + | | | +--rw address? oc-inet:ip-address + | | | +--rw port? oc-inet:port-number + | | +--ro state + | | +--ro address? oc-inet:ip-address + | | +--ro port? oc-inet:port-number + +The above "source-intf" & "address" fields are supported in the scope of this work. + +module: sonic-system-dns + +--rw sonic-system-dns + +--rw DNS_SERVER + | +--rw DNS_SERVER_LIST* [ipaddress] + | +--rw ipaddress inet:ip-address + +--rw DNS + +--rw DNS_LIST* [type] + +--rw type enumeration ('global') + +--rw src_intf? union + +``` + +### 3.6.2 CLI + + +#### 3.6.2.1 Configuration Commands +All commands are executed in `configuration-view`: +``` +sonic# configure terminal +sonic(config)# +``` + +##### 3.6.2.1.1 Configure DNS server & source IP +``` +sonic(config)# ip name-server + source-intf Configure source interface to pick the source IP, used for the DNS query + A.B.C.D/A::B Domain name server + +sonic(config)# ip name-server source-intf + Ethernet Ethernet interface + Loopback Loopback interface + Management Management interface + PortChannel PortChannel interface + Vlan Vlan interface + +sonic(config)# ip name-server 10.11.0.1 + +sonic(config)# ip name-server 2001:aa:aa::a + +sonic(config)# ip name-server source-intf Loopback 0 + +``` + +##### 3.6.2.1.2 Delete DNS server & source IP + +``` +sonic(config)# no ip name-server + source-intf Configure source interface to pick the source IP, used for the DNS query + A.B.C.D/A::B Domain name server + +sonic(config)# no ip name-server 10.11.0.1 + +sonic(config)# no ip name-server 2001:aa:aa::a + +sonic(config)# no ip name-server source-intf + +``` +#### 3.6.2.2 Show dns hosts + +``` +sonic# show hosts +Source Interface : Loopback0 +Name servers are : 10.11.0.1, 2001:aa:aa::a +sonic# +``` + +#### 3.6.2.3 Debug Commands + +#### 3.6.2.4 IS-CLI Compliance + +### 3.6.3 REST API Support +``` +GET - Get existing DNS configuration information from CONFIG DB. + Get DNS peer states +POST - Add DNS configuration into CONFIG DB. +PATCH - Update existing DNS Configuration information in CONFIG DB. +DELETE - Delete a existing DNS configuration from CONFIG DB. +``` +### 3.6.4 Click command support +#### 3.6.4.1 Configuration support +Below click commands are supported for DNS configurations +oot@sonic:~# config dns +Usage: config dns [OPTIONS] COMMAND [ARGS]... + + DNS command line + +Options: + -?, -h, --help Show this message and exit. + +Commands: + add Specify a DNS name server + delete Delete a DNS name server + source_intf DNS source interface configurations +root@sonic:~# +root@sonic:~# config dns source_intf +Usage: config dns source_intf [OPTIONS] COMMAND [ARGS]... + + DNS source interface configurations + +Options: + -?, -h, --help Show this message and exit. + +Commands: + add DNS source interface add configuration + delete DNS source interface delete configuration +root@sonic:~# +``` +root@sonic:~# ping www.yahoo.com +ping: www.yahoo.com: Temporary failure in name resolution + +root@sonic:~# config dns add 10.11.0.1 + +root@sonic:~# ping www.yahoo.com +PING new-fp-shed.wg1.b.yahoo.com (98.137.246.8) 56(84) bytes of data. +64 bytes from media-router-fp2.prod1.media.vip.gq1.yahoo.com (98.137.246.8): icmp_seq=1 ttl=52 time=24.2 ms +64 bytes from media-router-fp2.prod1.media.vip.gq1.yahoo.com (98.137.246.8): icmp_seq=2 ttl=52 time=23.9 ms +64 bytes from media-router-fp2.prod1.media.vip.gq1.yahoo.com (98.137.246.8): icmp_seq=3 ttl=52 time=23.9 ms +^C +--- new-fp-shed.wg1.b.yahoo.com ping statistics --- +3 packets transmitted, 3 received, 0% packet loss, time 2001ms +rtt min/avg/max/mdev = 23.910/24.033/24.262/0.241 ms +root@sonic:~# +``` +#### 3.6.4.2 Show support +Below Click command is used to dump the DNS configurations +``` +root@sonic:~# show hosts +Source Interface : Loopback0 +Name servers are : 10.11.0.1, 2001:aa:aa::a +root@sonic:~# +``` +# 4 Flow Diagrams + +# 5 Error Handling + +# 6 Serviceability and Debug + +# 7 Warm Boot Support +This support is added in the hostcfgd and hence no explicit handling is needed. + +# 8 Scalability + +# 9 Unit Test + +The unit-test for this feature will include: +#### Configuration via CLI + +| Test Name | Test Description | +| :-------- | :----- | +| Configure DNS server | Verify DNS servers are present in the DNS_SERVER table (configDB) and the same is reflected in the resolv.conf file | +| Delete DNS server | Verify DNS servers are not present in the DNS_SERVER table (configDB) and the same is reflected in the resolv.conf file | +| Verify max DNS servers | Verify more than 6 servers are not allowed | +| Configure DNS source interface | Verify source interface is present in the DNS table (configDB) and DNS query packets are transmitted with the source IP present in source interface | +| Delete DNS source interface | Verify source interface is not present in the DNS table (configDB) and DNS query packets are not transmitted with the source IP present in source interface | +| Configure mgmt VRF | Verify that DNS query packets (from mgmt VRF) are transmitted based on nameservers present in resolv.conf file | +| Verify IPv4 DNS query is sent for ping to domain name | Verify whether DNS query is sent based on IPv4 source IP (based on source interface) & destination IP | +| Verify IPv6 DNS query is sent for ping to domain name | Verify whether DNS query is sent based on IPv6 source IP (based on source interface) & destination IP | +| show hosts | Verify source interface and nameservers are displayed correctly | + +#### Configuration via gNMI + +Same test as CLI configuration Test but using gNMI request. +Additional tests will be done to set DNS configuration at different levels of Yang models. + +#### Get configuration via gNMI + +Same as CLI show test but with gNMI request, will verify the JSON response is correct. +Additional tests will be done to get DNS configuration and DNS states at different levels of Yang models. + +#### Configuration via REST (POST/PUT/PATCH) + +Same test as CLI configuration Test but using REST POST request +Additional tests will be done to set DNS configuration at different levels of Yang models. + +**URIs for REST configurations:** + +Source interface - /restconf/data/openconfig-system:system/dns/config/openconfig-system-ext:source-intf +Name server - /restconf/data/openconfig-system:system/dns/servers/server[8.8.8.8]/config + +#### Get configuration via REST (GET) + +Same as CLI show test but with REST GET request, will verify the JSON response is correct. +Additional tests will be done to get DNS configuration and DNS states at different levels of Yang models. + + +# 10 Internal Design Information diff --git a/doc/mgmt/SONiC_Design_Doc_Unified_FRR_Mgmt_Interface.md b/doc/mgmt/SONiC_Design_Doc_Unified_FRR_Mgmt_Interface.md new file mode 100644 index 0000000000..ff620b4d3c --- /dev/null +++ b/doc/mgmt/SONiC_Design_Doc_Unified_FRR_Mgmt_Interface.md @@ -0,0 +1,1069 @@ +# SONiC FRR-BGP Extended Unified Configuration Management Framework +## High Level Design Document +### Rev 0.4 + +## Table of Contents + * [List of Tables](#list-of-tables) + * [Revision](#revision) + * [About This Manual](#about-this-manual) + * [Scope](#scope) + * [Definition/Abbreviation](#definitionabbreviation) + * [Table 1: Abbreviations](#table-1-abbreviations) + * [1 Feature Overview](#1-feature-overview) + * [1.1 Requirements](#11-requirements) + * [1.1.1 Functional Requirements](#111-functional-requirements) + * [1.1.2 Configuration and Management Requirements](#112-configuration-and-management-requirements) + * [1.1.3 Scalability Requirements](#113-scalability-requirements) + * [1.1.4 Warm Boot Requirements](#114-warmboot-requirements) + * [1.2 Design Overview](#12-design-overview) + * [1.2.1 Basic Approach](#121-basic-approach) + * [1.2.2 Container](#122-container) + * [2 Functionality](#2-functionality) + * [3 Design](#3-design) + * [3.1 Overview](#31-overview) + * [3.2 DB Changes](#32-db-changes) + * [3.3 SwSS Design](#33-swss-design) + * [3.4 SyncD](#34-syncd) + * [3.5 SAI](#35-sai) + * [3.6 User Interface](#36-user-interface) + * [3.6.1 Data Models](#361-data-models) + * [3.6.2 CLI](#362-cli) + * [3.6.2.1 Configuration Commands](#3621-configuration-command) + * [3.6.2.2 Show Commands](#3622-show-command) + * [3.6.2.3 Debug Commands](#3623-debug-command) + * [3.6.2.4 IS-CLI Compliance](#3624-is-cli-compliance) + * [3.6.3 REST API Support](#363-rest-api-support) + * [4 Flow Diagrams](#4-flow-diagrams) + * [4.1 Configuration Sequence](#41-configuration-sequence) + * [4.2 CLI Show Command Sequence](#42-cli-show-command-sequence) + * [4.2.1 CLI Show Sequence - config only](#421-cli-show-sequence-config-only) + * [4.2.2 CLI Show Sequence - State/Statistics](#422-cli-show-sequence-state-statistics) + * [4.3 REST Get Sequence](#43-rest-get-sequence) + * [4.3.1 REST Get Sequence - config only](#431-rest-get-sequence-config-only) + * [4.3.2 REST Get Sequence - State/Statistics](#432-rest-get-sequence-state-statistics) + * [5 Error Handling](#6-error-handling) + * [6 Serviceability and Debug](#7-serviceability-and-debug) + * [7 Warm Boot Support](#8-warm-boot-support) + * [8 Scalability](#9-scalability) + * [9 Unit Test](#10-unit-test) + * [APPENDIX](#APPENDIX) + + +## List of Tables +[Table 1: Abbreviations](#table-1-abbreviations) + +## Revision +| Rev | Date | Author | Change Description | +|:---:|:-----------:|:------------------:|-----------------------------------| +| 0.1 | 09/25/2019 | Karthikeyan Arumugam | Initial version | +| 0.2 | 10/30/2019 | Venkatesan Mahalingam | Config DB schema changes | +| 0.3 | 11/20/2019 | Venkatesan Mahalingam | Added various fields in config DB | +| 0.4 | 12/18/2019 | Venkatesan Mahalingam | Addressed the comments on error handling and method of testing | + +## About this Manual +This document provides general information about the implementation of Extended Unified Configuration and Management framework support for FRR-BGP feature in SONiC. +## Scope +This document describes the high level design of FRR-BGP Extended Unified Configuration and Management feature. + +## Definition/Abbreviation + +### Table 1: Abbreviations +| **Term** | **Meaning** | +|--------------------------|-------------------------------------| +| FRR | Free Range Routing Stack | +| CVL | Config Validation Library | +| VRF | Virtual routing forwarding | +| RIB | Routing Information Base | +| PBR | Policy based routing | +| NBI | North Bound Interface | + + + +# 1 Feature Overview + +This feature extends and provides unified configuration and management capability for FRR-BGP features used in SONiC. This allows the user to configure & manage FRR-BGP using SONiC Management Framework with Open Config data models via REST, gNMI and also provides access via SONiC Management Framework CLI as well. + + +## 1.1 Requirements + + +### 1.1.1 Functional Requirements + + 1. Extend Unified mode for full FRR-BGP config and management in SONiC + 2. Extend sonic-cfggen, bgpcfgd and integrate with FRR-BGP for features supported in SONiC + 3. Support for retrieval of FRR-BGP state and statistics information + 4. For backward compatibility retain access to FRR UI (vtysh) for managing features that are NOT in conflict with SONiC features + +### 1.1.2 Configuration and Management Requirements + +1. Support Open Config data models for BGP config and Management +2. Provide IS-CLI/gNMI/REST support for config and management of FRR-BGP features used in SONIC +3. Enhance with Custom YANG models for features used in BGP that are not supported via Open Config data model +4. Define ABNF schema for BGP features used in SONiC + +### 1.1.3 Scalability Requirements +N/A + +### 1.1.4 Warm Boot Requirements +As state and statistics information is retrieved from FRR-BGP on demand there is no Warm Boot specific requirements for this feature. + +## 1.2 Design Overview +SONiC FRR-BGP Extended Unified config and management capability makes use of Management framework to implement the backend and transformer methods to support Open Config data models for BGP and route policy feature. The backend converts the incoming request to Redis ABNF schema format and writes the configuration to Redis DB. Then from DB events, bgpcfgd will configure FRR-BGP using FRR CLI commands. +It also uses management framework's transformer methods to do syntactic and semantic validation of the requests using ABNF JSON before writing them into the Redis DB. + +### 1.2.1 Basic Approach + +* This enhancement takes comprehensive approach to support BGP features used in SONiC: + * Standard based YANG models and custom YANG models + * Open API spec + * Industry standard CLI + * Config Validation +* REST server, gNMI server, Transformer methods - all in Go +* Marshalling and unmarshalling using YGOT +* Redis updated using CAS(Check-and-Set) trans. (No locking, No rollback) +* Config Validation by using YANG model from ABNF schema + +### 1.2.2 Container + +There will be changes in following containers, +* sonic-mgmt-framework +* sonic-frr + +### 1.2.3 SAI Overview +N/A - software feature + + +# 2 Functionality +## 2.1 Target Deployment Use Cases +Configure and manage FRR-BGP via gNMI, REST and CLI interfaces using SONiC Management Framework. + +## 2.2 Functional Description +Provide GNMI and REST support for config get/set, state get and statistics get, CLI config and show commands for FRR-BGP features used in SONiC. + + +# 3 Design +## 3.1 Overview +The extended unified config and management framework for FRR-BGP in SONiC is represented in below diagram. + +![FRR-BGP Unified Mgmt Framework](images/FRR-BGP-Unified-mgmt-frmwrk.png) + +1. Transformer common app owns the Open config data models related to BGP (which means no separate app module required for handling BGP open-config and augmented YANG objects). + +2. Provide annotations for required objects so that transformer core and common app will take care of handling them. + +3. Provide transformer methods as per the annotations defined to take care of model specific logics and validations. + +4. Define SONiC YANG and Redis ABNF schema for the supported Open Config BGP models & objects. + +5. KLISH CLI and REST clients provide extensive BGP configurations and hence there should not be any need for BGP configurations via vtysh. + +6. In bgpcfgd register for Redis DB events for the BGP and other related objects, so as to translate the Redis DB events to FRR-BGP CLI commands to configure FRR-BGP, similarly, separate config daemons can be present to configure individual features like OSPF, BFD..etc + +7. Update /usr/share/sonic/templates/bgpd.conf.j2 template for new FRR-BGP configurations supported in SONiC which will be used by sonic-cfggen to generate /etc/frr/bgpd.conf file. + +## 3.2 DB Changes +Following section describes the changes to DB. + +### 3.2.1 CONFIG DB + +Added new tables to configure following information: + + * BGP router & address family configurations + * BGP neighbor & address family configurations + * BGP peer group & address family configurations + * BGP listen prefix configuration for peer group + * Route map configurations + * Route redistribute configurations + * Route policy sets (prefix list/community/ext-community/as-path/neighbor-set/tag-set configurations + +Enhanced following table to configure additional attributes: + + * BGP Neighbor table + +#### 3.2.1.1 BGP_GLOBALS +```JSON +;Defines BGP globals table +; +;Status: stable + +key = BGP_GLOBALS|vrf ; +vrf = 1\*15VCHAR ; VRF name +local_asn = 1*10DIGIT ; Local ASN for the BGP instance +router_id = \*IPv4prefix ; Router ID IPv4 address +load_balance_mp_relax = "true" / "false" ; +grace_restart = "true" / "false" ; +always_compare_med = "true" / "false" ; +load_balance_mp_relax = "true" / "false" ; +graceful_restart_enable = "true" / "false" ; +gr_restart_time = 1*4DIGIT ; {1..3600 }; +gr_stale_routes_time = 1*4DIGIT ; {1..3600 }; +ebgp_route_distance = 1*2DIGIT ; {1..255 }; +ibgp_route_distance = 1*2DIGIT ; {1..255 }; +external_compare_router_id = "true" / "false" ; +ignore_as_path_length = "true" / "false" ; +log_nbr_state_changes = "true" / "false" ; +rr_cluster_id = 1*64VCHAR ; Route reflector cluster id +rr_allow_out_policy = "true" / "false" ; Router reflector allow outbound Policy +disable_ebgp_connected_rt_check = "true" / "false" ; +fast_external_failover = "true" / "false" ; +network_import_check = "true" / "false" ; +graceful_shutdown = "true" / "false" ; +route_flap_dampen = "true" / "false" ; +route_flap_dampen_half_life = 1*2DIGIT; {1..45} +route_flap_dampen_reuse_threshold = 1*5DIGIT; {1..20000} +route_flap_dampen_suppress_threshold = 1*5DIGIT; {1..20000} +route_flap_dampen_max_suppress = 1*3DIGIT; {1..255} +rr_clnt_to_clnt_reflection = "true" / "false" ; +max_dynamic_neighbors = 1*4DIGIT; {1..5000} +read_quanta = 1*2DIGIT; {1..10} Indicates how many packets to read from peer socket per I/O cycle +write_quanta = 1*2DIGIT; {1..10} Indicates how many packets to write to peer socket per run +coalesce_time = 1*10DIGIT; Subgroup coalesce timer value in milli-sec +route_map_process_delay = 1*3DIGIT; { 0..600} +deterministic_med = "true" / "false" ; +med_confed = "true" / "false" ; Compare MED among confederation paths when set to true +med_missing_as_worst = "true" / "false" ; Treat missing MED as the least preferred one when set to true +compare_confed_as_path = "true" / "false" ; Compare path lengths including confederation sets & sequences in selecting a route +as_path_mp_as_set = "true" / "false" ; Generate an AS_SET +default_ipv4_unicast = "true" / "false" ; Activate ipv4-unicast for a peer by default +default_local_preference = "true" / "false" ; Configure default local preference value +default_show_hostname = "true" / "false" ; Show hostname in BGP dumps +default_shutdown = "true" / "false" ; Apply administrative shutdown to newly configured peers +default_subgroup_pkt_queue_max = 1*2DIGIT; {20..100} Configure subgroup packet queue max +max_med_time = 1*5DIGIT; {5..86400} Time (seconds) period for max-med +max_med_val = 1*10DIGIT; Max MED value to be used +max_delay = 1*4DIGIT; {0..3600} Maximum delay for best path calculation +establish_wait = 1*5DIGIT; {Maximum delay for updates} +``` +#### 3.2.1.2 BGP_GLOBALS_AF +```JSON +;Defines BGP Address family table +; +;Status: stable + +key = BGP_GLOBALS_AF|vrf|af_name ; +vrf = 1\*15VCHAR ; VRF name +af_name = "ipv4_unicast" / "ipv6_unicast" / "l2vpn_evpn" ; address family +max_ebgp_paths = 1*3DIGIT ; {1..256} +max_ibgp_paths = 1*3DIGIT ; {1..256} +aggregate_prefix = IPv4Prefix / IPv6prefix ; +aggregate_as_set = "true" / "false" ; +aggregate_summary_only = "true" / "false" ; +network_prefix = IPv4Prefix / IPv6prefix ; +network_policy = 1*64VCHAR ; +network_backdoor = "true" / "false" ; +route_download_filter = 1*64VCHAR ; +ebgp_route_distance = 1*3DIGIT; { 1.255} +ibgp_route_distance = 1*3DIGIT; { 1.255} +ibgp_equal_cluster_length = "true" / "false" ; + +``` + +#### 3.2.1.3 BGP_LISTEN_PREFIX +```JSON +;Defines BGP Listen Prefix table +; +;Status: stable + +key = BGP_GLOBALS_LISTEN_PREFIX|vrf|IPprefix ; +vrf = 1\*15VCHAR ; VRF name +IPprefix = IPv4Prefix / IPv6prefix ; +peer_group_name = 1*64VCHAR ; Peer group this listen prefix is associated with +``` + +#### 3.2.1.4 BGP_NEIGHBOR + +```JSON +;Defines BGP neighbor table +; +;Status: stable + +key = BGP_NEIGHBOR|vrf|neighbor_ip ; +vrf = 1\*15VCHAR ; VRF name +neighbor_ip = IPv4Prefix / IPv6prefix ; +local_asn = 1*10DIGIT ; Local ASN for the BGP neighbor +name = 1*64VCHAR ; BGP neighbor description +asn = 1*10DIGIT; Remote ASN value +ebgp_multihop = "true" / "false" ; Allow EBGP neighbors not on directly connected networks +ebgp_multihop_ttl = 1*3DIGIT ; {1..255} EBGP multihop count +auth_password = STRING ; Set a password +enabled = "true" / "false" ; Neighbor admin status +keepalive_intvl = 1*4DIGIT ; {1..3600} keepalive interval +hold_time = 1*4DIGIT ; {1..3600} hold time +local_address = IPprefix ; local IP address +peer_group_name = 1*64VCHAR ; peer group name +peer_type = "internal" / "external" Internal/External BGP peer +conn_retry = 1*5DIGIT ; {1..65535} Connection retry timer +min_adv_interval = 1*3DIGIT ; {1..600} Minimum interval between sending BGP routing updates +passive_mode = "true" / "false" ; Don't send open messages +capability_ext_nexthop = "true" / "false" ; Advertise extended next-hop capability +disable_ebgp_connected_route_check = "true" / "false" ; one-hop away EBGP peer using loopback address +enforce_first_as = "true" / "false" ; Enforce the first AS for EBGP route +solo_peer = "true" / "false" ; Solo peer - part of its own update group +ttl_security_hops = 1*3DIGIT ; {1.254} BGP ttl-security parameters +bfd = "true" / "false" ; Enable BFD support +capability-dynamic = "true" / "false" ; Advertise dynamic capability +dont-negotiate-capability = "true" / "false" ; Do not perform capability negotiation +enforce-multihop = "true" / "false" ; Allow EBGP neighbors not on directly connected networks +override-capability = "true" / "false" ; Override capability negotiation result +peer-port = 1*5DIGIT ; {0..65535} Neighbor's BGP port +shutdown-message = "true" / "false" ; Add a shutdown message +strict-capability-match = "true" / "false" ; Strict capability negotiation match + +``` +#### 3.2.1.5 BGP_NEIGHBOR_AF +```JSON +;Defines BGP Neighbor table at an address family level +; +;Status: stable + +key = BGP_NEIGHBOR_AF|vrf|neighbor_ip|af_name ; +vrf = 1\*15VCHAR ; VRF name +neighbor_ip = IPv4Prefix / IPv6prefix ; +af_name = "ipv4_unicast" / "ipv6_unicast" / "l2vpn_evpn" ; address family +enabled = "true" / "false" ; Neighbor admin status +send_default_route = "true" / "false" ; +default_rmap = 1*64VCHAR ; Filter sending default routes bsaed on this route map. +max_prefix_limit = 1*10DIGIT; Maximum number of prefixes to accept from this peer +max_prefix_warning_only = "true" / "false" ; Only give warning message when limit is exceeded +max_prefix_warning_threshold = 1*3DIGIT; Threshold value (%) at which to generate a warning msg +max_prefix_restart_interval = 1*5DIGIT; Restart bgp connection after limit is exceeded +route_map_in = 1*64VCHAR ; Apply route map on incoming routes from neighbor +route_map_out = 1*64VCHAR ; Apply route map on outgoing routes to neighbor +soft_reconfiguration_in = "true" / "false" ; Per neighbor soft reconfiguration +unsuppress_map_name = 1*64VCHAR ; Route-map to selectively un-suppress suppressed routes +route_reflector_client = "true" / "false" ; Configure a neighbor as Route Reflector client +weight = 1*5DIGIT ; {0..65535} Set default weight for routes from this neighbor +as_override = "true" / "false" ; Override ASNs in outbound updates if aspath equals remote-as +send_community = "standard" / "extended" / "both" / "none" ; Send Community attribute to this neighbor +add_path_tx_all = "true" / "false" ; +add_path_tx_bestpath = "true" / "false" ; +unchanged_as_path = "true" / "false" ; +unchanged_med = "true" / "false" ; +unchanged_nexthop = "true" / "false" ; +filter_list_name = 1*64VCHAR ; +filter_list_direction = "inbound" / "outbound"; +nexthop_self_enabled = "true" / "false" ; +nexthop_self_force = "true" / "false" ; +prefix_list_name = 1*64VCHAR ; +prefix_list_direction = "inbound" / "outbound"; +remove_private_as_enabled = "true" / "false" ; +replace_private_as = "true" / "false" ; +remove_private_as_all = "true" / "false" ; +allow_as_count = 1*3DIGIT ; Number of occurences of AS number +allow_asin = "true" / "false" ; Accept as-path with my AS present in it +allow_as_origin = "true" / "false" ; Only accept my AS in the as-path if the route was originated in my AS +capability_orf_send = "true" / "false" ; Capability to receive the outbound route filtering from this neighbor +capability_orf_receive = "true" / "false" ; Capability to send the outbound route filtering to this neighbor +capability_orf_both = "true" / "false" ; Capability to send and receive the outbound route filtering to/from this neighbor +route-server-client = "true" / "false" ; Configure a neighbor as Route Server client +``` + +#### 3.2.1.6 BGP_PEER_GROUP +The existing BGP_PEER_RANGE (peer group) table does not have vrf-name as the key (but added as the field in the table as per VRF HLD) but FRR and open config models have the peer-group under VRF context, so, the new table BGP_PEER_GROUP has been introduced for configurations from management framework (This is not a backward compatible change, expecting the user to migrate to this table in the near future). + +```JSON +;Defines BGP peer group table +; +;Status: stable + +key = BGP_PEER_GROUP|vrf|pgrp_name ; +vrf = 1\*15VCHAR ; VRF name +pgrp_name = 1*64VCHAR ; alias name for the peer group , must be unique +local_asn = 1*10DIGIT ; Local ASN for the BGP neighbor +name = 1*64VCHAR ; BGP neighbor description +asn = 1*10DIGIT; Remote ASN value +ebgp_multihop = "true" / "false" ; Allow EBGP neighbors not on directly connected networks +ebgp_multihop_ttl = 1*3DIGIT ; {1..255} EBGP multihop count +auth_password = STRING ; Set a password +enabled = "true" / "false" ; Neighbor admin status +keepalive_intvl = 1*4DIGIT ; {1..3600} keepalive interval +hold_time = 1*4DIGIT ; {1..3600} hold time +local_address = IPprefix ; local IP address +peer_group_name = 1*64VCHAR ; peer group name +peer_type = "internal" / "external" Internal/External BGP peer +conn_retry = 1*5DIGIT ; {1..65535} Connection retry timer +min_adv_interval = 1*3DIGIT ; {1..600} Minimum interval between sending BGP routing updates +passive_mode = "true" / "false" ; Don't send open messages +capability_ext_nexthop = "true" / "false" ; Advertise extended next-hop capability +disable_ebgp_connected_route_check = "true" / "false" ; one-hop away EBGP peer using loopback address +enforce_first_as = "true" / "false" ; Enforce the first AS for EBGP route +solo_peer = "true" / "false" ; Solo peer - part of its own update group +ttl_security_hops = 1*3DIGIT ; {1.254} BGP ttl-security parameters +bfd = "true" / "false" ; Enable BFD support +capability-dynamic = "true" / "false" ; Advertise dynamic capability +dont-negotiate-capability = "true" / "false" ; Do not perform capability negotiation +enforce-multihop = "true" / "false" ; Allow EBGP neighbors not on directly connected networks +override-capability = "true" / "false" ; Override capability negotiation result +peer-port = 1*5DIGIT ; {0..65535} Neighbor's BGP port +shutdown-message = "true" / "false" ; Add a shutdown message +strict-capability-match = "true" / "false" ; Strict capability negotiation match +``` +#### 3.2.1.7 BGP_PEER_GROUP_AF + +```JSON +;Defines BGP per address family peer group table +; +;Status: stable + +key = BGP_PEER_GROUP_AF|vrf|pgrp_name|af_name" ; +vrf = 1\*15VCHAR ; VRF name +af_name = "ipv4_unicast" / "ipv6_unicast" / "l2vpn_evpn" ; address family +pgrp_name = 1*64VCHAR ; alias name for the peer group template, must be unique +af_name = "ipv4_unicast" / "ipv6_unicast" / "l2vpn_evpn" ; address family +enabled = "true" / "false" ; Neighbor admin status +send_default_route = "true" / "false" ; +default_rmap = 1*64VCHAR ; Filter sending default routes bsaed on this route map. +max_prefix_limit = 1*10DIGIT; Maximum number of prefixes to accept from this peer +max_prefix_warning_only = "true" / "false" ; Only give warning message when limit is exceeded +max_prefix_warning_threshold = 1*3DIGIT; Threshold value (%) at which to generate a warning msg +max_prefix_restart_interval = 1*5DIGIT; Restart bgp connection after limit is exceeded +route_map_in = 1*64VCHAR ; Apply route map on incoming routes from neighbor +route_map_out = 1*64VCHAR ; Apply route map on outgoing routes to neighbor +soft_reconfiguration_in = "true" / "false" ; Per neighbor soft reconfiguration +unsuppress_map_name = 1*64VCHAR ; Route-map to selectively un-suppress suppressed routes +route_reflector_client = "true" / "false" ; Configure a neighbor as Route Reflector client +weight = 1*5DIGIT ; {0..65535} Set default weight for routes from this neighbor +as_override = "true" / "false" ; Override ASNs in outbound updates if aspath equals remote-as +send_community = "standard" / "extended" / "both" / "none" ; Send Community attribute to this neighbor +add_path_tx_all = "true" / "false" ; +add_path_tx_bestpath = "true" / "false" ; +unchanged_as_path = "true" / "false" ; +unchanged_med = "true" / "false" ; +unchanged_nexthop = "true" / "false" ; +filter_list_name = 1*64VCHAR ; +filter_list_direction = "inbound" / "outbound"; +nexthop_self_enabled = "true" / "false" ; +nexthop_self_force = "true" / "false" ; +prefix_list_name = 1*64VCHAR ; +prefix_list_direction = "inbound" / "outbound"; +remove_private_as_enabled = "true" / "false" ; +replace_private_as = "true" / "false" ; +remove_private_as_all = "true" / "false" ; +allow_as_count = 1*3DIGIT ; Number of occurences of AS number +allow_asin = "true" / "false" ; Accept as-path with my AS present in it +allow_as_origin = "true" / "false" ; Only accept my AS in the as-path if the route was originated in my AS +capability_orf_send = "true" / "false" ; Capability to receive the outbound route filtering from this neighbor +capability_orf_receive = "true" / "false" ; Capability to send the outbound route filtering to this neighbor +capability_orf_both = "true" / "false" ; Capability to send and receive the outbound route filtering to/from this neighbor +route-server-client = "true" / "false" ; Configure a neighbor as Route Server client +``` +#### 3.2.1.8 ROUTE_MAP +```JSON +;Defines route map table +; +;Status: stable + +key = ROUTE_MAP|route_map_name|stmt_name ; +route_map_name = 1*64VCHAR ; route map name +stmt_name = 1*64VCHAR ; statment name +route_operation = "ACCEPT" / "REJECT" +match_interface = 1*64VCHAR ; Match interface name +match_prefix_set = 1*64VCHAR ; Match prefix sets +match_med = 1*10DIGIT ; Match metric of route +match_origin = 1*64VCHAR ; Match BGP origin code +match_local_pref = 1*64VCHAR ; Match local-preference of route +match_community = 1*64VCHAR ; Match BGP community list +match_ext_community = 1*64VCHAR ; Match BGP/VPN extended community list +match_as_path = 1*64VCHAR ; Match BGP AS path list +call_route_map = 1*64VCHAR ; Jump to another Route-Map after match+set + +set_origin = 1*64VCHAR ; Set BGP origin code +set_local_pref = 1*64VCHAR ; Set BGP local preference path attribute +set_next_hop = 1*64VCHAR ; Set IP address of next hop +set_med = 1*64VCHAR ; Set Metric value for destination routing protocol +set_repeat_asn = 1*3DIGIT ; NO of times the set_asn number to be repeated +set_asn = 1*10DIGIT ; Set ASN number +set_community_inline = 1*64VCHAR ; Set BGP community attribute inline +set_community_ref = 1*64VCHAR ; Refer BGP community attribute from community table +set_ext_community_inline = 1*64VCHAR ; Set BGP extended community attribute inline +set_ext_community_ref = 1*64VCHAR ; Refer BGP extended community attribute from extended community table +``` +#### 3.2.1.8 ROUTE_REDISTRIBUTE +```JSON +;Defines route redistribution table +; +;Status: stable + +key = ROUTE_REDISTRIBUTE|vrf|src_protocol|dst_protocol|addr_family ; +vrf = 1\*15VCHAR ; VRF name +src_protocol = "connected" / "static" / "ospf" / "ospf3" +dst_protocol = "bgp" +addr_family = "ipv4" / "ipv6" +route_map = 1*64VCHAR ; route map filter to apply for redistribution +``` + +### 3.2.1.9 IP_PREFIX_SET +```JSON +;Defines prefix set table +; +;Status: stable + +key = IP_PREFIX_SET:name ; prefix_set_name must be unique +name = 1*255VCHAR ; community set name +mode = "IPv4"/"IPv6" ; mode of prefix set. + +```` +#### 3.2.1.9.1 IP_PREFIX +```JSON +;Defines prefix table +; +;Status: stable +key = IP_PREFIX:set_name:ip_prefix:masklength_range; an instance of this key will be repeated for each prefix + ; an instance of this key/value pair will be repeated for each prefix +set_name = 1*255VCHAR ; community set name +ip_prefix = IPv4prefix / IPv6prefix ; prefix, example 1.1.1.1/32 +masklength_range = 1*255VCHAR ; exact or (masklength_range..low-masklength_range_high). example 8..16 or exact +``` +### 3.2.1.10 BGP_COMMUNITY_SET +```JSON +;Defines community table +; +;Status: stable +key = BGP_COMMUNITY_SET|name ; name must be unique +set_type = "STANDARD"/"EXPANDED" +match_action = "ANY/ALL" +community_member = string list ; community member list + ; Acceptable List of communities as ("AA:NN","local-AS", "no-advertise", "no-export" | regex) + +```` +### 3.2.1.11 BGP_EXT_COMMUNITY_SET +```JSON +;Defines extended community table +; +;Status: stable +key = BGP_EXT_COMMUNITY_SET|name ; name must be unique +set_type = "STANDARD"/"EXPANDED" +match_action = "ANY/ALL" +community_member = string list; community member list + ; Acceptable List of communities as ("route-target/route-origin:AA:NN" or "IP_Address" or regex) +```` +### 3.2.1.12 BGP_AS_PATH_SET +```JSON +;Defines extended community table +; +;Status: stable +key = AS_PATH_SET|name ; name must be unique + +as_path_member = string list; AS path list + ;Acceptable List of as paths "string, string" +```` + +### 3.2.2 APP DB +N/A + +### 3.2.3 STATE DB +No changes to State DB, State and statistics information will be retrieved directly from FRR-BGP. + +### 3.2.4 ASIC DB +N/A + +### 3.2.5 COUNTER DB +N/A + +## 3.3 Switch State Service Design + +### 3.3.1 Orchestration Agent +No changes to Orch agent. + +### 3.3.2 Other Process + +#### 3.3.2.1 FRR Template Changes + +FRR template must be enhanced to contain FRR-BGP related configuration that are supported via FRR-BGP extended unified (Config DB is propagated to FRR config at startup) with non-integrated mode (FRR configuration is saved in individual files: “bgpd.conf”, “zebra.conf” and ospfd.conf....etc) config management framework. + +On startup, sonic-cfggen will use "/usr/share/sonic/templates/bgpd.conf.j2" to generate "/etc/frr/bgpd.conf". + +## 3.4 SyncD +No changes to SyncD + +## 3.5 SAI +No changes to SAI APIs. + +## 3.6 User Interface +### 3.6.1 Data Models +List of Open-config YANG models required for FRR-BGP Unified Configuration and Management are, + + 1) openconfig-network-instance.yang + + 2) openconfig-routing-policy.yang + +BGP and "routing policy" related augmented and not-supported fields are available in openconfig-bgp-ext.yang and openconfig-routing-policy-ext.yang files respectively. +Supported YANG containers: +``` +module: openconfig-network-instance + +--rw network-instances + +--rw network-instance* [name] + +--rw table-connections + | ... + +--rw protocols + +--rw protocol* [identifier name] + +--rw bgp + +--rw global + | | ... + | +--rw afi-safis + | +--rw afi-safi* [afi-safi-name] + | ... + +--rw neighbors + | +--rw neighbor* [neighbor-address] + | | ... + | +--rw afi-safis + | +--rw afi-safi* [afi-safi-name] + | ... + +--rw peer-groups + | +---rw peer-group* [peer-group-name] + | | ... + | +--rw afi-safis + | +--rw afi-safi* [afi-safi-name] + | ... + +--ro rib + +--ro afi-safis + ... + +module: openconfig-routing-policy + +--rw routing-policy + +--rw defined-sets + | ... + +--rw policy-definitions + +--rw policy-definition* [name] + ... + +``` +### 3.6.2 CLI + 1. For all configuration commands, the CLI request is converted to a corresponding REST client SDK request based on the Open Config data model that was generated by the Swagger generator, and is given to the REST server. + + 2. From there on it will follow the same path as a REST config request for create, update and delete operations. + + 3. The Swagger generated REST server handles all the REST requests from the client SDK and invokes a common handler for all the create, update, replace, delete and get operations along with path and payload. This common handler converts all the requests into Transformer arguments and invokes the corresponding Transformer APIs. + + 4. For show commands, the CLI request is converted to a corresponding REST client SDK get request based on Open Config data model's config or state object on a case by case basis. + + 5. For show commands that requires retrieval of the data that doesn't contain any state information (information only based on the configuration), the backend callback will fetch the data from CONFIG_DB. + + 6. For show commands that requires retrieval of state or statistics information the backend, managemnt framework executes the FRR CLI using "docker exec bgp vtysh -c " to fetch the data in JSON format from FRR-BGP. + + 7. At transformer the JSON output (retrived from FRR BGP) is converted back to corresponding open config objects and returned to the caller. + + 8. For CLI show, the output returned in object format is then translated back to CLI Jinga template for output display in CLI. + + +#### 3.6.2.1 Configuration Commands + +##### 3.6.2.1.1 BGP Router mode commands + +|Command Description |CLI Command | +|:-----------------|:---------------| +|Enable BGP routing instance |sonic(config)# router bgp \ [vrf \] | +|Override configured BGP router-id |sonic(config-router-bgp)# router-id \ | +|Configure default best path selection |sonic(config-router-bgp)# bestpath {as-path { confed \| ignore \| multipath-relax \[as-set] \| med { confed \| missing-as-worst } } | +|Configure graceful restart capability params |sonic(config-router-bgp)# graceful-restart preserve-fw-state
sonic(config-router-bgp)# graceful-restart restart-time <1-3600>
sonic(config-router-bgp)# graceful-restart stalepath-time <1-3600>| +|Configure BGP IPv4/IPv6 neighbor |sonic(config-router-bgp)# neighbor { \ \| \ } | +|Configure BGP peer group |sonic(config-router-bgp)# peer-group \| +|Enter address family command mode|sonic(config-router-bgp)# address-family { ipv4 unicast \| ipv6 unicast \| l2vpn evpn} | +| Subgroup coalesce timer | sonic(config-router-bgp)# coalesce-time \ | +| How many packets to read from peer socket per I/O cycle | sonic(config-router-bgp)# read-quanta \ | +| How many packets to write to peer socket per run | sonic(config-router-bgp)# write-quanta \ | +| Configure client to client route reflection | sonic(config-router-bgp)# client-to-client reflection | +| Configure Route-Reflector Cluster-id | sonic(config-router-bgp)# cluster-id { <32-bit-val> \| } | +| Log neighbor up/down and reset reason(default) | sonic(config-router-bgp)# log-neighbor-changes | +| Pick the best-MED path among paths advertised from the neighboring AS | sonic(config-router-bgp)# deterministic-med | +| Enable route-flap dampening | sonic(config-router-bgp)# dampening \ \ \ | +| Disable checking if nexthop is connected on ebgp sessions | sonic(config-router-bgp)# disable-ebgp-connected-route-check | +| Graceful shutdown parameters | sonic(config-router-bgp)# graceful-shutdown | +| Configure BGP defaults | sonic(config-router-bgp)# bgp listen \{ limit \ \| range \ } | +| Advertise routes with max-med | sonic(config-router-bgp)# max-med on-startup [\