# Configuring Nokia SR OS with NETCONF
_netlab_ uses gNMI to configure Nokia SR OS devices. The gNMI Ansible collection has to be installed from a GitHub repository together with an outdated version of the **protobuf** library (`netlab install grpc`), which makes the installation fragile.
This document describes how we could replace gNMI with NETCONF while reusing the existing SR OS configuration templates.
## Current Implementation
All SR OS configuration templates render YAML data describing a gNMI SetRequest:
* **updates**, **replace** and **delete** lists
* Every list element is a dictionary with a **path** (gNMI path) and a **val** (YANG-JSON data in the `nokia-conf` YANG model)
The `deploy-config/sros.yml` task renders the template, parses the YAML data, and passes the lists to the `nokia.grpc.gnmi_config` module.
```
updates:
- path: configure/router[router-name=Base]/interface[interface-name=system]
val:
admin-state: enable
description: "System interface"
ipv4:
primary:
address: "10.0.0.1"
prefix-length: 32
```
## Proposed Approach
SR OS exposes the same `nokia-conf` YANG model over NETCONF. The gNMI data generated by the configuration templates can thus be converted into NETCONF XML with a mechanical transformation that does not need the YANG schema. The configuration templates (including the users' custom configuration templates) do not have to change.
The conversion would be implemented as an Ansible filter (for example, `netsim/ansible/filter_plugins/sros_netconf.py`; Ansible loads filter plugins from the `filter_plugins` directory next to the playbooks).
{.table-wrap}
| gNMI data | NETCONF XML |
|-----------|-------------|
| `elem[key=val]` path element | `val...`; key leaves are emitted first (taken from the path, duplicates in **val** are skipped) |
| Keys containing slashes (`port-id=1/1/c1`) | Bracket-aware path parsing |
| `configure` top-level element | `` |
| **updates** | Default `merge` operation |
| **replace** | `nc:operation="replace"` on the target node |
| **delete** | `nc:operation="remove"` on the target node |
| JSON list of dictionaries or leaf-list | Repeated elements |
| `[null]` (YANG _empty_ type) or `{}` | Empty element |
| `True`/`False` | `true`/`false` |
| Scalar **val** (gNMI path points to a leaf) | Leaf element text |
| Multiple items sharing a path prefix | Merged into a single XML tree |
The whole module configuration is sent in a single `` RPC to the candidate datastore followed by a commit. gNMI applies all delete operations, then all replace operations, then all update operations within a single atomic transaction. The converter processes the items in the same order and resolves overlapping operations to get the same end result:
* A **replace** or **delete** operation discards the earlier changes within its subtree
* An update or replace below (or at) a deleted node changes the **remove** operation into **replace**: deleting a node and then creating some of its contents is equivalent to replacing the node with the new contents
* An update below (or at) a replaced node is merged into the replacement contents
Converted configuration snippet:
```
Base
eth-1/1/c3
enable
500
500
3
```
### Prototype Converter
A prototype converter has been tested on the configurations generated for an SR-SIM lab using the **initial**, **ospf**, **bgp**, **bfd**, and **vrf** modules (`netlab initial -o`). All configuration files were converted; the results have not been tested on a live device.
```python
import re
import xml.etree.ElementTree as ET
NS = { 'configure': 'urn:nokia.com:sros:ns:yang:sr:conf' }
NC = 'urn:ietf:params:xml:ns:netconf:base:1.0'
PATH_ELEM = re.compile(r'([^/\[]+)((?:\[[^\]]*\])*)')
def split_path(path): # 'a/b[k=1/1/c1]/c' -> [(a,{}),(b,{k:1/1/c1}),(c,{})]
return [ (m[1], dict(kv.split('=',1) for kv in re.findall(r'\[([^\]]*)\]',m[2])))
for m in PATH_ELEM.finditer(path) ]
def scalar(v):
return 'true' if v is True else 'false' if v is False else str(v)
def add_value(parent, name, val):
if isinstance(val, list):
if val == [None]: # YANG 'empty' in JSON encoding
ET.SubElement(parent, name); return
for item in val: # list entries or leaf-list values
add_value(parent, name, item)
return
e = ET.SubElement(parent, name)
if isinstance(val, dict):
for k, v in val.items():
add_value(e, k, v)
elif val is not None:
e.text = scalar(val)
def find_or_add(parent, name, keys):
for c in parent.findall(name): # merge entries sharing the same path prefix
if all(c.findtext(k) == v for k, v in keys.items()):
return c
e = ET.SubElement(parent, name)
for k, v in keys.items(): # keys from the path go first
ET.SubElement(e, k).text = v
return e
def to_xml(cfg):
OP = '{%s}operation' % NC
root = ET.Element('{%s}config' % NC)
for op in ('delete', 'replace', 'updates'): # gNMI processing order
for item in cfg.get(op) or []:
elems = split_path(item['path'])
keys = elems[-1][1]
node = root
for idx, (name, path_keys) in enumerate(elems):
node = find_or_add(node, name, path_keys)
if idx == 0:
node.set('xmlns', NS[name])
if op != 'delete' and node.get(OP) == 'remove':
node.set(OP, 'replace') # delete followed by replace/update is a replace
if op != 'updates': # replace/delete supersedes earlier changes in the subtree
for c in [ c for c in node if c.tag not in keys ]:
node.remove(c)
node.set(OP, 'remove' if op == 'delete' else 'replace')
val = item.get('val')
if isinstance(val, dict):
for k, v in val.items():
if k not in keys: # already emitted from path predicate
add_value(node, k, v)
elif val is not None and val != [None]:
node.text = scalar(val) # scalar leaf value
ET.register_namespace('nc', NC)
ET.indent(root)
return ET.tostring(root, encoding='unicode')
```
The prototype does not handle gNMI paths pointing to a leaf-list (a list of scalar values in **val**); none of the current templates use them.
## Required Changes
* **deploy-config/sros.yml**: render the template as today, then use `ansible.netcommon.netconf_config` with `ansible_connection: ansible.netcommon.netconf` and port 830 (already defined as `netconf: 830` in `defaults/ports.yml`):
```
ansible.netcommon.netconf_config:
content: "{{ yaml_config | from_yaml | sros_netconf_xml }}"
target: candidate
commit: true
lock: always
```
* **readiness-check/sros.yml**: remove the gRPC plugin check and wait for TCP port 830.
* **readiness-check/srsim.yml**: convert the SR-SIM hardware configuration with the same filter and replace `gnmi_get` with `netconf_get` using a `` filter.
* **fetch-config/sros.yml**: use `netconf_get` with a `` filter (XML output) or the `md-cli-raw-command` RPC executing `admin show configuration` (MD-CLI output).
* **Installation**: replace `netlab install grpc` with installing **ncclient** (**ansible.netcommon** is already part of the Ansible distribution).
* Remove the **sros_grpc_port** group variable.
* Remove the unused OpenConfig templates (`sros.openconfig.j2`) and the **sros_use_openconfig** group variable instead of adding module-prefix-to-namespace mapping to the converter.
## Open Questions
These have to be verified on a live SR-SIM device:
* The NETCONF connection selects the **ncclient** device handler based on **ansible_network_os** (`sros`). **ncclient** has an `sros` device handler, but we have to check that the Ansible default NETCONF plugin uses it.
* Is NETCONF enabled in the default containerlab SR-SIM startup configuration, and is the **admin** user allowed to use it?
* The converter knows the list keys only for path elements. Within **val**, it relies on the ordering of the template data. That ordering could be wrong when the template data is generated with `to_nice_yaml` (which sorts the keys), for example in the SR-SIM hardware configuration. Does SR OS reject list entries in which the keys are not the first child elements?