Let us learn Networking

[AmitKumarDas]

• FreeBSD 9.x CARP (Common Address Redundancy Protocol)

  - implemented as a separate cloned interface
  - create such interface in a host and delegate it to jail(8)
  - then automatically assign a jail's IP address alias to it

• OpenStack Networking Parts

  - DHCP doesn't imply inconsistent IP addresses. 
  - If you boot a VM without a specific IP address, 
    - it keeps the IP chosen for it until you destroy the VM. 
  - If you boot a VM with a specific (static) IP address, DHCP serves that IP to the VM. 
  - You can disable DHCP, but that tends to break the IP metadata service and
    - therefore requires implementing config drive. 
  - The provider network scenarios in the networking guide assume use of config drive.

  - You don't need to consider DVR unless you intend to use private/internal/project networks 
    - (i.e., virtual networks that reside completely within OpenStack) 
    - in addition to public/external/provider networks
      - you require higher performance and 
      - redundancy for private-to-public and private-to-private network routing. 

  - If you intend to use private networks, 
    - but only for internal comm between two or more VMs on the same private network, 
    - you can easily augment the provider networks scenario 
      - to support conventional private networks. 

  - Alternatively, L3HA provides redundancy for private-to-public and 
    - private-to-private network routing, 
      - but exchanges the performance increases of DVR for a simpler architecture.

  - Also, if you plan to keep your virtual networking simple, 
    - did you consider the Linux bridge agent?

• SCTP

  - SCTP end-point can have multiple IP addresses
  - When using multihoming you use multiple local and remote IP-addresses 
    - to provide network fault tolerant

• What is FIB

  - to have different routing policies for different kinds of traffic
  - one application can even use more than one FIB
  - different applications can use different FIBs at the same time

• Automation w.r.t FIB

  - run the tests @ tests/sys/netinet

• Routing table

  - netstat -nr                     # default routing table
  - setfib 2 netstat -nr        # routing table for fib 2

• Identify Issues

  - TSO/LRO
    - tcpdump    do you see "segments" > 32K

• Logs

  - Flags [P.], seq 1:657, ack 1, win 1040, options [..,TS val 478100 ecr 3553932482], 
    - length 656
  - Flags [S.], seq 1819579546, ack 15244367, win 65535, options
    - [mss 1460,nop,wscale 6,sackOK,TS val 3553932482 ecr 478099], length 0
  - seq 2897:4345, ack 1, win 1040, options [..,TS val 478102 ecr 2829000022], length 1448
  - ack 5793, win 1018, options [nop,nop,TS val 2829000023 ecr 478102], length 0

• Tuning & Claims / Reports

  - removing rxcsum/txcsum did not provide differences
  - removing RSS + disabling rx/cxsum support provided better numbers.
  - with netmap and i40e on linux & enabling FlowDirector killed the pps throughput 
    - (from 32 down to 18 Mpps)
  - running on FreeBSD 10.2 amd64 got 9.6 GB/s with 1 netperf stream
    - run 4 streams & you get 36 GB/s
  - the performance drop happened on dpdk poll mode
  - RSS performance
    - RSS@4, RSS@16, RSS disabled
  - if avg pkt size of 768 bytes then not a card resource starvation