High CPU load by Netmap-IPFW (90-100 %)

[GoogleCodeExporter]

Dear Netmap-Ipfw Team.
We tried to use netmap-ipfw in production (as filtering bridge) for 
traffic sanity and bandwidth limitation.

kipfw starts as:
/usr/local/netmap-ipfw/kipfw netmap:ix0 netmap:ix1
current traffic:
netstat -bdh -w1 -I ix1  (ix0 the same)

          input            ix1           output
    packets  errs idrops      bytes    packets  errs      bytes colls 

     607K     0     0       753M       452K     0        88M     0     
     603K     0     0       750M       449K     0        87M     0     
     604K     0     0       751M       448K     0        88M     0     
     604K     0     0       747M       452K     0        92M     0     

all traffic:
netstat -bdh -w1

          input        (Total)           output
    packets  errs idrops      bytes    packets  errs      bytes colls 

       2M     0     0       1.6G         2M     0       1.6G     0     
       2M     0     0       1.6G         2M     0       1.6G     0     

current CPU:
CPU 0: 31.1% user,  0.0% nice, 56.1% system, 5.1% interrupt,  7.7% idle
CPU 1:  0.0% user,  0.0% nice,  0.5% system, 8.2% interrupt, 91.3% idle
CPU 2:  0.0% user,  0.0% nice,  0.0% system, 4.6% interrupt, 95.4% idle
CPU 3:  0.0% user,  0.0% nice,  0.5% system, 7.1% interrupt, 92.3% idle

THE Question:
is it normal for kipfw to take so much resoures ?
660 root        99    0   873M   325M CPU0    0 272:03  91.46% kipfw

In addition we can inform, that kipfw uses 88-93% , even if firewall is fully 
open (with first rule "allow ip from any to any"):

60 root       100    0   885M   342M CPU0    0 621:31  92.38% kipfw

In any cases kipfw occupes not more than 885M in RAM.

Below I place real ruleset from our bridge with our comments.
This bridge serves about 25K subscribers with IP from sub-network 
192.168.0.0/16.

100 allow ip from 192.168.254.0/24 to 192.168.254.0/24
200 allow ip from any to 192.168.0.0/16 - traffic to subsribers (SCr)
300 allow ip from 192.168.0.0/16 to 212.112.124.192/26-from SCr to GGC
350 deny tcp from table(25) to any dst-port 25 - block spammers
360 deny tcp from 192.168.0.0/16 to table(26) dst-port 25 - block bot-n
400 pipe 665 udp from 192.168.0.0/16 to any dst-port 6881
500 pipe 666 tcp from 192.168.0.0/16 to any tcpflags syn
750 allow ip from 192.168.0.0/16 to any - we have to use this rule
800 pipe 10 ip from 192.168.0.0/16 to any - the main rule for bridge
65535 allow ip from any to any

pipes:
# BW for packets with SYN flag and UDP-6881
${fw} pipe 665 config mask src-ip 0xffffffff bw 384Kbit/s
${fw} pipe 666 config mask src-ip 0xffffffff bw 64Kbit/s
# Outgoing BW for each IP
${fw} pipe 10 config mask src-ip 0xffffffff bw 5120Kbit/s

We have to use rule #750, cos when rule #800 in work - kipfw hits 100% in top 
and whole traffic downs from 750M (~6Gbps) to 250M (~2Gbps) with packets drops 
and delay increase.


What version of the product are you using? On what operating system?
FreeBSD-Stable 10.1
last code of netmap-ipfw
CPU: i5-4690 CPU @ 3.50GHz
RAM: 8GB x 1800Mhz
NET: Intel DA 520 (2 x 10Gbps)


Also, this issue I sent to 
http://lists.freebsd.org/pipermail/freebsd-net/2014-December/040778.html

if netmap-ipfw will take such resources, it can't be used with medium volumes 
of traffic. But we thought, that we can use it up to 10Gbps.

With best regards
Azamat B. Umurzakov
AkNet ISP

Original issue reported on code.google.com by [email protected] on 1 Jan 2015 at 12:21

[GoogleCodeExporter]

Hello, Azamat!

I executed performance tests on Linux, netmap, ixgbe 10GE.

I builded following scheme:
Flood server - eth4/filter server/eth6 - client 

And run ./kipfw netmap:eth4 netmap:eth6 and flood server with 6MPPS of syn 
flood with trafgen.

And got following results:

top - 13:33:05 up 3 min,  2 users,  load average: 0.44, 0.13, 0.05
Tasks:  78 total,   2 running,  76 sleeping,   0 stopped,   0 zombie
%Cpu(s): 10.0 us, 14.8 sy,  0.0 ni, 74.6 id,  0.0 wa,  0.0 hi,  0.7 si,  0.0 st
KiB Mem:  32980328 total,   546592 used, 32433736 free,    45320 buffers
KiB Swap:  8387580 total,        0 used,  8387580 free,    32204 cached

  PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND                                                                                  
 3105 root      20   0  660m 9060 8908 R  99.5  0.0   0:25.36 kipfw     


  1  [|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||100.0%]     Tasks: 26, 3 thr, 52 kthr; 2 running
  2  [                                                             0.0%]     Load average: 0.60 0.19 0.07 
  3  [                                                             0.0%]     Uptime: 00:03:48
  4  [|                                                            0.7%]
  Mem[|||                                                   458/32207MB]
  Swp[                                                         0/8190MB]

As you can see, kipfw uses only single thread and can't process more.

If you want more performance we should introduce threads support to kipfw. 

Original comment by pavel.odintsov on 4 Mar 2015 at 1:57

• Added labels: ****
• Removed labels: ****