controller: Reduce the time it takes to remove multiple ports. #2533

Job	Run time
linux rpm fedora	6m 51s
osx clang --disable-ssl	2m 0s
Prepare container	3m 27s
linux gcc --disable-ssl	2m 36s
linux clang --disable-ssl	2m 41s
linux gcc test -300	9m 36s
linux gcc test 301-600	6m 21s
linux gcc test 601- unstable	6m 44s
linux clang test sanitizers -300	12m 54s
linux clang test sanitizers 301-600	9m 5s
linux clang test sanitizers 601- unstable	11m 37s
linux gcc test -ljemalloc -300	9m 49s
linux gcc test -ljemalloc 301-600	6m 39s
linux gcc test -ljemalloc 601- unstable	6m 46s
linux gcc system-test-dpdk dpdk -100	16m 54s
linux gcc system-test-dpdk dpdk 101- unstable	14m 59s
linux gcc system-test-userspace -100	15m 51s
linux gcc system-test-userspace 101- unstable	13m 51s
linux gcc system-test -100	18m 22s
linux gcc system-test 101- unstable	18m 8s
linux clang system-test sanitizers -100	20m 26s
linux clang system-test sanitizers 101- unstable	19m 46s
linux x86 gcc --disable-ssl	2m 36s
	3h 57m 59s

Provide feedback