Network Functions Virtualisation (NFV) Release 3; Testing; Specification of Networking Benchmarks and Measurement Methods for NFVI

The widespread adoption of virtualised implementation of functions has brought about many changes and challenges for the testing and benchmarking industries. The subjects of the tests perform their functions within a virtualisation system for additional convenience and flexibility, but virtualised implementations also bring challenges to measure their performance in a reliable and repeatable way, now that the natural boundaries and dedicated connectivity of physical network functions are gone. Even the hardware testing systems have virtualised counterparts, presenting additional factors to consider in the pursuit of accurate results.
 
The present document specifies vendor-agnostic definitions of performance metrics and the associated methods of measurement for Benchmarking networks supported in the NFVI. The Benchmarks and Methods will take into account the communication-affecting aspects of the compute/networking/virtualisation environment (such as the transient interrupts that block other processes or the ability to dedicate variable amounts of resources to communication processes). These Benchmarks are intended to serve as a basis for fair comparison of different implementations of NFVI, (composed of various hardware and software components) according to each individual Benchmark and networking configuration evaluated. Note that a Virtual Infrastructure Manager (VIM) may pay a supporting role in configuring the network under test. Examples of existing Benchmarks include IETF RFC 2544 Throughput and Latency (developed for physical network functions).