祝超博士 - DPDK on Power-ChaoZhu-En

DPDK on POWER: A New Enhanced Packet Processing ArchitectureDr. Zhu Chao IBM Research– China 2015.04.21

Agenda1.DPDK on multi-architecture 2.IBM POWER architecture 3.Migration experience 4.Summary and future work

Why do we want a multi-architecture DPDK? Fast packet processing is crucial for industries––––– Telecom operator/ manufacturer - NFV Cloud/ data center operator Security companies Video sharing companies Government

DPDK leads to significant performance improvement–––– Pure software-based optimization General purpose computing architecture Intel Xeon E5 L3 forwarding1: 10Mpps/core 6X faster than Linux native stack

Packet Processing

1. http://www.77cn.com.cn/content/dam/www/public/us/en/documents/solution-briefs/communications-packet-processing-brief.pdf

Why do we want a multi-architecture DPDK? Trend: DPDK should be hardware-independent– Common optimization techniques for difference CPU architecture– Common optimization techniques for difference NICsDPDK Library Memory management Ring/ Queue Multicore/ Multithread Poll Mode Driver

Platform HardwareIntel NICs Intel x86 CPUs NonIntel NICs

NonIntel CPUs

Hardware accelerator

NFV on IBM POWER– IBM cloud platform to support NFV

NFV is a big trend and opportunity– IBM invested to it from 2010– The TD-LTE NFV prototype was shown on IMIC2013 and MWC 2014– POWER processor is already involved in the NFV prototype

NFV on IBM POWER– IBM cloud platform to support NFVData Plane Optimization plays a vital role in NFVService Orchestration& Workflow Management Serving Gateway (SWGW) UPLINK Firewall 4G PGW Load Balancers Content Filter Charging Services Gateway HTTP Proxy Web/Video Caching Optimization

Service Engine Carrier Network Cloud ServicesWorkload orchestration service Network function service (LB, firewall, etc.) High Availability service

Resource optimization service

Advisor& monitor service

IBM OpenStack APIVM of NFV VM of NFV

…

NOVAVM of NFV

Neutron

Cinder

Ceilometer

HEAT

Data plane optimization/ DPDK

Virtual Switch Hypervisor (POWERVM, KVM)Accelerator CPU (POWER, x86) Network

Key technology focus High performance in cloud High availability in cloudStorage

Elastic resource autoscaling

HW system framework (Rack server)

Easy of management

POWER ArchitecturePOWER: Performance Optimization with Enhanced RISC

IBM WatsonPOWER 1 POWER 2

Mars roversPOWER 3

Embedded processor Apple computerPOWER 5

POWER 7

1990 1992

1993 1995

1998

2001

2004

2007

2010

2013

POWERP C

First 64 bit POWER

POWER 4

POWER 6

POWER 8

Modern POWER: From POWER6 OpenPOWER: From POWER8, provide customized processor and platform Open source: DPDK

IBM POWER8 v.s. Intel E5 v3CPUMax Freq. Max Pcore Max Lcore Cache

Intel E5 v33.7GHZ 18 36 L2: 256K LLC: 2.5MB/Core

POWER85GHZ 12 96 L2: 512 K L3: 8MB/Core eDRAM L4: Max 128 MB off chip

High performanc

e packet processing

P8 has excellent computation ability

Computation power Fast I/O and memory bandwidth

Core Core Core L2

Local SMP Links Accelerators

Core Core Core

Max Memory sizeMax Memory speed On-chip PCIe Gen3 Chip interconnection Accelerator IF.

768GB68 GB/s BW 40 lanes QPI: 9.6 GT/s x 2=19.2GT/s N/A

1TB230 GB/s sus BW 48 lanes 150 GB/s x 12 seg= 3.6TB/s CAPI: Coherent Accelerator Processor Interface

L2 L2 L2 L2 L2 8M L3 Region L3 Cache and Chip InterconnectMemCtrl MemCtrl Remote SMP Links PCI Gen 3 Links L2 L2

L2

L2

L2

L2

Core Core Core

Core Core Core

Migration experiences– Challenges

DPDK is tightly coupled with x86 architecture Low level code migration– Compiler micros doesn’t help– X86 and POWER ISA doesn’t have 1-1 mapping– X86 hardware specific instructions rdtsc, cpuid, sse, etc.– POWER ISA changes from POWER7 Half-word and byte atomic operations

Performance optimization Virtualization technology– Hypervisor

– Optimization should leverage CPU features

Migration experiences– Considerations

Start from April. 2014. Minimize the migration efforts– Migrate from a common Linux version– Both x86 and POWER platform support Linux– The compiler can be GCC

Physical host first, virtual guest later Porting first, optimizing later Migrate POWER 64 bit first

Migration experiences– Know Architecture Programming Difference

The basic difference: ISA– Most of the difference can be hided by compiler

The compiler difference The byte order difference The data width difference Data alignment differences Hardware differences

Migration experiences– Find Out Architecture-Specific Code

ISA related code– ASM code asm volatile{…}

Compiler related code– Architecture specific compiling options DPDK/config DPDK/mk Gcc: -march is not supported on POWER

– Compiler built-in micros

Migration experiences– Find Out Architecture-Specific Code

Platform specific featuresArchitecture Endianness Cache line size Huge page size Logic cores/ processor Vector instructions CPU flag register Time register X86_64 Little endian 64 Byte 4KB/2MB/1GB Max 36 Lcores SSE Yes TSC register PPC_64 Big endian/ little endian 128 Byte 64KB/16MB/16GB Max 96 Lcores VMX/ VSX N/A Time base register

HW CRC

Yes

N/A

Migration experiences– Find Out Architecture-Specific CodeArchitecture specific files/lib/librte_eal/common/include/rte_atomic.h/lib/librte_eal/common/include/rte_prefetch.h/lib/librte_eal/common/include/rte_byteorder.h/lib/librte_eal/common/include/rte_cycles.h/lib/librte_eal/common/include/rte_cpuflags.h/lib/librte_eal/common/include/rte_spinlo …… 此处隐藏：4937字，全部文档内容请下载后查看。喜欢就下载吧 ……