End-to-End Qos Implementation - Home - Cisco Community
End-to-End Qos Implementationand Operation with NexusRohan Grover, Technical Marketing ManagerBRKDCT-3346
Session Objectives Provide a refresh of QoS and Queuing Basics Understand the basic hardware architecture for the Nexus Platforms Provide a detailed understanding of QoS on Nexus platforms Examine real-world deployment examples3
Session Non-Objectives Nexus hardware architecture deep-dive Nexus 3000/9000 QOS (in Backup slides) Application Centric Infrastructure (ACI) QOS4
Related SessionsSession IdSession NameBRKARC-3470Cisco Nexus 7000/7700 Switch ArchitectureBRKARC-3452Cisco Nexus 5600 and 6000 ArchitectureBRKRST-2509Mastering Data Center QoS5
Agenda Introduction QoS and Queuing Basics QoS Implementation on Nexus Nexus 7000/7700 QoS Nexus 5x00/6000 QoS Nexus 2000 QoS Real World Deployment Conclusion6
Introduction7
Wikipedia.org8
Margaret RouseWhatis.com9
Congestion Happens Everyday!10
Why QoS in the Data Center?AssignColor to TrafficManageCongestionMaximizeThroughputMaximize Throughput and Manage Congestion!11
Agenda Introduction QoS and Queuing Basics QoS Implementation on Nexus Nexus 7000/7700 QoS Nexus 5x00/6000 QoS Nexus 2000 QoS Real World Deployment Conclusion12
QoS and Queuing Basics13
22nd AnniversaryThe QoS ToolsetClassificationPolicingIdentify andSplit Trafficinto DifferentClassesDiscardMisbehavingTraffic to MaintainNetwork IntegrityMarkingQueuing &SchedulingMark TrafficPrioritize,According to Protect andBehavior and Isolate TrafficBusinessBased onPoliciesMarkingsShapingControlBursts andConformTraffic14
Traffic Management ToolsClassification Traffic CategorizationMarking Traffic Re-categorizationPolicing Limit misbehaving flows15
Classification and Marking – Two sides of a coinIdentify traffic DSCP CoS ACLs IP PRECBefore:Uncontrolled ackupRemap Traffic Like to Like (i.e CoS to CoS) Like to Unlike (i.e DSCP to COS) Needs mapping tablesAlso called MutationVoIP/LyncAfter:Controlled BandwidthEmail/HTTPSAPBackupMad Max (1979)16
Policing – limit misbehaving trafficTokensSingle rate Two Color Policer Conform Action (permit) Exceed Action (drop)Two rate Three Color Policer Conform Action (permit) Exceed Action (markdown) Violate Action (drop)BurstLimitPolicerConformingTrafficExcess TrafficMad Max (1979)17
Congestion Management ToolsBuffering Storing packets in memoryQueuing Buffering packets according to traffic classScheduling Order of transmission of buffered packetsShaping Smooth bursty traffic18
Buffering – Why do we need it?Many to One Conversations Client to Server Server to Storage Aggregation Points10 GE10 GE1 GESpeed Mismatch Client to WAN to Server1 GE1 GE1 GE19
Scheduling – Who goes first? Defines Order of transmission The Priority-Queue always serviced first Normal Queues serviced only after Priority Queue empty Different Scheduling algorithms for normal queues20
Common Scheduling AlgorithmsRound Robin (RR) Simple and Easy to implement Starvation-freeWeighted Round Robin (WRR) Serves n packets per non-emptyqueue Assumes a mean packet sizeDeficit Weighted Round Robin Variable sized packets Uses a deficit counterShaped Round Robin More even distributed ordering Weighted interleaving of flows21
4 Class Queuing Model exampleClassCoSQueuesPriority5-7PQNo-Drop3, 4Q22Q10, 1Default-QBetter than Best-EffortBest-Effort22
Congestion Avoidance ToolsTail Drop (TD) Drop packets at tail of the queue Single threshold per queueWeighted Random Early Drop (WRED) One or more thresholds per queue Threshold associated with DSCP or COS23
Putting it all together!12Packet is placed in the EgressQueue according to CoS/DSCPvalue.Priority Queuealways served firstCOSQueue5, 6, 7PQ13, 4Q32Q20, 1Q-DefaultEgress PortNormal QueueDrop Threshold 4PQ14Normal Queues haveThreshold and droppacket accordinglyDrop Threshold 3dataL3DSCP [24]L2CoS [3]Q3SchedulerDWRRDrop Threshold 2Q2Drop Threshold 13Normal Queues servedafter PQ is emptyaccording to SchedulerQ-Default24
Congestion could result in?HeadofLineBlocking25
Head of Line BlockingWhat is the Problem?Red Carsblocked!No Congestion26
Head of Line BlockingSolution – Create Separate Lanes but StadiumFullCars turnedback27
Head of Line BlockingStadiumFullSolution – Have an arbiterSupermarketNot Full28
Head of Line BlockingWhat is the Problem?Ingress ModuleEgress ModulebufferIngress ModuleIngress ModuleCrossbarFabricEgress ModuleEgress Module29
Virtual Output QueuesAvoid Head of Line BlockingIngressVOQIngress ModuleIngress ModuleIngress ModuleEgress ModuleCrossbarFabricEgress ModuleEgress Module30
Virtual Output QueuesAvoid Head of Line BlockingIngressVOQIngress ModuleEgress ModulebufferIngress ModuleIngress ModuleCrossbarFabricEgress ModuleEgress Module31
Agenda Introduction QoS and Queuing Basics QoS Implementation on Nexus Nexus 7000/7700 QoS Nexus 5x00/6000 QoS Nexus 2000 QoS Real World Deployment Conclusion32
QoS Implementation on Nexus33
Nexus uses Modular QOS CLI (MQC)3 Block ConstructClass-Map What Traffic do we care about?DSCPCoSIPPRECACLsPolicy-MapWhat action do I take on theseclasses? Policing Marking SchedulingService-PolicyWhere do I apply this policy? System Wide Vlan Interface (in/out) Port-channels34
New Data Center QoS CapabilitiesPriority Flow Control (802.1Qbb) Enables Lossless Ethernet using perCOS pause During congestion, no-drop COS ispaused No effect on other COS values35
New Data Center QoS CapabilitiesDCBXP (802.1Qaz) LLDP with new TLV Values Negotiates capabilities (like PFC) with otherdevicesECN (Explicit Congestion Notification) Congestion Notification without droppingpackets Uses two LSB bits in DiffServ field in IP headerDiffServ Field Values0x00 – Non ECN Capable0x10 – ECN Capable Transport (0)0x01 – ECN Capable Transport (1)0x11- Congestion Encountered36
Data Center ApplicationsFCoE (Fiber Channel over Ethernet) Replaces the FC0 and FC1 of the Fiber Channel stack withEthernet Reduces cabling and applicable to server virtualizationapplicationsRoCE (RDMA over Converged Ethernet) Remote Direct Memory Access allows network adapters todo server-to-server data transfer between applicationmemory RoCE is a IBTA (InfiniBand Trade Association) standard thatbrings RDMA capabilities over Converged Ethernet37
Network-QoS Policy Define global queuing and scheduling parameters for all interfaces in switchIdentify drop/no-drop classes, MTU and WRED/TD, etc. One network-QoS policy per system, applies to all ports Assumption is network-QoS policy defined/applied consistently network-wideNetwork QoS policies should be applied consistentlyon all switches network wideSwitch 2Switch 1IngressModuleIngressModuleIngressModuleSwitch leFabricEgressModuleIngressModule38
Three Different TypesClass-mapQoS CoS DSCP PREC ACLsQueuing CoS DSCPNetwork-QoS CoS Protocol (fcoe)Policy-mapQoS Marking Policing MutationQueuing Buffering Queuing SchedulingNetwork-QoS Congestion-Control Pause / MTU per VLService-policyQoS Interfaces Vlans Port-channel System-qosQueuing Interfaces Port-channels System-qosNetwork-QoS System-qos39
To Trust or Not To Trust? Data Centre architecture provides a newset of trust boundaries Virtual Switch extends the trustboundary into the Hypervisor Hence Nexus Switches always trustCoS and DSCP40
Agenda Introduction QoS and Queuing Basics QoS Implementation on Nexus Nexus 7000/7700 QoS Nexus 5x00/6000 QoS Nexus 2000 QoS Real World Deployment Conclusion41
Nexus 7000/7700 QoS42
Nexus 7000 / 7700 I/O Module FamiliesM2 10G / 40G / 100GM1 1G and 10GF1 10GF3 40GF2E 10GF2 10GF2E 10GF3 10G / 40G / 100G43
M2 I/O Module ArchitectureEOBCTo Fabric ModulesTo Central ArbitersArbitrationAggregator Fabric 2 gineReplicationEngine12 X 10G MAC / LinkSec12345VOQs678912 X 10G MAC / LinkSec10 BRKARC-347011 1213 14 15 16 17 18 19 20 21 22 23 24Cisco Nexus 7000/ 7700Switch ArchitectureFrontPanel Ports44
M2 I/O QoS Packet-Flow (10G)1CoS to Queue-Mapping Policy Type “queuing”(Scheduling, Buffering)3 Policy Type “qos”(Mutation, Classification,Marking, Policing)58Credit Grant forFabric AccessReturn Creditto PoolArbiter9CoS to Queue-Mapping Policy Type “queuing”(Scheduling, Buffering)ForwardingEnginedataL3DSCP [24]L2CoS [2]Ingress Module4VoQ gets assignedbased on Dest CoS value1 2 3 4 5 6 7 88CrossbarFabricEgress ModuleL3DSCP [24]L2CoS [2]7 Traffic getsReceived onEgress VoQ6 Traffic getsforwarded overFabricIngressdata1 2 3 4 5 6 7 88q2t1 2 33 4Ingress port-ASICVOQ21p3q1tDefault CoS-to-Queue map:CoS [2] to Default-Q (8)Ingress VoQ-ASICEgress1p7q4t11EgressFIFOEgress VoQ-ASIC10Egress port-ASICDefault CoS-to-Queue map:CoS [2] to Default-Q (8)45
Nexus 7700 F3 48-Port 1G/10G Module ArchitectureTo Fabric ModulesEOBC6 X 110GbpsFSACPUTo Central Arbiters6 X 110GbpsArbitrationAggregator x61G per SOCFabric ASICLC InbandFabric ASIC55Gbps55Gbpsto LCCPU8 X 10GSoC 11328 X 10GSoC 25476981110 128 X 10GSoC 313 1514 1617 198 X 10GSoC 421 2325 278 X 10GSoC 529 3118 2022 2426 2830 32Front Panel Ports (SFP/SFP )33 3534 368 X 10GSoC 637 3938 4041 4345 4742 4446 4846LinkSec-capable
F3 SoC ArchitectureTo CentralArbiterTo LocalFabricFrom LocalFabricDecision EngineEgress ParserVirtualQueuingIngressBufferLayer 2 LookupsLayer 3 LookupsClassificationMarking / PolicingEgress BufferIngress ParserPort LogicF3 SoCFront Panel Ports8x10G2x40G1x100G47
Supervisor EngineBuffer creditgrantedCredit CentralF2E/F3 QoS Packet WalkTransmitto fabricRequest buffercredit for dest priorityArbiterFabric Module 1Fabric Module 2Fabric Module 3Fabric Module 4Fabric Module 5Fabric ASICFabric ASICFabric ASICFabric ASICFabric ASICFabric ASICCheckVQ dropthresholdsReturn buffer credit(dest priority)Fabric ASICFinal lookup result:destination priorityReqDecision EnginePKTe2/1VirtualQueuingHDRe1/1Queue packetdescriptor in VOQ(dest priority)IngressBufferPayload queued inport buffer based onCOS / DSCPCL TCAMLayer 2 LookupsLayer 3 LookupsClassificationMarking / PolicingIngressParserPKT HDRReceivefrom fabricCL TCAM containsQoS classificationrulesAll L2 L3 lookups /Ingress egress QoS classification /Ingress egress marking and policingSPPacket headerssent to DEBuffer on egress basedon dest priorityPort LogicModule 1DWRRe2/1Egress BufferPort LogicF2E/F3 SoCReceive packetfrom wireCreditIngress packetparsingScheduleand transmiton wireF2E/ F3 SoCModule 2PKT HDRe1/1e2/148
Ingress Queuing – Logical View8e Template8e-4q4q Template7e Template6e TemplateCoS 5-7CoS 5-7CoS 5-7CoS 5-7(Q1)(Q1)(Q1)(Q1)4e TemplateCoS 5-7(Q1)CoS 0(Q-Default)CoS 3-4CoS 2,4CoS 0-2(Q3)(Q3)(Q-Default)CoS 4(Q3)CoS 0-4CoS 2CoS 0,1CoS 4(Q-Default)(Q4)(Q-Default)(Q3)CoS 0-1CoS 3CoS 3(Q-Default)(Q4)(Q4)High (Pause)ThresholdLow (Resume)ThresholdCoS 1-3(Q4)49
Egress Queuing – Logical ViewPQ2. PQ3 PQ1 Q-Def.(4)(3) (5,6,7) (0-2)PQ1 Q-Def. PQ2 Q3(5,6,7) (0)(4) (1,2,3)DWRRDWRRPriorityPrio50%50%Q2red indicates no-dropPrio100%PrioDWRRDWRRPQ2 (H)Q-Def33%Q3Q-Def33%PQ1Q333%Egress PortQ233%PQ1Q-Def33%Egress PortQ333%PrioDWRRDWRRQ2PQ1Egress PortDWRRPriorityEgress PortPriority100%DWRRQ3DWRRPQ2 (L)PQ1 Q3 Q-Def.(5,6,7) (2,4) (0,1)Q-DefQ2(3)PQ1 (H)Q3 Q-Def.(2) (0,1)4e TemplateEgress PortPQ1 Q2(5,6,7) (3,4)6e TemplatePQ1PQ1 Q2 Q3 Q-Def.(5,6,7) (3,4) (2) (0,1)7e TemplateQ-Def8e-4q4q TemplatePQ3 (L)8e Template100%50
Shared Buffer Queuing on F3 Disabled by Default and enabled on permodule-basis Default ratio is 50:50 for dedicated/shared Ratio can be changed per port-group(SOC) basis6.2.10onwardsSOC Ingress BufferPort 1DedicatedPort 2DedicatedShared Packet BufferPort 3DedicatedPort 4DedicatedSwitch(config)# hardware qos shared-buffer module module-numberswitch(config)# hardware module module-number port-group port-group-numberswitch(config-port-group)# qos shared-buffer queue-limit percent51
CoS or DSCP to Queue Mapping Default CoS to Queue Mapping for Nexus 7000 (F- and M-Series I/O Module) Ingress: CoS to Queue Egress: CoS to Queue Default DSCP to Queue Mapping for Nexus 7700 Ingress: DSCP to Queue Egress: CoS to Queue Global Configuration (Admin/Default VDC) required to enable DSCP to QueueMapping:N7k(config)# hardware qos dscp-to-queue ingress module type {all f-series m-series}52
Changing the Default Trustrouted/bridged802.1q Trunk802.1q Trunk /Access PortL3DSCP [24]CoS [2]L2EgressQueuesCoSdataCoSIngressQueuesq-nq-1Set CoS 0dataL3L2Set DSCP 0L3L2q-default(will only set CoS to 0)dataDSCP [0]CoS [0]DSCP[24]DSCP[0]CoS [0]Note: CoS is used for egressqueue selection, even if theegress interface does NOTcarry CoS in the frame(will set DSCP CoS to 0)53
Nexus 7000 QoS Golden Rules QoS is enabled by default and cannot be disabled CoS and DSCP are TRUSTED by default Default Queuing and QoS policies are applied to all physical interfaces acrossall VDCs For bridged traffic, CoS is preserved, DSCP is unmodified For routed traffic, DSCP is copied to CoS (first 3 bits) Ex: DSCP 40 (b101000) becomes CoS 5 (b101)54
Agenda Introduction QoS and Queuing Basics QoS Implementation on Nexus Nexus 7000/7700 QoS Nexus 5x00/6000 QoS Nexus 2000 QoS Real World Deployment Conclusion55
Nexus 5000/6000 QoS56
Ingress Buffering ModelNexus 5500/5600/6000 compared to Nexus 7000 F-Series I/O ModuleIngressVOQ bufferIngressVOQ bufferEgressVOQ bufferEgressVOQ bufferIngress ModuleNexus 7000F-SeriesI/O ModuleCrossbarIngress ModuleEgress ModuleFabricIngress ModuleIngress VOQ buffer – Manages congestion toward egress destinationsEgress VOQ buffer – Receives frames from fabric; also buffers multi-destination frames57
Key Concepts – Common PointsNexus 7000 compared to Nexus 5000/6000 QoS Nexus 5000/6000 & Nexus 7000 F-Series I/O Modules share the Ingress BufferModel Ingress buffering and queuing occur at VOQ of each ingress port Egress scheduling enforced by egress port No Egress QOS Policies58
Cisco Nexus 5600/6000 QoS Features Traffic classification‒ DSCP, CoS, IP Precedence and ACL Packet marking‒ DSCP, CoS, and ECN Strict Priority Queuing and DWRR‒ Priority Flow Control‒ DCBX 802.1Qaz Ingress policing (No egress policing)‒ 4096 policers per ASIC Flexible buffer management59
Cisco Nexus 5672UP Internal ArchitectureFabricSupervisorUPC 1UPC 2UPC 3UPC 4UPC 5UPC 6UPC-0FC PHY24xSFP 24xSFP Slot 16xQSFP CPUSlot 260
Packet Buffering 25MB packet buffer is shared by every three 40 GE ports or twelve 10 GE ports. Buffer is 16MB at ingress and 9MB at egress. Unicast packet can be buffered at both ingress and egress.16MBMulticast VOQIngressUPCUnicast VOQ Multicast Buffered at egress onlyEgress UPC224GbpsUnifiedCrossbarFabric448Gbps9MB61
Nexus 5600 QoS ProcessingIngressUPCTrust CoS/DSCP andMatch on PMarkingIf Buffer Usage Crosses Threshold: Tail drop for drop class Assert Pause for no-drop COSIngressPolicingForwardingPer-classBuffer UsageMonitoringIngress BufferEgress QueuesMACEgress UPCWRED/ECNMarkingStrict Priority DWRR SchedulingCrossbarFabricMake a forwarding decisionVoQs(8 per egress port)EgressSchedulingEgress Buffer62
Flexible Buffer ManagementIngress Buffer Shared buffer is good for burstabsorption.UPC Ingress Buffer (16MB)Dedicated buffer is good forpredictable performance for each port. On by default, no configurationneeded Long-distance FCoE, video editing(i.e., AVID), Big Data, and distributedstorageSPANControlShared Packet BufferPort 1DedicatedPort 2DedicatedPort 3Dedicated63
Tune Buffer Allocation at Ingress “queue-limit” under “network-qos” policy specifies the dedicated buffer for each port and each class.The dedicated buffer can be used by the port for only that class of service. Without “queue-limit” each class of service will get 100 KB of dedicated buffer. The size of dedicated buffer can be different for different classes of service. The policy applies to allports in the chassis. Total ingress buffer minus the dedicated buffer and buffer for control and SPAN will be in the sharedbuffer pool. The following example sets the dedicated buffer for “class-default” to be 400 KB for all ports.switch(config)# policy-map type network-qos Policy-bufferswitch(config-pmap-nq)# class type network-qos class-defaultswitch(config-pmap-nq-c)# queue-limit 400000 bytesswitch(config-pmap-nq-c)# system qosswitch(config-sys-qos)# service-policy type network-qos Policy-buffer64
Flexible Buffer ManagementEgress Buffer 9-MB packet buffer is shared amongthree 40 GE or twelve 10 GE. CLI is provided to allocate bufferbetween unicast and multicast (future). Unicast traffic can be buffered at egressand ingress. Multicast is buffered at egress in caseof interface oversubscription.UPC Egress Buffer (9MB)Unicast BufferMulticast Buffer65
WRED/ ECN Configuration ECN parameters are configurable only at system level. ECN is enabled by default along with WRED Packet Threshold below minimum – Transmit Packet Threshold between minimum and maximum – Mark ECN bits Packet Threshold above maximum – Dropswitch(config)# hardware random-detect min-thresh 10g 10g-min-threshold 40g 40g-minthreshold max-thresh 10g 10g-max-threshold 40g 40g-max-threshold ecn qos-group qosgroup-number66
Nexus 5600/6000 QoS Configuration Model Uses QOS-Groups to tie together QoS, Queuingand Network-QoS policies QoS-Group has no direct relation with CoS value QoS-Groups defined (set) in policy-map typeqos. QoS-groups referenced (match) in policy typequeuing and policy-map type networkqos67
Putting it all togetherCreate class-maptype qos andmatch oncos/dscp/aclsAttach policy-mapqueuing tointerfaceCreate policy-maptype qos and setqos-group and/oradd policing ruleCreate policy-maptype queuing andcreate actionsAttach policy-maptype qos as input toan interfaceCreate class-maptype queuing andmatch on qosgroupclass-map type qos class foomatch cos 3-4policy-map type qos pm1class type qos class fooset qos-group 1police cir 20 mbytes conform transmit violate dropclass type qos class-defaultset qos-group 0interface ethernet 1/1service-policy type qos input pm1class-map type queuing class-foomatch qos-group 1policy-map type queuing policy-fooclass type queuing class-foobandwidth percent 20class type queuing class-defaultbandwidth percent 80interface ethernet 1/3service-policy type queuing input policy-foo68
Nexus 5600/6000 QoS Golden Rules ECN/WRED is enabled by default and cannot be disabled CoS and DSCP are TRUSTED by default Use QoS-Groups to tie policies together No Egress QOS policies69
Agenda Introduction QoS and Queuing Basics QoS Implementation on Nexus Nexus 7000/7700 QoS Nexus 5x00/6000 QoS Nexus 2000 QoS Real World Deployment Conclusion70
Nexus 2000 QoS71
FEX Overview Scalable and Extensible Fabric Single point of management Homogeneous and consistent policies72
Cisco Nexus 2000 QoS Features Traffic classification‒ DSCP, CoS‒ ACL classification (FEX offload) –only on 5600/6000 Strict Priority Queuing and DWRR‒ Priority Flow Control Queue-limit Carving73
FEX QoS Policies Support for ingress port-based QoS policies on FEX HIF ports Support for ingress/egress VLAN-based QoS policies on FEX VLANs FEX QoS policies applied at ingress module of parent switch No support for remarking, policing policiesIngress portbased QoSpolicyIngress/egressVLAN-based QoSpolicyNexus ParentFEXIngress FEXModule/PortIngress ModuleModule/PortEgress ModuleFEXEgress FEX74
FEX QoS Packet Flow Example (CoS2Q)Ingress queue /egress schedulebased on COS 0Egress schedulebased on COS 0Ingress queue atinput port based onCOS 0Ingress queue /egress schedulebased on COS 0Nexus ParentPKTDSCP 01Q trunkCOS 0FEXIngress FEX(FEX 101)ModuleIngress ModuleFabricModuleEgress ModuleFEXEgress FEX75
FEX QoS Packet Flow Example (DSCP2Q)Ingress queue /egress schedulebased on DSCP 0Egress schedulebased on COS 0Ingress queue atinput port based onDSCP 0Ingress queue /egress schedulebased on DSCP 0Nexus ParentPKTDSCP 0COS 0FEXIngress FEX(FEX 101)ModuleIngress ModuleFabricModuleEgress ModuleFEXEgress FEX76
FEX QoS Packet Flow Example (With Ingress Marking Policy and DSCP-to-Queue)Ingress QoS policy:policy-map type qos markerclass ClassAset dscp cs3!interface ethernet 101/1/1service-policy type qos input markerIngress queue /egress schedulebased on DSCP 0Egress schedulebased on COS 3Ingress queue atinput port based onDSCP 0Ingress queue /egress schedulebased on DSCP CS3Nexus 7000PKTDSCP 0COS 0FEXIngress FEX(FEX 101)PKTDSCP CS3ModuleCOS 3FabricIngress ModuleModuleEgress ModuleFEXEgress FEXClassify and remark toDSCP CS3/COS 3 atingress Decision Engine77
FEX Policy Offload (Nexus 5600/6000 only) TCAM resources on a FEX to perform ACL-based classification The feature is disabled by default By default, a FEX classifies packets on CoS value Both system level and interface level policies are offloaded to the FEXswitch# configure terminalswitch(config)# fex chassis IDswitch(config)# hardware card-type qos-policy-offload78
FEX Queuing Policies FEX queuing driven implicitly by parent switch queuing configuration Network QoS template drives: Number of queues MTU Ingress queuing class-maps drive: Both ingress and egress COS/DSCP-to-queue mappingOn Nexus 7000 with FEX M-Series parent modules, network-qos and F-seriesingress queuing class-maps still drive FEX queuing configuration79
DSCP-to-Queue on FEX Enabling DSCP-to-queue on parent switch enables DSCP-to-queue on FEX Currently active DSCP mappings pushed to FEX when enabled DSCP-to-queue only active in the HIF NIF direction NIF HIF direction always uses COS-to-queue mapping, based on COS transmitted byparent switch to FEX80
FEX Queue-Limit Provides FEX queue-limit configuration option Manages buffer thresholds on FEX based on platform capabilities Default has queue-limit enabled, disabling not recommeded Configuration applied per-VDC (on Nexus 7000/7700) Different FEX models have different capabilities81
Nexus 2000 QoS Golden Rules FEX QOS classification on COS or DSCP unless FEX offload enabled FEX queuing driven implicitly by parent switch queuing configuration No support for per-queue shaping, policing or marking Drop thresholds are tail-drop only, no WRED support82
Agenda Introduction QoS and Queuing Basics QoS Implementation on Nexus Nexus 7000/7700 QoS Nexus 5x00/6000 QoS Nexus 2000 QoS Real World Deployment Conclusion83
Real World Deployment84
What do we want to achieve?Company XYZ’s Business Intent Make sure no disruption in network services Put control traffic in priority queue Video/voice hosting also an business objective Put voice traffic in priority queue Dedicated bandwidth to video traffic Flexibility in moving applications across servers Dedicated bandwidth to vmotion/mobility Everything else best-effort85
Translating to the language of ffer)CharacterBest Effort0, 1BW remaining50%60%High Volume /Less ImportantvMotion / LiveMigration2BW remaining20%10%Medium Volume /ImportantMultimedia3, 4BW remaining30%20%Medium VolumeVery ImportantStrict Priority5Priority QueueNetwork Control6,710%Low Volume /Important /Delay SensitiveLow Volume /Very important86
TopologyCoreM2 cards facingcore and F2/F3cards facingaccessNexus 7000Nexus 7000vpc peer-linkVPC fromaccess toaggregationNexus 5000Straight-throughFEX, No VPCNexus 5000vpc peer-linkNexus 5000vpc peer-link Nexus 5000Host VPC,Straight-throughFEXNexus 2000Nexus 2000Nexus 2000Nexus 5000vpc peer-link Nexus 5000Enhanced VPC(evPC)Nexus 2000Nexus 2000Nexus 200087
Classification, Marking & Trust on Nexus 5000/7000CoreMark Traffic fromCore(policy-map type qos)or just TRUST(default)vpc peer-linkvpc peer-linkBetween thedifferent Tiers,all ports areTRUSTEDvpc peer-linkvpc peer-linkMark Trafficfrom Servers(policy-maptype qos)or just TRUST(default)88
Classification & Marking: Nexus 7000ip access-list ACL QOS LOWPRIO10 permit ip access-list ACL QOS VMOTION10 permit ip access-list ACL QOS MULTIMEDIA10 permit ip access-list ACL QOS SCRICTPRIO10 permit !class-map type qos match-any CM QOS LOWPRIO COS1match access-group name ACL QOS LOWPRIO!class-map type qos match-any CM QOS VMOTION COS2match access-group name ACL QOS VMOTION!class-map type qos match-any CM QOS MULTIMEDIA COS4match access-group name ACL QOS MULTIMEDIA!class-map type qos match-any CM QOS STRICTPRIO COS5match access-group name ACL QOS STRICTPRIOpolicy-map type qos PM QOS MARK COS INclass CM QOS STRICTPRIO COS5set cos 5class CM QOS MULTIMEDIA COS4set cos 4class CM QOS VMOTION COS2set cos 2class CM QOS LOWPRIO COS1set cos 1!interface Ethernet1/1service-policy type qos input PM QOS MARK COS IN!vlan configuration 100service-policy input PM QOS MARK COS IN89
Classification & Marking: Nexus 5600/6000 (1)ip access-list ACL QOS LOWPRIO10 permit ip access-list ACL QOS VMOTION10 permit ip access-list ACL QOS MULTIMEDIA10 permit !class-map type qos match-any CM QOS LOWPRIO COS1match access-group name ACL QOS LOWPRIO!class-map type qos match-any CM QOS VMOTION COS2match access-group name ACL QOS VMOTION!class-map type qos match-any CM QOS MULTIMEDIA COS4match access-group name ACL QOS MULTIMEDIA!class-map type qos match-any CM QOS STRICTPRIO COS5match cos 5policy-map type qos PM QOS MARK COS INclass CM QOS STRICTPRIO COS5set qos-group 5class CM QOS MULTIMEDIA COS4set qos-group 4class CM QOS VMOTION COS2set qos-group 3class CM QOS LOWPRIO COS1set qos-group 2!system qosservice-policy type qos input PM QOS MARK COS INQoS-Group # is mapping between Slide 1 & Slide 290
Classification & Marking: Nexus 5500/6000 (2)class-map type network-qosmatch qos-group 2class-map type network-qosmatch qos-group 3class-map type network-qosmatch qos-group 4class-map type network-qosmatch qos-group 5CM N-QOS MATCH QG2 COS1CM N-QOS MATCH QG3 COS2CM N-QOS MATCH QG4 COS4CM N-QOS MATCH QG5 COS5policy-map type network-qos PM N-QOS SYSTEMclass type network-qos CM N-QOS MATCH QG2 COS1set cos 1class type network-qos CM N-QOS MATCH QG3 COS2set cos 2class type network-qos CM N-QOS MATCH QG4 COS4set cos 4class type network-qos CM N-QOS MATCH QG5 COS5set cos 5queue-limit 20480 bytes!system qosservice-policy type network-qos PM N-QOS SYSTEMQoS-Group # is mapping between Slide 1 & Slide 291
Classification & Marking (Nexus 2000)Corevpc peer-linkvpc peer-linkvpc peer-linkvpc peer-linkTrafficClassificationbased on CoShappens here!TrafficClassificationbased on ACLhappens here!92
Queuing (M2-cards)Corevpc peer-linkAssign a egress policymap to each interfacewith priority, bandwidthand queue-limit to alterdefault queuing policyvpc peer-linkModify CoS toQueue mappingin Admin/Default-VDCAssign ingresspolicy-map to eachinterface withbandwidth andqueue-limitvpc peer-linkvpc peer-link93
CoS to Queue Mapping – M2 I/O e-Limit(Buffer)Queue(8q2t / 1p7q4t)CharacterBest Effort0, 1BW remaining50%60%8q2t-in-q-default /1p7q4t-out-q-defaultHigh Volume /Less ImportantvMotion / LiveMigration2BW remaining20%10%8q2t-in-q6 /1p7q4t-out-q6Medium Volume /ImportantMultimedia3, 4BW remaining30%20%8q2t-in-q2 /1p7q4t-out-q2Medium VolumeVery ImportantStrict Priority5Priority QueueNetwork Control6,710%8q2t-in-q1 /1p7q4t-out-pq1Low Volume /Important /Delay SensitiveLow Volume /Very important94
CoS to Queue Mapping (on M2 cards)Exmple (Admin- / Default-VDC)class-map type queuingmatch cos 5-7class-map type queuingmatch cos 3-4class-map type queuingmatch cos 2class-map type queuingmatch cos 0-1!class-map type queuingmatch cos 5-7class-map type queuingmatch cos 3-4class-map type queuingmatch cos 2class-map type queuingmatch cos 0-1match-any 8q2t-in-q1Corematch-any 8q2t-in-q2match-any 8q2t-in-q6AdminVDCAdminVDCmatch-any 8q2t-in-q-defaultmatch-any 1p7q4t-out-pq1match-any 1p7q4t-out-q2match-any 1p7q4t-out-q6match-any 1p7q4t-out-q-defaultChanges apply to ALL ports of specified type in ALL VDCsChanges are traffic disruptive for ports of specified type95
Ingress Queuing Configuration on M2 cardsExample (Payload-VDC)policy-map type queuing PM QUEUE 10G-40G-100G INclass type queuing 8q2t-in-q1queue-limit 10bandwidth remaining percent 10class type queuing 8q2t-in-q2queue-limit 20bandwidth remaining percent 30class type queuing 8q2t-in-q6queue-limit 10bandwidth remaining percent 10class type queuing 8q2t-in-q-defaultqueue-limit percent 50bandwidth percent 50!interface Ethernet1/1service-policy type queuing input PM QUEUE 10G-40G-100G INAll Policy-Map and Service-Policy are done in relevant Payload-VDC andonly affect the interface to which they get appliedCore96
Egress Queuing Configuration on M2 cardsExample (Payload-VDC)policy-map type queuing PM QUEUE 10G-40G-100G OUTclass type queuing 1p7q4t-out-pq1priority level 1queue-limit percent 10class type queuing 1p7q4t-out-q2queue-limit percent 20bandwidth remaining percent 30class type queuing 1p7q4t-out-q6queue-limit percent 10bandwidth remaining percent 20class type queuing 1p7q4t-out-q-defaultqueue-limit percent 50bandwidth remaining percent 40!interface Ethernet1/1service-policy type queuing output PM QUEUE 10G-40G-100G OUTAll Policy-Map and Service-Policy are done in relevant Payload-VDC andonly affect the interface to which they get appliedCore97
Network-QoS Configuration on F2/F3-SeriesCorevpc peer-linkvpc peer-linkvpc peer-linkUse the networkqos policy for 8e4q4q i.e 4 ingressqueuesvpc peer-link98
Network-QoS Configuration –F2/F3 cardsExample (Admin- / Default-VDC)Coresystem qosservice-policy type network-qos default-nq-8e-4q4q-policypolicy-map type network-qos default-nq-8e-4q4q-policy template 8e-4q4qclass type network-qos c-nq-8e-4q4qmatch c
Fabric ASIC e1/1 e2/1 e2/1 SP DWRR F2E/F3 QoS Packet Walk PKT HDR PKT HDR Req Credit Credit PKT HDR Receive packet from wire Ingress packet parsing Packet headers sent to DE . Low (Resume) Threshold 8e-4q4q Template CoS 3-4 (Q3) CoS 5-7 CoS 2 (Q4) CoS 0-1 (Q-Default) 49. Egress Queuing –L
Catalyst 3750 Switch Software Configuration Guide 78-15164-04 29 Configuring QoS This chapter describes how to configure quality of service (QoS) by using automatic QoS (auto-QoS) commands or by using standard QoS commands on the Catalyst 3750 switch. With QoS, you can provide preferenti
HowtoConfigureAuto-QoS 106 ConfiguringAuto-QoS 106 EnablingAuto-QoS 106 TroubleshootingAuto-QoS 108 MonitoringAuto-QoS 109 ConfigurationExamplesforAuto-Qos 110 Examples:GlobalAuto-QoSConfiguration 110 QoS Configuration Guide, Cisco IOS Release 15.2(2)E (Catalyst 2960-X Switches) vi OL-32552-01 Contents
QoS technologies, or tools, each have speciic roles that are used in conjuncion with one another to build end-to-end network QoS policies. Skype QoS A Powerful Duo Together, Skype and QoS make a powerful duo for two reasons: 1. QoS has a signiicant impact on the network when it comes to high-bandwidth video and mulimedia transmissions.
LTE - QoS LTE architecture supports hard QoS, with end-to-end quality of service and guaranteed bit rate GBR for radio bearers. Just as Ethernet and the internet have different types of QoS, for example, various levels of QoS can be applied to LTE traffic for different applications. Because the LTE MAC is fully scheduled, QoS is a natural fit.File Size: 381KB
times of congestion. Quality of Service (QoS) management is the collection of mechanisms that control how traffic is prioritized and handled during these times. QoS technologies, or tools, each have specific rolls that are used in conjunction with one another to build end-to-end network QoS policies. The two most common QoS tools used to
Configuring Auto-QoS FindingFeatureInformation,page1 PrerequisitesforAuto-QoS,page1 RestrictionsforAuto-QoS,page2 InformationAboutConfiguringAuto-QoS .
Chapter-7 VoIP Case Study Voice Bandwidth Calculators . www.itttc.net 3 Quality of Service (QoS) Chapter 1 Cisco VoIP Implementations . 1.3 Encapsulating Voice Packets 1.4 Bandwidth Calculation 1.5 Implementing VoIP Support in Enterprise Network Chapter 2 IP QoS 2.1 Introduction to QoS 2.2 Identifying and Comparing QoS Models 2.3 QoS .
Implementing a QoS solution is not a one-time task that is complete upon policy deployment. A successful QoS policy rollout is followed by ongoing monitoring of service levels and periodic adjustments and tuning of QoS policies. Short-term monitoring is useful for verifying that the deployed QoS policies are having the desired end-to-end effect.
