CASE STUDY: DATA CENTER REFRESH PROJECT - Dell

CASE STUDY: DATA CENTER REFRESHPROJECTJason GatesData Center ArchitectPresidiojgates@presidio.comRandall BorgeSenior Network ArchitectPresidiorborge@presidio.com

Table of Contents1.0Abstract .52.0Scope .53.0Solution Overview .63.1Project Overview . 63.2Overview of Network Devices . 63.3Network Diagrams . 83.4Host Table . 93.5Chassis-Based Hardware Platforms . 103.5.1Nexus 7000 BC1-CR-1. 103.5.2Nexus 7000 BC1-CR-2. 113.5.3Nexus 7000 BC2-CR-1. 113.5.4Nexus 5596 BC1-DS-1. 113.5.5Nexus 5596 BC1-DS-2. 123.5.6Nexus 5596 BC2-DS-1. 123.6Software Versions . 133.6.1Nexus 7000. 133.6.2Nexus 5000. 133.6.3Nexus 2000. 133.7Licensing . 143.7.1Nexus 7000. 143.7.2Nexus 5000. 164.0Physical Layer . 174.1Management Interfaces. 174.1.1Nexus 7000 Supervisor Management Interface (Mgmt 0) . 174.1.2Nexus 5000 Switch Management Interface (Mgmt 0) . 184.1.3Nexus Fabric Interconnect Switch Management Interface (Mgmt 0) . 185.0Layer 2 Features and Design . 185.1Virtual Device Context (VDC) Configuration . 185.1.1VDC Information . 195.2VLAN Database. 195.2.1BC1-CR-1 . 195.2.2BC1-CR-2 . 205.3VLAN Trunk Protocol (VTP) Configuration . 205.3.1Atlanta VTP Configuration . 215.3.2Santa Clara VTP Configuration . 215.4Nexus 7000 Spanning Tree Configuration . 215.4.1BC1-A Spanning Tree Configuration. 215.4.2BC1-B Spanning Tree Configuration . 215.5Spanning Tree Extensions . 215.6Virtual Port Channel Configuration . 225.6.1vPC Domain Configuration Details . 222016 EMC Proven Professional Knowledge Sharing2

5.6.2BC1-CR-1 . 225.6.3BC1-CR-2 . 235.6.4Port Channel Configuration . 235.6.5BC1-CR-1 Port Channel Configuration Table . 245.6.6BC1-CR-2 Port Channel Configuration Table . 246.0Layer 3 Configuration . 256.1Layer 3 Interface Configurations. 256.1.1BC1-CR7K1. 256.1.2BC1-CR7K2. 256.1.3BC2-CR7K1. 266.2Routing Configuration . 266.2.1EIGRP Configuration . 266.2.2BGP Configuration . 266.2.3Static Routing Configuration . 277.0Multicast Routing Configuration . 278.0Device Management and Security Configuration . 288.1Nexus 7000 Control Plane Policing (CoPP). 289.0Quality of Service (QoS) Configuration . 299.1QoS Configuration . 3010.0 Advanced Features . 3010.1OTV Configuration. 3010.1.1 Multihomed Sites and Load Balancing . 3010.1.2 OTV BCS1. 3210.1.3 OTV BCS-2 . 3210.1.4 OTV BCS2-1 . 3311.0 Device Management and Security Configuration . 3311.1System Management Configuration . 3311.1.1 Message of the Day (MOTD) Banner . 3311.1.2 Network Time Protocol (NTP) Server . 3311.1.3 Timezone Settings . 3411.1.4 DNS Settings . 3411.1.5 Syslog Configuration . 3411.2Smart Call Home Configuration . 3411.2.1 Example Call Home Configuration . 3511.3Simple Network Management Protocol V2 (SNMP) Configuration . 3511.3.1 SNMPv2 Configuration . 3511.4NetFlow Configuration . 3512.0 Compute & Storage Architecture . 3612.1Overview . 3612.2Data Center Diagrams . 3712.2.1 World Pay Architecture Design Diagram . 3712.3Overview of Platform Features . 3712.3.1 Cisco Unified Computing System . 372016 EMC Proven Professional Knowledge Sharing3

12.4Overview of Cisco UCS . 3712.4.1 EMC VNX Series Storage Arrays . 39Chapter 1 . 41Chapter 2 . 41Chapter 3 . 41Chapter 4 . 41Chapter 5 . 41Chapter 6 . 41Chapter 7 . 41Chapter 8 . 41Chapter 9 . 41Chapter 10 . 4110.5.1 Virtual Blended Storage Pool (FAST VP). 4110.5.2 FAST Cache . 443.3.1 Pool-based Thin Provisioning . 4612.4.2 Storage Configuration . 4712.5EMC VNX 5400 Configuration . 4712.5.1 Basic Configuration Information . 4712.5.2 Storage Processor Summary . 4812.5.3 Disk Summary . 4812.5.4 Storage Pools / RAID Groups. 4812.5.5 Initial Proposed RAID Group Layout . 4913.0 Cisco Unified Computing System (ucs) Production Environment . 4913.1Cisco UCS Configuration. 4913.1.1 Fabric Interconnects . 4913.1.2 Server Configuration . 5013.1.3 Admin Configuration . 5114.0 Conclusion . 51Disclaimer: The views, processes or methodologies published in this article are those of the authors.They do not necessarily reflect Dell EMC’s views, processes or methodologies.Dell, EMC and other trademarks are trademarks of Dell Inc. or its subsidiaries.2016 EMC Proven Professional Knowledge Sharing4

1.0ABSTRACTTechnology refreshes can be challenging for companies due to overwhelming choices of technologyplatforms, vendor choice, and complexity. The main drivers for tech refreshes are cost savings,consolidation, replacing aging equipment, maintenance renewal, increasing applicationHA/performance, and other factors. "World of Art" is a company selling high end art, collectibles, and24/7 auction events with locations around the globe. World of Art has decided to refresh the technologyin their main Atlanta data center and co-location Santa Clara; which includes compute, networking,storage, and backups. The objective of this case study is to detail the design considerations for theproject such as advanced concepts, load balancers, iSCSI, virtual port channels, etc. This designdocument will detail the architecture, implementation strategy, and expected results of following bestpractices combined with real world examples. It considers steps during the high and low level designphases.This design document is intended for EMC Proven Professionals, network engineers, SAN engineers, ITmanagers, and IT directors.2.0SCOPEThe EMC Proven Professional Firm has developed a solution that addresses the requirements fromWorld of Art personnel obtained from the RFP documentation and detailed design meetings.This document contains the detailed design elements associated with the deployed solution: Build-out of two new data centers including Nexus 7000, 5500, and 2000 series switches.Storage Area Network build-out with VNX 5400.Physical network design, which includes physical connections between all of the components.Logical design of the network, which takes into account World of Art requirements listed in theRFP.VNX array information and detailed feature analysis.Cisco Unified Computing System information and detailed feature analysis.2016 EMC Proven Professional Knowledge Sharing5

3.0SOLUTION OVERVIEW3.1Project OverviewThe Datacenter Design Project provides the following features and benefits: 3.2Separation of traffic between different networks.High Availability and Redundancy.Quality of Service for Voice and Business Applications (identified by World of Art).Increased throughput through the use of up to 40 Gigabit interfaces.Overview of Network DevicesNexus 7000 Series SwitchThe modular Cisco Nexus 7000 Series Switches deliver the highest-density 10 Gigabit Ethernet ports inthe market, with up to 768 10-Gbps ports and more than 17 terabits per second (Tbps) of switchingcapacity, support for 40 and 100 Gigabit Ethernet interfaces, and a comprehensive Cisco NX-OS Softwarefeature set. The Cisco Nexus 7000 Series Switches are designed to meet the requirements of nextgeneration data centers, Cisco Data Center Interconnect (DCI) solutions, data center aggregation, spineand leaf architecture, and data center end-of-row (EoR) access designs.Built on a zero-service-loss hardware and software architecture, the Cisco Nexus 7000 Series offers thekind of high availability needed in a next-generation data center, in which virtualization increases thescope of downtime and unified fabric demands Fibre Channel-like availability to properly supportstorage services. The Cisco Nexus 7000 Series was built with manageability in mind and incorporates anumber of unique features, including integrated lights-out management and integrated packet captureand decoding. The Cisco Nexus 7000 also offers innovative switch virtualization capabilities, which, incombination with the switch's density, allows customers to greatly simplify their switchinginfrastructure, reducing costs, power and cooling load, and management complexity.Cisco Nexus 5000 Series SwitchEven with its svelte rack switch form factor, the Cisco Nexus 5000 Series of switches offers numerousinnovations. The low-latency, low-cost, 10 Gigabit Ethernet switch was first to market with support forData Center Ethernet, which improves the reliability and scalability of Ethernet for data center purposes.The switch was also the first to deliver Fibre Channel over Ethernet (FCoE), which allows storage trafficto be reliably carried over an Ethernet infrastructure. These features are prime examples of Cisco'sholistic approach, with Cisco working with companies such as Emulex, Intel, QLogic, and VMware tobring a complete implementable solution to market. While the Cisco Nexus 5000 Series is designed formost data center environments, its low-latency characteristics also make it an ideal candidate for highperformance computing applications. The Cisco Nexus 5000 Series is available in two variations. TheCisco Nexus 5020 Switch offers 40 ports of lossless 10 Gigabit Ethernet and two uplink slots that cansupport a combination of 10 Gigabit Ethernet and Fibre Channel ports. The Cisco Nexus 5010 Switch2016 EMC Proven Professional Knowledge Sharing6

offers the same features and capabilities as the Cisco Nexus 5020, but in a smaller form factor: 20 fixed,lossless, 10 Gigabit Ethernet ports and one uplink slot.The Cisco Nexus 5500 platform extends the industry-leading versatility of the Cisco Nexus 5000 Seriespurpose-built 10 Gigabit Ethernet data center-class switches and provides innovative advances towardhigher density, lower latency, and multilayer services. The Cisco Nexus 5500 platform is well suited forenterprise-class data center server access-layer deployments across a diverse set of physical, virtual,storage-access, and high-performance computing (HPC) data center environments.The Cisco Nexus 5596UP Switch is a 2RU 10 Gigabit Ethernet, Fibre Channel, and FCoE switch offering upto 1920 Gbps of throughput and up to 96 ports. The switch has 48 unified ports and three expansionslots.Cisco Nexus 2000 Series Fabric ExtendersThe Cisco Nexus 2000 Series Fabric Extenders offer a unique approach designed specifically to givecustomers a means of granularly transitioning from Gigabit Ethernet to 10 Gigabit Ethernet and to aunified fabric. The Cisco Nexus 2000 Series sits on top of a server rack and essentially acts as a remoteline card for an upstream switch and becomes an extension of the switch, so software, configuration,and policy are all inherited from the upstream switch; even advanced features such as FCoE and CiscoVN-Link support are inherited. This approach offers two primary benefits to the customer. First, totalcost of ownership (TCO) is reduced because of simpler cabling requirements (primarily intrarack) andbecause there are fewer switches to manage. Second, the Cisco Nexus 2000 Series allows customers tosupport their existing Gigabit Ethernet attached servers while providing access to advanced features andmaintaining a consistent management and operations environment across the data center. The initialmodel of the fabric extender is the Cisco Nexus 2148T Fabric Extender, which supports 48 GigabitEthernet downlinks and 4 10 Gigabit Ethernet uplinks. The combination of the Cisco Nexus 2248PQ andupstream Cisco Nexus 5000 Series Switches and Cisco Nexus 6000 Series Switches provides a costeffective access layer and scalable strategy for 10 GE and FCoE at the server access layer. It also helps toenable a smooth migration to a 40 GE network fabric.2016 EMC Proven Professional Knowledge Sharing7

3.3Network DiagramsFigure 1 Primary Atlanta Data Center2016 EMC Proven Professional Knowledge Sharing8

Figure 2 Santa Clara and Atlanta ConnectivityFigure 3 WAN Connection3.4Host TableHostnameMake/ModelDescriptionData CenterLocationBC1-CR-1Cisco Nexus 7004Core SwitchAtlantaBC1-CR-2Cisco Nexus 7004Core SwitchAtlantaBC2-CR-1Cisco Nexus 7004Core SwitchSanta ClaraBC1-DS-1Cisco Nexus 5596Distribution/Access switchAtlantaBC1-DS-2Cisco Nexus 5596Distribution/Access switchAtlantaBC2-DS-1Cisco Nexus 5596Distribution/Access switchSanta Clara2016 EMC Proven Professional Knowledge Sharing9