February, 2015 Bill Loewe - HPC Advisory Council
February, 2015Bill Loewe
AgendaFile System Metadata, a growing issueParallel File System - Lustre OverviewMetadata and Distributed NamespaceTest setup and implementation for metadata testingScaling Metadata ServersHigh AvailabilitySeagate Confidential
Metadata PerformanceFile System Performance typically viewed in BandwidthBandwidth problem largely addressed, but metadata is agrowing issue.We see this in workloads with high numbers of files toaccess and process. Genome processing CPU Chip manufacturing Video compositing/renderingSeagate Confidential
Lustre Parallel File SystemLustre is an open source,distributed parallel file system Object-based design providesextreme scalability Compute clients interact directlywith storage servers Comprised of: Clients Metadata Servers and Targets Storage Servers and Targets
Lustre Distributed NamespacE (DNE)Distributed NamespacE (DNE) is a new feature availablein Lustre 2.5 that allows multiple MDS / MDTcomponents to participate in a single file system.DNE allows the namespace to be divided across multiplemetadata servers.Enables the size of the namespace and metadatathroughput to be scaled with the number of servers.The Lustre DNE project is comprised of 2 phases.Seagate Confidential
Phase 1, Lustre 2.5 ReleaseRemote Directories -- Lustre sub-directories aredistributed over multiple metadata targets (MDTs).Sub-directory distribution is defined by anadministrator.Remote DirectoriesRootdir bdir aFiledir cdir b2dir c2dir eFileFileFileFiledir ddir d2dir e2Seagate Confidential
Phase 2, Lustre 2.7Striped Directories -- The contents of a given directoryare distributed over multiple MDTs.Striped DirectoriesStriped DirectoryFileFiledir c2dir e2Seagate Confidential
Engineered Storage Solutions for HPC, Big Data & CloudHigh speed networking(IB/40GB/e)ClusterStorParallelfile system/ObjectData protectionHigh availabilityFlash optimizationFile system (Ext4)Linux OSBIOS/IPMIGEM diagnosticsCustom X86embedded serverSeagate storageplatformsSeagate Storage Devices agate Confidential
Lustre ComponentsClientsDirectoryOperations, Fileopen/close,metadata, andconcurrencyMDSFile I/O andlockingFile creation, filestatus, andrecoveryOSSOSSOSSSeagate Confidential
ClusterStor Management Unit (CMU):Management and Metadata (MDS/MDT)CSM Manager and MDS/MGSNodes 2RU 4-node Sandy Bridge Servers– Server 1: CSM Mgmt– Server 2: Boot– Server 3: MGS– Server 4: MDSFault Tolerance (active/passive)Serviceability2U24 JBOD – MDT SAS JBOD forMDS/MGS/Management Disk Configuration– Qty 4 Lustre Management (MGS)– Qty 4 ClusterStor Managementand NFS– Qty 2 Global Hot spares– Qty 14 Drives for MDT
Scalable Storage Unit (SSU)SSU 5U84 Enclosure 2 Object Storage Servers’s perSSU Two (2) trays of 42 HDD’s eachfor Object Storage Targets H/A on each SSU Infiniband QDR/FDR and 40GbEthernet data networkconnectivity
ClusterStor & Lustre 2.5 DNE HardwareDNE is available in ClusterStor v2.0 MDT0 is master and default in DNE environmentDNE Servers are configured in active / active pairs Seagate 2U24 with 2 MDS embedded server modulesScale Metadata Capacity / Performance with DNE ServerpairsRootdir aBase MDSFiledir bdir cdir b2dir c2dir eFileFileFileFiledir ddir d2dir e2
ClusterStor Hardware and the Lustre File SystemMeta Data andManagementServers2U x 4 Servers1) Where is file?Client2) File is at .Meta Data TargetSeagate 2U24 JOBD3) Single File(3,072Kb)File4) File is broken into blockstripe segments (1,024Kb)Object Storage ServerSeagate EmbeddedApplication ServerObject Storage TargetSeagate 5U84 StorageBay Bridge Enclosure5a) File blockstripe 1 of 3(1,024Kb)5b) File blockstripe 2 of 3(1,024Kb)5c) File blockstripe 3 of 3(1,024Kb)
Scaling MDS and DNEs MDS 4 DNE Servers(2 ADUs)mdtest scaling MDS 4 DNEs mdtest create/stat/del600,000500,000 Mean of 5 iterationsOp/s400,000300,000Mean Create200,000Mean Stat100,000Mean Remove0Seagate Confidential
Metadata High AvailabilityMDT failover will ensure that theLustre filesystem remainsavailable in the face of MDS nodefailureHigh Availability andPerformance140,000120,000Based on existing OSS pairfailover modelFailover is graceful, quick, andnon-disruptiveOp/s100,00080,000Mean Create60,000Mean Stat40,000Mean Remove20,0000BeforeFailoverFailed over After FailoverFailback is automatic and nondisruptiveSeagate Confidential
Green Machine: Environmentally-Aware Cold Storage SolutionSpaceLight weightSmall foot printCold storage optimized designCoolingZero heat emissionAmbient cooling/No fansHigh operating temp. tolerant HDDsPowerDynamic power managementLow power serversAggressive TCO goalsGreenRecyclable chassisReduced metalResponsible disposal of old chassisLowest Operating CostReduced Carbon footprint“Best for the Planet”15
Typical Use cases Retrieve content, photographs etc. from deep archive while maintainingconsistent user experience Online pictures/Social media store use cases Pictures 45 days in cold storageRetrieve MRIs/X-rays of a patientUse cases leveraging Tape-based solutions16
Thank you !
Seagate Embedded Application Server Object Storage Target Seagate 5U84 Storage Bay Bridge Enclosure ClusterStor Hardware and the Lustre File System Meta Data and Management Servers 2U x 4 Servers Meta Data Target Seagate 2U24 JOBD 1) Where is file? 2) File is at . Client File 3) Single File (3,072Kb) 5a) File block stripe 1 of 3
XSEDE HPC Monthly Workshop Schedule January 21 HPC Monthly Workshop: OpenMP February 19-20 HPC Monthly Workshop: Big Data March 3 HPC Monthly Workshop: OpenACC April 7-8 HPC Monthly Workshop: Big Data May 5-6 HPC Monthly Workshop: MPI June 2-5 Summer Boot Camp August 4-5 HPC Monthly Workshop: Big Data September 1-2 HPC Monthly Workshop: MPI October 6-7 HPC Monthly Workshop: Big Data
Motor Vehicle Bill of Sale Template. Click the following link to find out more details about . bill of sale form sc, bill of sale form south carolina, sc bill of sale pdf, bill of sale south carolina, sc bill of sale form, fillable bill of sale sc, south carolina bill of sale pdf, free bill of sale form sc,
Motor Vehicle Bill of Sale Template. Click the following link to find out more details about . does oregon require a bill of sale, bill of sale form oregon, oregon bill of sale fillable, bill of sale oregon template, state of oregon bill of sale, bill of sale oregon form oregon dmv bill of sale, dmv bill of sale
Microsoft HPC Pack [6]. This meant that all the users of the Windows HPC cluster needed to be migrated to the Linux clusters. The Windows HPC cluster was used to run engineering . HPC Pack 2012 R2. In order to understand how much the resources were being used, some monitoring statis-tics were extracted from the cluster head node. Figure 2 .
Bill No. 5 Precast concrete 70.67 Bill No. 6 Masonry 31,566.68 Bill No. 7 Carpentry 10,281.72 Bill No. 8 Sheet roof coverings 599.79 Bill No. 9 Tile and slate roof and wall coverings 15,427.20 Bill No. 10 General joinery 1,545.26 Bill No. 11 Windows, screens and lights 17,675.61 B
The Bill-to-Bill 300XE Housing joins all the other modules. It is meant to be permanently secured inside a host machine. Several Bezel styles are available for the Bill-to-Bill 300XE recycler. Software updates can be easily uploaded into the Bill-to-Bill 300XE with a Memory Card or through network. 1.4.1. Additional Systems:
DEFAULT BILL ACCEPTOR: Coinco Vantage for 5 - 20 CAN Bills REAR LOAD CHANGE MACHINES Mars AE2612 5 - 20 CAN Bill Acceptor N/C Cashcode Currenza 5 - 100 CAN Bill Acceptor 195.00 Coinco 1,100 Bills Stacker Upgrade * 100.00 Mars 700 Bill Stacker Upgrade 140.00 Mars 1K Bill Stacker Upgrade 140.00
used when setting windows hooks . Relies on the user api hook . User Api Hook Special hooking mechanism introduced to support Windows themes RegisterUserApiHook Can only be registered by privileged processes Requires the TCB privilege Caller must be running as SYSTEM Allows Windows to load a theme client module into every GUI application . Smashing the Atom . Theme Subsystem Introduced in .
