SWIFTPage:1Openstack SwiftObject Store Cloudbuilt from the grounds upDavid HadasSwift ATCHRLdavidh@il.ibm.com 2012 IBM Corporation
SWIFTPage:2Object Store Cloud ServicesExpectations:PUT/GET/DELETE Huge Capacity (Scale) Always Available Safe High Performance More PUTs than GETs Low Cost per TB 2012 IBM Corporation
SWIFTPage:3High Level: Object Store vs. File SystemFile SystemObject Store Interface:– Web based: GET/PUT/DELETE– RESTful: Stateless– Metadata Synchronization– Eventual Consistency– No Distributed Locking Software Defined Storage– Commodity hardware– Designed to Fault but never Fail– Built to auto-recover by design Features– Basic services that scale (KISS)– SW extendible with web interfaces Interface:– Posix:Open,Seek/Read/Write,Close– Stateful Synchronization– Always consistent– Uses Distributed Locking Hardware and Software– Best of breed hardware– Designed not to fault– Admin controlled recovery Features– Abundant enterprise features builtinto the products 2012 IBM Corporation
SWIFTPage:4Object Store Cloud Options Amazon S3– Proprietary API Rackspace Cloud Files (Swift)– Swift API Microsoft Blob Storage– Proprietary API OpenStack Object Services– Written in Python- Apache license– Interfaces:- Native Swift 1.0- S3, CDMI interfaces Google Cloud Storage– Proprietary API HP Cloud Object Store (Swift)– Swift API 2012 IBM Corporation
SWIFTPage:5Historical PerspectiveAmazon S3 LaunchedRackspace Cloud Files V2 (‘Swift’)Distributed Storage and ns Swift’AustinOpenStackRackspace contributes SwiftNASA contributes NovaOpenStackFoundationGrizzlyRackspace Cloud Files V1Distributed StorageCentralized MetadataHard to scale – while growingrapidlyOctDec2006Rackspace deployment of Swift3 Data CentersMigrated all customersAprS3 holds 14 Billions Objects2013 2012 IBM Corporation
SWIFTPage:6OpenStack - The Cloud operating System 550K lines of code 300K Downloads 550 Developers 7000 Individuals from 100 Countries 850 Organizations 10M Funding 3000 participants in the last summit 2012 IBM Corporation
SWIFTPage:7SwiftA massively scalable redundant storage system– Replicate objects to disk drives spread as far apart as possible Scales horizontally (adding new servers)– Known to work with- ‘Thousands of servers’- ‘Petabytes of data’ Designed to contain high rate of failures– Inexpensive commodity hard drives and servers can be used– Maintains replication level following faults An Open Source Software 2012 IBM Corporation
SWIFTPage:8Swift General Layout Swift Cluster– Proxy Nodes- Smart– Storage Nodes- Simple– (Can be the same node)PUTProxy Nodes Object GET/PUT– Web Clients use HTTP– Approach a Proxy Node– The Proxy node maps to storage devices- Located on Storage Nodes– Request is propagated to Storage Node A Cut-through technology– Proxy and Storage Nodes never store theentire object in memoryStorage Nodes 2012 IBM Corporation
SWIFTPage:9Swift API Examples GET MyAccount/MyContainer/MyObject– Responder provides the object in the body metadata in http headers PUT MyAccount/MyContainer– Requester indicates container metadata in http headers Responder indicates 201 POST MyAccount/MyContainer– Requester indicates container metadata in http headers Responder indicates 200 PUT MyAccount/MyContainer/MyObject– Requester indicates object content in the message body metadata in http headers– Responder indicates 201 GET MyAccount/MyContainer– Responder provides the list of objects in the body metadata in http headers DELETE MyAccount/MyContainer/MyObject– Responder indicates 200 2012 IBM Corporation
SWIFTPage:10Principles of Operation - Mapping Mapping must allow us to stay:––––DistributedScalableLoad BalancedHighly Available Fortunately, “All problems in computer science can be solved byanother level of indirection (David Wheeler)“ Double mapping to allow virtualization of Disk space– Objects are stored in virtual partitions– Partitions are stored in “devices”- Typically a device is a local hard drive in a server- Device can be anything- Allowing another level of indirection 2012 IBM Corporation
SWIFTPage:11Mapping Implementation Consistent Hashing conceptsMD5(‘FQ Object Name’)– Only K/n keys are remapped– Evenly distributed load during changesPartition 3 Mapping of Objects to Partitions– Partition determined by MD5(Fully Qualified ObjectName)– Uses Partition-Power highest bits of the MD5 result– Can elastically grow and shrink Mapping Partitions to devices– Using a ‘Ring’ which maps each partition to D devices(i.e. D replications)– The Ring is consistent allowing addition of removal ofdevices- Used mainly to grow the clusterDevices:5, 2, 14Device TableID IP:PORT Device12 220.127.116.11:6010 sde1 sde1 partitions2, 3, 8, 18.104.22.168:6010 2012 IBM Corporation
SWIFTPage:12Principles of Operation - Eventual Consistency Faults are assumed– Disk faults– Server faults Service continues normally during faults– Continue servicing both PUTs and GETs– Continue offering the same level of replication Distributed, on-going recovery– Handoff replicas are created following lost of replica Always strive for consistency– Replicas look for the way ‘back home’ 2012 IBM Corporation
SWIFTPage:13Eventual Consistency Implementation PUT/POST/DELETE ObjectAdd/Update an object at D replicas– Respond after RoundDown(D/2 1)- On failure:place on a handoff device- Any device may serve as ahandoff for any object– Update the container- On failure: Queue inasync pending for later update– Delete marks an object for deletion GET ObjectGet one of the replicas– Would only try the D replicas at theRing Replicator Daemon– Crawl local devices– Replicate locally found replicas to itsdesignated place- Delete successful and handoff- Signatures for objects partitionsused for efficiency and speed Updater Daemon– Crawl local devices looking forasync pending– Updates container about an object Auditor Daemon– Crawl local devices– Audit the object signatures Expirer Daemon– Deletes cold objects 2012 IBM Corporation
SWIFTPage:14Principles of Operation - Containers and Accounts Containers and Accounts are Objects––––The content of the object is the list of items it containsGET /myaccount/mycontainer retrieves the list of objectsPUT /myaccount/mycontainer creates a containerPUT /myaccount/mycontainer/myobject adds an object to a container 2012 IBM Corporation
SWIFTPage:15Containers and Accounts - Implementation The Container/Account is an SQLite3 DB.– Today used for Separate Rings– Allow store on separate devices (SSDs for containers/accounts):- Objects- Containers- Accounts Containers and Accounts have their own set of eventualconsistency irer 2012 IBM Corporation
SWIFTSwift 6ClientAuthenticationAuthorizationLoggingCORSCDMI and S3 APIsPUTLoad BalancerPUTObject ServerContainer ServerAccount ServerDevice TableID IP:PORT Device12 22.214.171.124:6010 sde1 Scale-out architectureCommodity iftProxySwiftProxySwiftProxyPUTSwiftStorage NodeXFS NodeReplicatorUpdaterAuditorExpirer 2012 IBM Corporation
SWIFTPage:17IBM Participation – Create Enterprise Grade Swift Contributions to Open Source– Done:- CDMI Support- Swift as a request processor of Apache– On going:- Account ACLs- Account Listing- Enable Ring Size to Grow- Swift Crawler– Future- Object ACLs- Metadata Extendibility- Undelete 2012 IBM Corporation
SWIFTPage:18 2012 IBM Corporation
Openstack Swift Object Store Cloud built from the grounds up David Hadas Swift ATC HRL . Swift Architecture M D 5 Swift Proxy Swift Proxy Swift Proxy Load Balancer Client PUT PUT PUT Extensions Extensions Extensions Swift Storage Node . Micr
1.4. set environment variables using the openstack rc file c a t o e st c o an - i e c n 2.1. openstack usage 2.2. openstack optional arguments 2.3. openstack acl delete 2.4. openstack acl get 2.5. openstack acl submit 2.6. openstack acl user add 2.7. openstack acl user remove 2.8. openstack action definition create 2.9. openstack action .
SWIFT Fidelity Integrity Assessment [SWIFT-FIA] v.1.3 for SWIFT Partner Schools rev. January, 2016 1 Purpose of SWIFT-FIA SWIFT Fidelity Integrity Assessment (SWIFT-FIA) is a s
This book describes Swift 5.5, the default version of Swift that's included in Xcode 13. You can use Xcode 13 to build targets that are written in either Swift 5.5, Swift 4.2, or Swift 4. When you use Xcode 13 to build Swift 4 and Swift 4.2 code, most Swift 5.5 functionality is available. That said, the following changes are available
Swift 4 1 Swift 4 is a new programming language developed by Apple Inc for iOS and OS X development. Swift 4 adopts the best of C and Objective-C, without the constraints of C compatibility. Swift 4 makes use of safe programming patterns. Swift 4 provides modern programming features. Swift 4 provides Objective-C like syntax.
Modi ed IBM IBM Informix Client SDK 4.10 03/2019 Modi ed IBM KVM for IBM z Systems 1.1 03/2019 Modi ed IBM IBM Tivoli Application Dependency Discovery Manager 7.3 03/2019 New added IBM IBM Workspace Analyzer for Banking 6.0 03/2019 New added IBM IBM StoredIQ Suite 7.6 03/2019 New added IBM IBM Rational Performance Test Server 9.5 03/2019 New .
VMware's OpenStack Initiative Contribute to OpenStack Integrate VMware compute, network, storage SW with OpenStack. Make OpenStack better, helping customers succeed with their cloud effort. Help customers understand how VMware technology helps them build the best possible OpenStack cloud.
Web-based Lab Guide Presentation PDF 3. Agenda Module 1 SDN In OpenStack Module 2 OpenStack Networking Module 3 Neutron L2 Module 4 OpenStack Services Module 5 Cisco & OpenStack A Short History of SDN Dynamic 'overlay' networks SDN Controllers SDN in OpenStack
10 Annual Book of ASTM Standards, Vol 02.02, in the Related Material section (gray pages). 11 Available from Standardization Documents, Order Desk, Bldg. 4, Section D, 700 Robbins Ave., Philadelphia, PA 19111-5094, Attn: NPODS. TABLE 1 Chemical Composition Limits NOTE 1—When single units are shown, these indicate the maximum amounts permitted. NOTE 2—Analysis shall be made for the elements .