Object-based SSD (OSSD): Our Practice And Experience

FSWDSAMSUNGSAMSUNGObject-based SSD (OSSD):Our Practice and ExperienceJaesoo Leejaesu.lee@samsung.comFlash Solution Team, Memory DivisionSamsung Electronics Co.

Outline Part 1. OSD and Object-based SSD Part 2. Application-Aware Storage 2IntroductionOur Practice and ExperienceIntroductionOur Practice and Experience Future Directions Summary and ConclusionSAMSUNG

Outline Part 1. OSD and Object-based SSD Part 2. Application-Aware Storage 3IntroductionOur Practice and ExperienceIntroductionOur Practice and Experience Future Directions Summary and ConclusionSAMSUNG

Outdated Storage Stack SAMSUNGHDD as main storage for decadesStructure is relatively simple Physical location can be easily derived LBA (c N heads N sec tors ) (h N sec tors ) s 1 Storage stack has remained staticNarrow block interfaces (ATA, SCSI) No information flow except block reads/writes File subsystems make HDD-specific assumptionsSequential read is much faster than random read No write amplification, wear-out, background activity, 4What if the underlying device changes ?[Excerpted from Block Management in SSD, Usenix 2009 ]

Challenges in SSD Storage StackSAMSUNG Host-side optimization is no more feasible Confidential, subject to changes, different among SSDs, Striping methodBuffer cache management policyLogical-to-physical mappingGarbage collection policyWear-leveling policyBad block managementProcessorCore5[figure by Sang Lyul Min @ SNU] # of channels# of ways# of planes per chip# of blocks per plane# of pages per blockBlock sizePage size

SSD-Aware File System SAMSUNGPreliminary results with in-house SSD-aware file system Up to 1700% improvement for random writes13% degradation in sequential write performance18SSD-Aware FSext3Normalized Throughput1614121086420468163264128 256 512 1024 2048 4096 8192I/O Size (KB)

OSD: The Basic Concept Optional command set defined for SCSI device Provide object-based interface instead of traditional blockbased interfaceIn OSD, an object is a flexible-sized data container 7SAMSUNGUnique object IDA set of attributes

SSD as OSD OSD manages space for objects Informed cleaning (utilize delete info)Stripe aligned accessesLogical to physical mappingOSD supports object attributes SAMSUNGWear-leveling using cold datainformationPriority assigned to objects (i.e., QoS)OSD handles low-level operations Block management in SSD[Source: Block Management in SSD, Usenix 2009 ]8

Linux Support for OSD SAMSUNGLinux provide a ready-to-use OSD prototype open-osd: open source initiator and exofs filesystemOSC-OSD: iSCSI OSD targetOpen-iSCSI: iSCSI transportvfsexofsbsgosd uld (osd.ko)osd lld (libosd.ko)ext3udfsdsrSCSI core (scsi mod.ko)iscsiinitiator9scsi initiatorSCSI/SATAscsisatast

Target Platform OSD is the most promising for mobile storages Storage and system vendors are decoupled inherentlyQoS provisioning is essentiale.g., managed flash memory applications including 10SAMSUNGeMMC, UFD, miniSD, SD, Our primary target is a Linux-based mobile platform

Outline Part 1. OSD and Object-based SSD Part 2. Application-Aware Storage 11IntroductionOur Practice and ExperienceIntroductionOur Practice and Experience Future Directions Summary and ConclusionSAMSUNG

OSSD Prototype Prototyped in the Linux-based host SAMSUNGMLD: object mapping, space management, New type of SSD developed (called rawSSD)VFSfrom open-osdand Linux kernel exofsOSSD ULDOSSD MLDGet rawSSD informationErase all blocksErase a blockRead a flash pageProgram a flash pageOSSD LLDFlash Cmd I/FSCSI/SATAFlash Mng.Host (Linux)raw SSDSATA Link12

MLD: Object Management LayerSAMSUNG Overall architectureOSD Commands{PID, OID} t descriptorMeta dataUser dataOSSD LLDREAD / PROGRAM / ERASEFlashmedia13

MLD: Object DescriptorSAMSUNGContaining metadata including extents and attributes Cached in memory (LRU) Descriptor Header1page(16KB)14 Object ID PPN Object informationBlock Table PPNs for object data Extent: support 9MB Indirect table: support 4GBAttributes Attr. data of object Usually contain i-node and length Managed using original code

MLD: Object Mapping Support page-wise mapping Based on page extentsAssociation policies of an update block Fully associative Object associative (called per object)Garbage collection Victim selection 15Separation of index and user dataNo separation (called unified)Highest invalid pagesBitmap for invalid informationBackground GC supportedSAMSUNG

Low Level Driver Provide block Raw SSD specificinterfaces to upper layersSAMSUNGBlock interface- submit bio()Sanity check Raw SSD interface- obtain config()- erase block()- format()OSSD LLDPage overwriteOut-of-order page writeSanity CheckerBackend dispatcher Provide backend flexibility RawSSD 16SCSIsubsystemVia SCSI-ATA Translation Layer (SATL)LoopRAMRAMLoopRawSSD

Experimental Setup Read ThreadHost system SAMSUNGWrite ThreadsQuad Q9650 3GHz4GB RAMLinux kernel 2.6.334GB partition for OSSD VFSexofsOSSD DriverFlash Cmd I/F17Flash Mng.SCSI/SATAraw SSDHost

Experiments 1. Better performance for fragmented/multi-streamwrites Effect of incorporating space management in OSSD2. QoS support (read prioritized service) 18SAMSUNGEffect of having knowledge on the contexts of data blocksand a request

Exp1. Multi-thread File WriteSAMSUNG Scenario Write 800 MB 4 thread delete 2 file write 800 MB 2threadShow the effect of fragmentationUnified updateblockIndex / DataIndex / Per objectWrite 800MB4 thread3m48.470s3m52.263s3m49.769sAfter delete,write 800MB2 050EraseValid copy19

Exp2. Read Priority Scenario 20SAMSUNG200MB read over total 2.4G write (e.g., 300MB x 8)

Outline Part 1. OSD and Object-based SSD Part 2. Application-Aware Storage 21IntroductionOur Practice and ExperienceIntroductionOur Practice and Experience Future Directions Summary and ConclusionSAMSUNG

Capacity Outran BandwidthSAMSUNGCapacity of HDD grows exponentially Ratio of the capacity to the interface bandwidth alsogrows exponentially C/B Ratio100000Capacity (MB, log scale)1000000Capacity1000001000010000Seagate Barracuda(SATA/300)Seagate Barracuda IVIBM Deskstar 16GP (UltraATA/100)(UltraATA/33)Quantum Fireball ST(UltraATA/33)100010010100010010Maxtor 7000(IDE)11990119952000Year2220052010Capacity / Interface Bandwidth (sec)1000000

Application-Aware Storage With a small database proxy, the amount of datatransferred can be decreased significantly scan, aggregation, join, sorting, Other promising areas 23SAMSUNGData mining, search indexing, image processing, Anti-Virus,

Application-Aware Storage Application-awareness can be easily achieved in OSSD 4 Cortex-R4 CPUs, SATA 6Gbps, 512MB RAM, AXI bus matrix,16 flash memory controllers, What about moving some of application’s work toOSSD? 24Integrated object (i.e., file) managementFluent attribute mechanismFurthermore, SSD is no more dumb SAMSUNGIssues are models for programming, execution, anddeployment

Outline Part 1. OSD and Object-based SSD Part 2. Application-Aware Storage 25IntroductionOur Practice and ExperienceIntroductionOur Practice and Experience Future Directions Summary and ConclusionSAMSUNG

DB-Aware OSSD SAMSUNGPostgreSQL Free and open sourceObject-relational databasePostgreSQLQuery ParserQuery PlannerOSSDPlug-inQueryOptimizer Developed a plug-in foraccelerating: AggregationSelectionExecutorVFSioctlEXOFSOSD DriverDB Proxy 26Query is processed in a similarway to the active diskKernelDB-Aware OSSD

DB Operators SAMSUNGAggregation Count, Sum, AverageSELECT count(*) FROM emp where age 30;Count(*)-------17ResultResultAggregateSeq. Read27Seq. ReadAnd g281400Lim25400

DB Operators SAMSUNGSelection, Projection Filtering (not yet)SELECT * FROM emp where age 30;Name Age Salary----- ----- -------Na34 4000ResultResultSelectionSeq. Read28Seq. ReadAnd g281400Lim25400

Implementation MechanismsSAMSUNG Currently, Running on existing exofs file systemLinux HostPostgreSQLSELECT AGGREGATE(target col)FROM t1 OSDWHERE cond col COND value;exofsTranslate inode to Object IDOSSD ULDOSSD MLDOSSD LLDSCSI/SATA29Sequential ReadandProcessing Aggregate Function

Experiments Experimental setup Hardware AMD Atholon64x2 7750 2.7 Ghz3GB RAM (PC6400)HDD WD3200AAKS (SATA2/7200/16M)OS and drivers Linux kernel 2.6.33 (Fedora Core 13)Target : OSC-OSDFile system and Driver : Open-OSD exofs and driversDatabase 30SAMSUNGPostgreSQL 8.4.4

Experiments 31Results will be available soon!!SAMSUNG

Outline Part 1. OSD and Object-based SSD Part 2. Application-Aware Storage 32IntroductionOur Practice and ExperienceIntroductionOur Practice and Experience Future Directions Summary and ConclusionSAMSUNG

Future Directions of Work 1. OSD SSD Virtually any communication protocol over SATA3. i2SSD: open storage software platform 33Native support of OSD commands in SSDOSD command tunneling2. Bi-directional communication extension in SATA SAMSUNGJVM and deployment middlewareStorage programming framework

1. OSD SSD Architecture (Revised)SAMSUNGLinux HostOSSDexofsOSSD ULDOSSD Cmd HandlerOSD Cmd I/F (via Tunneling Tech.)Buffer ManagerOSSD LLDSATASCSI/SATAFTLSATA Link34Object Layer

1. OSD SSD: OSD Cmd TunnelingSAMSUNG 2-phased protocol OSD command is transferred as a payload of the 1stcommandHostSSD1st phase(cmd transfer)SATA vendorcommand(register FIS)H2D Data35D2H Data2nd phase (optional)(data transfer)

2. Bi-Directional Comm ExtensionSAMSUNGATA/SCSI protocol is too limited for application-specificextension of OSSD We extended SATA with a simple bi-directionalcommunication protocol Based on reserved LBAs and notification mechanismServe as a link layer of other advanced protocol, e.g., TCP/IPHOSTApplication a lMessaging Cmd.Bi-directionalMessaging Cmd.ATA/SCSI TrasportATA/SCSI TrasportSATA/SAS/ Link36Application AgentFTL

3. i2SSD: Intelligent and Innovative SSD What is SAMSUNG?Object-based SSD capable of extending its functionalities viadynamic application deploymenti.e., “Open Storage Architecture for Enabling Storage-AwareApplication Deployment”Benefit is significantly saved time and energyComputation at the near of the datai2SSD 37i2SSDdynamically deployedapplicationi2SSD Run-timeFrameworkservice library components

3. Preliminary i2SSD ArchitectureSAMSUNGApplication LayerService Component LayerSecurityi2SSD am EditorRegEx MatchHILObject L

Outline Part 1. OSD and Object-based SSD Part 2. Application-Aware Storage 39IntroductionOur Practice and ExperienceIntroductionOur Practice and Experience Future Directions Summary and ConclusionSAMSUNG

Conclusion OSD fits well for SSD Free from variations in SSD internalsEasy application-specific extensionAdvanced features (e.g., QoS) made possibleWe have developed a proof-of-concept OSSD anddemonstrated the benefits 40SAMSUNGThe results look promising

Call for Participation Standardization SCSI/SATA extensions for OSD command tunnelingSCSI/SATA extensions for bi-directional communicationcommand seti2SSD framework and programming interfacesCall for participation of Linux community 41SAMSUNGOSSD bring-up issues in LinuxEnd-to-end storage QoS support in Linux

FSWDSAMSUNGSAMSUNGThanks

Object-based SSD capable of extending its functionalities via dynamic application deployment i.e., "Open Storage Architecture for Enabling Storage-Aware Application Deployment" Benefit is significantly saved time and energy Computation at the near of the data 37 dynamically deployed application service library components i 2 SSD i2SSD Run .