Avoiding OOM (Out Of Memory) On Embedded Linux

2y ago
31 Views
2 Downloads
903.44 KB
23 Pages
Last View : 1m ago
Last Download : 2m ago
Upload by : Xander Jaffe
Transcription

Avoiding OOM (Out of Memory)on Embedded LinuxApril 15,15 2008YoungJun Jang yj03.jang@samsung.comSoftware LaboratoriesSamsungg Electronics

ContentsProblem StatementBackgroundDemand PagingOvercommitAddress Space Usage: Virtual vs. PhysicalOvercommit ProblemApproach & ImplementationRSS QuotaLimitation & Future WorksAnother Approaches2/23

Problem statementWhat’s the problem of this code?On Linux systemy64M RAM availablemain(void){char *p;f (i 0;for(i 0 i 100;i 100 i ){p malloc( 1M );if ( p NULL ){// error handling}}}[ CODE 1 ]3/23

Problem statement (cont.)Then how about that code?On Linux systemy64M RAM availablemain(void){char *p;f (i 0;for(i 0 i 100;i 100 i ){p malloc( 1M );if ( p NULL ){// error handling}}}memset( p, 0, 1M );[ CODE 2 ]4/23

Demand PagingSeparated virtual and physical addressReserve virtual address first, and allocate physical memory when accessingAllocated pages to other processesvirtual memory later.Psize:i4KB s Spacechar *p;p malloc(10M);Physical Memory0x08048000TextAllAllocatedt d pages tot memtestt tFree pages2MBData0xC0000000BSS HeapStack1. do mmap() expands BSS/Heap region(Virtual Addr. Space)0x00000000memtest’sVirtualAddress Space.memset( p 2M, 0, 5M );memtest.cText10MBData0xC0000000BSS HeappStack2. NO physical memoryconsumption occursPhysical ss Space0x08048000Text10MBDataBSS Heap0xC0000000StackPage faults occur5*256 times(do page fault())Physical Memory5MB5/23

OvercommitThe total aggregation of virtual address space can be larger than physicalmemory size.Application can allocate ‘large’ memory without considering physicalmemory size.0x00000000Process A’sVirtual Address Space0x00000000Process B’sVirtual Address Space0x00000000Process C’sVirtual Address Space0x08048000Text0xC0000000DataBSS HeapStack0x08048000Text0xC0000000DataBSS HeapStack0x08048000Text0xC0000000DataBSS HeapStackTotal Aggregation ofVirtual Address SpacesVirtual Memory used by process AVirtual Memory used by process BVirtual Memory used by process COvercommit !!Physical MemoryPhysical Memory used by process APhysical Memory used by process BPhysical Memory used by process CFree Physical Memory6/23

Address Space Usage: Virtual vs. Physical (1/2)Memory Usage Example.(92MB)7/23

Address Space Usage: Virtual vs. Physical (2/2)Memory Usage Example. (per each area)8/23

Overcommit Problem (1/2)Allocation request (ex. malloc()) larger than available memory size always succeeds.Linux invokes OOM killer and kills application.Allocated pages to other processesAllocated pages to memtestFree pages (FreeMem,(FreeMem PageCache,.)PageCache )0x00000000main(void){.char *p;p malloc(10M);.memtest’sVirtualAddress SpaceVirtualAddress Space.Text2MBData0xC0000000BSS Heap Page size: 4KBStack6MB freePhysical MemoryMalloc() succeeds0x00000000even when memory isnot sufficient.memtest’smemset(p,0,10M);0x08048000do mmap() expands BSS/Heap region10MB0x08048000TextDataBSS Heap0xC0000000StackPhysical Memory0x00000000memtest’sVirtualAddress Space10MB0x08048000TextDataBSS HeapStackAfter page faults occur6*256 times,timesPhysical Memory6MB allocatedApp will be killedby OOM-killer9/23

Overcommit Problem (2/2)Generally, SW developer thinks :malloc() will return NULL, if the system has no available memory.Then, developers add error handling code when malloc() returns NULL.But indeed linux kernel doesn’tNULLBut,doesn t return NULL.Instead, invokes OOM killer when allocated spaces are really used in low memorycondition.So, the system gains no opportunity to handling ‘allocation failure’ error.It’s a problem for all embedded linux systems.10/23

Approach & Implementation (1/5)Limit maximum physical memory size per each application.If memoryy usageg exceeds limitation,, malloc()() returns NULL.Data Structurecurrss : current rss ( current physical memory usage )cur rssmax rss : maximum rss ever been used since process createdlimit max : rss limitation of each process can usePhysical Memorycurrent rssmaximum rss ever usedlimit valuecur rss max rss11/23

Approach & Implementation (2/5)Why ‘max rss’ is needed?max rss : maximum rss ever been used since process created[example : trace physical memory usage]max rss 70: peak value of phy. mem. usagecur rsscurrss 40: current phy. mem. usageWe can sayy that this application’sppreal memoryy usageg is 70.With the ‘max rss’ value, we can limit application’s memory usage.12/23

Approach & Implementation (3/5)2 phases : memory usage profiling, run-time allocation controlProfiling: collect max RSS for each address section (text, data, .) of targetprocessRun-time memory allocation control: admission control of memory area allocationBased on profiling result, we can do allocation control.Text, RO DataRW DataBSS HeapBSS,HStackApp.p malloc(req)Glibcmalloc()library functionmemset(p,0,req)UserSpaceText.Size Data.SizeHeap.SizeStack.SizeHSProcess ATDText.RSSData.RSSHeap.RSSStack.RSS App’s maximum memory usageappM required i.T . max i.D. max i.H . max i.S . max App’s’ totall sum off virtuall addressddspaceappM virttotal i.T .size i.D.size i.H .size i.S .sizeKernelSpacedo mmap(req) syscall{.if ( !check region mem() )return –ENOMEM;if ( !check available mem() )return –ENOMEM;if( req vm enough memory() )return –ENOMEM;.create or expand a vmvm area structarea struct.}appappM required M virttotalMemory Usage ProfilingRun-time Memory Allocation Control13/23

Approach & Implementation (4/5)With the limit value and current rss, we can decide allocation will succeedor not.To succeed in allcation, 2 conditions must be fulfilled.allocation size limit – rssallocation size system free memoryBecause another process can use free memory.Data.RSSHeap.Limit 20MBHeap.RSS 15MBmain(void){.char *p;p alAddress imitTextDataHeapStackPhysical Memory20MB freeMalloc() FAILES !!because 10M (Heap.Limit-Heap.RSS)14/23

Approach & Implementation (5/5)Other regions (text, data, stack region) uses memory as well as heap.Æ Keep memory usage per each region.Need for allocation control of fork(),(), exec(),(), mremap(),p(), Max. 6MBMax. 20MBCurrent 2MBVirtualAddress SpaceText2MBDataBSS HeapHeap will useadditional 2MB memoryStackPhysical MemoryText will useadditional 4MB memory6MB freeA malloc(4MB) should be failed,failedeven though free memory (6MB) isstill enough for requested memory(4MB).15/23

RSS Quota (1/3)System DiagramApplicationhc Setting RSS QuotaSystem Call InterfacecfRSS QuotaParameterManagerdRSSLimit ValuesgVAS AdmissionC t lControlFile SystemeTTrace & updated t physicalh i lmemory page countersVirtual Memory Mgmt.Proc FSExt3d Save settings into memorydescriptorRSSCountersProcess MemoryDescriptorf Memory allocation requestsePhysical MemoryTracerPhysical Memory Mgmt.g Compare the counter withthe limit valueOSh Return results (success orfail)H/W – Physical Memory16/23

RSS Quota (2/3)FilesystemData Structurecur rss[]max rss[] : used for profiling.limit max[] : used for allocation controllimit S HBSS nt main(void) {.pthread create( &tid, );.}task structtask structtask structcomm[] “testapp”*mmpid 1000, tgid 1000comm[] “testapp”*mmpid 1001, tgid 1000comm[] “testapp”*mmpid 1002, tgid 1000mm struct0x00000000.vm area structvm area struct*mmapvm start, vm end*vm nextvm start, vm end*vm 0xC0000000.limit max[4];limitmax[4];cur rss[4];max rss[4];pgd t[]pte t[]*pgdpte t[]t t[]0xFFFFFFFF17/23

RSS Quota (3/3)Results# ./rssq test 10malloc start. : 1M x 06)(007)(008)(009)& memset completedmemory freeing.memory free complete.## psPID Uid1020.316 0317 0318 0VmSize Stat Command528 S initSW [ksoftirqd/0]620 S404 S664 R/bin/sh./rssq test 10ps# cat /proc/317/rss268490134Process Max : 10736KCode Max : 360KData Max :52KStack Max :16KOther Max : 10308K# echo "2684 90 13 4 /test/rssq test“ /proc/sys/vm/rss quota# ./rssq test 11malloc start. : 1M x 11malloc (000)malloc (001)malloc (002)malloc (003)malloc (004)malloc (005)malloc (006)malloc (007)malloc (008)malloc (009)malloc errormemory freeing.memory free complete.## cat /proc/sys/vm/rss quota2684 90 13 4 /test/rssq test#test program(1M x 10 allocation)Restrict memorybased on profilingAfter restrict,allocate 1M x 1118/23

Limitation & Future WorksLimitationWe must profile each application.Inaccuracy of calculating system free memory.Future WorksCalculates free memory more accurately.Improves PFRA (Page Frame Reclaiming Algorithm)Shared Memory Accounting.19/23

Another Approaches (1/3)No overcommitIt doesn’t use overcommit policy.“echo 2 /proc/sys/vm/overcommit memory”ProsWe can sure that OOM will never occur.ConsThere’s no merit of demand paging.Some or more applications may not run.20/23

Another Approaches (2/3)OOM notify to applicationWhen occurred lack of kernel memory, kernel notify it to user applications. Andeach application manages OOM /archive/2006/08/16/702746.aspxmem notify patchhttp://lwn.net/Articles/267013/ProsEffective manipulation of OOM.– Because application knows well which memory allocation is useless than the otherallocations.ConsApplication developer should consider about OOM.Also, the existing application code must be changed.21/23

Another Approaches (3/3)Improves OOM policyImproves victim selecting method.Android platformAndroid has its own victim selecting method.There’s “importance hierarchy” based on the state of components.htt // dl/ d id/i t /lifl ht mlProsEffective than kernel OOM killerkiller’ss victim policy.policyConsOOM still existsexists. Because it changes only victim policypolicy.22/23

The EndThank You.Q&A23/23

2 phases : memory usage profiling, run-time allocation control Profiling: collect max RSS for each address section (text, data, .) of target process Run-time memory allocation control: admission control of memory area allocation Based on profiling result, we can do allocation control. Text, RO

Related Documents:

We can t meet and greet them as we have in the past, so let me introduce you to your new neighbors. Daryle Lessard, R oom 113 Aleen Freeman, Room 1 02 Carol Sullivan, R oom 111 John Rogers, R oom 228 Jim and Carolyn Bro wn, Room 325 Cletus Baudendistel, R oom 221 If yo

(Daikin's products are subject to continuous improvements. Daikin reserves the right to modify product design, specifications and information in this data sheet without notice and without incurring any obligations) Submittal Revision Date: May 2020 Page 5 of 7 oom oom oom oom g ) 2 9 9 — 0 0 3 9 9 — 0 0 4 7 9 — 0 0 4 7 8 — 0 0 4 8 8 .

Zilveren Penseel Mijn wonderlijke oom Yvonne Jagtenberg tekst en illustraties Rubinstein 2018 15,99 isbn 978 90 476 2600 8 Als de ouders van Gerard op stap gaan past zijn oom op hem, zijn hond Dali en hun huis. Ze mogen absoluut geen kattenkwaad uithalen. Maar dat is moeilijk met een oom die vroeger in het circus heeft gewerkt

In memory of Paul Laliberte In memory of Raymond Proulx In memory of Robert G. Jones In memory of Jim Walsh In memory of Jay Kronan In memory of Beth Ann Findlen In memory of Richard L. Small, Jr. In memory of Amalia Phillips In honor of Volunteers (9) In honor of Andrew Dowgiert In memory of

Memory Management Ideally programmers want memory that is o large o fast o non volatile o and cheap Memory hierarchy o small amount of fast, expensive memory -cache o some medium-speed, medium price main memory o gigabytes of slow, cheap disk storage Memory management tasks o Allocate and de-allocate memory for processes o Keep track of used memory and by whom

MPA Guide to Avoiding Contact with Moving Machinery and Isolation The PDCA cycle 10 MPA Guide to Avoiding Contact with Moving Machinery and Isolation MPA Guide to Avoiding Contact with Moving Machinery and Isolation 11 The PDCA cycle This guide follows the Plan, Do, Check, Act (PDCA) approach to explore a variety of factors aimed at reducing incidents involving contact with moving

Chapter 2 Memory Hierarchy Design 2 Introduction Goal: unlimited amount of memory with low latency Fast memory technology is more expensive per bit than slower memory –Use principle of locality (spatial and temporal) Solution: organize memory system into a hierarchy –Entire addressable memory space available in largest, slowest memory –Incrementally smaller and faster memories, each .

ASP .NET (Active Server Pages .NET) ASP .NET is a component of .NET that allows developing interactive web pages, which are typically GUI programs that run from within a web page. Those GUI programs can be written in any of the .NET languages, typically C# or VB. An ASP.NET application consists of two major parts: