DEVELOPER DAY - Khronos Group

Shader Permutations From each material graph we generate fragment and vertex shader permutations- “Vertex Factory” (mesh type)- Static mesh, skeletal mesh, particle, terrain, - Forward lighting pass- Base forward pass with CSM shadow- Base forward pass, unshadowed- Shadow depths- Shared for opaque objects- Unique for alpha masked objects- Translucency & effects Result is over 28,000 individual shadersThis work is licensed under a Creative Commons Attribution 4.0 International License The Khronos Group Inc. 2018 - Page 27

Shaders – OpenGL ES Set of PSOs encountered while playing Fortnite gathered offline using automatedand manual gameplay- 4,300 PSOs actually needed for renderingOn OpenGL ES we must compile from GLSL source code First launch of Fortnite- Compile all shader programs- Save the resulting shader program binary to the user’s phone usingGL OES get program binary Subsequent launches- Recreate shaders with glProgramBinaryOESThis work is licensed under a Creative Commons Attribution 4.0 International License The Khronos Group Inc. 2018 - Page 28

Shaders – OpenGL ES – LRU Cache Ideally we would have all shader programs created before gameplay starts- Shaders measured to expand to more than 10x their binary size in driver RAMallocations- Instead we use an LRU cache to keep only a limited number of shader programsresident- Saves over 400MB on some devicesThis work is licensed under a Creative Commons Attribution 4.0 International License The Khronos Group Inc. 2018 - Page 29

Shaders – OpenGL ES – LRU Cache Shader eviction strategies- When resident shader program count exceeds some threshold- Estimation of resident shader memory- On object destruction-Not great for transient but frequent uses like particles, so we add an extra delay Shader restoration strategies- Stream shader binary from storage, creates hitches in practice- Recreate shader from compressed binary in RAM- On Adreno, shaders total about 20MB compressed so it’s feasible to always keep themresident- On Mali we keep binaries in RAM for non-resident shader programs- Create create binary from shader program on eviction and store in RAM- Restore shader program from RAM binary and free RAM binaryThis work is licensed under a Creative Commons Attribution 4.0 International License The Khronos Group Inc. 2018 - Page 30

Shaders - Vulkan We gather Vulkan PSOs using the same mechanism as for OpenGL GL On Vulkan we create pipelines on first launch and save vkPipelineCache to storage Vulkan mode also has a runtime PSO cache in memory with LRU Kostiantyn will provide some detailsincluding shader memory savingsThis work is licensed under a Creative Commons Attribution 4.0 International License The Khronos Group Inc. 2018 - Page 31

Recent UE4 Renderer ImprovementsUnreal Engine has been evolving to support new platforms and rendering APIs since itsinception.The Render Hardware Interface abstraction layer (RHI) has had some recentimprovements made to better support modern graphics APIs like Vulkan.1. Explicit render passes2. Vulkan subpasses3. High-level rendering refactorThis work is licensed under a Creative Commons Attribution 4.0 International License The Khronos Group Inc. 2018 - Page 32

1. Explicit Render PassesStarting a new renderpass can be expensive on mobile tiled GPUs- Save out the results of the previous render pass from the GPU core to RAM- Load an existing render target from RAM back into the GPU core Render passes in the high level code were originally implict- Engine code set render targets and then the RHI guessed if we were starting anew render pass- Each rendering operation (eg shadows, base color, translucency) called functionsat the beginning and end of their operations to set render targets and resolve theresults New in UE 4.22- RHI functions have been added to explictly begin and end render passes- UE4 mobile renderer now makes use of these to remove of unnecessarytransitions eg base pass translucency post processingThis work is licensed under a Creative Commons Attribution 4.0 International License The Khronos Group Inc. 2018 - Page 33

2. Vulkan Sub-passes Use case: Soft Particle Translucency and Deferred Decals- These rendering techniques require access to an existing fragment depth value- In GLES we use EXT shader framebuffer fetch toget an existing depth value- Depth previously written to alpha in base pass, or weuse ARM shader framebuffer fetch depth stencilwhere available Compare fragment depth against existing depthThis work is licensed under a Creative Commons Attribution 4.0 International License The Khronos Group Inc. 2018 - Page 34

2. Vulkan Sub-passes Very Vulkan-specific feature, and only applicable tomobile GPUs No general UE4 support for subpasses, instead: RHIBeginRenderPass() call provides a hint that thefollowing passes will use depth- Vulkan RHI sets up 2 subpasses- RHINextSubpass()- VulkanRHI calls VkNextSubpass()- OpenGLRHI could use this to callFramebufferFetchBarrierQCOM() to supportQCOM shader framebuffer fetch noncoherentThis work is licensed under a Creative Commons Attribution 4.0 International License The Khronos Group Inc. 2018 - Page 35

2. Vulkan Sub-passesUnfortunately extra PSO permutations are necessary to support MSAA The depth is fetched using GLSL subpassLoad(input), but when using MSAA you mustuse subpassLoad(input, sampleindex) So toggling MSAA requires alternate shaders Targeting UE 4.23This work is licensed under a Creative Commons Attribution 4.0 International License The Khronos Group Inc. 2018 - Page 36

3. High Level Rendering Refactor Much more aggressive caching for static scene elements The full state of each drawcall is cached when mesh added to the scene- Pipeline State ObjectBound resources, shader constants and uniform buffers Much reduced Render thread cost After calculating visibility it simply walks the drawlist and applies the cached state Initial release in UE 4.22This work is licensed under a Creative Commons Attribution 4.0 International License The Khronos Group Inc. 2018 - Page 37

3. High Level Rendering Refactor Automatic geometry instancing support- Sort draw list by PSO, mesh and bound resources- Examine for sets of matching PSOs and bound resources- Requires all per-instance constants (eg transform matrices) to be stored in asingle buffer- Look up per-instance parameters in the shader- Potentially bad for mobile performance.- Previously measured 30% cost. Work in progress.This work is licensed under a Creative Commons Attribution 4.0 International License The Khronos Group Inc. 2018 - Page 38

Part 2Vulkan for FortniteMobileKostiantyn Drabeniuk,Samsung ElectronicsThis work is licensed under a Creative Commons Attribution 4.0 International License The Khronos Group Inc. 2018 - Page 39

Galaxy GameDev Provide the best gaming experience to customers on Samsung devices Promote new technologies usage Contribute to the most popular game engines Support game developers all over the worldThis work is licensed under a Creative Commons Attribution 4.0 International License The Khronos Group Inc. 2018 - Page 40

Agenda Advantages of using Vulkan in mobile games How to get more FPS - performance optimizations How to get stable FPS - hitching/memory optimizationsThis work is licensed under a Creative Commons Attribution 4.0 International License The Khronos Group Inc. 2018 - Page 41

OpenGL ESFPS ChartFPS39RHI ThreadTime (ms)16.94RHI Thread time60403040ms 20201000051015202530SecondsThis work is licensed under a Creative Commons Attribution 4.0 International License01002003004005006007008009001000Frames The Khronos Group Inc. 2018 - Page 42

Vulkan Balanced CPU/GPU usage Lower CPU overhead Parallel tasking Explicit control No error checking at runtimeThis work is licensed under a Creative Commons Attribution 4.0 International License The Khronos Group Inc. 2018 - Page 43

Vulkan vs GLESFPS Chart6040GLES20Vulkan0051015202530SecondsRHI Thread time4030ms 20GLES10Vulkan00100 200 300 400 500 600 700 800 900 1000FramesThis work is licensed under a Creative Commons Attribution 4.0 International LicenseGLESVulkanFPS3947( 8)RHI Thread Time (ms)16.948.25(-51%) The Khronos Group Inc. 2018 - Page 44

Performance optimizations DescriptorSet cache Merge RenderPasses Remove useless barriers Remove extra depth copy Occlusion query Buffer uploadThis work is licensed under a Creative Commons Attribution 4.0 International License The Khronos Group Inc. 2018 - Page 45

DescriptorSet cache Reuse already updated DescriptorSets AllocateDS UpdateDSBindDS Draw AllocateDS UpdateDSBindDS Draw AllocateDS UpdateDSBindDS Draw AllocateDS UpdateDSBindDS Draw Allocate and UpdateDescriptorSets before eachdraw call AllocateDS UpdateDSBindDS Draw BindDS Draw BindDS Draw BindDS Draw AllocateDS UpdateDSBindDS Draw BindDS Draw Reuse Descriptors Sets fromcacheThis work is licensed under a Creative Commons Attribution 4.0 International License The Khronos Group Inc. 2018 - Page 46

DescriptorSet cache There were a lot of cache misses due to storing buffer offset inside DescriptorSetAllocate DescriptorSetUpdate rSet3Binding 0offset .Binding 0offset Binding 0offset DescriptorSet1DescriptorSet2DescriptorSet3Binding 0Binding 0offset 128offset 224offset 352ShaderShaderShaderBinding 0Binding 0Binding 0Binding 0Bind DescriptorSet0128UBO1224UBO2This work is licensed under a Creative Commons Attribution 4.0 International License352UBO3VkBuffer The Khronos Group Inc. 2018 - Page 47

DescriptorSet cache Hit rate can be improved by using Dynamic Uniform BufferDescriptorSet1Binding 0offset Allocate DescriptorSetDescriptorSet1DescriptorSet1 can be used for all binds,no need to allocate and update new oneBinding 0offset 0Update DescriptorSetDynamicOffset 128DynamicOffset 224DynamicOffset 352ShaderShaderShaderBinding 0Binding 0Binding 0Bind DescriptorSet0128UBO1224UBO2This work is licensed under a Creative Commons Attribution 4.0 International License352UBO3VkBuffer The Khronos Group Inc. 2018 - Page 48

DescriptorSet cache Do not use Vulkan Handle for hash calculation- Vulkan can use same handles for different types- Vulkan can reuse handles from destroyed resources Generate own Handle ID for all Vulkan resources and use it for hash calculationHashInfoHashInfo BufferInfoBufferInfoVkBufferRangeHashValueBuffer Handle IDRangeOffsetOffset This work is licensed under a Creative Commons Attribution 4.0 International LicenseHashValue The Khronos Group Inc. 2018 - Page 49

DescriptorSet cachePercentile vkUpdateDescriptorSets() callsPercentile RHI Thread time6002045015300ms eOriginalDSCacheUpdates (avg calls per frame)2522(-99.2%)RHI Thread Time Avg (ms)10.129.15(-0.97)This work is licensed under a Creative Commons Attribution 4.0 International License70% The Khronos Group Inc. 2018 - Page 50

Merge LoadDecalRenderTargetUpscale reLoadDecal oreThis work is licensed under a Creative Commons Attribution 4.0 International LicenseStoreStore The Khronos Group Inc. 2018 - Page 51

Remove useless barriersREAD ONLY - COLOR ATTACHMENTCOLOR ATTACHMENT - READ ONLYREAD ONLY - READ ONLYRenderDoc captureThis work is licensed under a Creative Commons Attribution 4.0 International License The Khronos Group Inc. 2018 - Page 52

Merge RenderPasses/Remove extra barriersFPS 0260SecondsOriginalMergeRPMedian FPS41This work is licensed under a Creative Commons Attribution 4.0 International License44 The Khronos Group Inc. 2018 - Page 53

Remove extra depth copyBase passColorBase passDepthColorDepthBarrier to SRCCopyNew DepthBarrier to ReadBarrier to OptimalDecal,Translucency passes(Z write off)ColorDecal,Translucency passes(Z write off)DepthColorDrawDraw.DrawDepth.fetchThis work is licensed under a Creative Commons Attribution 4.0 International LicenseDrawfetch The Khronos Group Inc. 2018 - Page 54

Occlusion query Get occlusion query result for 3 frames back- UE4 gets occlusion results for 2 frames back by default- 3 swapchain back buffers are used in Android, so sometimes waiting happensSwapchain queue size 2Frame N-3 RenderOcclusionFrame N-2 RenderOcclusionFrame N-1 RenderOcclusionFrame N GetQueryResult Sometimes, CPU need to waitocclusion querySwapchain queue size 2Frame N-3 RenderOcclusionFrame N-2 RenderOcclusionFrame N-1 RenderOcclusionFrame N GetQueryResult DO NOT need to waitocclusion queryThis work is licensed under a Creative Commons Attribution 4.0 International License The Khronos Group Inc. 2018 - Page 55

Occlusion query Query management in original version- Use one global query poolfree queriessubmitted in N-3 framesubmitted in N-2 framesubmitted in N frameN-2 frameGlobalPoolRequest M queriesN-1 ResetN frameK calls ofvkGetQueryPoolResults()K calls ofvkCmdResetQueryPool()GlobalPoolThis work is licensed under a Creative Commons Attribution 4.0 International License The Khronos Group Inc. 2018 - Page 56

Occlusion query Query management after optimization- Use separate pool for each framefree queriessubmitted in N-3 framesubmitted in N-2 framesubmitted in N frameN-2 frame Pool1Pool2 Pool3N-1 frameRequest M queries Pool1Pool21 call of vkGetQueryPoolResults( ,0,K, )by specifying firstQuery and queryCountGetResult Pool3N frame1 call of vkCmdResetQueryPool( ,0,K, ) byspecifying firstQuery and queryCountReset Pool1This work is licensed under a Creative Commons Attribution 4.0 International License Pool2 Pool3 The Khronos Group Inc. 2018 - Page 57

Occlusion query Performance n FPS2729( 2)FPS Stability75%90%( 15%)CPU Usage16.32%15.55%GPU Usage70.90%79.52%This work is licensed under a Creative Commons Attribution 4.0 International License The Khronos Group Inc. 2018 - Page 58

Buffer upload Remove staging buffer usage- Mobile GPUs usually have unified memory. Such memory allow direct host access- For mobile GPUs staging buffer is not needed and extra copying can be removedBuffer’s rawdataBuffer’s rawdatacopyStaging VkBuffervkCmdCopyBuffer()HOST VISIBLEcopyThis work is licensed under a Creative Commons Attribution 4.0 International LicenseVkBufferDEVICE LOCALVkBufferHOST VISIBLE /DEVICE LOCAL The Khronos Group Inc. 2018 - Page 59

Buffer uploadRHI Thread timePercentile RHI Thread veStagingBuffer40%50%60%70%80%90% 100%RemoveStagingBufferRHI Thread Time (avg)14.96 msThis work is licensed under a Creative Commons Attribution 4.0 International License14.00 ms The Khronos Group Inc. 2018 - Page 60

Hitching/memory optimizations Asynchronous Vertex/Index buffer create Upload Texture DescriptorSetLayout cache miss Remove shader duplication Purge ShaderModules PSO cache missThis work is licensed under a Creative Commons Attribution 4.0 International License The Khronos Group Inc. 2018 - Page 61

Asynchronous Vertex/Index buffer create Allow asynchronous Vertex/Index buffer creation- The basic versions of CreateVertex/IndexBuffer() use needless RHI Thread stall- Vulkan RHI allows asynchronous Vertex/Index buffer creation StallRHIThreadWait until RHI Thread finishexecution of current task Task No waiting Render ThreadCreateVertexBufferRT Render ThreadCreateVertexBufferRTTask OriginalAsyncBufferCreationFrame Time9289(-3)Render ThreadTime3816(-22)RHI ThreadTime4138(-3)RHI Thread Number of HitchesRHI ThreadThis work is licensed under a Creative Commons Attribution 4.0 International License The Khronos Group Inc. 2018 - Page 62

Upload Texture Texture uploading process:UploadTextureImage’s rawdatacopyStagingVkBufferHOST VISIBLERecord commandsto copy Buffer intoImageVkImageDEVICE LOCALSubmitUploadCmdBufferBarrier to SRCvkCmdCopyBufferToImage()Barrier to READ ONLYThis work is licensed under a Creative Commons Attribution 4.0 International License The Khronos Group Inc. 2018 - Page 63

Upload Texture Record texture upload commands into one command bufferFrame N SubmitUploadCmdBuffer2 UploadTextureNSubmitUploadCmdBufferN N calls of vkQueueSubmit()Frame N xture4 UploadTextureNSubmitUploadCmdBuffer1 1 call of vkQueueSubmit()This work is licensed under a Creative Commons Attribution 4.0 International License The Khronos Group Inc. 2018 - Page 64

Upload TextureExecution time of SubmitUploadCmdBuffer()12010080ms ionNumber of HitchesOriginalUploadTextureOptimizationFrame Time8940(-49)Render Thread Time168(-8)RHI Thread Time384(-34)This work is licensed under a Creative Commons Attribution 4.0 International License The Khronos Group Inc. 2018 - Page 65

PSO cache miss Do not hash pointers Hash only info which is used for Pipeline creation Calculate shader’s key from ShaderCode and use it for hashingHashInfoShader’s pointers Pipeline Create InfoPointer to Depth stateShadersPointer to Rasterizationstate Extra dataHashValueDepth stateRasterization stateExtra dataCalculated frompointers and extra dataCache hit rateBeforeAfter1 session33.5%34.9%2 session77.4%95.4%3 session80.8%99.6%HashInfoShader’s keys Depth stateRasterization stateThis work is licensed under a Creative Commons Attribution 4.0 International LicenseHashValue The Khronos Group Inc. 2018 - Page 66

DescriptorSetLayout cache miss Hash data instead of pointer

- Dequeue state change into game object render proxies - Create or update render resources - Render scene 1. Retrieve occlusion queries from a previous frame 2. Calculate object visibility 3. Render shadow maps 4. Render opaque geometry including lighting and shadows (base pass) 5. Render occlusion queries testing on depth layed down by base .