Past GPU Memory Limits with Unified Memory On Pascal > 자유게시판

• 목록
• 답변
• 글쓰기
• 게시판 리스트 옵션
  • 수정
  • 삭제

Trendy computer architectures have a hierarchy of reminiscences of varying measurement and efficiency. GPU architectures are approaching a terabyte per second memory bandwidth that, coupled with excessive-throughput computational cores, creates a super system for information-intensive tasks. Nonetheless, all people is aware of that quick memory is costly. Fashionable applications striving to solve bigger and larger issues can be limited by GPU memory capability. Since the capacity of GPU memory is significantly lower than system memory, it creates a barrier for builders accustomed to programming just one memory house. With the legacy GPU programming model there is no such thing as a straightforward option to "just run" your application when you’re oversubscribing GPU memory. Even if your dataset is simply slightly larger than the available capability, improve neural plasticity you would still need to handle the energetic working set in GPU memory. Unified Memory is a way more intelligent memory management system that simplifies GPU development by offering a single memory area straight accessible by all GPUs and CPUs in the system, with automatic web page migration for data locality.

Migration of pages permits the accessing processor to profit from L2 caching and the lower latency of native memory. Furthermore, migrating pages to GPU memory ensures GPU kernels benefit from the very excessive bandwidth of GPU memory (e.g. 720 GB/s on a Tesla P100). And web page migration is all fully invisible to the developer: the system automatically manages all data motion for you. Sounds nice, right? With the Pascal GPU architecture Unified Memory is even more highly effective, thanks to Pascal’s bigger virtual memory deal with house and Web page Migration Engine, enabling true digital memory demand paging. It’s additionally price noting that manually managing memory motion is error-prone, improve neural plasticity which affects productivity and delays the day when you may finally run your entire code on the GPU to see these nice speedups that others are bragging about. Builders can spend hours debugging their codes because of memory coherency issues. Unified memory brings enormous advantages for developer productivity. In this post I'll present you the way Pascal can enable functions to run out-of-the-box with larger memory footprints and achieve nice baseline efficiency.

For a moment you can completely forget about GPU memory limitations while creating your code. Unified Memory was introduced in 2014 with CUDA 6 and the Kepler structure. This comparatively new programming mannequin allowed GPU functions to use a single pointer in both CPU features and GPU kernels, which tremendously simplified memory administration. CUDA eight and the Pascal architecture significantly improves Unified Memory performance by adding 49-bit virtual addressing and on-demand page migration. The large 49-bit virtual addresses are adequate to allow GPUs to entry the complete system memory plus the memory of all GPUs in the system. The Web page Migration engine permits GPU threads to fault on non-resident memory accesses so the system can migrate pages from anywhere within the system to the GPUs memory on-demand for efficient processing. In other words, Unified Memory transparently enables out-of-core computations for any code that's utilizing Unified Memory for allocations (e.g. `cudaMallocManaged()`). It "just works" without any modifications to the application.

CUDA eight additionally adds new ways to optimize information locality by providing hints to the runtime so it is still doable to take full management over information migrations. Nowadays it’s exhausting to discover a high-performance workstation with only one GPU. Two-, 4- and eight-GPU methods are becoming common in workstations in addition to giant supercomputers. The NVIDIA DGX-1 is one example of a excessive-performance integrated system for deep studying with 8 Tesla P100 GPUs. For those who thought it was troublesome to manually handle knowledge between one CPU and one GPU, now you could have eight GPU memory areas to juggle between. Unified Memory is essential for such systems and it permits more seamless code improvement on multi-GPU nodes. At any time when a particular GPU touches data managed by Unified Memory, this data might migrate to native memory of the processor or the driver can set up a direct entry over the accessible interconnect (PCIe or NVLINK).