SHF: Small: CPU-GPU Collaborative Execution in Fusion Architectures

The most recent trend in chip design is to integrate general purpose central processing units (CPUs) with graphics processing units (GPUs) onto a single microprocessor chip. Looking beyond the obvious benefits of simply putting components closer together, such integration presents an unprecedented opportunity for the CPU and GPU to collaborate, yielding a system whose performance far exceeds the sum of its parts. Whereas, currently, the CPU and GPU are delegated different tasks that each is suited for, this project explores new ways for the CPU and GPU to tackle and collaborate on the same task. The collaboration is fundamentally different from conventional parallel processing, because the CPU and GPU have radically different architectures. In particular, the CPU performs novel meta-computation that assists a GPU task, or vice versa. This innovative approach uncovers new opportunities for emerging heterogeneous architectures.

The project investigates CPU/GPU collaborative execution paradigms to overcome fundamental limitations of both CPU and GPU computing tasks. The GPU achieves high computational throughput and energy efficiency by executing a single instruction on many data items. Its efficiency is severely degraded if some data items are not available due to long memory access latency or different data items require different operations. The CPU/GPU collaboration leverages the CPU to run far ahead of the GPU to prefetch the data and reorganize the operations needed for different data items so as to drastically improve the GPU efficiency. Conversely, on the CPU side, the CPU/GPU collaboration leverages the GPU's parallel processing power to accelerate auxiliary computations that greatly enrich the CPU program. Locality analysis, for instance, reveals the nature of memory accesses but requires high computation time when running on a CPU. GPU acceleration makes it possible to perform locality analysis simultaneously with the CPU program and adapt the memory hierarchy on-the-fly to improve CPU performance. The research cuts through software and hardware layers. From the software perspective, the project develops automated approaches to generate code for collaborative execution. From the hardware perspective, future architectures are defined to facilitate more effective CPU/GPU collaboration. The automated software approach adds value to current and upcoming microprocessors by enabling them to run more efficiently. The performance improvement and energy savings translate directly into enhanced user experience.