High Performance Computing with GPUs

An Introductory Document

Contents
Introduction to Graphics Processing Units (GPUs) .............................................................................. 3 Adoption and Applications of GPUs ................................................................................................... 5 Bio-Informatics and Life Sciences ............................................................................................... 5 Computational Electromagnetic and Electrodynamics ................................................................ 5 Computational Finance .............................................................................................................. 5 Computational Fluid Dynamics................................................................................................... 5 Data Mining, Analytics, and Databases ....................................................................................... 5 Imaging and Computer Vision .................................................................................................... 6 MATLAB Acceleration ................................................................................................................ 6 Medical Imaging ........................................................................................................................ 6 Molecular Dynamics .................................................................................................................. 6 Weather, Atmospheric, Ocean Modeling, and Space Sciences .................................................... 6 Our Experience with GPGPU .............................................................................................................. 7 Planned Offerings .............................................................................................................................. 8 Consulting, Migration and Training ................................................................................................ 8 GPU Cloud Computing ................................................................................................................... 8 GPU enabled Software Products .................................................................................................... 8 Resources and References ................................................................................................................. 9

Introduction to Graphics Processing Units (GPUs)

Graphics Processing Units (GPU) are dedicated hardware used to offload graphics related computations. GPUs have been targeted at applications like, computer games, visualization, computer aided design (CAD) etc. Rapid growth has been recorded in terms of processing power and ease of programming the massively parallel hardware. With the state of the art GPUs packed with more than a billion transistors, the amount of processing power offered is an order higher when compared to state of the art CPUs. Due to their successful architecture design for parallel computing, GPUs have today evolved into desk side supercomputers. A single NVIDIA S1070 Tesla GPU which retails for as low as $1200 provides 1 TERAFLOPS of computational power. GPUs deliver supercomputing power at 1/10th the cost and 1/20th the power compared to an equivalent CPU based system. Because of reduced power and cooling costs, the GPUaccelerated cluster racks up additional savings over the lifetime of the system.

GPGPU: General Purpose Computation on GPU

GPU has evolved into a parallel computing hardware driven by graphics requirements. As a result, GPUs today comprise of 100s of small cores working in parallel. In comparison to state of the art CPUs comprising of 4 and 8 cores, current line of Nvidia GPUs offer 240 cores with upcoming GPUs offering up to 512 cores. With increasing computation power of the GPUs, the architecture has been extended beyond the fixed function graphics processing. The classic video game rendering GPU is now made more flexible to provide the user with ample control. General Purpose Processing on the GPU is now possible resulting into huge performance gains when compared against the CPU counterparts. Given the raw power provided by 100s of cores, various data parallel applications could now take advantage of this massively parallel hardware. With the growing success of low cost powerful computation resources, GPUs are now exposed as massively parallel processors with dedicated APIs and interface for general purpose computing. With

the introduction of NVIDIA's Compute Device Unified Architecture which is a C like interface to the GPU, more and more data and compute intensive applications are now able to harness the power of the GPU. Computation Platforms: NVIDIA CUDA Architecture NVIDIA CUDA is at the forefront of the high performance computing revolution. With a new compute and graphics architecture of the NVIDIA GPUs one can now have a TERAFLOP of processing power at his desk side for a few hundred dollars. With widely available CUDAcapable GPUs, thousands of software programmers are already using the free CUDA software tools to accelerate applicationsfrom video and audio encoding to oil and gas exploration, product design, medical imaging, and scientific research. OpenCL, an open standard inspired by CUDA is aimed to use various computing resources on a machine like the multi-core CPUs and one or more GPUs for computational purposes. Recent version of Mac OSX operating system now uses OpenCL to speed up various applications for better OS experience. Microsoft DirectX Compute is another upcoming API to join the parallel computing ecosystem enabling parallel computing easier and accessible to larger number of audience.

Adoption and Applications of GPUs

High-performance visual computing can benefit enterprises running applications with highly parallel code that uses floating-point arithmetic. Fewer systems are needed to give equal performance, resulting in savings on cost, space and power. Enterprise applications currently utilizing GPUs include finance, medical imaging, product design, movie and digital-video production, and oil and gas exploration. The applications have been or are in process of being, moved from super computers and computing grids to these many-core processors. The emergence of new programming tools and relatively low-cost hardware is aiding in these efforts, and is introducing many-core computing to a broader range of developers. Nvidia's CUDA, Microsoft's DirectX Compute, and the industry standard OpenCL are making, developing and implementing many-threaded algorithms more straightforward and portable across a variety of hardware platforms. However, a general lack of expertise and familiarity with massively parallel computing techniques will take several years to overcome, and likely will require a new crop of computer scientists entering the industry to bring it to the mainstream. Various applications in the below listed domains, had highly computation intensive requirements. Even with the evolution of multi-core CPUs such problems could not be addressed in real-time or practical durations. Due to the inherent data parallel nature of domains below, they fit very well to the GPU architecture, thus resulting into performance gains of up to 100x when performed using the GPUs. Bio-Informatics and Life Sciences Sequencing and protein docking are very compute-intensive tasks that see a large performance benefit by using a CUDA-enabled GPU. There is quite a bit of ongoing work on using GPUs for a range of bio-informatics and life sciences codes. Computational Electromagnetic and Electrodynamics Electromagnetic and electrodynamics applications using CUDA-enabled GPUs are widely deployed with customers such as Motorola, Kodak, Nokia, etc. GPU-based acceleration for Finite-difference time-domain (FDTD) scales with the number of GPUs. Computational Finance There is ongoing work in options pricing, risk analysis, and algorithmic trading using CUDA. Work on random number generators and Monte-Carlo simulations have boosted various applications in domain of computing for finance solutions. Computational Fluid Dynamics Several ongoing projects on Navier-Stokes models and Lattice Boltzman methods have shown very large speedups using CUDA-enabled GPUs. CUDA accelerated PDE (Partial Differential Equation) solvers over regular grids are used for fluid simulations. Data Mining, Analytics, and Databases Databases are the workhorse of the enterprise today. Searching through databases and finding useful information has become a big computational challenge. Researchers from academia and Microsoft, Oracle, SAP, and many other corporations are looking to CUDA-enabled GPUs to find a scalable solution.

Imaging and Computer Vision Computer vision and image processing algorithms are computationally intensive. With CUDA acceleration, applications can achieve interactive video frame-rate performance. Large number of applications could now run at real-time and near-real-time rates for better user experience and save on precious computing time. MATLAB Acceleration Mathematical computing software applications like MATLAB, Mathematica, and LabView benefit greatly by using CUDA-enabled GPUs. These very-high level scripting and language software applications can use the CUDA FFT and BLAS libraries besides writing CUDA functions for key kernels. Medical Imaging Medical imaging is one of the earliest applications to take advantage of GPU computing to get acceleration. The use of GPUs in this field has matured to the point that there are several medical modalities shipping with NVIDIAs Tesla GPUs now. Molecular Dynamics Molecular dynamics applications are extremely amenable to the massively parallel architecture of NVIDIAs GPUs. Work has been done on VMD and also molecular dynamics leading to software packages such as NAMD and HOOMD. Weather, Atmospheric, Ocean Modeling, and Space Sciences Several weather and ocean modelling applications such as WRF (Weather Research and Forecasting model) and Tsunami simulations are achieving tremendous speedups that enable savings in time and improvements in accuracy.

Our Experience with GPGPU

GPUs have come a long way from video game rendering hardware to C language driven massively parallel co-processors. The same has been achieved over the period of less than a decade. We bring forward our experience over the last 6 years in programming the GPUs for graphics and high performance computing applications. Our team has over 5 years of experience with programmable GPUs for graphics and visualization applications. With the introduction of programmable GPUs we have been among the first movers to exploit the state of the art GPU features to introduce and advance various rendering techniques. Our team has been conceived at Center for Visual Information Technology (CVIT), IIIT Hyderabad. CVIT is the largest research group in India working on a wide spectrum of visual computing problems. The lab under the supervision of Prof. P. J. Narayanan was one of the first movers in the field of GPU Computing for which he was awarded the NVIDIA fellow award in year 2008. The lab has around 60 research and undergraduate students working on problems ranging from Medical Imaging, Visualization, Optical Character Recognition and Video and Image search solutions. CVIT has been working closely with NVIDIA Corporation and have produced state of the art GPU solutions. The lab has made significant contributions in the field of Image and Video Processing algorithms, Graph algorithms, Data-Parallel and Inherently linear algorithms, and various visualization solutions dealing with gigabytes of data. The above work was initiated with the introduction of NVIDIAs CUDA technology and has been used with various other internal projects to speed up existing algorithms and applications. In the era of classic GPGPU prior to the launch of CUDA, general purpose computation required the user to present its computations as a graphics rendering problem. Our deep understanding of the programmable graphics pipeline helped us quickly get on board with various non-graphics application development on the GPU. Exploiting different programmable units of the rendering pipeline efficiently helped us deliver many efficient parallel versions of the otherwise CPU oriented applications. Given that not all applications are inherently data-parallel, various algorithms and techniques were needed to develop efficient GPU based solutions of the same. With a strong background in graphics and general purpose programming, we were ready to be at par with CUDA development. Over the last 2 years, we had the chance to work with rapidly changing NVIDIA CUDA feature set exploiting all major and minor modifications introduced with different versions. It is now commonly understood that efficient CUDA based solutions only come with extremely good understanding of the GPU architecture itself. Our experience with GPU programming over years empowers us with a unique opportunity to design efficient CUDA based solutions for the current and upcoming hardware. We have been involved with various Image/Video Processing, Database Processing and Graph Algorithms based solutions using the CUDA framework for the GPU. Our experience with a vast landscape of applications also provides us with ample knowledge and enables greater efficiency when attacking novel solutions for the GPUs.

Planned Offerings
Consulting, Migration and Training
GPU computing is only a 2-3 years old revolution. Currently, there is a scarcity of companies which could undertake tasks involving across domain knowledge along with deep understanding of efficient GPU computing. Due to its derivative nature from existing graphics hardware a very few companies provide CUDA Trainings and consultancy. Companies also face the problem of finding employees with experience and understanding of such niche technology. Our office is located at IIIT Hyderabad campus itself and it positions us uniquely in collaborating with the highly experienced faculty and very capable research students working in various research centers at IIIT. We plan to closely collaborate with CVIT and work with people who are working with state of the art GPUs and leading the revolution. We offer not only the working knowledge of CUDA but also understand and deliver end to end GPU solutions right from selecting the correct hardware to efficient software solutions. http://www.nvidia.com/object/cuda_consultants.html http://www.nvidia.co.in/object/cuda_service_provider_in.html

GPU Cloud Computing

With more and more applications moving to the cloud (internet), large number of software and data is being managed on servers accessible at anytime from anywhere. A lot of hassle is saved on the user's part by doing away with hard to manage hardware, software and data. We propose a GPU Cloud Computing Cluster which provides remote computing resources. GPU hardware and software setup requires substantial expertise and cost. Not all upcoming and established business will find it logical and feasible to set up a cluster or a farm of GPUs for their computing needs. With growing benefits from GPU computing a lot of business will take advantage of this cheaper and faster computing platform. These businesses can be targeted with a service which allows them to quickly make use of GPU computing and confirm their migration to a GPU based computing platform.

GPU enabled Software Products

With the growing use of GPUs in PCs, laptops and now even on hand-held devices, a large number of consumer applications are being moved to fast parallel chips. Hand held devices are becoming more and more powerful to deal with high resolution image and video data. User interfaces and gaming on such devices are exploiting graphics chips for better realistic experience. URL Resources and References Description Common Forum for GPGPU http://gpgpu.org/ GPU Computing http://www.nvidia.com/object/GPU_Computin g.html CUDA Success Stories http://www.nvidia.com/object/tesla_testimoni als.html NVIDIA CUDA Home http://www.nvidia.com/object/cuda_what_is.h tml NVIDIA CUDA Applications http://www.nvidia.com/object/cuda_home.ht ml NVIDIA Tesla Computing http://www.nvidia.com/object/tesla_computin g_solutions.html CUDA Accelerated Applications http://www.nvidia.com/object/cuda_app_tesla .html