Why bother about CPU v GPU? Because this will
let you know how parallel processors making revolution in the Deep Learning \
AI world.
CPU or Central Processing Unit: is electronic
microprocessor which is designed to do sequential processing of tasks/programs
as described by von Neumann Architecture (1945). Neumann was working on
Manhattan project at Los Alamos National Laboratory, he wanted to device a
machine to do lot of calculations. So Neumann designed CPU architecture,
obviously he was influenced by the great Alan Turing, British
mathematician.
The reason I am trying to give background of CPU is to
understand the time its architecture was designed.
GPU or Graphics Processing Unit: many of us don’t
know that GPU exist in almost all modern computer, mobile phones or any display
units. It’s the work horse, does all the graphics related activities like
calculating vectors\polygons or all activities involved in the graphics
pipeline. In short to do math required to displaying video games\movies, images
is responsibilities of GPU.
Initially CPU is to do all graphics processing related
activities, but the raise of the sizes of movies, video games, images let to
the raise of GPU’s. Actually still CPU is central of a computer, but it just
outsource the graphics processing work to GPU. GPU is capable of doing many
times billions of calculations per second then CPU because of its enormous
number of its cores.
CPU v GPU
“A CPU consists of a few cores optimized for sequential serial
processing while a GPU has a massively parallel architecture consisting of
thousands of smaller, more efficient cores designed for handling multiple tasks
simultaneously” NVidia
CPUs & GPUs both have fundamentally different design
philosophies. We cannot compare apple to orange. Each excel at their own respective
domain. Here is the few basic difference between them:
CPU
|
GPU
|
Multiple cores
|
Hundreds or thousands of Cores
|
general purpose processors to handle all kind of activities
|
special purpose processors to handle polygon based 3d graphics
|
serial portions of the code run
|
parallel portions of the code run
|
have higher operating frequency, higher of number of registers
|
- lower operating frequency, lower number of registers
|
file handling, branching, serial tasks
|
better in math computer like polygons, vector math’s,
transformation
|
GPUs are in general targeted for gaming.
|
|
Sequential
|
Parallel
|
Deep learning:
With rapid advancement of GPU’s processing power researchers and
business started to use it in Deep learning \ AI
“Deep learning is the
fastest-growing field in artificial intelligence, helping computers make sense
of infinite amounts of data in the form of images, sound, and text” -
nvidia
Deep learning is nothing but trying to solve complex problems by
creating neural networks which works like human brain. It learns from the
process it does and try’s evolve or get better as it does more and more
operations. Basically try’s to become like one human brain.
Today’s advanced deep
neural networks use algorithms, big data, and the computational power of the
GPU to change this dynamic. Machines are now able to learn at a speed, accuracy, and
scale that are driving true artificial intelligence.
Deep learning is used in the research community and in industry
to help solve many complex problems like genetic simulation, molecular biology
simulation etc.
Parallel
Processors Framework
Following API’s helps programmers to manage parallelism and data
delivery in massively parallel processors (GPU’s):
OpenCL by Apple
DirectCompute by Microsoft
CUDA by NVidia
By using any above frameworks you can write programs to analyze
or solve problems like complex math calculation, molecular simulation, genetics
or even atom level simulations.
