One, I think this is the computer time to start the second session of the tutorial
track of electricity. Welcome, everyone. So we'll be using the webinar
functionality of zoom, such as yesterday. For those not familiar with it, the
speakers are there and then have the right to speak, why the attendees are nice to
raise their hands with the button on the zoom interface, so that they can raise the
attention to the co host. And we'll be unmuting you after the talks. So feel free
to raise your hands during the talk and after a turn so that we can unmute you and
you will that be able to ask your questions. If there is any problem then
you have also to chat, so that we can try to see if we can resolve any of the
problems
with this, and if
there is no major issue now, we can certainly start with the first
presentation decision from A to contest on GPU for boosting performance in head.
Okay, so can you hear me?
Yes. Let me hear. Oh, I'm
trying to share the slide.
Okay. You see those lights?
Yes.
Okay, so good afternoon, everybody. Today I'm pleased to report you my summary about
the usage of graphic processing unit for boosting performance in high energy
physics computational use cases is an outline. Okay. So let me Just to briefly
introduce the architecture of GPUs for whose is less familiar with with it. So,
graphic processing unit is a programmable architecture that offers a large number of
parallel and independent streams of instruction.
They were
initially designed for image processing and image rendering, but nowadays there
are also used for so called general purpose GPU computing that means to use
GPU to perform mathematical to solve mathematical problems in scientific field,
where we compare the structure the architecture of graphic processing unit
with the CPU, we see that the essential layout is quite similar. They both have a
dedicated memory a dedicated memory and they have the arithmetic logic units which
are the process are responsible to perform arithmetic and logic operations. Also they
have different number of control units. That's strange the instructions to the
processor. So the strip's the chips the structure of the cheapest, very similar.
Despite this fact, they were designed to cope with different type of computation.
So let me just add that both GPU and CPU use threads to distribute the parallel
workloads of workloads of a program among the multiple processor. Also, there is a
difference in the scale of the architectural CPU can manage up to
hundreds of processor and threads, and GPU can scale beyond many thousands of them.
So there are typical scopes are quite different. The CPU focuses more on
minimizing the latency in context switching among the application, reducing
the time spent in moving from one to another, and the GPU and more at
implementing the single instruction of multiple on multiple threads parodic that
aims at maximizing its throughput that means maximizing the amount of operation
performed on the memory
on data per time
time unit.
So, while a GPU is more flexible as a CPU is more flexible a GPU is only focused on
the on the on the aforementioned biobank, where each thread in a group performs the
same sequence of instruction, then its neighbors do. So, in some contexts, GPUs
are faster and less power consuming when compared to the CPU and the workflow that
fits this paradigm master comply with some kind of statistics. So, when we reached
when we successfully exploited you bill for this kind of computation, we can reach
also peak of performance that can be one order of magnitude higher than the CPU
wants. And one other thing to take into account that is that recently the number
of plots floating point operation that means The capability of the GPU has been
growing at a rapid pace compared on how much CPU are growing. So GPUs are becoming
more and more powerful with time passing by. So usually to use a GPU for
accelerating our
computing workloads is indeed a choice of convenience. Also, because from our point
of view, nowadays, based on our computing now can have up to eight GPUs and up to
four CPUs on the same motherboard, meaning that if we have a workload that can scale
that that can be well deployed both on GPU and on CPU. Indeed, we can, we can buy
less computing nodes by using more GPUs. So in the end, the engineering the
software that we currently have, and I think in the algorithm is compensated by
the cost of the performance related to the adoption of a GPU. So let's just talk
about the general purpose gpgpu computing, meaning that as I said before, For is the
usage of graphic cards in generic computing this is a reality in scientific
fields also outside the high energy physics, we have a lot of machine learning
and artificial intelligence successful application of the GPU, but also for real
time image processing. In those use cases, where activity is a is a requirement like
for instance, self driving cars. So any title algorithm that any inherently that
finally allow good it can, hypothetically scale that can particularly scale Well,
with parallel architecture is a good candidate to be deployed on a on a GPU
with a large core density. So also in the field of high energy physics, this is
true, and GPUs has been investigated over fireable that decade. And there are
currently many kinds of application that promoted the usage and the acceleration of
multiple acreage in different fields in different states. Audio inside the head
field, there are more or less advanced at the maturity level projects now are
currently ongoing. And to mention the forming larger experiments, LLC ATS, le
CMS and lsv have been evaluating solution that integrated GPUs since a while. And
the three main topics are indeed online and offline data reconstruction but also
physics simulation, and machine learning and deep learning application or supplied
to the analysis field. So now before looking at a broader overview about the
status of the of the current projects that we've tried to highlight, which are the
trait of an algorithm that can be well deployed on on a GPU. So in the field of
high energy physics, we have multiple application lights that are related to the
construction lift. For instance, we have succeeding models of truck fitting and
These are kind of the
other goods that also has
an inherent combinatorial computation that can benefit also from the usage of
graphics processing unit. So also today's and in, in recent times some experiment
are already moving to GPUs in order to sometime even meet requirements for their
upgrades. So let's just define which are the main feature of our high energy
physics algorithm that must comply with to be ported on a GPU GPU. And in the end,
what we find is that the algorithm must be static and predictable meaning that we in
the in its control stuck with control flow, we don't want to encounter too many
effects statements and also the the program should be intelligent enough in
using the local memory in order to minimize in the latency in the
computations. So will now present very briefly an interesting example that fits
very well to single instruction of multiple response, which is a Kalman
filter. And this algorithm is present in that many pop is a popular algorithm and
is present in many different scenarios. So, the idea is to have an iterative
algorithm that at every iteration, but for the same kind of operation operations and
this is true for every instance that can run concurrently on a on a parallel
architecture for instance, the one of the GPU and in this case we can see that
considering a hypothetical event we can perform the truck fitting of every truck
on different threads. So, the the only operation that Kalman filter performs are
mapping. As I said matrix multiplication. That means that data can change but the
operation not not doesn't do not so, We do not encounter branches in the control flow
means that we can reach very high throughput and in in deploying it in
scaling it on on a parallel architecture on a parallel architecture. So, in the
end, what we also consider is that to efficiently achieve a large throughput, we
have to fulfill the
day
to fulfill the occupancy
of the of the of the GPU. And the possibility to perform multiple
reconstructions also on multiple events allows us to implement a different
multiple levels of parallelism. So, in this case, GPU are very, very efficient in
performing
those kinds of computation.
So, let's talk about the broader overview. In recent times we observe a raised
interest in the topic of using GPU in high energy physics. This mainly is driven
because the The competition challenges a lot of the upcoming runs at the LSE are
more and more demanding in terms of the computing resources. So, what we observe
is also kind of consolidation is in a knowledge common knowledge base for the
development of of IoT our field and there is a an increasing interest in rethinking
existing algorithms in order to inspect if there is a possibility to move them
towards a parallel approach either that be on CPU or GPU. So, in the end, we have
also some critical point with the target integration with the existing and
consolidated frameworks, which sometimes might be not obvious and also the fact
that we do want to avoid code duplication wherever it is possible. So, in general,
any AI or physics algorithms whose implementation can provide a large benefit
These are good candidates for being uploaded on accelerator.
Okay, let's just start with the overview.
When we start with hcb lsv from country will operate its high level trigger one of
the construction entirely on a GPU. That means that the reconstruction of each sub
detector will be offloaded to an a graphics card. And this is thanks to the
Allen framework a self contained piece of software that can be deployed both on CPU
and GPU with the same result indeed, that implements efficient tracking and all the
other algorithms can be used in reconstruction and selection on on hL hL T
one three. So I advise you also to to check the talk on Wednesday should be
about this. So what it emerged in it emerged in the in this case is that the
results Good matching from the feature provide features provided by a GPU and the
requirements from a level trigger one in the electricity. And as you can see in the
plot of the plot on the right, the capability of the GPUs can increase but
Allen framework does not saturate in terms of throughput, meaning that they are
currently able to successfully and efficiently explore a exploit to the all
the resources that are being deployed it for them. So he's very promising. And now
we move from two to CMS. Historically, a lot of effort from their side has been
spent in the support to support the heterogeneity in a telogen 80 in the
framework, and currently up to 24% of the current aligner construction is GP ready
meaning that can be run on GPU And this is thanks to some frameworks like for
instance, a track which is responsible for the pixel based and tracks and vertex
reconstruction. You see in the bar plot on the right a comparison between the legacy
code legacy version on GPU and the new version that can run both on GPU and also
in this case, but they also have a calorimeter rocker local construction that
can be ported, that this has been ported on GPU. It is also interesting to observe
that also for for the high luminosity LFC, they have already they are going to, to to
to deploy your higher granularity calorimeter. And its reconstruction
algorithm is it's going to it's going to be on GPU as well and be observing the
port on the right that the the parallel version on GPU can scale up to 550 times
faster. And the interesting thing is that the same code can Compared to the second
version, so single core, the interesting fact is that the same code can be deployed
also on a CPU obtaining scaling by running in parallel. Indeed also in CMS, we have a
lot of machine learning application that can be run on a GPU and not only for the
training part, but also for the MCs and they are investigating possible techniques
for simulation
just
considering Atlas now no Atlas as historically already performed some some
tests of the of the implementation of their old tracking on GPU. You can see in
the plot on the right that the results were promising then in the end, they
decided not to use them, but it is not very relevant at this point, what is in
it, what is relevant is that 414 and five, they will also have the knowledge base to
allow us to investigate the use of GPUs, they are probably going to use them and
they will use it for like trigger on flatter construction and a lot of other
application like for instance, the simulation which is very interesting since
they have these two projects where they are porting part of the libraries to be
able to exploit the GPU
GPU to boost their,
their computation. So, similar GPUs for simulation, they're interested in not
easily implementable like other use cases and also in our in our class, we have
different machine learning techniques that are ready to be run on GPU backends. And
last but not least, at least the least experimental and three will move towards
our trigger less data acquisition mode, meaning that they will implement the so
called continuous readout and to obtain that they are going to
achieve that
larger data compression factors that are Going to reduce the input bandwidth from
3.5 terabytes per second down to hundreds of 100 of gigabytes per second. And GPUs
are the pivotal architectures to efficiently process and operate these
reduction. Considering for instance, the most expensive computationally expensive
part which is the time projection chamber tracking, we observe on the plot on the
right, which is actually are normalized to speed up how we can in this case, compare
the computing power of a GPU to the computing power of CPUs in order to
somehow estimate which are the exchange factor between the two the two parts the
two devices, so since the working the working condition will be more towards the
right part on the plot, we can say that one GPU will replace up to 40 CPU cores in
the in In, in their online infrastructure, so, there are also more advanced scenarios
that foresee to include to deploy more and more part of the reconstruction on the
GPUs whenever they are available in order to efficiently exploit those resources,
but also to free some some resources on CPU. I will also advise you to attend all
David Ross presentation on this topic. So, here I come to my conclusion. Extract
metallics he will be extremely challenging in the in terms of computing requirements
and GPU might be one solution to this increasing demand to address the the lack
of computing power. There are ongoing many different efforts with diverse scope
carried out by all the alleged larger illiteracy experiments. And the idea is to
accelerate all the parallel workflows that you have in order to Performance beyond
what we can be currently achieved with CPUs. There are also some cases where the
GPUs are enabling the some scenarios that otherwise wouldn't be reachable to by only
using standard CPU. And one last point I'll leave you to consider is that the
next generation of data center, but also performance computing facility will
increase the number of GPUs on board that they're computing now. So, so to be
efficiently able to be able to efficiently exploit those kind of architecture will
will give us a lot of potential that that otherwise we will just ignore, since most
of our workflows are on CPU, and from my side is everything and thanks for your
attention.
Hey, Matteo,
thank you for taking time. And I'm opening the floor for questions if you find the
race. button
and interface, I see an article
and allowing you to talk to you to be able to unmute yourself and ask your question.
Yes. Can you hear me? Yes. Cool. So I was wondering about the use of CUDA and
particularly considering the development of apparently more vendors trying to
develop GP GPUs and trying to deploy them. And if we,
if it's still a good idea to to be vendor specific by using CUDA exclusively,
basically, also, considering that there's a lot of development in terms of
other parallelism frameworks, like is
that clear?
Yes, yes. So I think what what's your opinion on this what
my take on that is that in the end, this has to be considered depending from
experiment to experiment. So there are some use cases also in the past Were in
Vidya frameworks were considered just because of their level of maturity but
also their level of the performances. So the idea is that depending on on how much
manpower how much resources you have, you can investigate also different different
vendor vendor solution, but also different frameworks that claims to be cross
portable across across different architectures layer. I didn't went didn't
go very much into detail about this topic in the presentation because it is a more
broader one. But the idea is that more or less all the experiments are considering
frameworks that allows for portability, both on Nvidia or AMD, or Intel GPUs in
the future. I think that nowadays nobody is thinking anymore indeed. To a single a
single vendor solution in any case, they always leave some some other door open and
that for some tests and development on that in the end you have to converge for
production to one architecture, but it depends if you are able to support it, you
can also write one code and be able to run in different GPUs but you have to be sure
every time that the result of the same the performances are cross portable and etc.
So, my take on that is that depends on how how much are you are you are able to
support both of them or the three of them or it depends.
But would you say there's a trend like towards the trend
depends on who you consider normalize on how fluent they are. Because if you look
at the outside our field, indeed, the domination of the media is is adaptable.
When there are more and more convincing use cases that claims that they can pop,
let's say TensorFlow or AMD GPUs and whatever. So for the time being, I don't
have a very strong opinion on. But the trend on considering other GPUs may be
much cheaper. Maybe that just fulfilled What you need is, is present. Maybe it
isn't. It is growing.
Okay, thank you very much interesting.
Is there any other questions, I don't see any raised hands
and one for you.
So you've talked about mostly on premises usage of GPU, but then know that there is
also a line of work on trying to use cloud based tools queues are server based GPU on
the Computing Center. So something in between the online tilty type of
facilities and the HPC facilities where there is a GPU on each node and something
where you either have like just one server of GPU for the whole HPC or just the GPUs
on the cloud. Can you say a couple of words on this?
Well, I think that the usage of cloud GPUs will probably be more directed for those
offline kind of computation because
in the end hlp farms or
de facto h lt. clusters
are not really interested at the moment, at least, to the best of my knowledge to
the ploy some kind of a bus abstraction layers in the analyzing GPUs, for for use
cases. So usually what One experiment tries to do is to exploit edit master edit
maximum capabilities every single GPU that means that we are paralyzed in it. So,
having a cloud approach is not the goal you, you you aim for it is possible that
indeed there are some offline use cases that also are going to consider the
conversion of a cluster
in order to be able to
fully
locate part of GPUs for the smaller tasks but I'm not really sure this is very
convenient for our use cases. If with cloud the approaching you you meant why
GPUs indeed when you buy via GPUs on Amazon, they are we have 12 minutes that
there is a perhaps GPU that is partitioned, and you will see like it is a
less powerful one. But these, I don't think an experiment is going to have an
online workflow on on an Amazon or Google platform or whatever, cloud provider
solution cluster. So these are different use cases that indeed can be investigated.
But for the moment, we have more stick to the online construction and other
sounds good that I see your hands your hand up.
You want to I just, I just want to very briefly comment on this did while I mean,
everything you said is, of course, true material. But I think at least in LHB once
we have a large cluster of GPUs sitting there. Then at some point, kind of
exposing them for other non online workflows when you're not taking data
is something that you'd respond Think about isn't necessarily trivial, but I
think that there will be kind of increasingly work and reflection over the
next years on, on how we make sure that they're not going to sitting there is that
silicon particularly during Ellis three.
Good point. Good point. Yes, sir. All right, thank you Matteo for talking and
answers.