All right.
Uh, before I started my talk, let me think on organizer first of all this invitation.
The title of my talk is detecting new physics as nobody as the start of this
talk.
At the start of this talk, let me bring two papers to your attention which was
submitted to archive this month. The first one is adversarially learned anomaly
detection and CMS opening data rediscovering the top Quark, and another
one is from outlasts here is extract. This letter describes a search for retina new
Felix, using a machine learning anomaly detection procedure that doesn't rely on
our signal model hypothesis. And these two more of these two papers, an important
step about application of the machine learning techniques and nobody detection
from proof of concept to real data analysis, what is and nobody detection
here is the definition nobody detection is a task of classifying test data that
differ in some respect from the data that are available during training. Okay. Why
important step in nobody detection is to evaluate the noted response of the testing
data and then based on the note the response of the testing data, we're able
to analyze detection sensitivity Okay, this sounds like a PDE analysis. First we
analyze the PDS response or testing data then we are able to get summative analysis
sensitivity if we If we take a survey and the literature's are nobody detection then
we will say the history of the novelty detection is basically a history of
developing nobody evaluator or a methods for the testing sample, okay. Here it is.
It is Westwell corn out in literature's sometimes we say that terminology semi
supervised lending or fully unsupervised here, the point is this terminology
actually is something separate from the discussion here. They refer to how to
simulate or evaluate the backgrounds okay. So, look, you know, what evaluator or
methods suggest suggested in principle, patient and the roughly speaking, the
evaluators can be classified into two classes. based and classroom based, the
first one is isolation based evaluators, that is the novelty of a given testing
data point is evaluated according to its distance to or isolation from the
distribution of the no parent data in the feature space. So, here the point is, the
point is the evaluation is purely based on the relation between the given testing
data point and the distribution of the no pattern data in the feature space and all
the other testing points in the same sample are irrelevant for this evaluation.
Okay. And here is one example it is the forms the method areas the neighbors and
the left one is the mayor and right one is normalized the using the cumulative
distribution function. And in this definition, the D term in the numerator
represents the mean distance of a testing data point that tweets a king Aries the
neighbors And the deep fried trim represents the average of the mean
distance is defined for its k nearest neighbors k nearest neighbors okay and you
know neighbor represent a standard a standard deviation of the letter. So, here
the point is all of these three quantities are defined with respect to the training
sample, okay. So, you can imagine in the feature space one test important if you
fall away from the distribution of the training sample, then the value of the
train is big and these points will be scored high okay. So, this is idea and
another another proposal is the reconstruction error or loss function. And
this was proposed by two collaborations two years ago. And reconstruction error in
essence, is isolation based. So, this can be understood that since, for each test
important when we calculate it, we can throw an error somehow basically just
based on the relation between This has been point and no pattern data okay and it
has nothing to do with the other data points in the same has been sample okay
another class of evaluators are classroom based that is the nobody of a given has
important event evaluated according to the data clustering around this testing point
on top of the no pattern data distribution in the feature space. So, different from
the first one in this process. The evaluation takes into account evaluate
takes into account the correlation between the testing data points and the other
points around in the same data sample. So, this is the generic difference between
these two classes of evaluators. One example again is from the method of the
king areas, the neighbors, okay, the left one is isolation, isolation base The right
one is classroom based here detached in the denominator of the classroom based on
his minty stance of a testing data of this passing data point to the K nearest
neighbors in the state in the training data sample, okay. And the test is a mean
distance of the testing data point with King RS neighbors in the testing data as a
testing data sample. As for M is a hyper parameter, it can be chosen to be the
dimension of the feature space and afford that case t test two minus m represents
the local density of this testing data point in the testing sample and a four D
twin to minus n, okay. So in this case, the no water response is evaluated by
comparing the local densities of the testing point in the training sample and
the testing sample. Actually, there is the approximate the statistic interpretation
for such a evaluator is proportional to the s over r squared to be in the local
being of this testing data points. Another proposal was made recently by a Panama and
the DaVinci. And this proposal was made in the similar spirit, okay, the difference
between these two is just from the definition of the density was the first
one the density is defined to be the local density space and in the second case, the
density is replaced with probability density. This is the only difference
between that but this is not the whole story. We want you to synergize these two
different classes of evaluators, what do we like to do that? Okay, we can consider
example, a two dimensional Gaussian sample to what To show the reason the left column
shows the left column shows the performance of the isolation basic
evaluator. And we can say, we can say the
which, okay, for the samples, the red points represent the new pattern data. The
blue points represents the unknown pattern data, let's say, which read a strong
response to the isolation based evaluator. So, here the point is, in addition to the
red points in the original signal being at the two dimensional plane, the red points
between the two yellow circles also has also have a strong response to the
isolation based evaluator, okay, similarly, because for all red points
between these two yellow circles, how their local densities are comparable, in
the middle column, it shows the performance of the classroom based nobody
evaluator So, here look for red points. In addition, in addition to the the ones from
the original signal from the original signal being the red points from the non
signal raging with a prior fluctuation can also have a strong response to the
classroom based evaluator, this also can be understood because statistically
somehow it is not easy to distinguish the fluctuation upward and the resonance okay.
So this idea so here we can say in both cases somehow a being at the bottom
received contributions from non signal regions, okay. So here we want one is it
So here, we can say if somehow these two classes are invalid, evaluators can be
properly synergized then we would expect only in the intersection in the second set
of these red points will continue. built two tools have been at the bottom, okay,
for that case some others and acuity is expected to be improved. Okay, this is the
idea. And the first proposal is geometric mean of the isolation basic evaluator and
classroom basic evaluator, what are these hell? Let's consider one example in the
right panel, the black curve represents the normal response of the new pattern
data that one okay on events. So, for some of the events have to the Austin based
evaluator and these black points, some of them are from the generic second region
and some of them are from the non second region with upward fluctuation. And
However, for these two classes of contributions, they are scored differently
that is better to be scored differently by the isolation based evaluators. Why
because why is a sacred Undertale and another one is the Nobody. Okay, so here
we will expand the low scoring of the AAS database, the evaluator will compensate
for the high scoring of the cluster basic evaluator for these fake a signal points.
Okay, so basically the rough idea. And here is a two dimensional toy toy example.
And the bottom right panel shows some activities, significant performance of
these three different types of evaluators. In this panel, we can see the black curve,
it will represent the performance of supervised learning, okay, it is used as a
reference and the magenta curve represent performance of isolation based evaluator,
and the blue curve represents the significant, awesome based evaluate
evaluator. As for the red curve, it represents the signature a signature based
evaluator and we can say indeed, somehow in this simple contract Next, the
performance evaluator can be improved significantly by this as soon as you base
the evaluator okay. But, but they such a design is very intuitive Okay, this means
that it is not a fully optimized. So, here the design of this classroom based
evaluator and and this enhance the synergy base evaluator somehow breaks the
independence of the testing points, why because when we evaluate their response,
somehow we take into a correlation between the given testing point and its neighbors
okay. So, this will make So, this will laters will lose nice properties in
statistics of the original sample and then if we want to calculate the significance
based on based on the data response to this evaluators name, it means we have to
do something extra. Okay. And motivated by this, somehow we wanted to. So here we
were, we purposed some
aters. The idea is based on the nobody responds all the testing data sample, to
the isolation basic evaluator and to the classroom based evaluator, we find some
new set of signals and a new set of backgrounds. They are not real signal
events and the real spec one events instead, for the signal set and the
background to set the ratio between the real signal the real background events,
the ratios that are different. Okay, and then we introduce a deep neural network of
supervised learning and the two twin days to a twin, then the new signal set and the
new background set, and then we'll use auto neuron as this is a new signature
based evaluator and to compare the performance Band Aid hurts the isolation
isolation base to one class based one signature based based one. Here we can see
there are nine two dimensional Gaussian benchmarks, okay. So here the again red
points represent Background The blue points represents signals as well as the
rows, they represent the broadness of the distributions of the signal signal events
as well as the columns and the different center positions for the signal events.
Okay, here as our C curves and AUC values here, I will not go here I will not go
into details, but if you compare these curves, then we will see the performance
complementarity between the ICER isolation based evaluator and a classroom based
evaluator, and this complementarity somehow would result in improvements of
the AUC values for the signature based evaluators, especially for the, for this
alternative signature base, the evaluator somehow it has the best AUC universally in
all of these nine benchmarks. So, for example, for the ABC values supervised
learning versus this evaluator, the first one is point nine, nine versus point nine,
nine, second one is point nine versus point nine, nine. The third one is 1.0
versus 1.0. Second, second row, point nine one versus point nine 1.93 versus point a
9.99. Versus point nine, eight, last low point seven two versus point seven 1.84
versus point a one and point nine, six versus point nine, three. So you can say
the gap between the supervisor lending and this new evaluator somehow is very, is
very unnatural. Okay, so here is a list of efforts in the last eight years since I
work for Do you want all the time? Somehow I will stop here. Here is the
summary. Okay.
Okay, thank you very much, though.
Other any questions or comments for Tao?
Yeah, I have a question on the first, I think on the first few slides on the paper
that you showed from
what is it home
adversary learned anomaly detection the rediscovering of the top core. So do you
know how they did they have a pre selection applied and then and then run
the anomaly detector or did they just run on reconstructed data or raw data or
Wow, this is
this is a challenging question to me.
Yeah, and
I think it is not not easy to read out this substitute information from, from
these two papers. So
yes, I'm sorry, I don't know the answer. Okay.
And then I get when with a lot of these anomaly detectors, um, how do you mean you
train on a certain data set, but our data sets are not static because you are, you
know, the running conditions are constantly changing Well, not constantly,
but they change within the run quite often. And so, hasn't been validated that
these the each of these algorithms is able to detect that running for missions are
changing and change the expectation of the distribution.
Okay.
This is good question. The answer is yes, indeed for different scenarios, different
evaluators may have different performance and one example is the first type of
signature based evaluator that I just mentioned. And for that case, let me go to
the slide. Okay. So this evaluator work works pretty well, if the signal beam with
generic a single beam is essentially Undertale. And is somehow this this
signature based evaluator can works nicely. But if these two are switched, for
that case, it was a generic signal B is a seat on a Bock And the fluctuation in
front On the tail and then for that case and somehow the performance of this
evaluator is not that good actually you can also say this point from the outside
curves
in
Okay, in slide 19 and 20, but here, it is true that we are trying to find some, we
are trying to build up the evaluator which can work universally well. And this is
also one of the motivations for us to propose the second type of percentage
based evaluators
compare
the LC curves in this slide, you will say, indeed, these new signature based
evaluator works well, what's the universally better than the other ones?
Okay, thanks.
Oh, sorry.
Yeah, yeah, I'm afraid we are running a bit late. So if this is a very quick
question,
no further discussion.
Okay. Okay, so thanks again. Tau for for for the nice talk, and we move to the
following one, which is by Oscar.
And Hello. Hey, I cannot share why he says that.
No, it's