Artificial Intelligence for High Energy Physics

The following section is included:

• Contents

The following sections are included:

• Artificial Intelligence at the Frontiers of High-Energy Physics
• Why This Book, and Who Should Read It?
• Pre-requisites
• How to Use This Book
• References

Boosted decision trees are a very powerful machine learning technique. After introducing specific concepts of machine learning in the high-energy physics context and describing ways to quantify the performance and training quality of classifiers, decision trees are described. Some of their shortcomings are then mitigated with ensemble learning, using boosting algorithms, in particular AdaBoost and gradient boosting. Examples from high-energy physics and software used are also presented.

The following sections are included:

• Introduction: Pre-Deep Learning State-of-the-Art
• Application of Deep Learning to Four Vectors
• Parameterized Networks
• Handling Sets of Four Vectors
• Physics-aware Networks
• Conclusions
• References

Modern machine learning tools offer exciting possibilities to qualitatively change the paradigm for new particle searches. In particular, new methods can broaden the search program by gaining sensitivity to unforeseen scenarios by learning directly from data. There has been a significant growth in new ideas and they are starting to be applied to experimental data. This chapter introduces these new anomaly detection methods, which range from fully supervised algorithms to unsupervised, and include weakly supervised methods.

High energy physics (HEP) experiments involve large and complex detection apparatuses, which need to be operated with high availability to profit at best from the expensive beam time provided by modern particle accelerators. HEP data analysis relies on well understood experimental equipment to chase statistically rare phenomena. These requirements call for constant and reliable monitoring to spot potential malfunctioning. For this reason, HEP collaborations build complex data quality monitoring (DQM) infrastructures and procedures, targeting prompt identification of anomalies arising from either hardware problems or data processing. The HEP monitoring problems are usually challenging for conventional statistical-based methods due to their inherently multidimensional nature. The goal of this chapter is to explore how machine learning (ML) methods of anomaly detection can help improve the existing DQM pipeline, under the specific constraints derived from the critical role of data validation: interpretability of the results and long-term maintainability of the system.

Generative models are a class of machine learning methods that map random numbers into structured data. One potentially powerful application of generative models to high energy physics is as surrogate models for slow detector simulations. Interactions of particles with detector material is often the slowest component of the full simulation stack and so mimicking this component with a neural network may significantly accelerate the entire simulation. This chapter introduces various approaches to deep generative modeling and then provides examples of applications to high energy physics detector simulations.

LHC physics crucially relies on our ability to simulate events efficiently from first principles. Modern machine learning, specifically generative networks, will help us tackle simulation challenges for the coming LHC runs. Such networks can be employed within established simulation tools or as part of a new framework. Since some neural network architectures can be inverted, they also open new avenues in LHC analyses.

Deep learning models are yielding increasingly better performances thanks to multiple factors. To be successful, model may have large number of parameters or complex architectures and be trained on large dataset. This leads to large requirements on computing resource and turn around time, even more so when hyperparameter optimization is done (e.g. search over model architectures). While this is a challenge that goes beyond particle physics, we review the various ways to do the necessary computations in parallel, and put it in the context of high-energy physics.

With ever increasing data rates. The challenge of processing data at ever higher throughputs with more complex algorithms is daunting. However, new computing technology, and optimized algorithm design show a path towards resolving these large throughput issues. In this chapter, we present recent ideas on algorithm compression, quantization, and parallelization that allow for ultra low latency high throughput algorithms. Additionally, we present methods for longer latency algorithms that can be sped up through deep learning acceleration, leading to enhanced computational throughput.

End-to-end analyses of data from high-energy physics experiments using machine and deep learning techniques have emerged in recent years. These analyses use deep learning algorithms to go directly from lowlevel detector information directly to high-level quantities that classify the interactions. The most popular class of algorithms for these analyses are convolutional neural networks that operate on experimental data formatted as images. End-to-end analyses skip stages of the traditional workflow that includes the reconstruction of particles produced in the interactions, and as such are not limited by efficiency losses and sources of inaccuracy throughout the event reconstruction process. In many cases, deep learning end-to-end analyses have been shown to have significantly increased performance compared to previous state-of-theart methods.

Clustering methods are in the core of data reconstruction in particle physics. Recent advancements in machine learning and computer vision offer a number of new, powerful techniques to be explored including image segmentation techniques with convolutional neural networks and a generalization of clustering tasks using graph neural networks. Applications of modern machine learning techniques for clustering tasks in pipelines of physics data reconstruction are discussed.

Machine learning methods have a long history of applications in high-energy physics (HEP). Recently, there is a growing interest in exploiting these methods to reconstruct particle signatures from raw detector data. In order to benefit from modern deep learning algorithms that were initially designed for computer vision or natural language processing tasks, it is common practice to transform HEP data into images or sequences. Conversely, graph neural networks (GNNs), which operate on graph data composed of elements with a set of features and their pairwise connections, provide an alternative way of incorporating weight sharing, local connectivity, and specialized domain knowledge. Particle physics data, such as the hits in a tracking detector, can generally be represented as graphs, making the use of GNNs natural. In this chapter, we recapitulate the mathematical formalism of GNNs and highlight aspects to consider when designing these networks for HEP data, including graph construction, model architectures, learning objectives, and graph pooling. We also review promising applications of GNNs for particle tracking and reconstruction in HEP and summarize the outlook for their deployment in current and future experiments.

Image-based jet analysis is built upon the jet image representation of jets that enables a direct connection between high-energy physics and the fields of computer vision and deep learning. Through this connection, a wide array of new jet analysis techniques have emerged. In this text, we survey jet image-based classification models, built primarily on the use of convolutional neural networks, examine the methods to understand what these models have learned and what is their sensitivity to uncertainties, and review the recent successes in moving these models from phenomenological studies to real-world application on experiments at the LHC. Beyond jet classification, several other applications of jet image-based techniques, including energy estimation, pileup noise reduction, data generation, and anomaly detection, are discussed.

Particle identification is a central task in the analysis of data from experiments. This is particularly true for data from Neutrino detectors where observations are made only through the products of interactions. The information available for identification varies depending on detector type. We survey various machine learning methods for particle and interaction identification. In particular, we cover applications making use of recently developed deep learning algorithms.

Sequence-based modeling broadly refers to algorithms that act on data that is represented as an ordered set of input elements. In particular, machine learning algorithms with sequences as inputs have seen successful applications to important problems, such as natural language processing (NLP) and speech signal modeling. The usage of this class of models in collider physics leverages their ability to act on data with variable sequence lengths, such as constituents inside a jet. In this chapter, we explore the application of recurrent neural networks (RNNs) and other sequence-based neural network architectures to classify jets, regress jet-related quantities, and build a physics-inspired jet representation, in connection to jet clustering algorithms. In addition, alternatives to sequential data representations are briefly discussed.

Our predictions for particle physics processes are realized in a chain of complex simulators. They allow us to generate high-fidelty simulated data, but they are not well-suited for inference on the theory parameters with observed data. We explain why the likelihood function of high-dimensional LHC data cannot be explicitly evaluated, why this matters for data analysis, and reframe what the field has traditionally done to circumvent this problem. We then review new simulation-based inference methods that let us directly analyze high-dimensional data by combining machine learning techniques and information from the simulator. Initial studies indicate that these techniques have the potential to substantially improve the precision of LHC measurements. Finally, we discuss probabilistic programming, an emerging paradigm that lets us extend inference to the latent process of the simulator.

In this chapter we consider the impact of nuisance parameters on the effectiveness of machine learning in high-energy physics problems, and discuss techniques that allow to include their effect and reduce their impact in the search for optimal selection criteria and variable transformations. The introduction of nuisance parameters complicates the supervised learning task and its correspondence with the data analysis goal, due to their contribution degrading model performances in real data, and the necessary addition of uncertainties in the resulting statistical inference. The approaches discussed include nuisance-parameterized models, modified or adversary losses, semi-supervised learning approaches, and inference-aware techniques.

In recent times, neural networks have become a powerful tool for the analysis of complex and abstract data models. However, their introduction intrinsically increases our uncertainty about which features of the analysis are model-related and which are due to the neural network. This means that predictions by neural networks have biases which cannot be trivially distinguished from being due to the true nature of the creation and observation of data or not. In order to attempt to address such issues we discuss Bayesian neural networks: neural networks where the uncertainty due to the network can be characterized. In particular, we outline the Bayesian statistical framework which allows us to categorize uncertainty in terms of the ingrained randomness of observing certain data and the uncertainty from our lack of knowledge about how processes that are observed can occur. In presenting such techniques, we show how uncertainties which arise in the predictions made by neural networks can be characterized in principle. We provide descriptions of the two favored methods for analyzing such uncertainties. We will also describe how both of these methods have substantial pitfalls when put into practice, highlighting the need for other statistical techniques to truly be able to do inference when using neural networks.

We discuss the determination of the parton substructure of hadrons by casting it as a peculiar form of pattern recognition problem in which the pattern is a probability distribution, and we present the way this problem has been tackled and solved. Specifically, we review the NNPDF approach to PDF determination, which is based on the combination of a Monte Carlo approach with neural networks as basic underlying interpolators. We discuss the current NNPDF methodology, based on genetic minimization, and its validation through closure testing. We then present recent developments in which a hyperoptimized deep-learning framework for PDF determination is being developed, optimized, and tested.

A number of scientific competitions have been organized in the last few years with the objective of discovering innovative techniques to perform typical high-energy physics tasks, like event reconstruction, classification and new physics discovery. Four of these competitions are summarized in this chapter, from which guidelines on organizing such events are derived. In addition, a choice of competition platforms and available datasets are described.

The following section is included:

• INDEX