Machine Learning with Python

The following sections are included:

• About the Author
• Contents

The following sections are included:

• Naturally Learned Ability for Problem Solving
• Physics-Law-based Models
• Machine Learning Models, Data-based
• General Steps for Training Machine Learning Models
• Some Mathematical Concepts, Variables, and Spaces
  • Toy examples
  • Feature space
  • Affine space
  • Label space
  • Hypothesis space
  • Definition of a typical machine learning model, a mathematical view
• Requirements for Creating Machine Learning Models
• Types of Data
• Relation Between Physics-Law-based and Data-based Models
• This Book
• Who May Read This Book
• Codes Used in This Book
• References

This chapter discusses basics of Python language for coding machine learning models. Python is a very powerful high-level programming language with the need for compiling, but with some level of efficiency of machine-level language. It has become the top popular tool for the development of tools and applications in the general area of machine learning. It has rich libraries for open access, and new libraries are constantly being developed. The language itself is powerful in terms of functionality. It is an excellent tool for effective and productive coding and programming. It is also fast, and the structure of the language is well built for making use of bulky data, which is often the case in machine learning…

This chapter discusses typical scientific computations using codes in Python. We will focus on how numerical data are represented mathematically, how they are structured or organized, stored, manipulated, operated upon, or computed in effective manners. Subtleties in those operations in Python will be examined. At the end of this chapter, techniques for initial treatment of datasets will also be discussed. The reference materials used in the chapter include Numpy documentation, Scipy documentation, https://gluon.mxnet.io/, and https://jupyter.org/. Our discussion shall start with some basic linear algebra operations on data with a structure of vector, matrix and tensor.

This chapter discusses some topics of probability and statistics related to machine learning models and the computation techniques using Python. Referenced materials include codes from Numpy documentation (https://numpy.org/doc/), Jupyter documentation (https://jupyter.org/), and Wikipedia (https://en.wikipedia.org/wiki/Main_Page). Codes from mxnet-the-straight-dope (https://github.com/zackchase/mxnet-the-straight-dope) are also used under the Apache-2.0 License…

To build an ML model for predictions, one needs to use some hypothesis, which predefines the prediction function to connect the feature variables to the learning parameters. Thus, a proper hypothesis may follow the function approximation theory, which has been studied intensively in physics-law-based models [1–4]. The most essential rule is that the prediction function must be capable of predicting an arbitrary linear function in the feature space by a chosen set of learning parameters. Therefore, the prediction function is assumed as a complete linear function of the feature variables, and it is one of the most basic hypotheses…

This chapter discusses two fundamentally important supervised machine learning algorithms, the Perceptron and the Support Vector Machine (SVM), for classification. Both are conceptually related, but very much different in formulation and algorithm. In the opinion of the author, there are a number of key ideas used in both classifiers in terms of computational methods. These ideas and the resulting formulations are very inspiring and can be used for many other machine learning algorithms. We hope the presentation in this chapter can help readers to appreciate these ideas. The referenced materials and much of the codes are from the Numpy documentations (https://numpy.org/doc/), Scikit-learn documentations (https://scikit-learn.org/stable/), mxnet-the-straight-dope (https://github.com/zackchase/mxnet-the-straight-dope), Jupyter Notebook (https://jupyter.org/), and Wikipedia (https://en.wikipedia.org/wiki/Main_Page) …

Activation functions are some of the most essential elements in machine learning models based on artificial neural networks. Activation functions are used for the purpose of bringing in must-have nonlinearity to the neural network to enable it for complex problems. We have discussed about this in Chapter 5. This chapter takes a close look at this topic. In this book, the activation function is generally denoted using ϕ (z) with an argument of z that is often an input to a neuron in a neural network and often an affine transformed output from the previous layer of neurons…

The following sections are included:

• General Issues on Optimization and Minimization
• Analytic Differentiation
• Numerical Differentiation
• Automatic Differentiation
  • The concept of automatic or algorithmic differentiation
  • Differentiation of a function with respect to a vector and matrix
• Autograd Implemented in Numpy
• Autograd Implemented in the MXNet
  • Gradients of scalar functions with simple variable
  • Gradients of scalar functions in high dimensions
  • Gradients of scalar functions with quadratic variables in high dimensions
  • Gradient of scalar function with a matrix of variables in high dimensions
  • Head gradient
• Gradients for Functions with Conditions
• Example: Gradients of an L2 Loss Function for a Single Neuron
• Examples: Differences Between Analytical, Autograd, and Numerical Differentiation
• Discussion
• References

The following sections are included:

• Introduction
• Analytic Optimization Methods: Ideal Cases
  • Least square formulation
  • L2 loss function
  • Normal equation
  • Solution existence analysis
  • Solution existence theory
  • Effects of parallel data-points
  • Predictability of the solution against the label
• Considerations in Optimization for Complex Problems
  • Local minima
  • Saddle points
  • Convex functions
• Gradient Descent (GD) Method for Optimization
  • Gradient descent in one dimension
  • Remarks
  • Gradient descent in hyper-dimensions
  • Property of a convex function
  • The convergence theorem for the Gradient Decent algorithm
  • Setting or the learning rates
• Stochastic Gradient Descent
  • Numerical experiment
• Gradient Descent with Momentum
  • The most critical problem with GD methods
  • Formulation
  • Numerical experiment
• Nesterov Accelerated Gradient
  • Formulation
• AdaGrad Gradient Algorithm
  • Formulation
  • Numerical experiment
• RMSProp Gradient Algorithm
  • Formulation
  • Numerical experiment
• AdaDelta Gradient Algorithm
  • The idea
  • Numerical experiment
• Adam Gradient Algorithm
  • Formulation
  • Numerical experiment
• A Case Study: Compare Minimization Techniques Used in MLPClassifier
• Other Algorithms
• References

Regression is one of the most widely used tools in analyses of engineering systems. It is also an important technique in the construction of numerical models for both physics-law-based methods (such as the point interpolation methods PIM [1], and the moving least square or MLS used in the meshfree methods [2]) and machine learning models. In a machine learning setting, it takes inputs of a dataset and produces a prediction of a real number, once a regression model is established/trained…

In many practical applications, we often only need to make a “yes” or “no” type of decision. Probability prediction is often the best choice for making such a decision. For example, what is the likelihood that a machinery may fail in operation in the coming month? What is the probability that a virus will become a pandemic in a region in the coming season? This type of problem requires an assessment on probability of the occurrence of an event, and is a typical classification problem. The simplest classification problem is the binary classification: there are only two classes, “yes” or “no”, for decision making. This chapter shall focus on this type of problem, focusing on necessary methods, formulation, and techniques that can be used for best prediction for binary labels, using neural networks. The reference materials for this chapter include the following:

• Lecture series by David S. Rosenberg, Office of the CTO at Bloomberg.
• Hastie, Tibshirani, and Friedman’s book: The Elements of Statistical Learning.
• James, Witten, Hastie, and Tibshirani’s book: An Introduction to Statistical Learning.
• Shalev-Shwartz and Ben-David’s book, Understanding Machine Learning: From Theory to Algorithms.
• The mxnet-the-straight-dope (https://github.com/zackchase/mxnet-the-straight-dope) (under the Apache License 2).

In the previous chapter, we discussed neural networks for binary classification, useful for “yes” or “no” types of problems. It is also a basis for handling many types of multi-classification problems, because one can often change a multi-classification problem into a set of binary classification problems. The techniques include one-to-rest, all-pairs, and their variation, as we have discussed in the Chapter 6 on SVM. There are many well-established modules for this type of purpose, such as Scikit learn. In this chapter, we discuss a systematic means to handle multi-classification problems, also known as k-classification problems, where we have in general a dataset that has k classes.

In the previous chapter, we have seen examples of using a simple (input and output layers, only one layer has learning parameters) neural network for k-classification of handwritten digits. We found that there is room for improvements, and we have shown that other classifiers like the random forest and the extra-forest performed better than the one-layer neural network. This chapter will show that a neural network with more layers (by adding in some hidden layers) is one of the ways to make an improvement. This type of neural networks is often known as multilayer perceptron (MLP) [1–5]…

The following sections are included:

• Why Regularization
• Tikhonov Regularization
  • Demonstration examples: One data-point
  • Demonstration examples: Two data-points
  • Demonstration examples: Three data-points
  • Summary of the case studies
• A Case Study on Regularization Effects using MXNet
  • Load the MNIST dataset
  • Define a neural network model
  • Define loss function and optimizer
  • Define a function to evaluate the accuracy
  • Define a utility function plotting convergence curve
  • Train the neural network model
  • Evaluation of the trained model: A typical case of overfitting
  • Application of L2 regularization
  • Re-initializing the parameters
  • Training the L2-regularized neural network model
  • Effect of the L2 regularization
• A Case Study on Regularization Parameters Using Sklearn
• References

Convolutional neural network (CNN) or ConvNet [1–6] is one of the most powerful tools in machine learning, especially for processing multi-dimensional images and image-based classifications. It is currently widely used, and effective modules are made available at many packages/libraries, such as Scikit learn, PyTouch, TensorFlow, and Keras. The CNN uses quite a number of special techniques, which have very much different net configurations, compared to the neural networks discussed in the previous chapters. CNNs still use affine transformations, but these are specially designed with spatial filters to capture the spatial features of images. These details are often hidden in the black boxes of the modules available at various libraries. This chapter offers a concise description of the key ideas and techniques used in CNN and then a demonstration of CNN. More detailed information can be found in the rich open literature and the original papers given for each CNN network. An excellent online course by Andrew Ng is also available…

Recurrent neural networks (RNNs) are yet another quite special class of artificial neural networks (NNs) for datasets with features of sequence or temporal significance. An RNN typically has an internal state recorded in a memory cell, so that it can process sequential inputs, such as video and speech records. Affine transformations are used in an extended affine space that includes memorized states as additional features. RNNs have important applications in various areas including natural language processing, speech recognition, translation, music composition, robot control, and unconstrained handwriting recognition [1], to name just a few. A recent review on RNNs can be found in [2]…

The following sections are included:

• Background
• K-means for Clustering
  • Initialization of means
  • Assignment of data-points to clusters
  • Update of means
  • Example 1: Case studies on comparison of initiation methods for K-means clustering
  • Example 2: K-means clustering on the handwritten digit dataset
• Mean-Shift for Clustering Without Pre-Specifying k
• Autoencoders
  • Basic structure of autoencoders
  • Example 1: Image compression and denoising
  • Example 2: Image segmentation
• Autoencoder vs. PCA
• Variational Autoencoder (VAE)
• References

The following sections are included:

• Basic Underlying Concept
  • Problem statement
  • Applications in sciences, engineering, and business
  • Reinforcement learning approach
  • Actions in discrete time: Solution strategy
• Markov Decision Process
• Policy
• Value Functions
• Bellman Equation
• Q-learning Algorithm
  • Example 1: A robot explores a room with unknown obstacles with Q-learning algorithm
  • OpenAI Gym
  • Define utility functions
  • A simple Q-learning algorithm
  • Hyper-parameters and convergence
• Q-Network Learning
  • Example 2: A robot explores a room with unknown obstacles with Q-Network
  • Building TensorFlow graph
  • Results from the Q-Network
• Policy gradient methods
  • PPO with NN policy
  • Strategy used in policy gradient methods and PPO
  • Ratio policy
  • PPO: Controlling a pole staying upright
  • Save and reload the learned model
  • Evaluate and view the trained model
  • PPO: Self-driving car
  • View samples of the racing car before training
  • Train the racing car using the CNN policy
  • Evaluate and view the learned model
• Remarks
• References

The following section is included:

• Index