Pre-Conference Workshop đź“š

Mathematical Engineering of Deep Learning - Foundations

A full day pre-conference workshop will be held on 5 December 2023 at the Wallumattagal Campus.

You can sign up to the workshop as part of the IASC-ARS 2023 registration via: https://www.eventbrite.com.au/e/iasc-ars-2023-tickets-712519403717?aff=oddtdtcreator.

Summary

In this workshop, we first present the general feed-forward deep neural network. After exploring the expressive power of deep neural networks, we dive into the details of training by understanding the back-propagation algorithm for gradient evaluation and exploring other practical aspects such as weight initialisation, dropout, and batch normalization. The second part of the workshop concentrates on convolutional neural networks. Much of the success of deep learning is due to the strength of convolutional neural networks when applied to images and similar data formats. The concepts of channels and filter design are introduced, followed by an exploration of the common state of the art architectures that have made significant impacts and are still in use today. The last part of this workshop is about sequence models. These models are critical for data such as text with applications in natural language processing. We explore recurrent neural networks and their generalizations. These include long short-term memory models, gated recurrent units, auto-encoders for end-to-end language translation, and the attention model with transformers. The workshop includes deep learning demonstrations using R and Python software.

Prerequisite

The focus of this short course is on the basic mathematical description of deep learning models, algorithms and methods. A mathematically equipped participant can quickly grasp the essence of modern deep learning algorithms, models, and techniques. Deep learning is easily described through the language of mathematics at a level accessible to many professionals. Attendees armed with basic familiarity of mathematical notation and knowledge of basic calculus, probability, and linear algebra can go a long way in understanding deep learning quickly. Attendees from the fields of engineering, signal processing, statistics, epidemiology, biostatistics, physics, pure mathematics, econometrics, operations research, quantitative management, applied machine learning, or applied deep learning will quickly gain insights into the key mathematical engineering components of the field.

Learning objectives

  • Understand the fundamentals of feed-forward deep neural networks and their expressive power.
  • Demonstrate knowledge of the back-propagation algorithm and its role in training deep neural networks.
  • Explain the practical aspects of weight initialization, batch normalization, and dropout in deep neural networks.
  • Comprehend the concepts of convolutional neural networks (CNNs) and their significance in image and data analysis.
  • Apply the convolution operation to build a convolutional layer in a CNN.
  • Identify and describe state-of-the-art architectures in convolutional neural networks, such as Inception and ResNets.
  • Recognize the importance of sequence models in natural language processing and other related applications.
  • Explore recurrent neural networks (RNNs) and their variations, including long short-term memory (LSTM) models and gated recurrent units (GRUs).
  • Understand the encoder-decoder architecture for end-to-end language translation using RNNs.
  • Explain the attention model and its implementation with transformers in sequence models. Utilize R and Python software to demonstrate practical applications of deep learning concepts.

Presenter: Prof Benoit Liquet

Photo of Prof Benoit Liquet

Prof Benoit Liquet

Dr Liquet is Professor of Mathematical and Computational Statistics at Macquarie University in the School of Mathematical & Physical Sciences. In addition he is affiliated to the University of Queensland and to the Université de Pau et Pays de l’Adour (UPPA). He was previously affiliated with ACEMS (Centre of Excellence for Mathematical and Statistical Frontiers), Queensland University of Technology. Throughout his career he has extensively worked in developing novel statistical models mainly to provide novel tools to analyse clinical, health and biological data arising from epidemiological studies. Since 2011, he moved to the field of computational biology and generalised some of these methods so that they scale to high throughput (“omic”) data. He has been teaching an advanced course on the mathematical engineering of Deep Learning at the Australian Mathematical Sciences Institute (AMSI) summer school in 2021. A book draft of his new co-authored book on concepts of “Deep Learning” is available at https://deeplearningmath.org .

He recently ran two workshop for the SSA (Statistical Society Of Australia):

  1. A crash course on using machine learning methods effectively in practice( https://www.statsoc.org.au/event-5028135)
  2. Mathematical Engineering of Deep Learning - Part One Foundations (https://www.statsoc.org.au/event-5183897)

Benoit Liquet works on Applied Statistics and Biostatistics, as well as on the development of R packages (such as Machine Learning).