Skip to: Site menu | Main content

Course Information

Course Description:

The lecture will provide an introduction to the practical implementation of Deep Learning architectures and algorithms. We will discuss the Statistical Mechanics approach to understand the general principles which underly the success of such algorithms.
In particular, we will discuss the structure of deep neural networks and the back-propagation algorithm through minimization of a coast function. We use the MNIST dataset as an example to train different neural networks architectures using gradient descent.
Furthermore, we will study the analysis of Gibbs and online learning of a perceptron in the teacher-student configuration, calculation of quenched averages using the replica method, analysis of two-layer networks using the Committee Machine as an example, random matrix theory, bias-variance trade-off, analysis of weight matrices and the application of neural networks to solve physical problems. An introduction to the use of TensorFlow and Keras will be given.
Class Times:

Prof. Rosenow, Thursday 15:15-16:45 and Friday 11:0 -12:30, (Lecture)
  Assem Afanah, Thursday 09:15-10:45, (Tutorial)
Course Registration:

Module Registration is possible via email
einschreibung-physgeo@uni-leipzig.de

10 LP (credit points), 300 h workload
Module examination:

Written examination (180 min) with weighting 1,
Pre-requisite: 50% of the homework points
Recommended Reading:

 The introductory part of the lecture will follow the book by Michael Nielsen, Neural Networks and Deep Learning. Other parts of the lecture will be based on the book "Statistical Mechanics of Learning" by A. Engel and C. Van den Broeck, which is available in the library.
Introduction to Python :

 For beginners with Python, we recommend the introductory course Introduction to Computer-based Physical Modeling by Frank Cichos.
Exam Results:

 preliminary results
Contact:

 For any questions about the course, Please contact the tutor of the class, Assem Afanah
afanah@itp.uni-leipzig.de

Problem Sets and Lecture Notes

Problem sets will be posted online every week on Monday. Solutions must be mailed as single pdf file before Tuesday 23:59 to stp2.leipziguni@gmail.com

MNIST Data set : The handwritten digits database can be downloaded from here MNIST , to load the database use the following code MNIST loader.py

Network Python code : The Python code used to train neural network on MNIST dataset can be downloaded from here network.py .

Install Anaconda & Jupyter Notebook : quick guide to install Anaconda and introduction to Jupyter Notebook and Python Introduction to Jupyter Notebook .

Problem set 0  
Problem set 1  
Problem set 2 :  
Problem set 3  
Problem set 4  
Problem set 5  
Problem set 6  
Problem set 7  
Problem set 8  
Problem set 9  
Problem set 10  
Problem set 11    
Lecture 1 : notes   video   notebook
Lecture 2 : notes   video
Lecture 3 : notes   video
Lecture 4 : notes   video
Lecture 5 : notes   video
Lecture 6 : notes   video   notebook
Lecture 7 : notes   video   notebook
Lecture 8 : notes   video
Lecture 9 : notes   video
Lecture 10 : notes   video
Lecture 11 : notes   video
Lecture 12 : notes   video
Lecture 13 : notes   video
Lecture 14 : notes   video
Lecture 15 : notes   video
Lecture 16 : notes   video
Lecture 17 : notes   video
Lecture 18 : notes   video
Lecture 19 : notes   video
Lecture 20 : notes   video
Lecture 21 : notes   video
Lecture 22 : notes   video
Lecture 23 : notes   video
Lecture 24 : notes   video
Lecture 25 : notes   video
Lecture 26 : notes   video
Lecture 27 : notes   video
Lecture 28 : notes   video