Artificial Neural Networks, Machine Learning, Deep Thinking Training Course

DAY 1 - ARTIFICIAL NEURAL NETWORKS

Introduction and ANN Architecture.

• Comparison of biological and artificial neurons.
• Theoretical modeling of ANNs.
• Overview of activation functions employed in ANNs.
• Common classifications of network architectures.

Mathematical Foundations and Learning Mechanisms.

• Refresher on vector and matrix algebra.
• Understanding state-space concepts.
• Core principles of optimization.
• Approaches to error-correction learning.
• Techniques for memory-based learning.
• Overview of Hebbian learning.
• Examination of competitive learning.

Single Layer Perceptrons.

• Structural design and learning processes of perceptrons.
• Introduction to pattern classifiers and Bayes' classifiers.
• Utilizing perceptrons as pattern classifiers.
• Analyzing perceptron convergence.
• Identifying the limitations of perceptrons.

Feedforward ANNs.

• Architectural structures of Multi-layer feedforward networks.
• The Back Propagation algorithm.
• Training dynamics and convergence in Back Propagation.
• Application of Back Propagation for functional approximation.
• Practical considerations and design challenges in Back Propagation learning.

Radial Basis Function (RBF) Networks.

• Concepts of pattern separability and interpolation.
• Theory of Regularization.
• The role of Regularization in RBF networks.
• Design strategies and training methods for RBF networks.
• Approximation capabilities of RBF networks.

Competitive Learning and Self-Organizing ANNs.

• Standard procedures for general clustering.
• Learning Vector Quantization (LVQ) techniques.
• Algorithms and architectures for competitive learning.
• Self-organizing feature maps.
• Key properties of feature maps.

Fuzzy Neural Networks.

• Integration of Neuro-fuzzy systems.
• Fundamentals of fuzzy sets and logic.
• Methodologies for designing fuzzy systems.
• Development of fuzzy ANNs.

Applications

• Discussion of select Neural Network applications, highlighting their benefits and associated challenges.

DAY 2 - MACHINE LEARNING

• The PAC Learning Framework
  • Guarantees for finite hypothesis sets – consistent case
  • Guarantees for finite hypothesis sets – inconsistent case
  • General considerations
    • Comparing Deterministic vs. Stochastic scenarios
    • Understanding Bayes error noise
    • Analysis of Estimation and approximation errors
    • Strategies for Model selection
• Rademacher Complexity and VC-Dimension
• The Bias-Variance tradeoff
• Regularization techniques
• Managing Over-fitting
• Validation methods
• Support Vector Machines
• Kriging (Gaussian Process regression)
• PCA and Kernel PCA
• Self-Organizing Maps (SOM)
• Kernel induced vector space
  • Mercer Kernels and Kernel-induced similarity metrics
• Reinforcement Learning

DAY 3 - DEEP LEARNING

This section builds upon topics covered in Days 1 and 2

• Logistic and Softmax Regression
• Sparse Autoencoders
• Vectorization, PCA and Whitening
• Self-Taught Learning
• Deep Networks
• Linear Decoders
• Convolution and Pooling
• Sparse Coding
• Independent Component Analysis
• Canonical Correlation Analysis
• Demos and Applications