Reinforcement Learning Algorithms Implementation: From Zero To Hero

What is Reinforcement Learning about?

In contrast to supervised learning where machines learn from examples that include the correct decision and unsupervised learning where machines discover patterns in the data, reinforcement learning allows machines to learn from partial, implicit and delayed feedback. This is particularly useful in sequential decision making tasks where a machine repeatedly interacts with the environment or users. Applications of reinforcement learning include robotic control, autonomous vehicles, game playing, conversational agents, assistive technologies, computational finance, operations research, etc..

Disclaimer!

This repository mainly contains my assignments for this Reinforcement Learning course, which was offered in Fall 2021 at UWaterloo by Professor Pascal Poupart. Because of the academic integrity, I don't have the permission to post this repository publicly online; therefore, this repository is only accessible upon explicit request to me as defined in this document.

Download From Github With Explanations [PRIVATE REPO, ONLY ACCESSIBLE BY EXPLICIT REQUEST]

Part 1

Summary:

• Markov Decision Process [from scratch in Python]
  
  • value iteration
  • policy iteration
  • modified policy iteration
• Maze problem to test above algorithms
• Compare the performance of each algorithm
• Q-Learning [from scratch in Python]
• Use matplotlib to compare the effect of the Q-Learning parameters on the cumulative discounted rewards per episode
• deep Q-network to solve the CartPole problem from Open AI Gym
  • Using Agents library from TensorFlow
• Use matplotlib to compare the effect of the deep Q-network parameters on the average cumulative discounted rewards [also averaged across several runs to reduce stochasity]
• More details: https://cs.uwaterloo.ca/~ppoupart/teaching/cs885-fall21/assignments.html assignment 1 section

Part 2

Summary:

• Bandit algorithms from scratch in Python
  
  • epsilon-greedy
  • Thompson sampling
  • UCB
• REINFORCE algorithm from scratch in Python
• model-based RL algorithm from scratch in Python
  
• Soft Q-Learning in Pytorch
• Soft Actor Critic in Pytorch
• Discussion over the properties of each algorithms and their effect over the performance
• More details: https://cs.uwaterloo.ca/~ppoupart/teaching/cs885-fall21/assignments.html assignment 2 section

Part 3

• Partially Observable RL
  • Deep Recurrent Q learning (DRQN) algorithm in Pytorch
    • Using LSTM and MLP
    • Compare to Deep Q Network's performance
• Generative Adversarial Imitation Learning (GAIL) algorithm in Pytorch
  • Using deterministic policy gradient update technique
  • Compare to Behavior Cloning's (BC) performance
• Categorical (C51) distributional RL algorithm
  • Compare to DQN on the Cartpole domain with epsilon greedy exploration
• More details: https://cs.uwaterloo.ca/~ppoupart/teaching/cs885-fall21/assignments.html assignment 3 section

Download From Github With Explanations [PRIVATE REPO, ONLY ACCESSIBLE BY EXPLICIT REQUEST]

From dirty data to knowledge with the application of AI: Full pipeline

What is this repo about?

There is a dire need for effective methods to model and analyze the data and extract useful knowledge from it and to know how to act on it. In this series of notebooks you will learn the fundamental tools for assessing, preparing and analyzing data. You will learn to design a data and analysis pipeline to move from raw data to task solution. You will learn to implement a variety of analytical and machine learning algorithms to including supervised, unsupervised and other learning approaches.

Download From Github With Explanations

Part 1

Summary:

• Load and work with two famous datasets "Iris" and "Heart Disease"
• Data cleaning approaches: filling missing values, noise reduction, normalization, and visualization
• Visualization for understanding data: pair plots, scatter plots, correlation and data distribution analysis
• Statistical analysis on data: correlation coefficient, statistical variables
• KNN classifier with Sckit-learn: parameter tuning with cross validation, metrics analysis, plot analysis, AUC method analysis
• Further tuning KNN classifier: weighted KNN approaches, algorithm selection, speed, etc.

Part 2

Summary:

• Two datasets: John Hopkins University CSSE COVID-19 (https://github.com/CSSEGISandData/COVID-19/tree/master/csse_ covid_19_data), US 2020 Census
• Preprocessing data: data cleaning, outlier dealing, normalization, missing value, etc.
• Representation Learning: PCA, LDA, scree-plot and statistcial analysis, visualization insights, comparing the algorithms
• Data analysis for classification: original, hybrid, or LDA/PCA constructed data
• Tree based algorithms for classification with extensive analysis: Decision trees, Random forrest, parameter tuning, group k-fold cross validation, Gradient Tree Boosting
• Naive bayes classifier (NB): var smoothing analysis
• Comparing the performance of NB compared to the decision tree approaches

Part 3

• Preprocessing data (outlier removal, feature selection, normalization, train-test split, creating 3 different training sets for the 3 targets, etc.)
• Deep neural network: MLP, model and architecture analysis, tuning the hyperparameters, class weights
• LSTM networks: model optimization, L2 regularization, activation functions, dropout, batch normalization
• Deep MLP vs LSTM: thorough analysis (time, accuracy, number of parameters, etc.)
• Convolutional neural network: Parameter and architecture tuning, padding, activation, classificaiton layer
• ResNet CNN model: thorough analysis and comparison to the previous CNN model (time, depth, number of parameters)

Download From Github With Explanations

These codes were written by Navid Malekghaini and Soheil Johari.

An Introduction to Encrypted Traffic Classification with Application of ML

In this video you will learn about "Encrypted Traffic Classification with Application of Machine Learning" in less than 10 minutes!

----
Update Jan 2, 2022:
You may access this video on YouTube with the link below:
https://lnkd.in/eFs78rSs
----

You may download the slides from here.