Advanced Machine Learning

Summary

While machine learning on structured data lays an important foundation, a larger world of analytical opportunities becomes available through understanding advanced machine learning techniques and how to handle unstructured data. We explore techniques such as support vector machines, decision trees, random forests, neural nets, clustering, KMeans, expectation maximization, time series, and signal processing. Students come away with intuition about the suitability of different techniques for different problems. In addition to handling structured data, students directly apply these techniques to large volumes of real-world unstructured data, solving problems in natural language processing using Word2Vec, bag of words, feature hashing, and topic modeling.

Associated project work

Students will use NLP techniques to extract sentiment from English text. Working with 300MB of venue reviews, they will build a series of models to predict the star rating associated with a given review. They will also examine statistically improbable phrases that appear in the text corpus.

Students will examine methods of dealing with seasonality, as they build models to predict temperatures in several cities. The training data come from National Weather Service observations and must be cleaned before use.

This module is currently part of our Data Science Fellowship.

Decision Trees and Random Forests

Decision Trees Ensemble Methods Determining feature importance

Support Vector Machines (SVM)

Maximal Margin Classifier Linear SVM Non-linear SVM Multi-class SVM Approximating kernels Support Vector Regression Outlier detection using SVM

Naive Bayes

Predictive modeling using Naive Bayes Classifying mushrooms (an example)

Natural Language Processing

Text as a "bag of words" Word importance: Term frequency–inverse document frequency (TF-IDF) Document similarity metrics Engineering your features Building the Classifier Additional NLP topics and resources

Sentiment Analysis

Bag of Words Model Interpreting the Model Grammar and Other Tools

Time Series

Trends in time series data Cross-validation for time series Modeling drift Modeling seasonality Modeling "noise" Using external data sources as features More advanced time series modeling frameworks

Outlier Detection

Motivation Concepts Scikit-learn Implementation One-class SVM Isolation Forest Case Study: Anomaly Detection in Time Series Modeling the background Detecting seasonality with Fourier Transforms Detrending z-Score Moving-Window Averages Including windowed data in model Bayesian Changepoints Online Learning References

Recommendation Engine

Problem definition and data format Feature engineering Nearest Neighbors Tag Data Dimensionality Reduction Recommendation for a User Cooperative Learning Regression of Ratings Baseline Model Overfitting and Cross-validation Modeling Interaction Surpriselib References

Unbalanced Classes

Introduction: cancer detection case study Definition and common scenarios for unbalanced data Simple techniques to deal with unbalanced data Undersampling Oversampling Synthetic data augmentation Additional approaches Train/Test split with unbalanced data Probabilities with unbalanced data The Python imbalanced-learn package

Digital Signals

Sampling Noise & Filters Audio files Filters Frequency domain

Choosing the Correct Machine Learning Algorithm

Few Features Many Features Few Observations Many Observations Underfitting Overfitting Explicability Prediction Speed Parallelization Online Learning Feature Scaling Outlier Detection/Novelty Detection Comparing ML Algorithms

Prerequisites

Intermediate to advanced statistics

Intermediate linear algebra

Basic programming

The Data Incubator

Advanced Machine Learning

Summary

Associated project work