Machine Learning solutions on Amazon Web Services. Including Amazon EC2 Deep Learning AMI and frameworks, Amazon SageMaker and AI Services. All AWS ML material is at https://ml.aws .
- Get started on machine learning training with content built by AWS experts https://aws.amazon.com/training/learn-about/machine-learning/
- CRISP-DM: Cross Industry Standard Process - Data Mining https://www.the-modeling-agency.com/crisp-dm.pdf includes six phases: business understanding, data understanding, data preparation, modeling, evaluation, and deployment.
- Well-Architected Framework for Machine Learning https://d1.awsstatic.com/whitepapers/architecture/wellarchitected-Machine-Learning-Lens.pdf
Background material on Machine Learning
- Artificial neural network (ANN)
- Multilayer perceptron (MLP)
- Hidden Markov model (HMM)
- Feedforward neural network
- Support-vector machine
- Recurrent neural network (RNN)
- Long short-term memory (LSTM)
- Stochastic gradient descent (SGD)
- Convolutional neural network (CNN)
- Recurrent Neural Networks Tutorial http://www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-1-introduction-to-rnns/ from WILDML machine learning blog.
- Convolutional Neural Networks for Visual Recognition. https://www.youtube.com/watch?v=NfnWJUyUJYU CS231n: Andrej Karpathy. Stanford Winter Quarter 2016.
- Convolutional Neural Networks for Visual Recognition. http://cs231n.github.io/ notes to accompany the Stanford CS class.
- Gradient-based Learning applied to Document Recognition http://yann.lecun.com/exdb/publis/pdf/lecun-98.pdf by Yann LeCun & others, November 1998
- Machine Learning. https://www.coursera.org/learn/machine-learning Andrew Ng. Stanford/Coursera. 2017.
- An Introduction to Distributed Deep Learning https://seba-1511.github.io/dist_blog/ by Seb Arnold, December 15, 2016
- Understanding LSTM Networks http://colah.github.io/posts/2015-08-Understanding-LSTMs/ - Colah's blog.
- Geoffrey Hinton: The Foundations of Deep Learning https://www.youtube.com/watch?v=zl99IZvW7rE Godfather of artificial intelligence Geoffrey Hinton gives an overview of the foundations of deep learning.
- Start Here with Machine Learning https://machinelearningmastery.com/start-here/
- Interpretable Machine Learning https://christophm.github.io/interpretable-ml-book/
- Fast.AI https://www.fast.ai/ Making neural nets uncool again.
- Deep Learning for Developers: An Introduction https://www.youtube.com/watch?v=4SJBcZQX8uA AWS re:Invent 2018
- Dive Into Deep Learning https://d2l.ai/ An interactive deep learning book with code, math, and discussions.
- Foundations of Machine Learning https://bloomberg.github.io/foml/ by Bloomberg.
- Machine Learning tips and tricks cheatsheet https://stanford.edu/~shervine/teaching/cs-229/cheatsheet-machine-learning-tips-and-tricks By Afshine Amidi and Shervine Amidi of Stanford University.
User Interfaces and Jupyter Notebooks
- Jupyter/IPython Notebook Quick Start Guide http://jupyter-notebook-beginner-guide.readthedocs.io/en/latest/what_is_jupyter.html Notebook documentation.
- Built-in Magic commands http://ipython.readthedocs.io/en/stable/interactive/magics.html for IPython.
- Python Pandas Tutorial https://www.learndatasci.com/tutorials/python-pandas-tutorial-complete-introduction-for-beginners/
- Build Amazon SageMaker notebooks backed by Spark in Amazon EMR https://aws.amazon.com/blogs/machine-learning/build-amazon-sagemaker-notebooks-backed-by-spark-in-amazon-emr/
Additional Algorithms and Tools
-
XGBoost. Maintained by the distributed machine learning council (DMLC)
https://xgboost.ai - Protobuf defined: https://en.wikipedia.org/wiki/Protocol_Buffers
- Frameworks you can use with SageMaker https://sagemaker.readthedocs.io/en/stable/frameworks/index.html
Frameworks in AWS Machine Learning AMIs Reference MXNet https://aws.amazon.com/mxnet/ TensorFlow https://www.tensorflow.org/ Caffe2 https://www.caffe2.ai/ Torch https://github.com/torch Keras https://keras.io/ Theano http://deeplearning.net/software/theano/ Caffe http://caffe.berkeleyvision.org/ Microsoft Cognitive Toolkit (CNTK) https://www.microsoft.com/en-us/cognitive-toolkit/ MXNet, Pytorch, Tensorflow, Programming Styles
- 60-minute MXNet and Gluon crash course on Youtube https://www.youtube.com/watch?v=r4-Ynxw0X5w . The course content website is https://gluon-crash-course.mxnet.io/
- Deep Learning Programming Paradigm https://mxnet.apache.org/api/architecture/program_model
- NDArray Tutorial from Distributed (Deep) Machine Learning Community (github) https://github.com/dmlc/mxnet-notebooks/blob/89939c617191828b8172836341780ab7be8ceb7b/python/basic/ndarray.ipynb
- The N-dimensional array from SciPy.org https://docs.scipy.org/doc/numpy/reference/arrays.ndarray.html For more information on Python NumPy.
- MXNet - flexible and efficient library for deep learning https://mxnet.apache.org/
- About Gluon https://gluon.mxnet.io
- GluonCV: a Deep Learning Toolkit for Computer Vision https://gluon-cv.mxnet.io/
- GluonNLP: Natural Lanaguage Processing https://gluon-nlp.mxnet.io/
- GluonTS: Time Series Modelling https://gluon-ts.mxnet.io/
- DASK https://docs.dask.org is a flexible library for parallel computing in Python consisting of (1) Dynamic task scheduling similar to Airflow, Luigi, Celery, or Make, but optimized for interactive computational workloads. (2) "Big Data" collections like parallel arrays, dataframes, and lists that extend common interfaces like NumPy, Pandas, or Python iterators to larger-than-memory or distributed environments.
Amazon SageMaker
- An overview of Amazon SageMaker https://www.youtube.com/watch?v=ym7NEYEx9x4
- Amazon SageMaker Algorithms https://www.youtube.com/watch?v=VT4tM0-7L80 AWS re:Invent 2017: Infinitely Scalable Machine Learning Algorithms
- Amazon SageMaker built-in algorithms https://docs.aws.amazon.com/sagemaker/latest/dg/algos.html
- Algorithms Provided by Amazon SageMaker: Common Parameters https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-algo-docker-registry-paths.html
- Using Your Own Training Algorithms https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-training-algo.html
- Reinforcement Learning with Amazon SageMaker RL https://docs.aws.amazon.com/sagemaker/latest/dg/reinforcement-learning.html
- XGBoost Hyperparameters https://docs.aws.amazon.com/sagemaker/latest/dg/xgboost_hyperparameters.html
- Amazon SageMaker Examples https://github.com/awslabs/amazon-sagemaker-examples repository contains example notebooks that show how to apply machine learning and deep learning in Amazon SageMaker.
- Estimators https://sagemaker.readthedocs.io/en/latest/estimators.html
- Amazon SageMaker Python SDK https://sagemaker.readthedocs.io/en/stable/
- Amazon SageMaker Pipelines https://aws.amazon.com/sagemaker/pipelines/ Purpose-built CI/CD service for machine learning
- Amazon SageMaker Feature Store https://aws.amazon.com/sagemaker/feature-store/ for machine learning
- Amazon SageMaker Edge Manager https://aws.amazon.com/about-aws/whats-new/2020/12/aws-introduces-amazon-sagemaker-edge-manager-model-management-edge-devices/ for managing Edge Devices
- Amazon SageMaker Debugger https://aws.amazon.com/sagemaker/debugger/
- Amazon SageMaker Clarify https://aws.amazon.com/sagemaker/clarify/ Bias Detection and Explainability
- Amazon SageMaker Workshop https://sagemaker-workshop.com/
Algorithms
- Algorithms Provided by Amazon SageMaker: Common Parameters https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-algo-docker-registry-paths.html
- Using Your Own Training Algorithms https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-training-algo.html
- CIFAR-10 Demo https://cs.stanford.edu/people/karpathy/convnetjs/demo/cifar10.html trains a Convolutional Neural Network on the CIFAR-10 dataset in your browser, with nothing but Javascript.
- Tinker With a Neural Network http://playground.tensorflow.org/ in Your Browser.
- 2D Visualization of a Convolutional Neural Network http://scs.ryerson.ca/~aharley/vis/conv/flat.html
- Interpretable Machine Learning https://christophm.github.io/interpretable-ml-book/ A Guide for Making Black Box Models Explainable.
- Evaluating ML Models https://docs.aws.amazon.com/machine-learning/latest/dg/evaluating_models.html from the Amazon Machine Learning Developer Guide.
- Introduction to Transformers and BERT on Amazon SageMaker https://www.youtube.com/watch?v=D9Qo5OpG4p8
Reinforcement Learning
- AWS DeepRacer - Highlights from AWS re:Invent 2018 https://www.youtube.com/watch?v=jD0Z9D9Y0Sc
- Developers, start your engines https://aws.amazon.com/deepracer/
Test Data Sets and Resources for Deep Learning
- About ImageNet https://qz.com/1034972/the-data-that-changed-the-direction-of-ai-research-and-possibly-the-world/
- The MNIST database of handwritten digits http://yann.lecun.com/exdb/mnist/ has a training set of 60,000 examples, and a test set of 10,000 examples. It is a subset of a larger set available from NIST. The digits have been size-normalized and centered in a fixed-size image. It is a good database for people who want to try learning techniques and pattern recognition methods on real-world data while spending minimal efforts on preprocessing and formatting.
- CIFAR-10 and CIFAR-100 are labeled subsets of the 80 million tiny images dataset https://www.cs.toronto.edu/~kriz/cifar.html . collected by Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton.
- Open Data on AWS https://aws.amazon.com/opendata/ and Registry of Open Data https://registry.opendata.aws/
- Kaggle Datasets https://www.kaggle.com/datasets
- Box Plot: or graphically depicting groups of numerical data https://en.wikipedia.org/wiki/Box_plot
- SMOTE = Synthetic Minority Over-sampling Technique https://en.wikipedia.org/wiki/Oversampling_and_undersampling_in_data_analysis Oversampling and undersampling in data analysis
- AWSLabs Deep Learning Benchmark (github) https://github.com/awslabs/deeplearning-benchmark
- AWSlabs reference Lambda function that predicts image labels for a image using an MXNet-built deep learning model https://github.com/awslabs/mxnet-lambda The repo also has pre-built MXNet, OpenCV libraries for use with AWS Lambda.
- Open Neural Network Exchange (ONNX) https://onnx.ai/ enables AI developers to more easily move models between state-of-the-art tools and choose the combination that is best for them.
Feature Engineering and Tuning
- Pandas Get_dummies https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.get_dummies.html
- Compare the effect of different scalers on data with outliers https://scikit-learn.org/stable/auto_examples/preprocessing/plot_all_scaling.html
- Best Practices for Hyperparameter Tuning https://docs.aws.amazon.com/sagemaker/latest/dg/automatic-model-tuning-considerations.html
- Why Is Random Search Better Than Grid Search For Machine Learning? https://analyticsindiamag.com/why-is-random-search-better-than-grid-search-for-machine-learning/
- Perform Automatic Model Tuning with Amazon SageMaker https://docs.aws.amazon.com/sagemaker/latest/dg/automatic-model-tuning.html
AWS Services for Machine Learning and Deep Learning
- Amazon Elastic Inference https://aws.amazon.com/machine-learning/elastic-inference/ allows you to attach low-cost GPU-powered acceleration to Amazon EC2 and Amazon Sagemaker instances or Amazon ECS tasks.
- AWS Inferentia https://aws.amazon.com/machine-learning/inferentia/ is a machine learning inference chip, custom designed by AWS to deliver high throughput, low latency inference performance at an extremely low cost.
- Amazon Forecast https://aws.amazon.com/forecast/ is a fully managed service that uses machine learning to deliver highly accurate forecasts, based on the same technology used at Amazon.com. No machine learning experience required
- Amazon Textract https://aws.amazon.com/textract/ is a service that automatically extracts text and data from scanned documents. Amazon Textract goes beyond simple optical character recognition (OCR) to also identify the contents of fields in forms and information stored in tables
- Amazon SageMaker Neo https://aws.amazon.com/sagemaker/neo/ enables developers to train machine learning models once and run them anywhere in the cloud and at the edge. It optimizes models to run up to twice as fast, with less than a tenth of the memory footprint, with no loss in accuracy.
- Amazon SageMaker Ground Truth https://aws.amazon.com/blogs/aws/amazon-sagemaker-ground-truth-build-highly-accurate-datasets-and-reduce-labeling-costs-by-up-to-70/ Build Highly Accurate Datasets and Reduce Labeling Costs by up to 70%
- Amazon Comprehend Medical https://aws.amazon.com/comprehend/medical/ is a natural language processing service that makes it easy to use machine learning to extract relevant medical information from unstructured text.
- Amazon Personalize https://aws.amazon.com/blogs/aws/amazon-personalize-real-time-personalization-and-recommendation-for-everyone/ Real-Time Personalization and Recommendation for Everyone
- Announcing Amazon Forecast Weather Index https://aws.amazon.com/about-aws/whats-new/2020/12/announcing-amazon-forecast-weather-index-include-local-weather-increase-forecasting-model-accuracy/
- Announcing Amazon Lookout for Metrics https://aws.amazon.com/about-aws/whats-new/2020/12/announcing-amazon-lookout-for-metrics/
Deploying Machine Learning Models
- Deploying models with SageMaker https://docs.aws.amazon.com/sagemaker/latest/dg/deploy-model.html
- Deploying a Trained Model for Prediction on AWS Lambda http://model-hosting.s3-website-eu-west-1.amazonaws.com The model is stored in Amazon S3 and will be downloaded to the lambda function for predictions.
- Dynamic training with Apache MXNet https://github.com/awslabs/dynamic-training-with-apache-mxnet-on-aws reduces cost and time for training deep neural networks by leveraging AWS cloud elasticity and scale.
- boto3 documentation of AutoScaling https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/application-autoscaling.html#ApplicationAutoScaling.Client.put_scaling_policy
- Load test and optimize an Amazon SageMaker endpoint using automatic scaling https://aws.amazon.com/blogs/machine-learning/load-test-and-optimize-an-amazon-sagemaker-endpoint-using-automatic-scaling/
- Amazon SageMaker Model Monitor https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor.html
- Amazon SageMaker Serverless Inference https://docs.aws.amazon.com/sagemaker/latest/dg/serverless-endpoints.html
TECHNICAL BLOGS and PRESENTATIONS
- Use the Amazon SageMaker local mode to train on your notebook instance https://aws.amazon.com/blogs/machine-learning/use-the-amazon-sagemaker-local-mode-to-train-on-your-notebook-instance/ allows you to switch seamlessly between local and distributed, managed training by simply changing one line of code.
- Spatial Localization and Detection http://cs231n.stanford.edu/slides/2016/winter1516_lecture8.pdf lecture at Stanford University.
- Create a model for predicting orthopedic pathology https://aws.amazon.com/blogs/machine-learning/create-a-model-for-predicting-orthopedic-pathology-using-amazon-sagemaker/ using Amazon SageMaker
- Deep Learning for Developers https://www.youtube.com/watch?v=ThWdx4h_8aE (AWS re:Invent 2018)
- Operationalizing Machine Learning to Deliver Content at Scale https://www.youtube.com/watch?v=mDbH6H1VGSI (AWS re:Invent 2018)
- Youtube channel for Computerphile https://www.youtube.com/results?search_query=Computerphile+Convolution
- Learning Paths: Training and Certification https://aws.amazon.com/training/learning-paths/machine-learning/
- Now easily perform incremental learning on Amazon SageMaker https://aws.amazon.com/blogs/machine-learning/now-easily-perform-incremental-learning-on-amazon-sagemaker/
- Save on inference costs by using Amazon SageMaker multi-model endpoints https://aws.amazon.com/blogs/machine-learning/save-on-inference-costs-by-using-amazon-sagemaker-multi-model-endpoints/
- Easy ML PoCs https://www.easymlpocs.com/ - find building blocks to help create machine learning PoCs
CUSTOMER REFERENCES
- Cats, Rats, A.I., Oh My! https://www.youtube.com/watch?v=1A-Nf3QIJjM - a presentation by Ben Hamm at Ignite Seattle about how he taught myself soldering, coding, and machine learning, all to thwart his sweet murderous cat.
- Zillow Zestimate https://d0.awsstatic.com/case-studies/PDF%20Case%20Studies/AWS-Casestudy_Zillow.pdf uses wide variety of public-record data, including tax assessments, sales transactions, images of homes, MLS listing data, and other information provided by homeowners.
- DXC Technology automates triage of support tickets https://aws.amazon.com/blogs/machine-learning/dxc-technology-automates-triage-of-support-tickets-using-aws-machine-learning/ using AWS machine learning
- Majority of Alexa Now Running on Faster, More Cost-Effective Amazon EC2 Inf1 Instances https://aws.amazon.com/blogs/aws/majority-of-alexa-now-running-on-faster-more-cost-effective-amazon-ec2-inf1-instances/
Webpage constructed by Dennis Adams
Sr AWS Technical Instructor
