CHAPTER1 INTRODUCTION -- Deep Learning Book Reading notes

1. AI system -> Deep learning

Background: programmable computer -> artifical intelligence

Problem: Abstract and formal problem(easy) -> Informal and intuitive problem(hard)

Projects:

Rely on knowledge from human: hard-code knowledge in formal language (unwieldy)

Gain knowledge from raw data: machine learning

Rely on representation of data

Representation learning:

Simple: Auto-encoder

Sophisticated, high-level, abstract feartures: Deep learning

Deep learning

Bulid complex concepts

how to measure the depth of the network

the number of sequential instructions (depth of computational graph)

deep probabilistic model (depth of related concepts)

A Venn diagram showing the relation of Deep learning, Representation learning, Machine learning and AI.



Flowcharts showing how AI system relate to each other.

2.Historcal Trends in Deep Learning

Three historical waves of artifical neural nets research

model	Training
Cybernetics (parallel distributed processing)	single neuron
Connectionism	neural network (only 1 or 2 hidden layers)
Deep learning	deep network

The figure of three historical waves of artifical neural nets research shows as follow:

Increasing Dataset Sizes & Model Sizes & Accuracy, Complexity and Real-Word Impact

Supervised learning

Unsupervised learning

Reinforcement learning

关于RL及其SL,UL的区别，参考此页.