Machine Learning Foundations - week3 key point

1. When Can Machines Learn?

1.3 Types of Learning

1.3.1 Learning with Different Output Space Y

• binary classification: Y = {−1,+1}

• multiclass classification: Y = {1,2,··· ,K}

• regression: Y = R (or a subset [lower, upper] of R)

• structured learning: Y = structures

• ... and a lot more!!

1.3.2 Learning with Different Data Label yn
• supervised: all yn
• unsupervised: no yn
• semi-supervised: some yn
• reinforcement: implicit yn by goodness(˜yn )
• ... and more!!

supervised learning: every xn comes with corresponding yn

unsupervised learning: diverse, with possibly very different performance goals

semi-supervised learning: leverage unlabeled data to avoid ‘expensive’ labeling

reinforcement: learn with ‘partial/implicit information’ (often sequentially)

1.3.3 Learning with Different Protocol f ⇒ (xn ,yn)

• batch: all known data

• online: sequential (passive) data

• active: strategically-observed data

• ... and more!!

Protocol ⇔ Learning Philosophy
• batch: ‘duck feeding’
• online: ‘passive sequential’
• active: ‘question asking’ (sequentially)
  —query the yn of the chosen x

online: hypothesis ‘improves’ through receiving data instances sequentially

active: improve hypothesis with fewer labels (hopefully) by asking questions strategically

1.3.4 Learning with Different Input Space X

• concrete: sophisticated (and related) physical meaning

• raw: simple physical meaning

• abstract: no (or little) physical meaning (image pixels, spee
4000
ch signal, etc.) 

• ... and more!!

concrete features: each dimension of X ⊆ R^d represents ‘sophisticated physical meaning’  --- the ‘easy’ ones for ML

raw features: often need human or machines to convert to concrete ones  --- difficult

abstract: again need ‘feature conversion/extraction/construction’  --- more difficult