机器学习基石 - Feasibility of Learning

机器学习基石上 (Machine Learning Foundations)—Mathematical Foundations

Hsuan-Tien Lin, 林轩田，副教授 (Associate Professor)，资讯工程学系 (Computer Science and Information Engineering)

Feasibility of Learning

Learning is Impossible?

Two Controversial Answers 多种合理的方式得到不同的答案

no-free-lunch problems

Probability to the Rescue

Inferring Something Unknown → sample

样本(in-sample)的概率与整体(out-of-sample)的概率大概是接近的

Hoeffding’s Inequality

Connection to Learning

抓弹珠类比学习



抓的一把弹珠是已知数据

橙色的代表错误

抽样测试，测试集上的正确率

增加部件



公式表述



Verification

Verification of One h



The Verification Flow

Connection to Real Learning

BAD sample: EinEin and EoutEout far away (can get worse when involving choice)

BAD Data for One h: Ein(h)Ein(h) and Eout(h)Eout(h) far away

不好的几率很小



BAD Data for Many h



for MM hypotheses, bound of PD[BADD]PD[BADD]



The Statistical Learning Flow



learning possible if |H||H| finite and Ein(g)Ein(g) small