Statistical Decision Theory and Bayesian Analysis 笔记

Statistical Decision Theory and Bayesian Analysis 笔记

目录

Statistical Decision Theory and Bayesian Analysis 笔记

目录

一些概念不太理解
Bayesian expected loss

Frequentist Risk

关于Decision principles
The conditional Bayes decision principle Minimize Bayesian expected loss

Frequentist Decision principles
Bayes Risk Principle minimizes Bayes Expected Risk

minimax principle针对randomized decision rule 暂不讨论

the invariance principle 暂不讨论

一些概念不太理解

对于很多概念的区别不太清晰，如：

Decision rule vs action

Bayesian expected loss vs Bayes risk vs frequentist risk vs risk function of a decision rule

Risk vs Loss

Bayes action vs Bayes rule

action vs Bayes action

Rule vs Bayes rule

Bayesian expected loss

Note: 这里为何叫Bayesian？

Frequentist Risk

Decision rule: A function from X(data) to A(action). For no-data problem, a decision rule is simply an action.



Risk function是Loss function在所有数据上的积分？



Bayes Risk是risk function在参数分布上的期望？

关于Decision principles

The conditional Bayes decision principle: Minimize Bayesian expected loss

Frequentist Decision principles

Bayes Risk Principle: minimizes Bayes Expected Risk.

minimax principle:针对randomized decision rule （暂不讨论）

the invariance principle （暂不讨论）

Questions:

为何这样分类，后面三个划分到Frequentist

针对到具体的机器学习算法，如逻辑回归等，这些原则是怎么体现的？

在书《Bayesian Reasoning and Machine Learning》中第13章介绍有监督学习的两种方法：Empirical risk minimisation和Bayesian decision theory，与这几个principles有什么联系和区别？