Mutual Information Bounds via Adjacency Events
Han, Ordentlich, Shayevitz · cs.IT,math.IT · 2016-09-08 · 原文
The mutual information between two jointly distributed random variables X and Y is a functional of the joint distribution P_{XY}, which is sometimes difficult to handle or estimate. A coarser description of the statistical behavior of (X,Y) is given by the marginal distributions P_X, P_Y and the adjacency relation induced by the joint distribution, where x and y are adjacent if P(x,y)>0. We derive a lower bound on the mutual information in terms of these entities. The bound is obtained by viewing the channel from X to Y as a probability distribution on a set of possible actions, where an action determines the output for any possible input, and is independently drawn. We also provide an alternative proof based on convex optimization, that yields a generally tighter bound. Finally, we derive an upper bound on the mutual information in terms of adjacency events between the action and the pair (X,Y), where in this case an action a and a pair (x,y) are adjacent if y=a(x). As an example, we apply our bounds to the binary deletion channel and show that for the special case of an i.i.d. input distribution and a range of deletion probabilities, our lower and uppe
讲义
讲义·推断 依据「原文」自动生成的结构化摘要（推断），非原文表述；以原文为准。
1. 人话版
The mutual information between two jointly distributed random variables X and Y is a functional of the joint distribution P_{XY}, which is sometimes difficult to handle or estimate.
A coarser description of the statistical behavior of (X,Y) is given by the marginal distributions P_X, P_Y and the adjacency relation induced by the joint distribution, where x and y are adjacent if P(x,y)>0.
2. 领域脉络
本文类目：cs.IT、math.IT，属于其所在研究脉络的最新进展。
3. 机制拆解
We derive a lower bound on the mutual information in terms of these entities.
We also provide an alternative proof based on convex optimization, that yields a generally tighter bound.
4. 证据与数字
摘要未给出量化结果——留意原文的实验与数据。
5. 反例与边界
The bound is obtained by viewing the channel from X to Y as a probability distribution on a set of possible actions, where an action determines the output for any possible input, and is independently drawn.
input distribution and a range of deletion probabilities, our lower and uppe
6. 跨领域连接与意外收获
横跨 2 个类目（cs.IT、math.IT），关注其在你兴趣板块间的迁移面。
7. 可复用方法
把本文机制与你手头项目对照，找一个两周内能验证的最小实验。
8. 术语表
精读时把不熟的术语记入此处，作为下次回忆的锚点。