Typicality of a Good Rate-Distortion Code

We consider a good code for a discrete memoryless source with a specified distortion level to be one whose rate is close to the corresponding rate-distortion function and which, with large probability, reproduces the source within the allowed distortion level. We show that any good code must contain an exponentially large set of codewords, of effectively the same rate, which are all typical with respect to the output distribution induced by the rate-distortion-achieving channel. Furthermore, the output distribution induced by a good code is asymptotically singular with respect to the i.i.d. output distribution induced by the rate-distortion-achieving channel. However, the normalized (Kullback–Leibler) divergence between these output distributions converges to the conditional entropy of the output under the rate-distortion-achieving channel.