Elements of Information Theory

Elements of Information Theory is an up-to-date introduction to the field of information theory and its applications to communication theory, statistics, computer science, probability theory, and the theory of investment. Covering all the essential topics in information theory, this comprehensive work provides an accessible introduction to the field that blends theory and applications. In step-by-step detail, the authors introduce the basic quantities of entropy, relative entropy, and mutual information and show how they arise as natural answers to questions of data compression, channel capacity, rate distortion, hypothesis testing, information flow in networks, and gambling. In addition, Elements of Information Theory investigates a number of ideas never before covered at this level in a textbook, including:The relationship of the second law of thermodynamics to Markov chainsCompetitive optimality of Huffman codesThe duality of data compression and gamblingLempel Ziv codingKolmogorov complexityPortfolio theoryInequalities in information theory and their consequences in mathematicsComplete with numerous illustrations, original problems and historical notes, Elements of Information Theory is a practical reference for communications professionals and statisticians specializing in information theory. It also serves as an excellent introductory text for senior and graduate students taking courses in telecommunications, electrical engineering, statistics, computer science, and economics.Cover, Thomas M. is the author of 'Elements of Information Theory' with ISBN 9780471062592 and ISBN 0471062596.