Fundamentals of Information Theory and Coding Design

ENTROPY AND INFORMATION Structure Structure in Randomness First Concepts of Probability Theory Surprise and Entropy Units of Entropy The Minimum and Maximum Values of Entropy A Useful Inequality Joint Probability Distribution Functions Conditional Probability and Bayes' Theorem Conditional Probability Distributions and Conditional Entropy Information Sources Memoryless Information Sources Markov Sources and n-Gram Models Stationary Distributions The Entropy of Markov Sources Sequences of Symbols The Adjoint Source of a Markov Source Extensions of Sources Infinite Sample Spaces INFORMATION CHANNELS What Are Information Channels? BSC and BEC Channels Mutual Information Noiseless and Deterministic Channels Cascaded Channels Additivity of Mutual Information Channel Capacity: Maximum Mutual Information Continuous Channels and Gaussian Channels Information Capacity Theorem Rate Distortion Theory SOURCE CODING Introduction Instantaneous Codes The Kraft Inequality and McMillan's Theorem Average Length and Compact Codes Shannon's Noiseless Coding Theorem Fano Coding Huffman Coding Arithmetic Coding Higher-Order Modelling DATA COMPRESSION Introduction Basic Concepts of Data Compression Run-Length Coding The CCITT Standard for Facsimile Transmission Block-sorting Compression Dictionary Coding Statistical Compression Prediction by Partial Matching Image Coding FUNDAMENTALS OF CHANNEL CODING Introduction Code Rate Decoding Rules Hamming Distance Bounds on M, Maximal Codes and Perfect Codes Error Probabilities Shannon's Fundamental Coding Theorem ERROR-CORRECTING CODES Introduction Groups Rings and Fields Linear Spaces Linear Spaces over the Binary Field Linear Codes Encoding and Decoding Codes Derived from Hadamard Matrices CYCLIC CODES Introduction Rings of Polynomials Cyclic Codes Encoding and Decoding of Cyclic Codes Encoding and Decoding Circuits for Cyclic Codes The Golay Code Hamming Codes Cyclic Redundancy Check Codes Reed-Muller Codes BURST-CORRECTING CODES Introduction Finite Fields Irreducible Polynomials Construction of Finite Fields Bursts of Errors Fire Codes Minimum Polynomials Bose-Chaudhuri-Hocquenghem Codes Other Fields Reed-Solomon Codes CONVOLUTIONAL CODES Introduction ASimple Example Binary Convolutional Codes Decoding Convolutional Codes The Viterbi Algorithm Sequential Decoding Trellis Modulation Turbo Codes INDEX Each chapter also contains a section of exercises and a section of references.