discrete source

discrete source A source of information whose output has an alphabet of distinct letters or, in the case of a physical source, whose output is a signal that is discrete in time and amplitude (see discrete and continuous systems). The size of the alphabet, or the number of amplitude levels, is usually finite, although for mathematical analysis it may conveniently be regarded as potentially infinite.

The discrete memoryless source (DMS) has the property that its output at a certain time does not depend on its output at any earlier time.

The discrete source with memory (DSM) has the property that its output at a certain time may depend on its outputs at a number of earlier times: if this number is finite, the source is said to be of finite order, otherwise it is of infinite order. DSMs are usually modeled by means of Markov chains; they are then called Markov sources.

An ergodic source has the property that its output at any time has the same statistical properties as its output at any other time. Memoryless sources are, trivially, always ergodic; a source with memory is ergodic only if it is modeled by an ergodic Markov chain.

See also information theory, Shannon's model, source coding theorem.