Information Theory

Possible Quiz 2 material of ENSC 100W

The information Function

• The information we gain in learning which out com has occurred

  • if N possible outcomes are all likely
    • ${\log }_2(N)$

      • Example

        • if there is a 1 in 10 chance
          • N=10 total information is 3.122 bits

How we communicate

• Over a distance

  • Fire

    • 0 = no fire, do nothing
    • 1 = WE ARE UNDER ATTACK SEND HELP

  • Aeneas Method

    • using water level to encode letters

  • Polybius square

    • using matrices arrays to communicate

• Rapid long range

  • optical, low-bandwidth

  • Chappe semaphore 1792

    • used by Napoleon still in use during the Crimean war

  • Electro chemical telegraph

  • Relay

  • Electric Telegraph

    • MORSE CODE :)

  • Baudot code

    • uses polybuis square
      • arrays to encode ascii letters

  • Transatlantic cable

    • wildman whitehouse
      • first cable had no relay switches . It relied on high

• Rapid mid rang

  • acoustic, medium bandwidth

• Slow

  • messenger

Digital VS Analog

• Digital was more favored because of its ease

• Analog

  • better noise to signal ration

• Digital

  • if signal is blocked/noise is introduced the system fails

Quantitation

• ”compresses” the information so it there is less information

  • like gray or dark gray
    • if you only need white and black just cap it to 2 bits and compress