ECE 6605 - Information Theory

Entropy

Mathematical vs operational view of entropy

What to expect in ECE 6605

• ECE 6605 is about the mathematical theory of information

  • We will talk a lot about probability
  • We will prove things formally (theorems, lemmas)
  • Exams and homework will be mathematical puzzles

• Most importantly…

  • ECE 6605 is a fun course and you will learn a lot of useful concepts
  • We will talk about applications to machine learning, communications, quantum information

• ECE 6605 will give you solid background to self-study for lots of other topics, including:

  • signal processing
  • machine learning
  • communications
  • quantum information

• Information theory was "born" 75 years ago in a landmark paper

  • A Mathematical Theory of Communication, The Bell System Technical Journal, Vol. 27, pp. 379–423, 623–656, July, October, 1948.

Logistics

• Class time and venue: Tuesday and Thursday 12:30pm:1:45pm in Clough
  • In-person live course
  • Asynchronously recorded lectures (DL and on-campus) + synchronous
  • Come to class

• Instructor: Prof. Matthieu Bloch
• Email: matthieu.bloch@ece.gatech.edu
• Office: TSRB 437 (appointments only, no walk-ins)
• Office hours: Thursdays 11am-12pm online and in person in TSRB, also recorded

• Websites
  • Canvas: announcements, homework, solutions
  • Piazza: open forum for (technical) discussion
  • Gradescope: assignment submission
  • My website: live slides

Electronic communication policy

• General guidelines
  • Email the Dean of Students if your personal situation requires special academic consideration
  • Use Piazza for technical questions
    • You can be anonymous to your peers, not to the instructors
    • You can use \(\LaTeX\) (\(\min_\beta\Vert y-X\beta\Vert_2^2\))
  • Be courteous in your electronic interactions
    • Avoid judgmental language, e.g., "The answer is obvious."
    • Try to be constructive
    • Avoid typos and use correct syntax
• If you really have to email me
  • Include "[ECE 6605]" in the subject of the email
  • I am usually reasonably fast

Writing

Grading

• Assignments (25%)
  • Due approximately every two weeks (approx. 5 assignments overall).
  • Mathematical in nature.
  • You can typeset in \(\LaTeX\), but do not waste time; handwritten and legible is fine.
  • Allocate time to submit on gradescope
• Midterm exam (25%)
  • In class on Thursday October 23, 2023 (DL students, GTPE will be in touch with you)
• Project exam (25%)
  • Report on information theory papers of your choice (see syllabus)
• Final exam (25%)
  • In person on Thursday, December 14, 2023 11:20am - 2:10pm

Assignments policy

• Two stage deadline policy

  • Soft official deadline with 2% bonus (conditions apply, read the fine print)
  • Hard deadline 48 hours after soft-deadline; no late homework accepted after hard deadline

• Abide by the Georgia Tech honor code

  • Reference all your sources
  • Do not plagiarize other sources (python code, homework solutions, etc.)
  • Do not upload course material on other websites
  • When in doubt regarding what constitutes plagiarism, ask!

• Assignments are individual but light collaboration permitted and encouraged

  • Piazza is here for that purpose
  • Small study groups are ok

Final thoughts

• I believe in accountability, integrity, and fairness
  • I will hold you to the same standards
  • I trust you be default
• Don't be shy and don't hesitate to talk to me !
  • I have little bandwidth for whining and complaining but I’m usually friendly and helpful
• I value your feedback - I use it to make improvements

• Don't obsess about grades