86-375/675 (15-387) Computational Perception

Carnegie Mellon University

Fall 2016

Course Description

The perceptual capabilities of even the simplest biological organisms are far beyond what we can achieve with machines. Whether you look at sensitivity, robustness, or sheer perceptual power, perception in biology just works, and works in complex, ever changing environments, and can pick up the most subtle sensory patterns. Is it the neural hardware? Does biology solve fundamentally different problems? What can we learn from biological systems and human perception?

In this course, we will first study the biological and psychological data of biological perceptual systems in depth, and then apply computational thinking to investigate the principles and mechanisms underlying natural perception. The course will focus primarily on visual perception this year. You will learn how to reason scientifically and computationally about problems and issues in perception, how to extract the essential computational properties of those abstract ideas, and finally how to convert these into explicit mathematical models and computational algorithms. The course is targeted to neuroscience and psychology students who are interested in learning computational thinking, and computer science and engineering students who are interested in learning more about the neural basis of perception. Prerequisites: First year college calculus, some linear algebra, probability theory and programming experience are desirable.

Course Information

Instructors	Office (Office hours)	Email (Phone)
Tai Sing Lee (Professor)	Mellon Inst. Rm 115	tai@cnbc.cmu.edu (412-268-1060)
Feng Wang (TA)	Mellon Inst. Rm 115	euphoria.wang@gmail.com
Jessica Lee (TA)	main campus	jklee@andrew.cmu.edu
Yi-Ching Lee (TA)	main campus	yichingl@andrew.cmu.edu

Class location and time: PH226A Monday/Wednesday 1:30 p.m - 3:00 p.m.
Website: http://www.cnbc.cmu.edu/~tai/cp16.html (course info)
Blackboard: http://www.cmu.edu/blackboard/ (Both 375/675 students should use 375 BB for access of course materials and announcements. If instructed to do so, 675 students may find additional materials relevant to them.

Handouts in Blackboard.
Frisby and Stone Seeing: The computational approach to biological vision . MIT Press, 2010 (recommended).
Supplementary Simon Prince Computer Vision: Models, Learning, and Inference . Cambridge University Press , 2012. Downloadable at: http://www.computervisionmodels.com. More relevant to graduate students.

Classroom Etiquette

Please turn OFF your laptop, cell phones or any other electronic devices in the classroom.

Grading Scheme

Evaluation	% of Grade
Assignments	60
Midterm	10
Final Exam	30
Term project	20 (substitition for homework)
675 Term Project	Required

Grading scheme: A: > 88, B: > 75. C: > 65.

Assignments

5 major assignments that can potentially be decomposed into smaller assignments. Each assignment will involve Matlab homework as well as reading/research assignments.

Term Project

Term project is optional for undergraduate versions of the course. A student can use the OPTIONAL term project (with a max score of 20 points) to replace up to two assignments or midterm , used toward the final exam grade. No collaboration is allowed. It will require a 6 pages written final report and a presentation to the class. Matlab codes and additional output should also be submitted as supplementary materials in a different pdf/doc file and/or matlab zip files.
Term project is required of 675 students. It is anticipated to take 20-30 hours to complete. Students can work on issues of their choice that are not covered in the course, but should discuss project proposals with the faculty instructors in advance for approval.

Examinations

There will be a midterm to test understanding of the materials covered discussed in the lectures and the reading assignments.

Late Policy

Each student will have 7 days grace period for late homework. This grace period can be used for one or multiple assignments or the term paper. Use it wisely and do not ask for more. Blackboard submission after the starting of class time is considered late.

Syllabus

Date	Lecture Topic	Relevant Readings	Assignments
	SENSORY CODING
M 8/29	1. Introduction and philosophy	ch. 1, Marr
W 8/31	2. Overview: visual system	ch. 9, 10, Van Essen
W 9/07	3. Retinal processing	ch 6 Meister	Homework 1
M 9/12	4. Linear Transform	ch 5 Abbot
W 9/14	5. Representations	handout
M 9/19	6. Primary visual cortex	paper handout
W 9/21	7. Sparse coding	paper handout, Olshausen	Homework 2
	EARLY PERCEPTUAL INFERENCE
M 9/26	8. Source separation	Sejnowski, Hyvarinen
W 9/28	9. Inferring What and Where	Sompolinsky	Homework 3
M 10/3	10. Edges and contours	ch 5, Mumford
W 10/5	11. Statistics and Bayesian inference	ch 13, Geisler
M 10/10	12. Lightness and Color	ch 16, Adelson
W 10/12	13. Retinex	ch 17, Land, Morel	&Homework 4
M 10/17	14. Review/Discussion
W 10/19	15. Midterm
	SURFACE PERCEPTION
M 10/24	16. Figure-ground segregation	Ch 7	Midterm grade due
W 10/26	17. Depth and Stereo	ch 18,19	Homework 5
M 10/31	18. Texture and surfaces	ch 2
W 11/2	19. Shape from Shading	Zucker, Potetz
M 11/7	20. Structure from motion	ch 14,15
M 11/14	22. Cue combination and selection	ch 20
	OBJECT AND SCENES
W 11/16	23. Object perception	ch 8	Homework 6
M 11/21	24. Context and scenes	Torrelba, Oliva
W 11/23	Thanksgiving
M 11/28	25. Hierarchical organization	ch 10
W 11/30	26. Attention and Saliency
M 12/5	27. Predictive and constructive vision
W 12/7	28. Review		Term paper due
X 12/X	Final Examination

Questions or comments: contact Tai Sing Lee
Last modified: March 1, 2016, Tai Sing Lee