15-386/686 Neural Computation

Carnegie Mellon University

Spring 2022

Course Description

Course Information

• Letures: POS 152 (in Person). Monday/Wednesday 1:25 p.m - 2:45 p.m. It will be on zoom and recorded
• 686 Journal Club location and time: Class zoom link. Friday 1:25/2:45 p.m
• Website: http://www.cnbc.cmu.edu/~tai/nc22.html (course info)
• Canvas: Both 386/686 students should use 386 Canvas for access of course materials and announcements. 686 additional readings are provided in this webpage.

Recommended Supplementary Textbook

• Reading List (below) and on Canvas.
• Trappenberg T.P. (TTP) Fundamentals of computational neuroscience, 2nd edition, Oxford University Press 2009 (highly recommended). Matlab codes associated with the book MIT Press. 2001
• Dayan, P and Abbott, L (DP) Theoretical Neuroscience Theoretical Neuroscience MIT Press. 2001 (reference)
• Hertz J, Krogh A, Palmer RG (HKP) Introduction to the theory of neural computation., Addison Wesley 1991 (reference).

Classroom Etiquette

• During remote session, please turn on your video if possible, particularly when you are talking.
• During in-person session, refrain from doing things unrelated to our class with your laptop.

386 Grading Scheme

686 Grading Scheme

Assignments

• There will be 5 or 6 assignments. Softcopy (pdf) should be submitted before class on the due day to CANVAS for on-time submission, or the next class for late submission.
• Collaboration in team of two (for 15386 only) is allowed for the first two assignments. Each student should submit a softcopy of the homework, with their partner's name on the front page as well.

Term Project

• 386 students may do a term project or paper to replace one quiz or assignment grade, up to 90% of the maximum grade of the midterm or assignment) Only Exceptionally brilliant paper or project might receive full credit. (top 3 386 submissions). Term paper will require a 6 pages written final report, in conference paper format, with introduction, methods, results, conclusion and references.
• 686 students who wish to do term project need to submit a proposal with feasibility findings by mid-term and should start discussing with the faculty asap. The project should be written as a conference paper, in conference format and is due the last day of class, in hard and soft copy (to CANVAS). Codes and additional output should also be submitted as supplementary materials in a different pdf/doc file and zip files to CANVAS or as a github page.

Examinations

• The midterm and final exam will cover materials covered in lectures and the problem sets. Problem set solutions are available to all students one week after due day.

Late Policy

• You will have approximately 2 weeks to do each homework assignment. For the semester, you have a total of seven free-late-days allowance to submit homework late without penalty. However, for each homework, no more than four late days can be used. Each day beyond four days will entail a 20 percent discount on the homework grade you earned. However, you might submit whatever you have done by due time, and the remaining part later. In that case, the late penalty discount will only be imposed on the grade you earned of the remaining part. No homework will be accepted one week after the due day. Late by one minute or twenty-three hours behinnd the deadline will use one free day allowance.
• There will be no exception to the late policy unless an extension is granted to the whole class or you have a legitimate medical excuse with doctor's letter. You may not submit a problem set once the solution set is released, typically one week after due day.
• No extension possible for the term project.

Syllabus

Journal Club 2022

Week 3 (2/18): Retinal Computation

• Tracy Ruijin Yang, Shuhao Zhang and Lucass Gil Nadolskis will examine complex computation in the retina.
• Gollisch and Meister (2010) Eye Smarter than Scientists Believed: Neural Computations in Circuits of the Retina Neuron 65: 151-164.
• Maheswaranathan et al. (2018) Deep learning models reveal internal structure and diverse computation in the retina under natural scenes. bioRxiv preprint.
• L. McIntosh, N. Maheswaranathan, A. Nayebi, S. Ganguli, S. A. Baccus (2016) Deep learning models of the retinal resposnes to natural scenes NIPS 2016.
• N. Maheswaranathan, S. A. Baccus, S. Ganguli (2018): Inferring hidden structure in multilayered neural circuits PLOS Computational Biology.

Week 4 (2/25): Sparse Coding

• Yuwei Fei and Zikai Zhao will present the following recent papers on sparse coding.
• Chalk, Marre and Tkacik (2018) Toward a unified theory of efficient, predictive and sparse coding PNAS 115(1):186-191.
• Beyeler, Rounds, Carlson, Dutt, Krichmar (2019) PLOS Computational Biology June 27.

Supplementary Reading List foru Journal Club (more to come)

Logical computation in Neurons

• Frank Rosenblatt (1958) The Perceptron: A Probabilistic model for information storage and organization in the brain. Psychological Review 65:6. 386-408.
• Gidon et al. (2020) Dendritic action potentials and computation in human layer 2/3 cortical neurons Science, Vol. 367, Issue 6473, pp. 83-87. Jan 2020.
• McCulloch and Pitts (1943) A Logical Calculus of Ideas Immanent in Nervous Activity Bulletin of Mathematical Biophysics 5: 115-133.

Biological Neural Circuits

• Larkum M. (2013) A cellular mechanism for cortical associations: an organizing principle for the cerebral cortex Trends in Neuroscience. 36 (3): 141-151.
• Manita et al. (2015) A Top-Down Cortical Circuit for Accurate Sensory Perception Neuron, 86(5) 1304-1316.
• Gollisch and Meister (2010) Eye Smarter than Scientists Believed: Neural Computations in Circuits of the Retina Neuron 65: 151-164.
• Maheswaranathan ... Ganguli (2018) Inferring hidden structure in multilayered neural circuits PLOS Computational Biology https://doi.org/10.1371/journal.pcbi.1006291

Neural Network models of Neural Circuits

• L. McIntosh, N. Maheswaranathan, D.B. Kastner, J. Melander, L. Brezovec, A. Nayebi, J. Wang, S. Ganguli, S. Baccus, (2020) Deep learning models reveal internal structure and diverse computations in the retina under natural scenes. Under review
• N. Maheswaranathan, S. A. Baccus, S. Ganguli, (2018) Inferring hidden structure in multilayered neural circuits PLoS Comput Biol 14(8): e1006291
• L. McIntosh*, N. Maheswaranathan*, A. Nayebi, S. Ganguli, S. A. Baccus, (2016) Deep learning models of the retinal responses to natural scenes Neural Information Processing Systems (NIPS) 2016.
• Burak, Y, Ronnie, U. Meister, M. Sompolinsky H. 2010. Bayesian model of dynamic image stabilization in the visual systems. PNAS. November 9, 2010 107 (45) 19525-19530
• Burak et al. PNAS. Supplementary Information PNAS.

Sparse Coding on computation and memory

• Babadi and Sompolinsky (2014) Sparseness and Expansion in Sensory Representations. Neuron.

Reinforcement Learning

• Radulescu, Niv and Ballard (2019) Holistic Reinforcement Learning: The Role of Structure and Attention. Trends in Cognitive NSciences
• Botvinick, Niv and Baro (2008) Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective. Cognition. ...
• Niv and Langdon (2016) Current Opinion in Behavioral Sciences 2016, 11:67–73
• Yael Niv (2009) Reinforcement learning in the brain. Journal of Mathematical Psychology , 53 (3), 139–154.

Biological Plausible Deep Learning Algorithms

• Akrout et al. Deep Learning without weight transport. NeurIPS 2019
• Guerguiev et al. (2017) Towards deep learning with segregated dendrites. eLIFE, 2017.
• Bartunov et al. (2018) Assessing the scalability of bioloigically-motivated deep learning algorithms and architectures. arXiv.1807.04587.

Casual inference

• Kording, Beierholm, Ma, Quartz, Tenenbaum and Shams (2007) Causal inference in multi-sensory perception. PLOS ONE.
• Sabyasachi Shivkumar, DeAngelis and Haefner. (2019) A causal inference model for the perception of complex motion in the presence of self-motion
• Orban, Berkes, Fiser and Lengyel (2016). Neural Variability and Sampling-Based Probabilistic Representations in the Visual Cortex. Neuron.

Inverse Rational Control

• Wu, Kwon, Daptarda, Schrater, Pitkow (2020) Rational Thoughts in Neural Codes. BioRXiv (with codes)
• Daptardar S, Schrater P, Pitkow X (2019). Inverse Rational Control with Partially Observable Continuous Nonlinear Dynamics. arXiv 1908.04696. [

Spiking Bayesian Circuit

• Rao (2004) Bayesian Computaiton in Recurrent Neural Circuits. Neural Computation 16 (1)
• Deneve (2008) Bayesian spiking neurons I: Inference. Neural Computation 20(1).
• Deneve (2008) Bayesian spiking neurons II: Learning. Neural Computation 20(1).

Curiosity and Imagination

• Gruber, Gelman and Ranganath (2014) States of curiosity modulates hippocampus-dependent learning via the dopaminergic circuit. Neuron 84(2). 486-496.
• Kidd and Hayden (2015). The Psychology and Neuroscience of Curiosity. Neuron 88(3). 449-460.
• Burda et al. .. Efros (2018) Large-Scale Study of Curiosity-Driven Learning ICLR 2018
• Pathak, Agrawal, Efros and Darrell (2017) Curiosity-driven Exploration by Self-supervised Prediction. CVPR 2017

Reinforcement Learning and Song Birds

• Gadagkar et al. (2016) Dopamine neurons encode performance error in singing birds Science 354: 1278-1282.
• Gadagkar et al. (2019) Dopamine neurons change their tuning according to courtship context in singing birds. bioRxiv https://doi.org/10.1101/822817
• Fee and Goldberg (2011) A hypothesis for basal ganglia-dependent reinforcement learning in the songbird. Neuroscience 198: 152-170.
• Mackevicius and Fee (2018) Building a state space for song learning Current Opinion in Neurobiology 49: 59-68.

Emotion

• Haixu Yu, Pei Yang (2019) An Emotion-Based Approach to Reinforcement Learning Reward Design IEEE conference in networking, sensing and control
• Moerland, Broekens and Jonker (2018) Emotion in reinforcement learning agents and robots: a survey. Machine Learning volume 107, pages443–480(2018)
• Sequerira, Melo and Paiva (2011) Emotion-based Intrinsic Motivation for Reinforcement Learning Agents ACII 2011 NCS 6974, pp. 326–336.
• Picard, Vyzas and Healey (2001) Toward machine emotional intelligence: analysis of affective physiological state. IEEE PAMI Vol 23 (10). 1175-1191

Consciousness

• Bengio (2019) The Consciouness Prior. arXiv:1709:08568v2
• van Hateren (2019) A theory of consciouness: computation, algorithm and neurbioligcal realization.
• Mumford (2019) Can an artificial intelligence machine be conscious?
• Blum (2018). Can machine be conscious? Toward a Conscious AI -- the Consciouness Turing Machine. Simon Institute Lecture (youtube)
• Crick and Koch (2003). A framework for consciouness Nature Neuroscience 6, 119-126.
• Koch (2018). What is consciouness? Nature 557 (S9): May 2018.
• Quantum approach to consciouness Also check https://en.wikipedia.org/wiki/Quantum_mind

Glia and their functions

• Fields et al. (2014) Glia Biology in Learning and Cognition. The Neuroscientist. Vol 20(5) 426-431
• Mu et al. (2019) Glia Accumulate Evidence that Actions Are Futile and Suppress Unsuccessful Behavior. Cell 178, 27–43
• Koob (2009) The root of thought: what do glial cells do? Scientific American Oct 2019.
• Fink (2016) Beyond the neuron: Emerging roles of glial cells in neuroscience. A PLOS student BLoGS that contains nice references to ideas about glia cells.
• Sajedinia and Helie (2018) A New Computational Model for Astrocytes and Their Role in Biologically Realistic Neural Networks.
• Manninen et al. (2018) Computational Models for Calcium-Mediated Astrocyte Functions. Front. Comput. Neurosci., 04 April 2018 | https://doi.org/10.3389/fncom.2018.00014.