15-386/686 Neural Computation

Week 1 (1/24): Logical computation in Neurons

• Songwei Ge will present McCulloch and Pitt and the new Science paper showing how a single neuron can perform XOR operation. AJ Parikh will present the first paper on neural network by Frank Rosenblatt.
• Frank Rosenblatt (1958) The Perceptron: A Probabilistic model for information storage and organization in the brain. Psychological Review 65:6. 386-408.
• Gidon et al. (2020) Dendritic action potentials and computation in human layer 2/3 cortical neurons Science, Vol. 367, Issue 6473, pp. 83-87. Jan 2020.
• McCulloch and Pitts (1943) A Logical Calculus of Ideas Immanent in Nervous Activity Bulletin of Mathematical Biophysics 5: 115-133.

Week 2 (1/31): Advances in Neural Circuits

• Peijia Yu will explore further on Larkum's works on the functional roles of apical dendrites and dendritic computations. Shili Wang will examin some of the detailed circuits that are known in the retina. In considering these recent advances in neural circuits, it will be worthwhile to think what elements are in the current neural network (deep learning) and what are still missing, and whether adding these new elements would make a difference.
• Larkum M. (2013) A cellular mechanism for cortical associations: an organizing principle for the cerebral cortex Trends in Neuroscience. 36 (3): 141-151.
• Manita et al. (2015) A Top-Down Cortical Circuit for Accurate Sensory Perception Neuron, 86(5) 1304-1316.
• Gollisch and Meister (2010) Eye Smarter than Scientists Believed: Neural Computations in Circuits of the Retina Neuron 65: 151-164.
• Maheswaranathan ... Ganguli (2018) Inferring hidden structure in multilayered neural circuits PLOS Computational Biology https://doi.org/10.1371/journal.pcbi.1006291

Week 3 (2/8): Computational models of Neural Circuits

• Ideas in Machine learning techniques such as Deep learning and Bayesian belief propagation have been used to explore the computations in the neural circuits. Matthew Wolff will present the work from Surya Ganguli's lab at Stanford on the use of deep learning to explore retinal neural circuits. Tianjin will present the work of Han Sompolinsky's work on how the retina might solve the chicken and egg problem. Could deep learning be uesful for solving Sompolinsky's problem?
• L. McIntosh, N. Maheswaranathan, D.B. Kastner, J. Melander, L. Brezovec, A. Nayebi, J. Wang, S. Ganguli, S. Baccus, (2020) Deep learning models reveal internal structure and diverse computations in the retina under natural scenes. Under review
• N. Maheswaranathan, S. A. Baccus, S. Ganguli, (2018) Inferring hidden structure in multilayered neural circuits PLoS Comput Biol 14(8): e1006291
• L. McIntosh*, N. Maheswaranathan*, A. Nayebi, S. Ganguli, S. A. Baccus, (2016) Deep learning models of the retinal responses to natural scenes Neural Information Processing Systems (NIPS) 2016.
• Burak, Y, Ronnie, U. Meister, M. Sompolinsky H. 2010. Bayesian model of dynamic image stabilization in the visual systems. PNAS. November 9, 2010 107 (45) 19525-19530
• Burak et al. PNAS. Supplementary Information PNAS.

Week 4 (2/17): Sparse Coding and Reinforcement Learning

• Sparse coding and reinformcent learning are two major advances in computational neuroscience and machine learning where there are good agreements between theory and experimental data. In this week, we will explore the theoretical foundation and neuroscience of sparse and distributed representations and reinforcement learning. Jacob Lee will present Yael Niv's work on Reinforcement Learning and Juntao Chen will present Haim Sompolinsky's work on Sensory Representations.
• Radulescu, Niv and Ballard (2019) Holistic Reinforcement Learning: The Role of Structure and Attention. Trends in Cognitive NSciences
• Babadi and Sompolinsky (2014) Sparseness and Expansion in Sensory Representations. Neuron.
• Botvinick, Niv and Baro (2008) Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective. Cognition. ...
• Niv and Langdon (2016) Current Opinion in Behavioral Sciences 2016, 11:67–73
• Yael Niv (2009) Reinforcement learning in the brain. Journal of Mathematical Psychology , 53 (3), 139–154.

Week 5 (2/24): Biological Plausible Deep Learning Algorithms

• Backpropagation has been deemed to be not biological plausibility for a number of reasons. Recently, a number of a more "biological plausible algorithms" have been proposed for deep learning. Yang Yi will discuss some of these papers.
• Akrout et al. Deep Learning without weight transport. NeurIPS 2019
• Guerguiev et al. (2017) Towards deep learning with segregated dendrites. eLIFE, 2017.
• Bartunov et al. (2018) Assessing the scalability of bioloigically-motivated deep learning algorithms and architectures. arXiv.1807.04587.

Week 6 (3/16): Neural Basis of Bayesian Inference

• This week, and next, we will study some classic and current litrature on the neural mechanisms for Bayesian inference. This week, Haodong will read a recent paper on causal inference and Peijia will read a paper on possibility that the brain might be storing priors and performing inference using the sampling strategy.
• Kording, Beierholm, Ma, Quartz, Tenenbaum and Shams (2007) Causal inference in multi-sensory perception. PLOS ONE.
• Sabyasachi Shivkumar, DeAngelis and Haefner. (2019) A causal inference model for the perception of complex motion in the presence of self-motion
• Orban, Berkes, Fiser and Lengyel (2016). Neural Variability and Sampling-Based Probabilistic Representations in the Visual Cortex. Neuron.

Week 7 (3/23): Bayesian Inference and Inverse Rational Control

• We will continue to study classical works on Bayesian inference as well as some recent work on inverse rational control. Jacob Lee will talk about inverse rational control. Xiaoqi Zhang will talk about spiking network or circuits for Bayesain inference.
• Wu, Kwon, Daptarda, Schrater, Pitkow (2020) Rational Thoughts in Neural Codes. BioRXiv (with codes)
• Daptardar S, Schrater P, Pitkow X (2019). Inverse Rational Control with Partially Observable Continuous Nonlinear Dynamics. arXiv 1908.04696. [
• Rao (2004) Bayesian Computaiton in Recurrent Neural Circuits. Neural Computation 16 (1)
• Deneve (2008) Bayesian spiking neurons I: Inference. Neural Computation 20(1).
• Deneve (2008) Bayesian spiking neurons II: Learning. Neural Computation 20(1).

Week 8 (3/30) : Curiosity and Learning

• This week, we explore curisoity learning or the relationship between curisoity and learning from neuroscience, psychology and machine leanring perspectives. Matthew Wolfe and Songwei Ge will present the following papers. Juntao is on hook for song-bird learning for a future journal club.
• Gruber, Gelman and Ranganath (2014) States of curiosity modulates hippocampus-dependent learning via the dopaminergic circuit. Neuron 84(2). 486-496.
• Kidd and Hayden (2015). The Psychology and Neuroscience of Curiosity. Neuron 88(3). 449-460.
• Burda et al. .. Efros (2018) Large-Scale Study of Curiosity-Driven Learning ICLR 2018
• Pathak, Agrawal, Efros and Darrell (2017) Curiosity-driven Exploration by Self-supervised Prediction. CVPR 2017

Week 9: (4/10) Reinforcement Learning and Song Birds

• This week, we will focus on reinforcement learning, and a particular interesting case of song-bird learning. Juntao Chen and Shili Wang will present the two papers published recently by Gadagkar.
• Gadagkar et al. (2016) Dopamine neurons encode performance error in singing birds Science 354: 1278-1282.
• Gadagkar et al. (2019) Dopamine neurons change their tuning according to courtship context in singing birds. bioRxiv https://doi.org/10.1101/822817
• Fee and Goldberg (2011) A hypothesis for basal ganglia-dependent reinforcement learning in the songbird. Neuroscience 198: 152-170.
• Mackevicius and Fee (2018) Building a state space for song learning Current Opinion in Neurobiology 49: 59-68.

Week 10: (4/17) Why do we have emotion?

• This week, we will investigate why we have emotion from a functional, computational and learning point of view. Xiaoqi Zhang and Haodong Liu will present.
• Haixu Yu, Pei Yang (2019) An Emotion-Based Approach to Reinforcement Learning Reward Design IEEE conference in networking, sensing and control
• Moerland, Broekens and Jonker (2018) Emotion in reinforcement learning agents and robots: a survey. Machine Learning volume 107, pages443–480(2018)
• Sequerira, Melo and Paiva (2011) Emotion-based Intrinsic Motivation for Reinforcement Learning Agents ACII 2011 NCS 6974, pp. 326–336.
• Picard, Vyzas and Healey (2001) Toward machine emotional intelligence: analysis of affective physiological state. IEEE PAMI Vol 23 (10). 1175-1191

Week 11: (4/24) Why does consicouness do for us and animals?

• This week, we will investigate the ideas about the computaitonal roles of consciouness. Yen-Tso Cheng will present a recent paper by Yoshua Bengio. Xiaoqi Zhang will read a paper by Van Hateren. I also put up some idea papers (blogs) by two great masters David Mumford and Manuel Blum that we can discuss.
• Bengio (2019) The Consciouness Prior. arXiv:1709:08568v2
• van Hateren (2019) A theory of consciouness: computation, algorithm and neurbioligcal realization.
• Mumford (2019) Can an artificial intelligence machine be conscious?
• Blum (2018). Can machine be conscious? Toward a Conscious AI -- the Consciouness Turing Machine. Simon Institute Lecture (youtube)
• Crick and Koch (2003). A framework for consciouness Nature Neuroscience 6, 119-126.
• Koch (2018). What is consciouness? Nature 557 (S9): May 2018.
• Quantum approach to consciouness Also check https://en.wikipedia.org/wiki/Quantum_mind

Week 12: (5/1) Glia -- the invisible essential workers

• Finally, we pondered about the functional role of glia cells that made up to 90% of the brain cells. Does glia cells just serve as insolators of neuronal cables, provide scaffolding, transport nutrients, removing debris, serving as policeman, firefighters, garbage collectors, nursses and doctors, i.e. essential workers, or are they also crucial to brain computation! Tianqi Li will read some papers on this subject, but it is a rich new area to explore.
• Fields et al. (2014) Glia Biology in Learning and Cognition. The Neuroscientist. Vol 20(5) 426-431
• Mu et al. (2019) Glia Accumulate Evidence that Actions Are Futile and Suppress Unsuccessful Behavior. Cell 178, 27–43
• Koob (2009) The root of thought: what do glial cells do? Scientific American Oct 2019.
• Fink (2016) Beyond the neuron: Emerging roles of glial cells in neuroscience. A PLOS student BLoGS that contains nice references to ideas about glia cells.
• Sajedinia and Helie (2018) A New Computational Model for Astrocytes and Their Role in Biologically Realistic Neural Networks.
• Manninen et al. (2018) Computational Models for Calcium-Mediated Astrocyte Functions. Front. Comput. Neurosci., 04 April 2018 | https://doi.org/10.3389/fncom.2018.00014.