AI News, BOOK REVIEW: NIPS Proceedingsβ

NIPS Proceedingsβ

Part of: Advances in Neural Information Processing Systems 27 (NIPS 2014) The persistent and graded activity often observed in cortical circuits is sometimes seen as a signature of autoassociative retrieval of memories stored earlier in synaptic efficacies.

The resulting networks operate in the balanced regime, are robust to corruptions of the memory cue as well as to ongoing noise, and incidentally explain the reduction of trial-to-trial variability following stimulus onset that is ubiquitously observed in sensory and motor cortices.

Neural oscillation

The first discovered and best-known frequency band is alpha activity (7.5–12.5 Hz)[5] that can be detected from the occipital lobe during relaxed wakefulness and which increases when the eyes are closed.[6] Other frequency bands are: delta (1–4 Hz), theta (4–8 Hz), beta (13–30 Hz), low gamma (30–70 Hz), and high gamma (70–150 Hz) frequency bands, where faster rhythms such as gamma activity have been linked to cognitive processing.

Quantitative models can estimate the strength of neural oscillations in recorded data.[12] Neural oscillations are commonly studied from a mathematical framework and belong to the field of 'neurodynamics', an area of research in the cognitive sciences that places a strong focus upon the dynamic character of neural activity in describing brain function.[13] It considers the brain a dynamical system and uses differential equations to describe how neural activity evolves over time.

Class I neurons can generate action potentials with arbitrarily low frequency depending on the input strength, whereas Class II neurons generate action potentials in a certain frequency band, which is relatively insensitive to changes in input strength.[10] Class II neurons are also more prone to display sub-threshold oscillations in membrane potential.

This thalamocortical network is able to generate oscillatory activity known as recurrent thalamo-cortical resonance.[18] The thalamocortical network plays an important role in the generation of alpha activity.[19][20] In a whole-brain network model with realistic anatomical connectivity and propagation delays between brain areas, oscillations in the beta frequency range emerge from the partial synchronisation of subsets of brain areas oscillating in the gamma-band (generated at the mesoscopic level).[21] Scientists have identified some intrinsic neuronal properties that play an important role in generating membrane potential oscillations.

resonance behavior that does not result in action potentials, may also contribute to oscillatory activity by facilitating synchronous activity of neighboring neurons.[22][23] Like pacemaker neurons in central pattern generators, subtypes of cortical cells fire bursts of spikes (brief clusters of spikes) rhythmically at preferred frequencies.

Oscillations recorded from multiple cortical areas can become synchronized to form large scale brain networks, whose dynamics and functional connectivity can be studied by means of spectral analysis and Granger causality measures.[26] Coherent activity of large-scale brain activity may form dynamic links between brain areas required for the integration of distributed information.[9] In addition to fast direct synaptic interactions between neurons forming a network, oscillatory activity is modulated by neurotransmitters on a much slower time scale.

In particular, models of interacting pyramidal cells and inhibitory interneurons have been shown to generate brain rhythms such as gamma activity.[30] Neural field models are another important tool in studying neural oscillations and are a mathematical framework describing evolution of variables such as mean firing rate in space and time.

It captures the activity of a local system (e.g., a single neuron or neural ensemble) by its circular phase alone and hence ignores the amplitude of oscillations (amplitude is constant).[34] Interactions amongst these oscillators are introduced by a simple algebraic form (such as a sine function) and collectively generate a dynamical pattern at the global scale.

The Kuramoto model is widely used to study oscillatory brain activity and several extensions have been proposed that increase its neurobiological plausibility, for instance by incorporating topological properties of local cortical connectivity.[35] In particular, it describes how the activity of a group of interacting neurons can become synchronized and generate large-scale oscillations.

a high- and low-amplitude mode, and hence shows that resting-state activity does not just reflect a noise process.[37] In case of fMRI, spontaneous fluctuations in the blood-oxygen-level dependent (BOLD) signal reveal correlation patterns that are linked to resting states networks, such as the default network.[38] The temporal evolution of resting state networks is correlated with fluctuations of oscillatory EEG activity in different frequency bands.[39] Ongoing brain activity may also have an important role in perception, as it may interact with activity related to incoming stimuli.

Phase resetting occurs when input to a neuron or neuronal ensemble resets the phase of ongoing oscillations.[43] It is very common in single neurons where spike timing is adjusted to neuronal input (a neuron may spike at a fixed delay in response to periodic input, which is referred to as phase locking[10]) and may also occur in neuronal ensembles when the phases of their neurons are adjusted simultaneously.

Increases in oscillatory activity are therefore often referred to as event-related synchronization, while decreases are referred to as event-related desynchronization.[47] It has recently been proposed that even if phases are not aligned across trials, induced activity may still cause event-related potentials because ongoing brain oscillations may not be symmetric and thus amplitude modulations may result in a baseline shift that does not average out.[48][49] This model implies that slow event-related responses, such as asymmetric alpha activity, could result from asymmetric brain oscillation amplitude modulations, such as an asymmetry of the intracellular currents that propagate forward and backward down the dendrites.[50] Under this assumption, asymmetries in the dendritic current would cause asymmetries in oscillatory activity measured by EEG and MEG, since dendritic currents in pyramidal cells are generally thought to generate EEG and MEG signals that can be measured at the scalp.[51] Neural synchronization can be modulated by task constraints, such as attention, and is thought to play a role in feature binding,[52] neuronal communication,[2] and motor coordination.[4] Neuronal oscillations became a hot topic in neuroscience in the 1990s when the studies of the visual system of the brain by Gray, Singer and others appeared to support the neural binding hypothesis.[53] According to this idea, synchronous oscillations in neuronal ensembles bind neurons representing different features of an object.

This phenomenon is best seen in local field potentials which reflect the synchronous activity of local groups of neurons, but has also been shown in EEG and MEG recordings providing increasing evidence for a close relation between synchronous oscillatory activity and a variety of cognitive functions such as perceptual grouping.[52] Cells in the sinoatrial node, located in the right atrium of the heart, spontaneously depolarize approximately 100 times per minute.

Neural oscillations could create periodic time windows in which input spikes have larger effect on neurons, thereby providing a mechanism for decoding temporal codes.[57] Synchronization of neuronal firing may serve as a means to group spatially segregated neurons that respond to the same stimulus in order to bind these responses for further joint processing, i.e.

Purely theoretical formulations of the binding-by-synchrony hypothesis were proposed first,[58] but subsequently extensive experimental evidence has been reported supporting the potential role of synchrony as a relational code.[59] The functional role of synchronized oscillatory activity in the brain was mainly established in experiments performed on awake kittens with multiple electrodes implanted in the visual cortex.

The frequency of these oscillations was in the range of 40 Hz and differed from the periodic activation induced by the grating, suggesting that the oscillations and their synchronization were due to internal neuronal interactions.[59] Similar findings were shown in parallel by the group of Eckhorn, providing further evidence for the functional role of neural synchronization in feature binding.[60] Since then, numerous studies have replicated these findings and extended them to different modalities such as EEG, providing extensive evidence of the functional role of gamma oscillations in visual perception.

Perceiving different odors leads to different subsets of neurons firing on different sets of oscillatory cycles.[61] These oscillations can be disrupted by GABA blocker picrotoxin,[62] and the disruption of the oscillatory synchronization leads to impairment of behavioral discrimination of chemically similar odorants in bees[63] and to more similar responses across odors in downstream β-lobe neurons.[64] Recent follow-up of this work has shown that oscillations create periodic integration windows for Kenyon cells in the insect mushroom body, such that incoming spikes from the antennal lobe are more effective in activating Kenyon cells only at specific phases of the oscillatory cycle.[57] Neural oscillations are also thought be involved in the sense of time[65] and in somatosensory perception.[66] However, recent findings argue against a clock-like function of cortical gamma oscillations.[67] Oscillations have been commonly reported in the motor system.

Pfurtscheller and colleagues found a reduction in alpha (8–12 Hz) and beta (13–30 Hz) oscillations in EEG activity when subjects made a movement.[47][68] Using intra-cortical recordings, similar changes in oscillatory activity were found in the motor cortex when the monkeys performed motor acts that required significant attention.[69][70] In addition, oscillations at spinal level become synchronised to beta oscillations in the motor cortex during constant muscle activation, as determined by cortico-muscular coherence.[71][72][73] Likewise, muscle activity of different muscles reveals inter-muscular coherence at multiple distinct frequencies reflecting the underlying neural circuitry involved in motor coordination.[74] Recently it was found that cortical oscillations propagate as travelling waves across the surface of the motor cortex along dominant spatial axes characteristic of the local circuitry of the motor cortex.[75] It has been proposed that motor commands in the form of travelling waves can be spatially filtered by the descending fibres to selectively control muscle force.[76] Simulations have shown that ongoing wave activity in cortex can elicit steady muscle force with physiological levels of EEG-EMG coherence.[77] Oscillatory rhythms at 10 Hz have been recorded in a brain area called the inferior olive, which is associated with the cerebellum.[11] These oscillations are also observed in motor output of physiological tremor[78] and when performing slow finger movements.[79] These findings may indicate that the human brain controls continuous movements intermittently.

Coupling between theta and gamma activity is thought to be vital for memory functions, including episodic memory.[81][82] Tight coordination of single-neuron spikes with local theta oscillations is linked to successful memory formation in humans, as more stereotyped spiking predicts better memory.[83] Sleep is a naturally recurring state characterized by reduced or absent consciousness and proceeds in cycles of rapid eye movement (REM) and non-rapid eye movement (NREM) sleep.

Neural oscillation has been applied as a control signal in various brain–computer interfaces (BCIs).[87] For example, a non-invasive BCI interface can be created by placing electrodes on the scalp and then measuring the weak electric signals.[88] Although individual neuron activities cannot be recovered through non-invasive BCI because the skull damps and blurs the electromagnetic signals, oscillatory activity can still be reliably detected.

Neurophysiological Bases of Exponential Sensory Decay and Top-Down Memory Retrieval: A Model

As discussed in the introduction, there are not (to our knowledge) single-cell neurophysiological experiments which have investigated explicitly and in a controlled manner the temporal bracketing between sensory stimulation and top-down control.

According to this interpretation, top-down control to T2 is only delivered once it has been released from T1 and thus the longer the time to complete the first task (RT1) the larger the gap between the presentation of T2 stimulus and the allocation of top-down control (see Figure ​Figure4B4B for a simple illustration of the scheme).

More precisely, following the assumptions of a sequential deployment of top-down control, the duration of the perceptual buffer can be obtained from experimental observables: as sketched in Figure ​Figure4B,4B, the duration of the perceptual buffer of S2 is determined by:

Rather, we make the simple assumptions that: (1) top-down to T2 is directed after the conclusion of the first task, (2) that this is indexed by RT1 and (3) that top-down control is implemented by a non-specific current to the network which sets it in a retrieval mode.

The experiments show that this single parameter derived from SOA and RT1 (the duration of the sensory buffer), is capable of capturing one of the main qualitative aspects of the dependence of performance with RT1 and SOA (Figures ​(Figures4C,D),4C,D), which captures most of the variability for intermediate SOA values.

more direct experimental psychological demonstration of the memory decay during the interval between stimulus presentation and top-down control comes from partial report experiments (Sperling, 1960).

Performance in many different variants of this experimental design has been shown to decay exponentially with the inter-stimulus interval (ISI), the time between the presentation of the stimulus and the spatial cue indicating the item to report (Loftus et al., 1992;

This asymmetry (local presynaptic and global postsynaptic connections of inhibitory neurons) turned out to be critical to assure that the network scaled correctly and generated a winner-take-all behavior during retrieval.

This can be intuitively understood with a simple qualitative calculation involving the balance of currents in excitatory populations and assuming that populations fire following a step function, if the input current is larger than a threshold T.

Of course, this algebraic equation needs to be iterated dynamically since once the population is active it changes the inputs to other populations, which in turn change the input to others populations and so on.

The important aspect of the proposed architecture is that inhibition to all neurons increases linearly with the number of excited neurons and thus the balance between inhibition and excitation can be easily controlled.

2 the inhibition current by a constant (the efficacy of synaptic inhibition) multiplied by the number of active populations (recall that the key aspect of this architecture is that for each active excitatory population, there is one active inhibitory population).

Thus, we could easily adjust the parameters, in a stable manner to set the network in a mode in which there is passive decay during the buffer and amplification to a single response after allocation of top-down control currents.

Thus, in future experiments and model it might be worth exploring the number of elements which can be correctly retrieved in iconic memory experiments and how this may relates in a more quantitative manner to the architecture of inhibition in recurrent memory networks.

Once we could assure a stable winner-take-all network for a large number of excitatory populations, we proceeded to explore whether retrieval in this network showed an exponential dependence with ISI, as observed in the experiments.

Here, we merely show that: (1) correct retrieval after passive delay accounts for the correct scaling observed in psychophysical experiments and (2) that a recurrent network can be configured to elicits passive decay of information in absence of top-down control and switch, with the allocation of top-down control, to a winter-take-all configuration for a large number of distinct excitatory populations.

NIPS: Oral Session 3 - Dylan Festa

Analog Memories in a Balanced Rate-Based Network of E-I Neurons The persistent and graded activity often observed in cortical circuits is sometimes seen as a signature of autoassociative retrieval of memories stored earlier in synaptic efficacies.

The resulting networks operate in the balanced regime, are robust to corruptions of the memory cue as well as to ongoing noise, and incidentally explain the reduction of trial-to-trial variability following stimulus onset that is ubiquitously observed in sensory and motor cortices.

NIPS: Oral Session 3 - Dylan Festa

Analog Memories in a Balanced Rate-Based Network of E-I Neurons The persistent and graded activity often observed in cortical circuits is sometimes seen as a signature of autoassociative...

Neural oscillation

Neural oscillation is rhythmic or repetitive neural activity in the central nervous system. Neural tissue can generate oscillatory activity in many ways, driven either by mechanisms within...

6/7/14 Circuits for Intelligence - Matt Wilson: Hippocampal Memory Reactivation