How is information encoded? (Single cell coding, Population coding)

Single cell coding. The amount of stimulus related information that a channel can carry is restricted by its statistics. I have shown that the channel capacity, estimated using statistics of recorded neural responses, is remarkably similar in early (LGN, V1) and late (IT,STS) visual areas (Oram et al., 1999, 2002; Richmond et al 1999), including under a variety of behavioural conditions (Wiener et al 2001). More recently, I have been involved in developing Bayesian analysis techniques for the assessment of mutual information of neuronal responses that provide more accurate estimates than other alternative methods (Endres et al., 2008, 2010; Endres & Oram, 2010).

An intriguing and powerful possibility that would increase the information carried by a neuronal channel is that of using two (or more) signals simultaneously. For neuronal processing, this implies signals at different temporal resolutions (i.e. spikes per second and the precise times of those spikes). I developed a model which generated artificial spike trains based on the coarse temporal statistics observed in the responses. Application of this model indicated, contrary to conclusions drawn by others, there was little if any evidence for such coding during responses in early or late visual areas (Oram et al., 1999, 2002), nor in motor cortices (Oram et al. 2001).

The precise time (±2ms) when visually evoked responses start is strongly related to the stimulus contrast but not other stimulus attributes (e.g. colour or orientation). The dependency of response latency on stimulus contrast is larger in later compared to early visual areas (Oram et al., 2001, 2002, 2005, 2010) and I have begun to develop computational models explaining this effect (van Rossum et al 2008). Preliminary analysis of EEG data using a novel method to correct for latency shifts (Oram & Jentzsch, in prep) supports the mechanism we used to obtain the latency shifts in our model over alternative mechanisms. Delaying signals from stimuli of reduced contrast allows for greater temporal integration of the incoming signals. This suggests that the contrast induced delay should depend on the difficulty (e.g. non-linearity) of the required computations, with a greater delay for more complex computations. There is some, albeit circumstantial, evidence for the hypothesis (Oram, 2010).

In summary, the application of computational and information theoretic analyses suggest that precise time of when the response starts (rather than precise times of patterns of individual spikes) may convey all of the available information contained in neural responses (Oram et al. 2001, 2002; Oram, 2010; Endres & Oram, 2010; Endres et al, 2010; Oram, under revision). This work will be extended to examine likely computational reasons for variable latency changes.

Population coding. It has long been recognised that the computations performed in the brain are a consequence of the activities of populations of neurones. How information is encoded in population activity is highly dependent on the correlation structure of the responses between the individual neurones. Using a combination of modelling and information theoretic analysis, I showed that correlation of the trial-by-trial variability between neurones frequently increases the mutual information between stimulus identity and population activity despite the necessary drop in overall entropy (Oram et al., 1998; Oram 2011; see also work by Panzeri, Latham, Averbeck & Lee).

There is considerable interest in the hypothesis that the precise times of spikes relative to the times of spikes in another neurone (e.g. synchrony) may form a signal that carries information that is independent of the information carried by the overall activity levels of the individual neurones (Oram et al., 1999; 2001, 2002, 2005). Extension of the model used to investigate the role of precisely timed spikes in single cells showed again that there was little or no evidence supporting the existence of such precisely timed spike codes in motor cortices once all of the lower resolution aspects of the responses are taken into account (Oram et al 2001).

The dynamics of population responses also reveal aspects of how information is encoded in populations of neurones. For example, there is a transitory decrease in the change in the statistics at response onset, with a marked decrease in trial-by-trial correlation (Oram et al. 2007; Oram 2011; See also work by Kohn & Smith). I have recently shown that this and related effects are captured by a spike-train generation model incorporating an instantaneous change of process within the individual neurones (Oram 2011). The simulated responses also allow for evaluation of the impact of the transitory changes on the mutual information carried by the population (Oram 2011). I intend to extend this work by developing spiking network models that implement a change of process, in the first instance within networks that propagate signals, then in models which perform simple (e.g. centre-surround) computations.

Research projects

What changes the encoded information?

Here I give two examples of changes in the encoded information: Behaviourally induced changes and Stimulus induced changes

Behaviourally induced changes in coding. It has been known for more than 20 years that paying attention to a particular location or visual feature results in increased activity of neurones responsive to that particular location or feature. The resultant increase in signal strength will trivially increase the information carried by the neurones’ responses. However, it is not so trivial to explain why such attention driven amplification is limited. Using simulated responses with known statistics, I modelled the impact of varying the “attention gain” on a population of neurones (Oram et al 2002). Bayesian decoding was applied to determine performance: how effective were different gains in increasing mutual information and accuracy? For gains up to 10, the greatest increase in performance was observed when two conditions were met. First, decoding had to be performed using what is now termed the “unaware decoder”: decoding assuming that no gain had been applied. Second, the gain was limited to the range 1.3 – 1.6, precisely the gains reported in the neurophysiological literature. To obtain equivalent or better performance using optimal decoding required gains larger than 10. Furthermore with such large gains, slight mis-match between actual gain and the gain assumed when decoding resulted in poor performance in detecting and identifying stimuli that were not attended. Thus, I argue that attention is limited by computational constraints (Oram et al 2002).

Temporal interactions between stimuli. One event ‘masks’ perception of and suppresses brain responses to subsequent events, yet paradoxically recent experience allows perceptual anticipation of future states. Thus, forward suppression or adaptation may have a positive role in perception, allowing neuronal activity to predict the immediate future during natural stimulus sequences. A computational model of the suppression of neuronal activity induced by presentation of an earlier stimulus allows accurate prediction of the responses of individual neurones to multiple stimuli presented in rapid sequence (Perrett et al 2009). Furthermore, Bayesian decoding of the model output, again assuming an unaware decoder (Oram et al 2002), indicates that the changes in the neural code induced by response suppression predicts anticipatory perception (seeing something before it actually appears) when the sequence of images follows a “natural order” across stimulus space.

Research projects

How does the encoded information relate to behaviour?

Learning & memory. Given the importance attached to memory in everyday life, the inability to recall items on demand can be problematic. An apparently ironic phenomenon has been identified however which suggests that in addition to retrieving desired memories, the act of remembering inhibits or suppresses related memories (retrieval induced forgetting). A a competitive model, designed to investigate the development of the cortical visual system (Oram & Foldiak 1996), provides an explanation for the suppression of some memories as a consequence of remembering others. Based on the model, a number of specific predictions as to when retrieval-induced forgetting effects should or should not occur are confirmed. The model suggests that the mechanisms by which memories are formed and adapted may also underlie retrieval-induced forgetting effects. In addition to having important practical implications, the model provides a theoretical base for the transfer of theories and ideas between two separate levels (cortical processing and memory formation and adaptation) of understanding brain function (Oram & MacLeod, 2001).

Selectivity & visual search. The process by which we find particular visual objects within the cluttered environment is termed visual search. The understanding of visual search is dominated by self-terminating serial search and guided search models. I show that a parallel processing diffusion model incorporating known neurophysiological responses to images of the face and head explains results from visual search experiments that cannot be explained by serial processing models. These results suggest that sensory systems are capable of processing multiple familiar stimuli in parallel and that the role of serial processing in visual search has been overestimated (Wattie et al, in prep; Oram, in prep).

Cognitive processes & information encoding. There has been relatively little work examining the relationship between different neuronal codes and the behavioural phenomena associated with cognitive processes. I generated predictions about reaction time distributions derived from an accumulator model incorporating known neurophysiological properties. Results from human experimental studies examining the effects of changing stimulus orientation, size and contrast are consistent with the model, including qualitatively different changes in reaction time distributions with different stimulus manipulations. The different changes in reaction time distributions depend on whether the image manipulation changes neuronal response latency or magnitude and can be related to parallel or serial cognitive processes respectively. The results indicate that neuronal coding can be productively incorporated into computational models to provide mechanistic accounts of behavioural results related to cognitive phenomena (Oram et al 2002; Oram 2005; Lorteije et al 2011).

Selectivity & binding. A fundamental problem in understanding perception concerns the correct binding of sets of attributes from multiple sources. Current models addressing the binding problem assume that the observed preponderance of illusory conjunctions – e.g. mistakenly associating the colour of one object with the shape of a second object – would not occur if memory contained objects whose attributes were bound together (red-square, blue-triangle) as opposed to a memory containing unbound attributes (red, blue, square and triangle) and, separately, their bindings. Bayesian decoding of simulated representations consisting of bound attributes with known neurophysiological response properties does, in fact, predict a preponderance of illusory conjunctions. Furthermore, such representations explain why illusory conjunctions become more likely as the similarity between the to-be-remember stimuli increases. Thus, detection of illusory conjunctions should not be interpreted as errors of binding: illusory conjunctions also occur from failures to differentiate the most likely representation from the next most likely, even when those representations have already solved the binding problem (Oram, in prep).

Research projects

Additionally...

I have recently been examining the neuronal representation of action (Barraclough et al 2006; Barraclough et al. 2009) and how responses to sequences of images differ from the responses of the same neurones to those images presented one at a time (Perrett et al, 2009). This has allowed modelling of the neuronal responses to image sequences (Perrett et al, 2009) and the development of models of predicitive behaviour (Oram in prep). We are also investigating the influence of eye gaze, hand position and hand posture on the perception of actions and the goals or intentions of others (Williams & Oram, in prep).

Research projects