We aim to test and develop hypotheses on how neural activity is shaped by context and expectation (i.e. learning to make predictions based on past experiences), on the scale of networks and individual neurons. More specifically, we desire to discover how bottom-up (sensory) and top-down (anticipatory) input into networks of neurons are combined to give rise to the experimental observations on expected and unexpected stimuli. These unexpected stimuli constitute deviant and omitted stimuli, and should, by definition, result in a prediction error somewhere in the brain. There are many theories regarding this “predictive processing”, and there have recently been exciting discoveries in this direction, but the theories are either incomplete or incompatible. To work towards a more cohesive theory, we analyze experimental data on deviant and omission paradigms to gain insights into the relevant patterns of neural activity and combine this with findings from existing studies, in order to develop computational models. These models enable us to generate novel, falsifiable hypotheses on how brains make predictions and consolidate them with existing theories.