Modern approaches to systems biology call for a tightly coupled iterative cycle of computational modelling and independent experimental validation of model predictions. A Bayesian formulation to model inference should be exceptionably amenable to this type of experimental paradigm. Given prior knowledge that has been encoded into a model, we can train the model on data from experiment A. The result is a posterior distribution over, say, gene regulatory networks which can act as a prior for the next model, trained on data from experiment B. The Bayesian model at each stage can be seen as a distillation of the experimental data obtained up to that point, and since it is a probabilistic model it can be used as an expert prior for the model trained on the next data set. A Bayesian sequential learning strategy can therefore be employed, instead of waiting for all the data to be collected before training the first model. We explore this paradigm using simulated data from a realistic in silico model network and experimental microarray time series data sets studying stress responses in Arabidopsis and E. coli.