PAC-Bayesian analysis is a general tool for deriving generalization bounds for a wide class of inference rules. Interestingly, PAC-Bayesian generalization bounds take a form of a trade-off between the empirical performance of the inference rule and the KL-divergence between the posterior distribution over the hypothesis space applied by the inference rule and a prior distribution over the hypothesis space. This form of a trade-off is closely related to the free energy in statistical physics. Moreover, PAC-Bayesian bounds can be used in order to determine the right "temperature" at which the system should be analyzed given a finite sample. In other words, PAC-Bayesian analysis introduces a principled way of treating finite samples in application of methods from statistical physics to inference. We present a generalization of PAC-Bayesian analysis to martingales. This generalization makes it possible to apply PAC-Bayesian analysis to time-evolving processes, including importance-weighted sampling, reinforcement learning, and many other domains. References: [1] Yevgeny Seldin, François Laviolette, Nicolò Cesa-Bianchi, John Shawe-Taylor, and Peter Auer. PAC-Bayesian inequalities for martingales. IEEE Transactions on Information Theory, 2012. Accepted. [2] Yevgeny Seldin, Peter Auer, François Laviolette, John Shawe-Taylor, and Ronald Ortner. PAC-Bayesian analysis of contextual bandits. In Advances in Neural Information Processing Systems (NIPS) 25, 2011.