Neural activity persisting for several seconds is thought to be the neural correlate of working memory in cortex. It was found recently that during persistent activity spike trains are highly irregular, even more irregular than in spontaneous activity. We show that this apparently innocuous feature raises a fundamental difficulty if one holds that neuronal nonlinearities combined with recurrent excitation underly activity persistence as usually assumed. Instead, we argue that the key nonlinearities involved are synaptic and not neuronal. We assess this proposal in the framework of a network model representing a circuit in prefrontal cortex involved in spatial working memory. This lead us to suggest that short term plasticity recently discovered in synapses made by pyramidal cells in prefrontal cortex is crucial in spatial working memory.