The deviation from a Random Walk behavior in financial time series have been identified as Stylized Facts (SF) and are common to all markets. The main ones are that fluctuations are much lager than those predicted from the standard economic theory (gaussian fluctuations), the clustering of volatility and a substantial non stationarity of all properties. Many Agent Based Models have been proposed to explain these phenomena and several are indeed able to reproduce some of them. However, the situation is still problematic becaus these models are typically rather complicated with various ad hoc assumptions. This has prevented a systhematic study of these effects. We have tried therefore to define a workable Agent based Model [1], which contains the essential elements, but in a mathematically simple and well defined framework. In addition we have considered some new important elements like the nonstationarity of the process with respect to the number of agents and the question of the self-organization. Namely why all markets evolve spontaneously towards the situation corresponding to the SF, considering that in all models this is restricted to a very narrow range of parameters. The SF are shown to correspond to finite size effects (with respect to time and to the number of agents N) which, however, can be active at different time scales. This implies that strict universality cannot be expected in describing these properties in terms of effecive critical exponents. The introduction of a threshold in the agents action (small price movements lead to no action) triggers the self-organization towards the intermittent state corresponding to the SF. From these studies the herding phenomenon seems to be a crucial one beyond the standard theory as a triggering element of bubbles and crashs which develop spontaneously without a cause-effect relation. The model can also be used backwards to derive the strategies of the agents from the price time series. Other applications are under consideration like the problem of finite liquidity and the possibility that the reference fundamental price is subject to large fluctuations if one cnsiders that all markets are linked into a large network [2].