The paper focuses on how to create an effective dynamic ranking service for IaaS, PaaS and SaaS cloud providers. It considers building of a quality model for this purpose along with definition of quality measurement procedures. The paper discusses several techniques known from already existing price comparison engines that could be modified and adopted for comparison of cloud providers. Furthermore, a technique for filtering measured data is proposed, in particular to avoid vendor lock-in issues.