As electronic communication, media and commerce increasingly permeate every aspect of modern life, real-time personalization of consumer experience through data-mining becomes practical. Effective classification, prediction and change modeling of consumer interests, behaviors and purchasing habits using machine learning and statistical methods drives efficiency, insights and consumer relevance that were never before possible.

The internet has brought on a rapid evolution in advertising. Everything about behavior on the internet can be quantified and responses to behavior can occur in real time. This dynamic interaction with the user has created opportunities to better understand the way in which individuals move from awareness of a product to considering a purchase, through to intent and ultimately a sale for the marketer. When a marketer can answer the question „did those TV ads cause consumers to switch shampoo brands?‟ they can model behavior change and adjust marketing strategies accordingly.

Underpinning this shift in how the world‟s trillion dollar marketing budget is spent is transactional data on an unprecedented scale, creating new challenges for software that must interpret this stream and make real time decisions tens, even hundreds of thousands of times every second.

I will explore advances in modeling media consumption, advertising response and the real-time evaluation of media opportunities through reference to Quantcast, a business launched in September 2006 which today interprets in excess of 10 billion new digital media consumption records every day. We will examine the challenges of applying machine learning to non-search advertising and in doing so explore the creation of business environments – organization, infrastructure, tools, processes (and costs considerations) – in which scientists can quickly develop new petabyte scale algorithmic approaches, migrate them rapidly to real-time production and deliver fully customized experiences for marketers, publishers and consumers alike.