The goal of data clustering is to organize a set of n objects into k clusters such that objects in the same cluster are more similar to each other than objects in different clusters. Clustering is one of the most popular tools for data exploration and data organization that has been widely used in almost every scientific discipline that collects data. Given the exponential growth in data generation (estimated to be over 35 trillion gigabytes by the year 2020), clustering is receiving renewed interest and use in applications such as social networks, image retrieval, web search and gene expression analysis. In this talk I will introduce the data clustering problem and discuss the challenges and opportunities in the research on large-scale clustering, with the focus on two main issues: (i) how to define pairwise similarity between objects? and (ii) how to efficiently cluster hundreds of millions of objects? I will present our recent work in approximation of the well known kernel k-means clustering algorithm. I show both analytically and empirically that the performance of approximate kernel k-means is similar to that of the kernel k-means algorithm, but with significantly lower run-time complexity and memory requirements.