Estimating parameters of object dynamics, such as viscosity or internal degrees of freedom, is key in the autonomous and dexterous robotic manipulation of objects. Oftentimes, it may be challenging to accurately and efficiently estimate these object parameters due to the complex highly nonlinear underlying physical processes. In an effort to improve the quality of hand-crafted solutions, we examine how control strategies can be automatically generated. We present an active learning framework that sequentially gathers data samples, using information-theoretic criteria to find the optimal actions to perform at each time step. Our framework is evaluated on a robotic hand-arm where the task involves optimizing actions (shaking frequency and rotation of shaking) in order determine viscosity of liquids, given only tactile sensory feedback. The active framework performs better than other simple strategies and speeds up the convergence of estimates.