In Gaussian process regression, both the basis functions and their prior distribution are simultaneously specified by the choice of the covariance function. In certain problems one would like to choose the covariance independently of the basis functions (e. g., in polynomial signal processing or Wiener and Volterra analysis). We propose a solution to this problem that approximates the desired covariance function at a finite set of input points for arbitrary choices of basis functions. Our experiments show that this additional degree of freedom can lead to improved regression performance.