Decoding subjective pain perception from functional magnetic resonance imaging (fMRI) data using machine learning technique is gaining a growing interest. Despite the well-documented individual differences in pain experience and brain responses, it still remains unclear how and to what extent these individual differences affect the performance of between-individual fMRI-based pain prediction. The present study is aimed to examine the relationship between individual differences in pain prediction models and between-individual prediction error, and, further, to identify brain regions that contribute to between-individual prediction error. To this end, we collected and analyzed fMRI data and pain ratings in a laser-evoked pain experiment. By correlating different types of individual difference metrics with between-individual prediction error, we are able to quantify the influence of these individual differences on prediction performance and reveal a set of brain regions whose activities are related to prediction error. Interestingly, we found that the precuneus, which does not have predictive capability to pain, could also affect the prediction error. This study elucidates the influence of interindividual variability in pain on the between-individual prediction performance, and the results will be useful for the design of more accurate and robust fMRI-based pain prediction models.