Recent large pre-trained music models have demonstrated competitive results on a variety of tasks. However, the potential benefits of large pre-training remain underexplored for many specific music tasks, including pitch estimation. This work systematically investigates the performance of Data2Vec-music when fine-tuned for pitch estimation. Our findings reveal the utility of its convolutional backbone, which we leverage to construct a computationally efficient, general-purpose pitch tracker. Our model achieves competitive performance with CREPE on monophonic data and NMP on polyphonic data. Along with first results, we compile a benchmark from various datasets to assess diverse aspects of pitch estimation.