Automatic Music Transcription (AMT) systems produce MIDI files of varying quality for audio samples with different acoustic conditions. Without filtering or postprocessing, transcription errors can propagate into downstream tasks. In this work, we present a data-driven approach to assessing the quality of piano MIDI. We analyze note alignment between MIDI performances and musical scores in ASAP and ATEPP datasets and show that alignment quality is a good proxy for music quality. Based on this insight, we label performances in ATEPP dataset and train a sequence classifier to categorize MIDI files into five classes: musical score, recorded, high quality, low quality, and corrupted. The final model demonstrates a perceptually meaningful understanding of music quality, with errors constrained to adjacent classes. This approach can be used to create cleaner datasets of transcribed music.