This paper explores the relationship between music and the color palettes used for designing their corresponding music cover images, providing a comprehensive analysis that bridges auditory and visual expression. Our findings reveal a relationship between musical pieces and certain colors, suggesting that the color palettes used in cover image design are carefully selected to reflect the auditory experience. Building on these findings, we propose a framework that estimates appropriate color palettes for musical pieces to support selecting colors for cover image design. Using a large private dataset of 582,894 pairs of a musical piece and its corresponding cover image from various music genres, our framework leverages deep learning techniques to train our color palette estimator. We demonstrate the effectiveness of our proposed framework in graphic design by showcasing an application that generates cover images using the estimated color palettes from given musical pieces.