I’ve done something similar in the past by using k-means to cluster the colors. Some posts that you might find interesting: