The things is, if you use the most advanced color model (which virtually no one is), this is a lot easier.
CIECAM02 models color perception and comparison (for humans) better than any other model.
I found a Python library that has a compare function in that color model, and wrote a script that sorts colors via that model. The results are better than using any other model I’ve seen.
I would love for such color comparisons to be available via a Processing library, and I want to suggest here, for anyone looking for a coding project, that this would be a worthy library to code