I suspect you will find the same in all programming languages. I have been programming for decades and I still have problems remembering programming languages / algorithms / concepts that are new to me. The problem is that to remember these things you have to use them often, repetition is the key and that takes time and patience.
A few years back I started to learn Javascript by creating one or two simple applications from scratch (i.e. no p5.js, no ProcessingJS and no third party libraries) and it was a struggle to create the basic boilerplate code for an application so I went back to Java. Now I have used Java for 20 years but 6 months ago I decided to try again and this time use p5.js. Now I still consider myself a Javascript novice but I have been able to create some nice stuff with p5.js (you can see some of it here).
So I have decided to use p5.js as my main language. It will take a long time for me to use the language fluently but I will use it in preference to Java.
For “flexible and interactive brand design systems” I would think p5.js is more useful because it works with the Internet out of the box but at the end of the day you must make your own choice.