Hi there!
I’m James, a current 1st year at NYU ITP. I’ve been an avid user of Processing and now p5 for a number of years, and I’d love to contribute somehow to the community during the summer of code.
As the title suggests, I’m interested in further improving p5’s WebGL mode. I was inspired by Aidan and Adil’s submissions last year, and think the work Aidan did is a great platform for further enhancing p5’s 3D capabilities, especially for beginners.
Specifically, I am interested in extending camera controls, debug mode, and - if it’s feasible - lighting.
-
Camera Improvements: The current orbitMode() could be improved with an optional pivot point, and other ‘modes’ could be included (e.g. ‘flythrough’ mode and ‘trackball’ mode). Different modes have different use cases, and flythrough mode could be particularly useful in helping users new to 3D intuit a scene.
-
Debug Improvements: in the spirit of making p5 more beginner friendly…
- Lighting overlays: graphical indicators for location of pointLight, direction of directionLight
- Measurements can be useful in 3D. Perhaps there’s a case to be made for debug mode to include options for measuring distances between objects/points.
-
Lighting Improvements: potentially implement a spot light in p5? I’m less sure on this point and am open to discussing more what “improving the lighting fidelity” from the project ideas page means.
These are the specifics that spoke out to me, but of course I am open to suggestions based on what’s necessary for p5! I’d love some feedback on how appropriate this all seems for a GSoC application - is it too vague, too small a scope, are there big red flags?
Thank you for your time and thoughts!
1 Like
Hi Processing folks!
Upon further discussion with some people in my life, I’ve veered my focus slightly onto an area of WebGL that has not gotten as much attention as camera or lighting: materials.
Here’s a draft of my application. I would be very grateful for any thoughts on whether these ideas could be a viable application.
Thanks for your time!
James
1 Like
Great!
Have you identified any of the specific people – on this forum or on github – that are involved with p5.js mode in general and/or with WebGL mode specifically, and that you want to @ tag or ping to request feedback?
https://discourse.processing.org/search?q=webgl
Thanks @jeremydouglass, that’s a great idea. I believe this may be the purview of @stalgiag? Or perhaps @kjhollen?
Hi @jameshosken thanks for your interest! I think the ideas you mention are good ones. I would also suggest taking a look at open webgl related issues in case there are any that you want to incorporate in your proposal: https://github.com/processing/p5.js/issues?q=is%3Aissue+is%3Aopen+label%3Aarea%3Awebgl
2 Likes
Hi @jameshosken thank you for the interest! Sorry for the late reply. I didn’t see this thread until now. I think that there are a lot of possibilities here that are really cool. My first thought is that the idea of procedural textures feels a bit more like a library as it starts to extend p5 WebGL into territory that is more aligned with game engines or fully fleshed out WebGL libraries like Three.JS. I think that with both Processing’s P3D and p5’s WebGL there is a certain amount of intentional limitations. This has to do with the mission of the foundation and also with the fact that there are already great libraries for more advanced users. That said these limitations are flexible because open-source software is flexible.
I know that there are some examples of folks doing cool things with bump mapping in Processing P3D floating around the internet.
Procedural textures could also provide a lot of opportunities. My first thought with this though is that this might exist more clearly as a set of examples that you publish. 3D geometry can already be textured with any p5.graphics object which allows any procedural patterns that can be made in 2D to be applied as textures with only a couple of lines of code. Additionally the ability to load in your own custom shaders allows for a large amount of expanded opportunities with GLSL. Adam Ferris has a great set of examples that might serve as inspiration here. I don’t know that much needs to be done to the core library to be able to make procedural textures so then it might be better to approach this as an instructive, artistic project or a library that enables something that isn’t possible currently such as custom uniforms/attributes.
Maybe this could all be neatly tied up in an ‘Advanced Materials’ p5 library that has a great set of examples and a bunch of helper functions and custom shaders for creating more elaborate materials. I know there isn’t much time yet but a little reformulation in that regard might make it more clear how this will fit in with the larger ecosystem of tools and examples for p5 webgl mode. Just to reiterate my first point this feels like an expansion beyond the bounds of what is core to p5’s WebGL mode and so it might be better to envision this as an expansion library. That is just a personal response and I am super open to arguments for these features being added to the core library.
Also, I know that this is last minute but I will say that some of the improvements that you mentioned in your first post are very welcome. It might be useful to think if there is any kind of direct improvement to WebGL that is needed that could become apart of the larger goal of the advanced materials library. Could you improve the way that the standard shaders work with lighting in order to open up new possibilities for cool procedural material examples? Could you offer ways to debug problems with shaders visually in the core library and then use these in an expansion library to test out the functionality of your custom methods? These are just off the cuff ideas but the point is that the WebGL portion of p5 still has plenty of room for improvement and there is significant overlap between your ideas for feature expansion and general refinement of the existing features.
Okay I think that is all for now! Thanks for the proposal! I’m excited by the ideas you have presented.
1 Like
Hi @stalgiag,
Thank you so much for such a thoughtful response! Totally my fault for not tagging you earlier, I apologise!
You’ve given me a lot to consider, and I think there are a few clear ways forward. Your ‘advanced materials’ suggestion sounds very compelling as a larger project outside the scope of GSoC, however I think what speaks most to me (and most to the Foundation’s mission?) is creating a more usable and friendly environment for beginners.
I think I shall reframe my application to focus more on debugging (in the spirit of Aidan Nelson’s debugMode() last year). I imagine this is too last minute to hope for any additional feedback, but from your comments above I’m assuming that it is at least an appropriate direction to head in?
I will focus on debugging lighting (based on the ‘overlays’ mentioned in my first iteration) and materials/shading, in the context of a potentially more advanced library. I definitely need to flesh out the details, but on principle I’d consider these improvements as extensions of the current debugMode(), intended to help users intuit lighting and shading in the same way that debugMode() helps intuit space and camera.
If you have any last minute thoughts on this direction I’d be forever grateful, but of course given the time I have no expectations!
Finally, some thanks I don’t want to leave unresolved:
- Thanks for explaining about the intentional limitations, that makes a lot of sense.
- The Adam Ferris resource is fantastic
- In general your suggestions are helpful and inspiring!
Cheers,
James
2 Likes
Hello @jameshosken
That sounds cool! I think my only advice is to check out the issues on the Github page and play around with the library for a bit. The webgl manual test examples might be a good jumping off point for this. The reason I suggest doing this is because there might be some balancing of new features with feature improvement. It might be difficult to spend the whole summer on tools for debugging lighting. This definitely depends on what you propose doing but it might be useful to look at Aidan Nelson’s summer project more holistically. Aidan added debugMode as part of a larger refactoring of how perspective and orientation is modified. This took the form of the p5.Camera class.. So perhaps there is an equivalent improvement to lighting + lighting debug feature that you could take on. For ideas on the former, you might try looking at Processing’s reference for lighting functions and see where you think the greatest disparity in features or quality with p5 exists. Some lighting functions still aren’t implemented, and some that are implemented don’t work as well as they do in Processing.
Anyways – these are just thoughts for how to approach brainstorming. I think the projects work best if they come from your own inspiration. So if you have time before the deadline, I’d say just spend sometime exploring and trying things out and hopefully inspiration strikes, be it for a new feature or refinement of something existing or both.
Good luck!
1 Like