Hi Everyone,
Myself Ashvin, and I’m excitedly looking forward in contributing to opensource for GSoC this summer. I am extremely interested on the Screen Reader for p5js canvas project as I think this could help a lot of people.
I am familiar with programming in HTML, Javascript, Python etc and have done a few odd projects using p5.js. Also have worked on projects using git and github. Primarily my interest is in machine learning and I think this is a great use case for implementing the same in p5js. My proposal includes
- An image-captioning model that can be used to describe the contents of the p5js using deep learning architectures such as transformers, visual-attention etc implemented via the tensorflow-js framework
- A suite of accessibility functions that can be added to p5js to implement the capabilities of the above AI model.
The possible challenges this project might face are
- Construction of a custom dataset specific to p5js components
- Quantizing/Reducing the size of the AI model so that it wont bloat the p5.js library.
I’ve also opened the following thread previously( Screen-reader accessible p5.js canvas ) regarding my queries on the project.
Just wanted to check if I have understood the project correctly and looking forward in discussing the project details with my mentor.
@sableRaph @Andres @hx2A could you please go through this and let me know if I’m in the right direction.
Thank you 