Dynamic image masking, I made it! But how can I improve it?

sundftig · March 6, 2019, 8:29pm

Hi everyone, this is my first post in this forum and I hope I will get some good help from you here!

I´ve been looking for a way to mask images with a custom made shape in both Processing and p5.js but I couldn’t find anything that really did what I wanted to do. In a way, the .mask() function works but it doesn’t allow you to animate the mask itself. Or at least I couldn’t make it work.

I don’t know if I looked well enough in the internet but basically I decided to do it by myself hoping, somehow, to help someone else other than me.

The problem here is that at this point in time the code is pretty heavy and I´m looking for some tips and suggestion on how I could optimize it to make it slimmer and more browser-friendly. Right now running the sketch on chrome takes around 100-130% of the cpu :S

This is the code:
https://editor.p5js.org/sundfitg/sketches/_IBgvElBD

Any thoughts or also questions??
I’d be so happy to hear from you guys and girls!

Cheers!!

Pr0tonX · March 6, 2019, 10:48pm

I have a very light solution for this, but you will have to betray p5.js for this, and it break my heart to say it, but for this type of case, I used to use anime.js to anime a svg. You just have to make a rectangle clipped by an octagon, and then import it and anime it.

I know, it’s an escape from your issue…but it could be useful if you want to do more with your sketch, because effectively, it’s a cool animation, but it takes a large part of my CPU !

Ohterwise, a good way to improve yours performances, it’s to create your canvas in WEBGL context, because it will be calculated by your graphic card, so for it can helps a lot if you work on a PC. But image() function doesn’t work in WEBGL, so you have to do a trick to display it : https://webglfundamentals.org/webgl/lessons/webgl-image-processing.html

With this, you will unload your CPU from these .

sundftig · March 7, 2019, 8:38am

Thank you @Pr0tonX for your reply! I didn’t know this “anime.js”! And thank you for the WEBGL link. I actually quickly tried to implement the WEBGL renderer but I encountered many problems so I gave up, but this link it is really interesting and I will try to take a look and see if I can come up with something.

I have one questions about anime tho:
I´m using p5.js because of the extreme simplicity to embed the code in an iFrame in Wix. So I can host my code on the p5.js server and call it in a specific fame any time I want to. Can I do the same with this anime? Do I need to host code and images in a private host? Would you recommend me something else?

Sorry for all this questions but I’m really new to the “online world” when it comes to use my code on a website, with all the limitations that wix has.

Thank you very much for your help!

jeremydouglass · March 9, 2019, 12:18am

Is the idea that the mask will always be a convex polygon, and is the common use case that the whole interior mask will be at the same fixed opacity? Because I could imagine that you might optimize it in that way, by converting a complex polygon into a set of array runs (index+length) and then doing array copies. That might save time compared to iterating over each pixel and checking the mask source – you wouldn’t even need a reference mask image at all, just the geometry.

sundftig · March 11, 2019, 11:10pm

Hey @jeremydouglass thank you for your reply. I don’t really get your first question but I understand that you are suggesting me an improvement, and I’m really curious about it! Do you have any example that could help me to better understand your suggestion? Also a link to some website would be really helpful. Sorry but my understanding of programming is not really deep and I still struggle to understand some terminology. Sorry about that. I will also google it and se if i find any result.
Thank you very much again! –

jeremydouglass · March 12, 2019, 12:49am

So, my first question was: will the mask always be a convex polygon?

My second question was, will the mask always have the same fixed opacity – e.g. will the mask opacity always be 0% or 50% etc., rather than different in different pixel locations?

sundftig · March 12, 2019, 10:09pm

Ok, I’m really sorry but I got confused with your previous message.
1 - To answer the first question I would say that at the moment I cannot be sure 100% that the shape will be always convex, since the noise values that are generating it could also create a situation in which the shape is slightly concave (I guess). And I think I now understand how that would be a problem .
2 - The opacity of the mask will always stay at 0 or 100%. No shades, to be clear.
But now I think I get what you mean by converting the mask, but I guess that would include some math right? Or maybe is there a library that would help me with that?

jeremydouglass · March 12, 2019, 10:23pm

I don’t know of a library.

One way to optimize lookups on a region (if it is a polygon) is to do encode ranged copying or checking rather than one pixel at a time. So, rather than checking and copying 400 pixels per row, you use mask lookups to copy the middle ranges – 1 range outside the mask, 1 inside the mask, 1 outside the mask. For each row of the mask, a tuple like this:

{ { 100, 200 }, { 99, 201}, {98, 202}, ... }

can be copied with an arrayCopy. But this method only has a chance of helping if your ranges are actually continuous ranges – if you have a starburst mask, encoding is more difficult because you need multiple ranges per row, and all that handling probably (?) won’t help performance.

Actually, looking at p5.js, it isn’t clear to me that using arrayCopy on pixels is even an option in p5.js right now – perhaps a p5.js expert can chime in…

https://github.com/processing/p5.js/search?q=arraycopy&type=Issues

sundftig · March 12, 2019, 10:54pm

Thank you so much! I completely understand now! Actually the pixel array is slightly more “complicated” then in processing (in my opinion) since the way the colours are considered is completely different.
In processing each array index (each pixel) contain all the colour and alpha informations, whereas in P5 you have those informations placed in order : index0 = red , index1 = green, index2 = blue, and index3 = alpha. I don’t know if this relate to the arrayCopy problem…but anyways, thanks a lot for all your help! I will check your link and I will post an update whenever I’ll find a way to improve performances.

Thanks!!

jeremydouglass · March 12, 2019, 10:58pm

In and of itself, an array with R,G,B,R,G,B,R,G,B that shouldn’t matter to an array copying strategy, as the per-pixels values are still continuous. So, instead of arrayCopy(100,200) you would do arrayCopy(100*3, 200*3) – assuming there is fast arrayCopy functionality at all!

Thanks for updating if you find anything!

sundftig · March 12, 2019, 11:13pm

I’ve just checked in the p5 reference page and apparently arrayCopy() (that apparently wouldn’t work for pixels[] anyways, see : https://goo.gl/B59NJ2 , line 54-56 ) is deprecated.

Sooo… I don’t know. We’ll see if I can come up with something else.
Thank you again.

GoToLoop · March 13, 2019, 6:17am

pixels[] is of datatype Uint8ClampedArray:

However, b/c the p5js API uses Array.isArray() in lotsa places:

We can say the library is pretty much incompatible w/ typed arrays!

Curiously, AFAIK, the only method that “welcomes” typed arrays within the p5js was contributed by moi:

github.com/processing/p5.js

Implements shuffle(arr, [bool])

processing:master ← GoToLoop:patch-1

opened 07:48PM - 09 Feb 15 UTC

GoToLoop

+44 -0

https://github.com/lmccart/p5.js/issues/513 https://github.com/lmccart/p5.js/tre…e/master/src/data/array_functions.js Randomizes the order of the elements of an array using "Fisher-Yates Shuffle Algorithm". By default, passed array argument isn't modified, unless setting a non-falsy 2nd parameter too. Otherwise, a shuffled **slice()** shallow clone array is returned.

github.com

processing/p5.js/blob/0.7.3/src/utilities/array_functions.js#L215-L231


      
          p5.prototype.shuffle = function(arr, bool) {
            var isView = ArrayBuffer && ArrayBuffer.isView && ArrayBuffer.isView(arr);
            arr = bool || isView ? arr : arr.slice();
          
            var rnd,
              tmp,
              idx = arr.length;
            while (idx > 1) {
              rnd = (Math.random() * idx) | 0;
          
              tmp = arr[--idx];
              arr[idx] = arr[rnd];
              arr[rnd] = tmp;
            }
          
            return arr;
          };

It checks ArrayBuffer.isView() 1st, so it doesn’t inadvertently invoke Array::slice() over a typed array:

Nowadays though, slice() is available for typed arrays as well.
But at that time when I was adding shuffle() to p5js it didn’t exist.

However today, I’d only check whether an object is an array-like container, rather than check it’s either a typed or an untyped array:

function isArraylike(obj) {
  return typeof obj === 'object' && obj !== null && obj.length >= 0;

However, p5.arrayCopy()'s incompatibility w/ typed arrays isn’t b/c it uses Array.isArray(), but b/c it relies on Array::splice() & Array::concat(), both of them unavailable for typed arrays:

github.com

processing/p5.js/blob/0.7.3/src/utilities/array_functions.js#L111


      
          
              start = 0;
              // rename  so we don't get confused
              dst = srcPosition;
              src = src.slice(0, end);
            }
          
            // Since we are not returning the array and JavaScript is pass by reference
            // we must modify the actual values of the array
            // instead of reassigning arrays
            Array.prototype.splice.apply(dst, [start, end].concat(src));
          };
          
          /**
           * Concatenates two arrays, maps to Array.concat(). Does not modify the
           * input arrays.
           *
           * @method concat
           * @deprecated Use <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/concat">arr1.concat(arr2)</a> instead.
           * @param {Array} a first Array to concatenate
           * @param {Array} b second Array to concatenate

Topic		Replies	Views
Does the mask-function behave differently from Processing to P5? Coding Questions	1	190	December 2, 2023
Add a "mask"/ "image" on top of these to work with a kind of cut out moving circle Libraries homework	6	383	January 4, 2023
someImage.mask (someMaskImage), not working (the mask is a createGraphic) Coding Questions	2	104	May 7, 2024
Masking a createGraphics object generates lag Coding Questions	3	374	January 27, 2021
How to use mask with animation loop? Coding Questions	0	296	February 17, 2022

Dynamic image masking, I made it! But how can I improve it?

Related Topics