# Paint on Webcam Stream

**URL:** <https://discourse.processing.org/t/paint-on-webcam-stream/44803>\
**Category:** Libraries\
**Created:** [July 23, 2024, 6:41pm UTC](https://discourse.processing.org/t/paint-on-webcam-stream/44803 "2024-07-23T18:41:27Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Atelierista](https://avatars.discourse-cdn.com/v4/letter/a/dbc845/32.png) [@Atelierista](https://discourse.processing.org/u/Atelierista)\
**Post date:** [July 23, 2024, 6:41pm UTC](https://discourse.processing.org/t/paint-on-webcam-stream/44803/1 "2024-07-23T18:41:27Z")

</div>

Hi there,

I need to write a small, straightforward program that allows children to draw into a webcam picture. I don’t have much time to do it, so I want to ask the pros before sinking valuable hours into code that doesn’t work or is just coded in a way that takes too much computing resources. (Also my girlfriend is gone for a couple days and I want the program to work when she’s back.)

I’m working on an exhibition where children can work with light, projectors, and projection surfaces. One of the “stations” we’ve been using combines a simple webcam stream and paint 3D, plus a digital drawing pad. Kids use it; it’s okay.

It has worked OK-ish, but I’m not happy with it. Right now, the projector screen is split in half; one half is the webcam, and the other half is the paint program. There are many unnecessary elements, and kids often don’t get to “experiment.” I have repeatedly proposed writing a small piece of software to superimpose whatever is drawn onto the webcam picture. (But the suggestion has been turned down because it’s “too much work.”) I don’t need fancy colors or tools; I have bought a small macro pad that I’d use to switch colors so that I can theoretically skip any GUI and give the drawing pad and the macro pad, and the kids can draw right into the camera stream.

I’ve tested the webcam stream in Processing, which works without issues. I bet I can also get the macro pad to work efficiently when switching colors.

So my question:  
What would be your general approach to writing this software?  
Can I use layers?  
Does something like this maybe already exist and I just haven’t found it?

Thanks for your ideas!!  
Best regards,  
Atelierista

---

<div class="post-metadata">

**Author:** ![Chrisir](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/chrisir/32/45_2.png) [@Chrisir](https://discourse.processing.org/u/Chrisir)\
**Post date:** [July 23, 2024, 7:06pm UTC](https://discourse.processing.org/t/paint-on-webcam-stream/44803/2 "2024-07-23T19:06:05Z")

</div>

> [@Atelierista](#):
>
> I’ve tested the webcam stream in Processing

so, you used p5.js?

And welcome to the forum!

---

<div class="post-metadata">

**Author:** ![Atelierista](https://avatars.discourse-cdn.com/v4/letter/a/dbc845/32.png) [@Atelierista](https://discourse.processing.org/u/Atelierista)\
**Post date:** [July 23, 2024, 7:51pm UTC](https://discourse.processing.org/t/paint-on-webcam-stream/44803/3 "2024-07-23T19:51:03Z")

</div>

Thank you!  
I’ve been using Processing 4.

for the webcam I used this code:

```auto
import processing.video.*;

Capture cam;

void setup() {
  size(640, 480);

  String[] cameras = Capture.list();
  
  if (cameras.length == 0) {
    println("There are no cameras available for capture.");
    exit();
  } else {
    println("Available cameras:");
    for (int i = 0; i < cameras.length; i++) {
      println(cameras[i]);
    }
    
    // The camera can be initialized directly using an 
    // element from the array returned by list():
    cam = new Capture(this, cameras[0]);
    cam.start();     
  }      
}

void draw() {
  if (cam.available() == true) {
    cam.read();
  }
  image(cam, 0, 0);
  // The following does the same, and is faster when just drawing the image
  // without any additional resizing, transformations, or tint.
  //set(0, 0, cam);
}

```

---

<div class="post-metadata">

**Author:** ![Atelierista](https://avatars.discourse-cdn.com/v4/letter/a/dbc845/32.png) [@Atelierista](https://discourse.processing.org/u/Atelierista)\
**Post date:** [July 23, 2024, 7:56pm UTC](https://discourse.processing.org/t/paint-on-webcam-stream/44803/4 "2024-07-23T19:56:45Z")

</div>

Interesting, I hadn’t even heard of p5.js

> **[Video Capture](https://p5js.org/examples/imported-media-video-capture/)**
>
> Display a live video feed from a camera.

does this also work offline?

---

<div class="post-metadata">

**Author:** ![jafal](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/jafal/32/19112_2.png) [@jafal](https://discourse.processing.org/u/jafal)\
**Post date:** [July 23, 2024, 8:10pm UTC](https://discourse.processing.org/t/paint-on-webcam-stream/44803/5 "2024-07-23T20:10:21Z")

</div>

Air drawing example just for learning

[https://funprogramming.org/150-Webcam-light-tracking-and-air-drawing.html](https://funprogramming.org/150-Webcam-light-tracking-and-air-drawing.html)

---

<div class="post-metadata">

**Author:** ![jafal](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/jafal/32/19112_2.png) [@jafal](https://discourse.processing.org/u/jafal)\
**Post date:** [July 23, 2024, 8:26pm UTC](https://discourse.processing.org/t/paint-on-webcam-stream/44803/6 "2024-07-23T20:26:57Z")

</div>

> [@Paint program using hands or touch](https://discourse.processing.org/t/paint-program-using-hands-or-touch/25213):
>
> Hey! I’m fairly new to processing and am trying to workout a project that I have in mind. I created a simple paint program where I can pick a color using the mouse and paint with that color in a rectangle. Now that is working fine, but I would like to be able to paint with my hands. Either on screen on which a projector projects the sketch and where I can paint on touch. Or have the webcam track my movement and paint like that. I just have no idea where to start and if the first option is even p…

---

<div class="post-metadata">

**Author:** ![glv](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/glv/32/18785_2.png) [@glv](https://discourse.processing.org/u/glv)\
**Post date:** [July 23, 2024, 11:07pm UTC](https://discourse.processing.org/t/paint-on-webcam-stream/44803/7 "2024-07-23T23:07:11Z")

</div>

Hello @Atelierista,

> [@Atelierista](#):
>
> I have repeatedly proposed writing a small piece of software to superimpose whatever is drawn onto the webcam picture. (But the suggestion has been turned down because it’s “too much work.”)

A simple example that will draw with the mouse on an offscreen buffer with a transparent background and superimpose it on the webcam image:

```auto
import processing.video.*;

Capture cam;

PGraphics pg;
boolean clear;

void setup() 
  {
  size(640, 480);
  pg = createGraphics(640, 480);

  String[] cameras = Capture.list();
  
  if (cameras.length == 0) 
    {
    println("There are no cameras available for capture.");
    exit();
    } 
  else 
    {
    println("Available cameras:");
    for (int i = 0; i < cameras.length; i++) 
      {
      println(cameras[i]);
      }
    
    // The camera can be initialized directly using an 
    // element from the array returned by list():
    cam = new Capture(this, cameras[0]);
    cam.start();     
    }      
    
  pg.beginDraw();
  pg.clear(); // Transparent background
  pg.endDraw();      
  }

void draw() 
  {
  if (cam.available() == true) 
    {
    cam.read();
    }
  
  pg.beginDraw();
  pg.strokeWeight(10);
  pg.stroke(mouseX*255.0/640, mouseY*255/480, 0);
  pg.point(mouseX, mouseY);
  pg.endDraw();
  
  // The following does the same, and is faster when just drawing the image
  // without any additional resizing, transformations, or tint.
  set(0, 0, cam);
  image(pg, 0, 0);
  }
  
void mousePressed()
  {
  pg.clear();
  }

```

References:

> **[Render Techniques](https://processing.org/tutorials/rendering/)**
>
> Tools for rendering geometries in Processing.

> **[PGraphics / Reference](https://processing.org/reference/pgraphics)**
>
> Main graphics and rendering context, as well as the base API implementation for processing "core". Use this class if you need to draw into an off-screen graphics buffer. A PGraphics object can be cons…

Drawing pad:

> [@Wacom Intuos drawing + pressure](https://discourse.processing.org/t/wacom-intuos-drawing-pressure/44590/2):
>
> Hello @lucnijs, I brushed the dust off of my Wacom Intuos (CTL-6100WL) and took it out for a spin! There is a Tablet library that seems to work: References to get me started: [Processing 1.0 - Processing Discourse - Using graphics tablet with JPen](https://forum.processing.org/beta/num_1252169588.html) \< JPen was a keyword I searched for [sdk - Wacom Integration into java - Stack Overflow](https://stackoverflow.com/questions/53067393/wacom-integration-into-java) \< I have not yet explored this! I need to work on my pen strokes! I will have to integrate it with my project: [GLV Paint A very simple dra…](https://discourse.processing.org/t/glv-paint-a-very-simple-draw-and-save-program/12669)

`:)`

---

<div class="post-metadata">

**Author:** ![Atelierista](https://avatars.discourse-cdn.com/v4/letter/a/dbc845/32.png) [@Atelierista](https://discourse.processing.org/u/Atelierista)\
**Post date:** [July 24, 2024, 11:04am UTC](https://discourse.processing.org/t/paint-on-webcam-stream/44803/8 "2024-07-24T11:04:38Z")

</div>

### You’re the man!

Seriously, thanks this will speed up development immensely. Girlfriend is coming back in a few hours and I feel like I’ll have something to show until then.

One question:  
This will probably just work in fullscreen and whatever resolution I’ll enter, right?  
Anything I should look out for?  
We’ve been using 1080p cameras to make use of the whole screen, but:

- I found that getting the aspect ratio right is more important than the resolution itself
- Better color rendering is better.

I’ve been looking at getting Sony IMX sensors in various packages, which tend to have the potential for better resolution, are all USB-OTG and UVC-compatible, and, first and foremost, come with nice C-mount packages. (Giving one more haptic feedback thing to play around with.)  
Any thoughts on that? Either way I’ll report back.

For reference, these are the ones I looked at. I’ll be testing with an IMX323 as a module with pinhole lens because it’s there,  
Sure! Here is an overview of the strengths and weaknesses of several Sony IMX sensors in a table format, including key specifications:

| **Sensor** | **Resolution** | **Low Light Performance** | **Max FPS (Full Res)** | **Dynamic Range** | **Strengths** | **Weaknesses** |
| --- | --- | --- | --- | --- | --- | --- |
| **IMX291** | 2 MP (1920x1080) | Excellent (0.01 Lux) | 60 fps | High | - Excellent low light performance  
- High sensitivity | - Lower resolution compared to newer sensors |
| **IMX323** | 2 MP (1920x1080) | Very Good (0.01 Lux) | 30 fps | High | - Good low light performance  
- Cost-effective | - Lower resolution  
- Moderate frame rate |
| **IMX335** | 5 MP (2592x1944) | Good | 30 fps | High | - Balanced performance  
- Good dynamic range | - Higher resolution can be taxing on older systems |
| **IMX298** | 16 MP (4656x3496) | Moderate | 30 fps | Moderate | - Very high resolution  
- Suitable for detailed imagery | - High data rate  
- Requires more processing power |
| **IMX415** | 8 MP (3840x2160) | Good | 60 fps | High | - High resolution with good low light performance  
- High frame rate | - Higher data rate and processing requirements |
| **IMX577** | 12 MP (4056x3040) | Good | 120 fps (1080p) | High | - High frame rate  
- Good resolution  
- Suitable for fast motion | - Higher cost  
- Requires substantial processing power |

---

<div class="post-metadata">

**Author:** ![Atelierista](https://avatars.discourse-cdn.com/v4/letter/a/dbc845/32.png) [@Atelierista](https://discourse.processing.org/u/Atelierista)\
**Post date:** [July 24, 2024, 11:12am UTC](https://discourse.processing.org/t/paint-on-webcam-stream/44803/9 "2024-07-24T11:12:37Z")

</div>

Also, another thing:  
I’ve used this in Processing 4 but also tried getting p5js to tun on VScode. Works! Not the same way I had in mind, but it’s baby steps from here to where I want to get. (Next step: change the color by tapping a key. Should be simple enough… )  
Am I correct that, theoretically, this would allow me to run the app on a website I can host (computing happening on the client side)?  
That could be really useful!