Create image from interaction with website

is it possible through processing, I connect to a website for example: google, and through my interaction on the google page, for example clicks and scroll, generate a graphic, abstract image in processing?


You can get information from the mouse with Robot (Java Platform SE 8 ), but I think that it will be very hard to do what you want with a browser.

You might want to use browser-side telemetry generation, like a Chrome extension, a Tampermonkey script, or a third-party tool, and then send clicks and coordinates to your sketch from the browser, rather than trying to collect data on the mouse from the sketch through the operating system.

Hi Fabiana, welcome to the forum.
I would do this with the help of AutoHotkey, which is C++ wrapper designed for this kind of things. It’s an easy to use language that can also interact in the background with Processiing.
If your gonna try, please let us know the results.:slight_smile:

1 Like

Yes, I want to generate an abstract image through the user’s interactivity with a website, I don’t necessarily need more technical info. just an abstract image. I don’t know much or almost nothing about code. I thought that there could be software that could generate an image within those parameters.


1 Like

If you can wait some time I’ll write a code in AutoHotkey that captures mouse clicks and, if you wish, also mouse-click-dragging and than saves the data in a file.
This data can then easily be used in a processing program and draw an art sketch like above.

1 Like

Hi @Fabiana
Sorry. I almost forgot my promise to post the Auto hotkey code, to capture mouse clicks and drags. But here it is. You can download the very small (30 Mb) Auto hotkey ide here. It installs easily and once finished, right-click on the desktop and choose ‘New \ Autohotkey Script’ Paste the code and save it. By double-clicking the icon, a message box will explain how to start and end, recording. The script will be running in the background., and when ended two files, mouse_clicks and mouse_drags will be saved in C: hard drive. If you don’t want to install AHK you can download an executable of the script here.
I also wrote a simple Processing sketch for you to start with. The image is not exactly an artwork, it’s just meant to be an example.

MsgBox,1, , To start recording press F1`nTo end and save the
and program press F2`nThe files will be saved in C:\MouseEvents

Allowed = A

FileAppend, %PCX%~%PCY%, C:\MouseEvents\mouse_click.txt
FileAppend, %PDX%~%PDY%, C:\MouseEvents\mouse_drag.txt

block = 0
if (Allowed != %A%){
  while GetKeyState("LButton", "D") { 
      If((SX != X) || (SY != Y)){
        PDX = %PDX%,%X%
        PDY = %PDY%,%Y%
        mouseDrag = D
      } else { 
        if(block = %0%){
          PCX =  %PCX%,%X% 
          PCY =  %PCY%,%Y%
          block = 1
  if(mouseDrag != %D%) {
    PDX = %PDX%,!
    PDY = %PDY%,!
    mouseDrag =


 //  Special random array with lower possibility off picking random high numbers for values up to 500
//  The table was made by a program you can find in this topic:
String[] c_x, c_y, split_dx, split_dy;
int t;

void setup() {
  size(1200, 600);

  // Extracting click coordinates from txt file into arrays and draw them
  String clicks[] = loadStrings("C:/mouseEvents/mouse_click.txt");
  String str_c = clicks[0];    // The one text srting devided in first half x values, second half y values
  String[] c_xy = split(str_c, "~");     // The one text srting devided in first half x values, second half y values
  c_x = split(c_xy[0], ",");    // The x value clicks array
  c_y = split(c_xy[1], ",");    // The y value clicks array

  // Extracting drag coordinares into arrays and draw them
  String drags[] = loadStrings("C:/mouseEvents/mouse_drag.txt");
  String str_d = drags[0];
  String[] d_xy = split(str_d, "~");          // The one text srting devided in first half x values, second half y values
  String[] d_x = split(d_xy[0], "!");         // The x value drags array
  String[] d_y = split(d_xy[1], "!");         // The y value drags array
  for (int i = 0; i < d_x.length; i++) {      // For each drag...  ( total number of drags)
    split_dx = split(d_x[i], ",");            // The x value array
    split_dy = split(d_y[i], ",");            // The y value array  
    for (int t = 1; t < split_dx.length-1; t++) {   // Draw the drags
      line(float(split_dx[t]), float(split_dy[t]), float(split_dx[t+1]), float(split_dy[t+1]));
   // Clicks drawn as circles with diameter values out of RandomTable below
  for (int i = 0; i < c_x.length-1; i++) {
    int rnd = num[int(random(0, 974))];
    fill(random(0, 255), random(0, 255), random(0, 255));
    circle(float(c_x[i]), float(c_y[i]), rnd/15);
    fill(255, 10);
    circle(float(c_x[i]), float(c_y[i]), rnd/2);


1 Like