Processing not writing on a file

Good morning, I am trying to save some data that my arduino is getting and save it into a file, the problem is the processing is not writing into the file, the data is correct and the processing is getting the data perfectly, the only problem is when it has to write into the file. The file is also created perfectly but is empty, here is my processing code:

import processing.serial.Serial;
import java.util.Scanner;
Serial myPort;

PrintWriter myfile;
String arduinoString = "NoValueYet"; // text received from the Arduino
int arduinoInt = 0; 
float arduinoFloat = 0.0;
// integer received from the Arduino
final color RED = color(255,0,0), GRAY = color(150), LIGHTGRAY = color(200);
color c = color(0);
int x = 0;
float y;

void setup() {
  size(400,400);
  printArray(Serial.list()); // print all port names
  String portName = Serial.list()[0];
  myPort = new Serial(this, portName, 9600);
  background(LIGHTGRAY);
  myfile = createWriter("data1.txt");
}

void draw() {
  /* storing data in myfile */
  myfile.println(arduinoString);
  myfile.println(arduinoInt);
  myfile.println(arduinoFloat);
  /* visualizing data in the Processing display window */
  stroke(c);
  y = map(arduinoInt, 0, 1023, 0, height);
  line(x, height, x, height-y);
  x = x + 1;
  if (x >= width) {
    x = 0;
    background(LIGHTGRAY);
  }

  delay(100);
}
void serialEvent (Serial myPort) {
  String s = myPort.readStringUntil('\n');
  if (s != null) {
    Scanner scanner = new Scanner(s);
    if (scanner.hasNextInt()) { 
      arduinoInt = scanner.nextInt();
      print(arduinoInt); // I added this file to check that the arduinoInt gets the value and it does
      myfile.print(arduinoString); // I added this to check if the problem is between serialEvent and draw, but is not, eventhough I add this line here the number is not saved in my file

    }
    else if (scanner.hasNextFloat()) { 
      arduinoFloat = scanner.nextFloat();
    }
    else if (scanner.hasNext("RED") || scanner.hasNext("YELLOW") || scanner.hasNext("GREEN") ) { 
      arduinoString = scanner.next();
      print(arduinoString);
    }
  scanner.close();
  } // end of if-statement with condition (s != null)
}
void keyPressed() {
  myfile.flush(); // writes remaining data
  myfile.close();
}
2 Likes

Obviously I cannot test it here

Observations

You don‘t have exit in keyPressed - at it is in the example.

https://processing.org/reference/PrintWriter.html

Also you define scanner every time - can’t you move this to setup () instead?

I’m suspicious about defining scanner every time, but haven’t tested.

Is the basic PrintWriter example working for you?
That would eliminate file permission / path problems as a culprit, and allow you to focus on code.

https://processing.org/reference/PrintWriter_print_.html

I suggest you explore previous arduino examples in the forum. To start, instead of saving to a file, you should print it in the console or within your sketch. Saving the data would be just an additional step.

Kf

i not see where you set that variable, how about

 myfile.print(s);

Did you manage to figure this one out, @begosustatx?

I’m facing the same problem. What I can add to the description in the OP is that I think the data does eventually get saved to the file (even without the keypress) or at least some of it does. So I’m suspecting that there might be some kind of write buffer that has to fill up before the data is actually written to the file. For obvious reasons, I don’t consider that a safe procedure for logging data, so what I’m trying to figure out is: how do I force progressing to save the data after every read (or after every tenth read, for that matter)?

Edit: So I guess I solved this adding myfile.flush(); to the loop. Not sure if there are reasons why you shouldn’t do this but at least the text appears in the file straight away… If someone tells me you shouldn’t be flushing at such a high frequency, my next idea would be to use millis() to flush every coupld of seconds (or whatever interval is suitable).

hi @tophee,
did you notice in the top post is a

void keyPressed() {
  myfile.flush(); // writes remaining data
  myfile.close();
}


next time better start your own topic and post your code…

yes, of course I did. But that code is only executed upon exit.

Did you notice that I said:

you mean
in the
draw(){}
you do a file write/save
( you understand processing can only complete replace a file / no OPEN FOR APPEND )
and that might be 60 times per second ( your FPS )

and i not say you shouldn’t, it is your drive ( i hope for you not a SSD drive )

  • but the timer would be one idea,
  • for this work ( Arduino data collection ) i use a RAM DISK (on RPI and WIN10 )
  • also a data reduction ( collect to array / save AVG to file ) might be good way
    all industrial process control systems work that way:
    • like 30min “current trend” for loop tuning, alarming …
    • forever ( see law for food industry ) “historic trend” 1 min resolution ( sample or averaged )
      using a tricky saving algorithm ( on delta )
1 Like

Yes.

Are you referring to this issue?

I am not sure if it is relevant here.

Yes, that’s kinda what I meant when I said:

But, of course, I didn’t just put the myfile.flush(); at the end of the loop so that it would be executed every single time. That would be nonsense. Instead, I put it right after the print command and the print command is only executed when data is received and that is only once every minute. So I’m flushing once per minute. I have no idea whether this is still to fast. The drive is a HDD.

I realize that my code is somewhat different from the OP example (which seems to be printing all the time), so I’m posting mine here in case someone finds it useful:

import processing.serial.*;

Serial mySerial;
PrintWriter output;
boolean first = true; 

void setup() {
   mySerial = new Serial( this, Serial.list()[0], 115200 );
   println("Waiting for data...");
   output = createWriter( "/srv/NAS/Arduino/data"+month()+"."+day()+"-"+hour()+"h"+minute()+".csv");
}

void draw() {
    if (mySerial.available() > 0 ) {
         String value = mySerial.readStringUntil('\n');
         if ( value != null ) {
              if (first) {first = false; // throw away first reading as it may be incomplete
              print(value);
              println("(not logged.)");
              println("Will start logging now...");
              }
              else {
              output.print(year()+","+month()+","+day()+","+hour()+","+minute()+","+second()+",");
              output.print( value );
              print(value);
              output.flush(); // Writes the data to the file
              }
         }
    }
}

void keyPressed() {
    output.flush();  // Writes the remaining data to the file
    output.close();  // Finishes the file
    exit();  // Stops the program
}

Note that the writing frequency is not evident from the code. It is determined by how often data arrives on the serial port and that is determined on the Arduino side.

Are you not worried to lose your precious data?

This is something I will have to look into. Thanks for the tip. Do you have any reference for how this is implemented on Arduino/processing? What does AVG stand for?

2 Likes

sorry ‘AVG’ a average, but if you worry data quality you might add like
AVG / MIN / MAX ( 3 data ) over a time period / of N samples…
as a data reduction.


ok, now as you say you get data every minute only ( and save a file ) ,
that takes the stress out of your hardware…

good luck with your project.


regarding your program structure ( and the wording confusion it caused for me )
https://processing.org/reference/libraries/serial/serialEvent_.html