I’ve got a question about processing video in Windows.
I’ve got an application that is connected to my motorcycle and shows real-time the throttle position and engine speed. This picture below is a typical view showing the throttle is 100% open.
I would like to go out for say a 1 hr ride, and then record the Android’s screen and have a 1 hour video file. Then, that video file will have my engine speed and throttle opening for the entire 1 hr. Maybe it would be better if I just took a series of images and then analyze those?
What I would like your feedback on, is how I would then process this 1 hour video file, into a heat map of throttle and exhaust position. I can work my way around excel no problem, but, what I need help for us getting the pixel data from the screen into some type of data table. The pixel is either black or not black…
Notice, on that graph, the lower white edge of the Throttle (%) could have a pixel equate to every percentage point, and then summarize how many frames in the total hour, are at that position. From there I can easily create a heat map in excel and perform analysis that way.
I’ve got programming experience but I’d rather no re-invent the wheel, and hope that there is a decently quick solution to this problem. Thanks for any input!
Hi @lucgallant
Why instead of recording the video, not just save the data in an array?
After that, you can display them in a sketch in the form of a video sketch, the way you want.
EDIT; Maybe I understood it wrong, and the app you are using is not your own but a third party and you record the app when working with a screen recording app. And you want to get certain pixels of the resulted video Is that it?
Sorry, yes, should have made that clear. The app is made by Yamaha for use with their engine control units. Ideally I’d prefer to have direct access to the data, but without putting in some effort in reverse engineering their datastream, I thought doing a video capture would be the best way. It’s pretty “ugly”, but I think it’d accomplish the goal.
A very crude way I was going to try is to just capture my cell phone’s screen (plenty of apps available), and then export every frame to pictures, and then just run a loop on all the files to get the EG and Throttle position on that… But that’s very ugly. Would be better to just process the video file directly…
I see.
On P4A PC the video lib won’t work. We have only the Ketai lib to extract and work on pixels in real time, but directly from the camera.
So you have to use the native Class MediaMetadataRetriever.
I can look into this tomorrow and write some code.
Tonight I will take say a 20 second video clip screen capture with my phone that shows exactly what real data looks like. What I will also do is describe in a table format what my ideal output looks like.
Really I don’t even care about relation to time, though that’s a nice feature, all I care about is just the raw data sample after sample, with both engine and throttle data.
And definitely this doesn’t have to be real time, as long as I can run my video through it after I get back from a ride and get the date of then that is what’s very important to me.
The reason this whole thing is a benefit is because the mapping software for the fuel injection for the engine control unit can be set and tuned based on engine RPM and throttle position. It is definitely good to know where you are operating to more finely tune the map at that position
I made every frame into an output file (for the crude/dirty/slow method)
Put a little word document together explaining what I’m after with pixel information
If anyone could give any hand or support that be great. If I’m left to it on my own I’d probably attempt something in C# and just loop through all the exported frames and attempt to collect the data that way… Pretty ugly and I’ll probably spend 12 hours trying to do it whereas someone here can probably do it in 15 mins…
Thanks in advance!
EDIT: Also I noticed this got moved to Processing for Android. To me I am much more inclined to take a video and process it in Windows, versus real time on Android. If it’s any more work to do in Android then I definitely wouldn’t ask that. If it’s easier to do in Android then could be an attractive thing, but this app is also available on iOS, so a solution on PC would be better suited, I feel.
When I saw the video I noticed immediately that the app “pauses” sometimes.
I suppose this is due to the recording app. So I started to write my own recording code, but in android it’s far more complex then I thought. Thus to give a code anyway, I wrote code as the video is, in P5.
To simplify I did a resize and a crop online here, but you can of course do it directly in the complete movie, given the right value to the ‘search’ variable. If you chose the right pixels value, you can even get the numeric values like Battery, etc. You can download the cropped video I used here.
After running the first code, you copy the data.txt in its data folder and paste it in the next display code folder. Also, use the data printed in the console for the next sketch.
I’ve changed the topic category because you’ll get answers more easily if it’s not Android. (But feel free to modify)
import processing.video.*;
Movie m;
PImage img;
int w = 252; // The video resolution
int h = 58;
int i;
int search = 40; // "y height" of line where you want to start to read
String str = "";
ArrayList<Integer> intgs;
void setup() {
size(252, 58, P2D); // w & h
line(0, search, width, search);
intgs = new ArrayList<Integer>();
m = new Movie(this, "sample.mp4");
m.play();
}
void draw() {
m.read();
m.loadPixels();
for (int y = 0; y < h; y++ ) {
for (int x = 0; x < w; x++ ) {
int loc = x + y * w;
if (y == search) {
color c = m.pixels [loc];
if(c != -16777216) i++; // black
}
}
}
intgs.add(i);
i = 0;
m.updatePixels();
image(m, 0, 0);
float mt = m.time();
float duration = m.duration();
if (mt > duration-1) {
m.stop();
println("Finished in "+mt+" seconds "+" with "+frameCount+" frames");
String[] list = new String[intgs.size()];
for (int i = 0; i < intgs.size(); i++) {
int temp = intgs.get(i);
list[i] = str(temp);
}
saveStrings("data/data.txt", list);
exit();
}
}
void movieEvent(Movie m) {
m.read();
}
Display code:
String[] lines;
String str = "";
int c, old_c;
int time_c = 48; // needed to display correct time
void setup() {
size(2368, 200); // frames = 2368 captured in other sketch
background(255);
lines = loadStrings("data.txt");
fill(0);
textAlign(CENTER);
for(int i = 0; i < lines.length; i++){
if(i % time_c == 0) { // duration in seconds
line(i, height-25, i, height-45);
text(str(i/time_c), i, height-8);
}
if(i % 4 == 0) line(i, height-25, i, height-35); // duration in settings
str = lines[i];
c = Integer.valueOf(str);
line(i, height-60-c, i, height-60-old_c);
old_c = c;
}
}
The app definitely does “pause”, and it’s definitely not from the capturing software (one would think). Even without the capturing software, there are the occasional pauses there. This app gets data over wi-fi from the engine control unit, so I guess there is occasional lost data.
I’ve got the Processing software and I’ve imported the video library, so I’ll give this a try for sure and get what I want out of it.
I definitely appreciate the time spent, and once I create my heat map in excel, I’ll give some feedback so you can actually see what I am doing with the data.
