Web Scraping code

Processing Coding Questions
2 
String page1 = "";
for (int i=0;i<ln.length;i++) {
  //println(i+" "+ln[i]);
    page1 = page1+=ln[i];
}

can be replaced with String page1 = join(ln,""); FYI

I look into your website. If you scroll down until you see the map on the left banner, you will notice there are two buttons right under the map: kml and xml. If you click on the xml button, you will see that it is the same url with an extra parameter: “&FcstType=dwml”

In short, you get an XML version. After a few iterations I get the snippet below.

I used an xml beautifier to make sense of the data. You can always try to hit the xml link and then right click on the page and click on View page source and your browser should be able to render the xml nicely for you.

Note there is not much info about the XML function in the Processing site. If you have code auto-completion working in your PDE, it will come handy this time.

By the way, check if the info I extracted contains the field of interest. You might need to explore the values of fields 13 and 15. Not sure if you will get Gale winds. You might need to work in the other XML fields.

Kf

XML xml = loadXML("https://forecast.weather.gov/MapClick.php?lat=42.3832&lon=-71.1018&unit=0&lg=english&FcstType=dwml");
//println(xml);

//FIRST level
XML[] c = xml.getChildren("data");
println("c len",c.length);
//for (int i = 0; i < c.length; i++) {
//  println(i+"\n",c[i]);
//}

//SECOND level
XML[] cc = c[1].getChildren("parameters");
//THIS next allowed me to identify fields 11,13 and 15 as fields of interest
//printArray(cc[0].listChildren());

printArray(cc[0].getChild(11));  //direction - see example at the end
printArray(cc[0].getChild(13));  //wind-speed - see example at the end
printArray(cc[0].getChild(15));  //wind-speed - see example at the end


//THIRD level
XML ccc=cc[0].getChild(11);
println("type:",ccc.getString("type"));
println("units:",ccc.getString("units"));
println("value:",ccc.getInt("value"));


println("Done");


////Example of positions 11,13,15
//
//<direction xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" time-layout="k-p1h-n1-1" type="wind" units="degrees true">  <value>300</value>  </direction>
//<wind-speed xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" time-layout="k-p1h-n1-1" type="gust" units="knots">  <value>NA</value>  </wind-speed>
//<wind-speed xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" time-layout="k-p1h-n1-1" type="sustained" units="knots">  <value>9</value>  </wind-speed>
2 Likes