Using Headers for loading Images

Hello,

I would like to use the discog API in order to get informations on music albums and the cover Image of the album.

The program I wrote handles the API correctly for fetching the String datas, but I have a problem with loading the Images from the URL provided in the JSON sent back by the Discog API:
The URL works fine in a browser but the server returns a 403 error when I try to load the image with processing using the same URL.

here is the code I would like to see working:

String searchString;
JSONObject json;

PImage cover;

void setup()
{
  searchString = ("https://api.discogs.com/database/search?q=Nirvana%20nevermind");
  testdiscogs();
}

void draw(){
  
}

void testdiscogs(){
  JSONObject json;
  int masterID = 0;
  
  println("test discogs");
  try{
    println(searchString);
    GetRequest get = new GetRequest(searchString);
    get.addHeader("Authorization", "Discogs key="+discogsConsumerKey+", secret="+discogsConsumerSecret);
    get.addHeader("user-agent","omnijukebot/0.0 bcrepet@yahoo.fr");
    get.send();
    json = parseJSONObject(get.getContent());
    JSONArray results = json.getJSONArray("results");
    if(results.size()>0){
      println("Au moins un master trouvé");
      masterID = results.getJSONObject(0).getInt("master_id");
      println("master_id:" + masterID);
      cover = requestImage(results.getJSONObject(0).getString("cover_image"));
  }
  catch (Exception e) {
    println("probleme test discogs e="+e);
  }
}

I asked for help in the Discop API forum, and I have been answered that the problem probably comes from the fact that I don’t send the necessary informations (Authorization and User-Agent) in headers when trying to load the image.

I know what to send in the headers but after searching for a while, I have not found any obvious solution for sending headers with loadImage() or requestImage(), or a library that could do the job. Now I am really stuck with that…

Does someone knows how to do this with processing?

1 Like

Understood.

By the way:

What gives println (results.getJSONObject(0).getString(“cover_image”));

println (results.getJSONObject(0).getString(“cover_image”); gives:

https://img.discogs.com/UoDAnoi3Cbp_JvOrz4CRgZBBSfg=/fit-in/600x592/filters:strip_icc():format(jpeg):mode_rgb():quality(90)/discogs-images/R-367084-1263095553.jpeg.jpg

Copied and pasted in a browser address bar, this shows the expected cover.

But a loadImage() right after this println() gives:
java.io.IOException: Server returned HTTP response code: 403 for URL: https://img.discogs.com/UoDAnoi3Cbp_JvOrz4CRgZBBSfg=/fit-in/600x592/filters:strip_icc():format(jpeg):mode_rgb():quality(90)/discogs-images/R-367084-1263095553.jpeg.jpg
… and a lot of other red lines…

I can’t help you with that

it’s not a straight forward, direct image url which looks more
like https://img.zeit.de/politik/deutschland/2021-07/sommerpressekonferenz-angela-merkel-bundeskanzlerin-bilder/2006-gettyimages-71746021.jpg/imagegroup/original__960x720__desktop

or just https://img.zeit.de/politik/deutschland/2021-07/sommerpressekonferenz-angela-merkel-bundeskanzlerin-bilder/2006-gettyimages-71746021.jpg

anyway, when the guys from discop are correct, either check out where the images are (when the folder is not protected) OR use GetRequest again, no?

I don’t understand what means checking where the images are. How can I do that with the URL I have?
Even trying to “cut” this URL - for example “https://img.discogs.com/UoDAnoi3Cbp_JvOrz4CRgZBBSfg=/fit-in/600x592/” this gives no results neither with a browser nor with processing.

However using GetRequest again with the URL gave something: A very long gibberish String that I can’t use as is but this is a step forward!

Do you think this String could be the image itself?

I recorded it in a .txt file here if you want to have a look. I tried to convert it to image with some online tools but did not manage to get an image.

If it is the image and if it is possible to convert it, is there a way to do that with processing?

I don’t know

Please check with the forum

Hey @Glenton I dont know if this helps:
Its a little class which downloads an Image and then loads it via loadImage() from processing

Click to see the my Example
String URL = "https://img.discogs.com/UoDAnoi3Cbp_JvOrz4CRgZBBSfg=/fit-in/600x592/filters:strip_icc():format(jpeg):mode_rgb():quality(90)/discogs-images/R-367084-1263095553.jpeg.jpg";
PImage img;
void setup() {
  size(100, 100);
  //img = requestImage(URL);//not working
  //img = loadImage(URL);//not working

  DownloadImage downImg = new DownloadImage(this);
  img = downImg.load(URL, sketchPath("test.png"));

  if (img == null) {
    println("Error");
    return;
  }
  println("Image loaded");
  surface.setSize(img.width, img.height);
}
void draw() {
  background(0);

  if (img == null) return;
  image(img, 0, 0);
}

import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.URL;
import java.net.URLConnection;


public class DownloadImage {
  PApplet parent;

  public DownloadImage(PApplet parent) {
    this.parent = parent;
  }
  public PImage load(String search, String path) {

    // This will get input data from the server
    InputStream inputStream = null;

    // This will read the data from the server;
    OutputStream outputStream = null;

    try {
      // This will open a socket from client to server
      URL url = new URL(search);

      // This user agent is for if the server wants real humans to visit
      String USER_AGENT = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36";

      // This socket type will allow to set user_agent
      URLConnection con = url.openConnection();

      // Setting the user agent
      con.setRequestProperty("User-Agent", USER_AGENT);

      // Requesting input data from server
      inputStream = con.getInputStream();

      // Open local file writer
      outputStream = new FileOutputStream(path);

      // Limiting byte written to file per loop
      byte[] buffer = new byte[2048];

      // Increments file size
      int length;

      // Looping until server finishes
      while ((length = inputStream.read(buffer)) != -1) {
        // Writing data
        outputStream.write(buffer, 0, length);
      }
    } 
    catch (Exception ex) {
      ex.printStackTrace();
    }

    // closing used resources
    // The computer will not be able to use the image
    // This is a must

    try {
      outputStream.close();
      inputStream.close();
    } 
    catch (IOException e) {
      e.printStackTrace();
    }


    PImage img = parent.loadImage(path);
    return img;
  }
}
Click to see the Example in your code
String searchString;
JSONObject json;

PImage cover;

void setup()
{
  searchString = ("https://api.discogs.com/database/search?q=Nirvana%20nevermind");
  testdiscogs();
}

void draw() {
  if (cover == null) return;
  image(cover, 0, 0);
}

void testdiscogs() {
  JSONObject json;
  int masterID = 0;

  println("test discogs");
  try {
    println(searchString);
    GetRequest get = new GetRequest(searchString);
    get.addHeader("Authorization", "Discogs key="+discogsConsumerKey+", secret="+discogsConsumerSecret);
    get.addHeader("user-agent", "omnijukebot/0.0 bcrepet@yahoo.fr");
    get.send();
    json = parseJSONObject(get.getContent());
    JSONArray results = json.getJSONArray("results");
    if (results.size()>0) {
      println("Au moins un master trouvé");
      masterID = results.getJSONObject(0).getInt("master_id");
      println("master_id:" + masterID);
      DownloadImage downImg = new DownloadImage(this);
      String url = results.getJSONObject(0).getString("cover_image");
      cover = downImg.load(url, sketchPath("test.png"));
    }
  }
  catch (Exception e) {
    println("probleme test discogs e="+e);
  }
  if (cover == null) return;
  surface.setSize(cover.width, cover.height);
}

import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.URL;
import java.net.URLConnection;


public class DownloadImage {
  PApplet parent;

  public DownloadImage(PApplet parent) {
    this.parent = parent;
  }
  public PImage load(String search, String path) {

    // This will get input data from the server
    InputStream inputStream = null;

    // This will read the data from the server;
    OutputStream outputStream = null;

    try {
      // This will open a socket from client to server
      URL url = new URL(search);

      // This user agent is for if the server wants real humans to visit
      String USER_AGENT = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36";

      // This socket type will allow to set user_agent
      URLConnection con = url.openConnection();

      // Setting the user agent
      con.setRequestProperty("User-Agent", USER_AGENT);

      // Requesting input data from server
      inputStream = con.getInputStream();

      // Open local file writer
      outputStream = new FileOutputStream(path);

      // Limiting byte written to file per loop
      byte[] buffer = new byte[2048];

      // Increments file size
      int length;

      // Looping until server finishes
      while ((length = inputStream.read(buffer)) != -1) {
        // Writing data
        outputStream.write(buffer, 0, length);
      }
    } 
    catch (Exception ex) {
      ex.printStackTrace();
    }

    // closing used resources
    // The computer will not be able to use the image
    // This is a must

    try {
      outputStream.close();
      inputStream.close();
    } 
    catch (IOException e) {
      e.printStackTrace();
    }


    PImage img = parent.loadImage(path);
    return img;
  }
}
2 Likes

Yes Floflo, this helps. You just totally solved my problem!
Thank you a lot! I have been stuck for a while on this…

I am just a processing user, I don’t really know Java.
I wish I could understand how your class works, especially what is this PApplet parent object and how it gets the image data with inputStream, outputStream an the buffer…
But I will dig on this later.

Thanks again!

2 Likes

That’s nice :smiley:

So the PApplet class is the base class for all sketches that use processing.core.
You can look at this link: PApplet

And I only looked for and adjusted the rest online together, I didn’t fully understand that im sorry :sweat_smile:

1 Like