HTTP GET command

Jeff Lockyear · November 2012

I'm having trouble getting XML data from a website. I've written various weather modules that use the same method, but in this instance I'm having some trouble. I can get the data via my browser with no problem, but when I connect to the site via my code and issue a GET with the URL I receive a "XML data not available at this URL" in my receive buffer.

Is it possible that the website is looking for a browser or mobile device and doesn't see my master as either? Is there an alternative to the GET command?

IP_CLIENT_OPEN (dvDevice.Port,'webservices.nextbus.com',80,1))

ONLINE: SEND_STRING dvDevice, "'GET /service/publicXMLFeed?command=predictions&a=ttc&stopId=4205',$0a"

Thanks,

Andrew G Welker · November 2012

You might need to give the unit/web server more information about what is requesting the information than you are right now. I would suggest using Wireshark to get the entire conversation between your browser and the web server, then adding pieces of the header until you get the data you are looking for.

jweather · November 2012

Jeff Lockyear wrote: »

SEND_STRING dvDevice, "'GET /service/publicXMLFeed?command=predictions&a=ttc&stopId=4205',$0a"

At a minimum your GET command needs to look like:

SEND_STRING dvDevice, "'GET /service/publicXMLFeed?command=predictions&a=ttc&stopId=4205 HTTP/1.1',$0a,$0d,'Host: webservices.nextbus.com',$0a,$0d,$0a,$0d"

That works for me, so you shouldn't need any extra headers.

Jeff Lockyear · November 2012

That worked great, thanks guys.

HARMAN_icraigie · November 2012

Encapsulate something like this in a Duet Utility Device module. Pass the urlToRead in as a parameterized command that passes back each line by default or optionally saves the return value as a file. The same module instance can be used for all http access requirements limiting the number of IP port devices needing to be defined.

The Duet version just worked without much fussy-ing about which I can't say about the Netlinx version I had developed previously.

   public String getHTML(String urlToRead) {
      URL url;
      HttpURLConnection conn;
      BufferedReader rd;
      String line;
      String result = "";
      try {
         url = new URL(urlToRead);
         conn = (HttpURLConnection) url.openConnection();
         conn.setRequestMethod("GET");
         rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
         while ((line = rd.readLine()) != null) {
            result += line;
         }
         rd.close();
      } catch (Exception e) {
         e.printStackTrace();
      }
      return result;
   }

HTTP GET command

Comments

Categories