Home >Blog

HTTP Programming With Java

Working with HTTP is a common task for professional Java developers. Almost all the applications developed nowadays will have some kind of interaction with the web. No matter if Java programming is your hobby or your profession, skill on working with HTTP(S) is essential. In this article I will show you how to work with HTTP/HTTPS in Java.

Unlike other languages Java does not depend on external libraries for working with HTTP. Again, it does not have any package called http or something like that. It's not that there are no third party packages or libraries to work with HTTP or HTTPS—it has some great third party libraries to work with it. But, today I am going to teach you how to work with HTTP without any third party package or library.

Java always provides us with the most convenient way of doing something good, sometimes even a magic way. In a previous article I discussed Understanding URLs for programmers and then, in another article, I showed you how to work with URLs in Java. After you have constructed a URL, you can can call a method on it called openConnection(). This method will return an object of URLConnection. An instance of URLConnection is the object that we need work with various aspects of HTTP(S). We can also cast that object to HttpURLConnection for using more specific HTTP stuff.

Create a Java project in your favorite IDE and create a public class called JavaHTTP where our main function will reside. At first we will construct a URL object. We are discarding MalformedURLException by putting throws Exception with the main method. Now, call the openConncetion() method on the URL object to get an instance of URLConnection. Your code should look like the following:

import java.net.*;

public class JavaHTTP {
   public static void main(String[] args) throws Exception{
       URL url = new URL("http://example.com");
       URLConnection con = url.openConnection();
   }
}

Now, build and run the project. You should not see any errors at this point.

We have our URL connection at hand and now we want an object of InputStream. To get the InputStream you should call getInputStream() on the URLConnection object. Also, you need to import the java.io package to work with InputStream.

import java.net.URL;
import java.net.URLConnection;
import java.io.InputStream;

public class JavaHTTP {
   public static void main(String[] args) throws Exception{
       URL url = new URL("http://example.com");
       URLConnection con = url.openConnection();
       InputStream inp = con.getInputStream();
   }
}

This time I did not import classes with wildcard import, but instead I imported every class specifically to demonstrate things more precisely and without ambiguity. We need the InputStram to get the contents from the connection. We can read every byte from the InputStream. The bytes are returned as integers. We need to cast them to char to show them on the screen. We read every byte by running a while loop and whenever the read method returns -1, we stop the loop. The code inside the main method should look like the following block of code.

URL url = new URL("http://example.com");
URLConnection con = url.openConnection();
InputStream inp = con.getInputStream();

int i;
while( ( i = inp.read() ) != -1 ){
   System.out.print((char) i);
}

If you are connected to the internet, you should see a result like below.

<!doctype html>
<html>
<head>
   <title>Example Domain</title>

   <meta charset="utf-8" />
   <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
   <meta name="viewport" content="width=device-width, initial-scale=1" />
   <style type="text/css">
   body {
       background-color: #f0f0f2;
       margin: 0;
       padding: 0;
       font-family: "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif;

   }
   div {
       width: 600px;
       margin: 5em auto;
       padding: 50px;
       background-color: #fff;
       border-radius: 1em;
   }
   a:link, a:visited {
       color: #38488f;
       text-decoration: none;
   }
   @media (max-width: 700px) {
       body {
           background-color: #fff;
       }
       div {
           width: auto;
           margin: 0 auto;
           border-radius: 0;
           padding: 1em;
       }
   }
   </style>    
</head>

<body>
<div>
   <h1>Example Domain</h1>
   <p>This domain is established to be used for illustrative examples in documents. You may use this
   domain in examples without prior coordination or asking for permission.</p>
   <p><a href="http://www.iana.org/domains/example">More information...</a></p>
</div>
</body>
</html>

So, we have successfully retrieved an URL and printed the HTML source code on the screen. It’s as simple as that. We just retrieved the content of the URL with the HTTP GET method. The default method that is used is GET.

To use HTTP(S) specific things we can cast the URLConnection object to HttpURLConnection. When you request some resources from a server with HTTP(S), the server sends some meta information as headers along with a status code. To get the headers we are going to invoke the getHeaderFields() method on the HttpURLConnection object. To get the HTTP status code, we invoke the method getResponseCode(). HTTP(S) 200 code means that everything is alright.

import java.net.URL;
import java.net.URLConnection;
import java.net.HttpURLConnection;
import java.io.InputStream;

public class JavaHTTP {
   public static void main(String[] args) throws Exception {
       URL url = new URL("http://example.com");
       URLConnection con = url.openConnection();
       InputStream inp = con.getInputStream();
       HttpURLConnection hcon = (HttpURLConnection) con;

       System.out.println(hcon.getResponseCode());
   }
}

The output we get on the screen is 200. Now, get all the headers.

import java.net.URL;
import java.net.URLConnection;
import java.net.HttpURLConnection;
import java.io.InputStream;
import java.util.Map;
import java.util.List;

public class JavaHTTP {
   public static void main(String[] args) throws Exception {
       URL url = new URL("http://example.com");
       URLConnection con = url.openConnection();
       InputStream inp = con.getInputStream();
       HttpURLConnection hcon = (HttpURLConnection) con;

       Map<String, List<String>> headers = hcon.getHeaderFields();

       for (String headerKey: headers.keySet()) {
           List<String> headerValues = headers.get(headerKey);
           if (headerValues.size() > 1){
               for (String headerValue: headerValues) {
                   System.out.print(headerKey + " : ");
                   System.out.println(headerValue);
               }
           }else{
               System.out.print(headerKey + " : ");
               System.out.println(headers.get(headerKey).get(0));
           }
       }
   }
}

The output should look like:

X-Cache : HIT
Server : ECS (rhv/818F)
Etag : "359670651+ident"
Cache-Control : max-age=604800
Vary : Accept-Encoding
Last-Modified : Fri, 09 Aug 2013 23:54:35 GMT
Expires : Tue, 08 Aug 2017 09:23:07 GMT
Content-Length : 1270
Date : Tue, 01 Aug 2017 09:23:07 GMT
Content-Type : text/html

I have excluded the first line of the result as that shows the status message of the HTTP response. There are other methods to get the same information but I have used getHeaderFields() for a quick and easy demonstration for you.

When we want to download a file, then how do we do that?

We want to download the Java logo from WikiMedia. The URL of the logo is https://upload.wikimedia.org/wikipedia/en/thumb/3/30/Java_programming_language_logo.svg/300px-Java_programming_language_logo.svg.png.

import java.io.*;
import java.net.URL;
import java.net.URLConnection;

public class JavaHTTP {
   public static void main(String[] args) throws Exception {
       URL url = new URL("https://upload.wikimedia.org/wikipedia/en/thumb/3/30/Java_programming_language_logo.svg/300px-Java_programming_language_logo.svg.png");
       URLConnection con = url.openConnection();
       InputStream inp = con.getInputStream();
       FileOutputStream out = new FileOutputStream("java-logo.png");

       int i;
       while( (i = inp.read()) != -1 ){
           out.write(i);
       }
       out.close();
   }
}

Look at the current working directory and you will see a PNG image file named java-logo.png. Open it with a photo viewer to verify.

The HTTP GET method is used to retrieve data. It can also be used to send data to the server with the help of a query string. But a query string cannot carry large amount of data. It is also not a safe method of transferring data. The most common use of the GET method for forms is various search engine search forms. As a consequence the search engine result page URLs are URLs with query string. For example, google with http methods. The search result page URL looks like https://www.google.com/search?q=http+methods. It's useful to use query strings for search result pages, because we can share the URL with the terms we used for searching. But, if we do not not want to expose the data in the URL, keep the URL clean or transfer larger amounts of data, we must use POST method when sending the request to the web server.

Sending HTTP POST requests with URLConnection or HttpURLConnection is not hard. It just requires a few more method calls to setup the connection to send data with the POST method. Without using a URLConnection object directly, let's use HttpURLConnection. So, again we need casting of URLConnection to HttpURLConnection. For testing our POST request I want to use httpbin.org's HTTP endpoints. I am going to use http://httpbin.org/post endpoint for testing our HTTP POST requests. This endpoint will return the sent data back along with some other data as JSON.

Cast the URLConnection to HttpURLConnection:

URL url = new URL("http://httpbin.org/post");
HttpURLConnection con = (HttpURLConnection) url.openConnection();

Tell the HttpURLConnection that the current request method is HTTP POST request method:

URL url = new URL("http://httpbin.org/post");
HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setDoOutput(true);
con.setRequestMethod("POST");

We are going to send the post data as application/x-www-form-urlencoded. We have to set the Content-Type as HTTP header:

URL url = new URL("http://httpbin.org/post");
HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setDoOutput(true);
con.setRequestMethod("POST");
con.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");

We need to prepare the data to be sent:

String name = "Java";
String suffix = "Land";
String data = String.format("name=%s&suffix=%s",
       URLEncoder.encode(name, "UTF-8"),
       URLEncoder.encode(suffix, "UTF-8")
);

To send the data over the connection use the following code:

con.getOutputStream().write(data.getBytes("UTF-8"));

Now, the data is sent over the connection. We can verify it by reading the data:

InputStream inp = con.getInputStream();
int i;
while( ( i = inp.read() ) != -1 ){
   System.out.print((char) i);
}

If we put the parts of the puzzle together, we get:

import java.io.*;
import java.net.URL;
import java.net.HttpURLConnection;
import java.net.URLEncoder;

public class JavaHTTP {
   public static void main(String[] args) throws Exception {
       // Setup the connection
       URL url = new URL("http://httpbin.org/post");
       HttpURLConnection con = (HttpURLConnection) url.openConnection();
       con.setRequestMethod("POST");
       con.setDoOutput(true);
       con.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");

       // Prepare data
       String name = "Java";
       String suffix = "Land";
       String data = String.format("name=%s&suffix=%s",
               URLEncoder.encode(name, "UTF-8"),
               URLEncoder.encode(suffix, "UTF-8")
       );

       // Write data to connection
       con.getOutputStream().write(data.getBytes("UTF-8"));

       // Reading data
       InputStream inp = con.getInputStream();
       int i;
       while( ( i = inp.read() ) != -1 ){
           System.out.print((char) i);
       }
   }
}

Hit build and run the program to see the following output:

{
 "args": {}, 
 "data": "", 
 "files": {}, 
 "form": {
   "name": "Java", 
   "suffix": "Land"
 }, 
 "headers": {
   "Accept": "text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2", 
   "Connection": "close", 
   "Content-Length": "21", 
   "Content-Type": "application/x-www-form-urlencoded", 
   "Host": "httpbin.org", 
   "User-Agent": "Java/1.8.0_05"
 }, 
 "json": null, 
 "origin": "103.242.219.100", 
 "url": "http://httpbin.org/post"
}

Look at this part of the data to verify that we got what we sent:

"form": {
   "name": "Java", 
   "suffix": "Land"
 },

So, now we have successfully completed our POST journey with URLConnection or HttpURLConnection. There are more intermediate and advanced topics that we are going to cover in future lessons. Remember, one thing that we did not do in this article is handle various exceptions. We bypassed all of them by the base exception Exception. Be sure to handle all the IOException, NullPointerException and other exceptions in your production app.

By Md. Sabuj Sarker | 8/25/2017 | General