Tuesday 30 June 2009

Practical Example of GSON

Please note that this page has moved to: http://www.javacreed.com/simple-gson-example/.

GSON is a Java API, developed by Google, used to convert between Java objects and JSON objects. This article discusses and provides examples about this API and how it can be used. More information about this API can be found at: http://sites.google.com/site/gson/.

This is the first from three articles about GSON. No GSON or JSON experience is required as this article acts as a primer for the other two articles. The second article provides more examples about the use of GSON deserializer (from JSON to Java) and the third and final article provides more examples about the GSON serializer (from Java to JSON). All code for all three articles is available at: http://code.google.com/p/gson-practical-examples/source/checkout.

Download and Install

Before you can do any work with this API, you need to download the library (jar file) and include it in the classpath. The library, together with the source and JavaDocs, can be downloaded from: http://code.google.com/p/google-gson/downloads/list. Once downloaded, add the gson-<version>.jar to the classpath. For those readers who prefer to use maven to manage the dependencies (JAR files), add the following dependency to the pom.xml.

    <dependency>
      <groupId>com.google.code.gson</groupId>
      <artifactId>gson</artifactId>
      <version>2.2.1</version>
    </dependency>

Change the <version>2.2.1</version> as required. All code examples shown in this article use the version listed above. A copy of the pom.xml file can be found here.

If this library is to be used in a web application, make sure to have a copy saved under the WEB-INF/lib folder. Alternatively, the library can be added to the application server and made available to the web application.

A Simple Example

The GSON API provides a stateless class, Gson, that handles the conversions between Java and JSON objects. An instance of this class can be created by invoking the default constructor or as shown in the simple example below, using the GsonBuilder class. The GsonBuilder class provides customisation and allows the developer to instantiate Gson as required.

package com.albertattard.examples.gson.part1_1;

import com.google.gson.Gson;
import com.google.gson.GsonBuilder;

public class SimpleExample1 {
  public static void main(String[] args) {
    Gson gson = new GsonBuilder().create();
    gson.toJson("Hello", System.out);
    gson.toJson(123, System.out);
  }
}
In the above example, we've created an instance of Gson and converted the Java String and int to JSON objects. The above code produces the following as its output to the command prompt:
"Hello"123
It's not rocket science, but it's a start. Note that the output of the above is all going to the command prompt. The toJason() method takes two parameters, the Java object to be converted to JSON and an instance of Appendable. We can easily change the out to a file or network stream.

package com.albertattard.examples.gson.part1_1;

import java.io.FileWriter;
import java.io.IOException;
import java.io.Writer;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;

public class SimpleExample2 {
  public static void main(String[] args) throws IOException {
    Writer writer = new FileWriter("Output.json");

    Gson gson = new GsonBuilder().create();
    gson.toJson("Hello", writer);
    gson.toJson(123, writer);

    writer.close();
  }
}
Why is the variable declared as a Writer when the actual type is FileWriter?
It is a good practice to have your variables as generic as possible. In the above example, we're only using methods defined by the Appendable and Writer interfaces. Having the variable type more specific than required will make the code less portable and harder to maintain as we'll see in the following example.

Note that in the above example we are not handling the steams (Writer) properly. Ideally the resources are closed within the finally block or used within the try-with-resources as shown below. We've ignored this to keep the code as simple as possible.


  public static void main(String[] args) throws IOException {
    try (Writer writer = new FileWriter("Output.json")) {

      Gson gson = new GsonBuilder().create();
      gson.toJson("Hello", writer);
      gson.toJson(123, writer);
    }
  }

The above code produces the file: Output.json with the JSON objects. Note that here we used the character streams and not the byte streams. We cannot use the byte streams as the toJson() method is expecting an Appendable instance and the byte streams are not descendants from the Appendable interface. The Appendable interface works with characters and not bytes. Java provides the InputStreamReader and the OutputStreamWriter classes that convert byte streams into character streams as illustrated in the following example.


package com.albertattard.examples.gson.part1_1;

import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStreamWriter;
import java.io.Writer;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;

public class SimpleExample3 {

  public static void main(String[] args) throws IOException {
    Writer writer = new OutputStreamWriter(
            new FileOutputStream("Output.json"));

    Gson gson = new GsonBuilder().create();
    gson.toJson("Hello", writer);
    gson.toJson(123, writer);

    writer.close();
  }
}
As you can see, we only needed to change the instantiation part. The rest (expect for the imports) of the code is unchanged.

Consuming JSON objects

Let's say that we need to consume JSON objects and load them as Java objects. Assume that a web server will produce the following JSON when queried:

{
  NAME:"Albert Attard",
  P_LANGUAGE:"Java",
  LOCATION:"Malta"
}
This JSON object contains three fields with their respective values. Let's say that we need to consume the JSON object and create a Java object that represents this data. To make this example more interesting, let assume that we're only interested from the name and the location fields.

First we need to create a Java class with the fields that we want to represent (name and location). Let's call the class Person1. The name of this class is irrelevant, but the name of the fields is not. The field names must match (including the case) with the names in JSON. Also, the class must include a default constructor. As shown below, the fields name and location are in uppercase as found in JSON. The JSON field P_LANGUAGE is ignored as the Java object does not include a field with this name. It is understandable that the fields' names do not follow the Java naming convention, but for the time being let's keep things simple. More about this is discussed in part 2.


package com.albertattard.examples.gson.part1_2;

public class Person1 {
  private String NAME;
  private String LOCATION;

  // Getters and setters are not required for this example.
  // GSON sets the fields directly.

  @Override
  public String toString() {
    return NAME + " - " + LOCATION;
  }
}
With the Java object ready, we can read the JSON objects and load them as Java objects as illustrated below. To simulate a real life situation, we're using a byte stream as input with default encoding. Also note that the JSON content is saved into a file (which is not usually the case) located in the same directory of the following class.

package com.albertattard.examples.gson.part1_2;

import java.io.IOException;
import java.io.InputStreamReader;
import java.io.Reader;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;

public class JsonToJava1 {

  public static void main(String[] args) throws IOException {
    Reader reader = new InputStreamReader(JsonToJava1.class
        .getResourceAsStream("Server1.json"));

    Gson gson = new GsonBuilder().create();
    Person1 p = gson.fromJson(reader, Person1.class);
    System.out.println(p);

    reader.close();
  }
}
This should produce: Albert Attard - Malta. Gson parsed the JSON object and created an instance of the Person1 class.

Nested JSON Objects

Let's take the above example one step further and include a nested object as illustrated in the following JSON code fragment.

{
  NAME:"Albert Attard",
  P_LANGUAGE:"Java",
  LOCATION:"Malta",
  EXAM: {
    SUBJECT:"Programming",
    GRADE:4.5
  }
}
Here we have an EXAM field which is made from two other fields: SUBJECT and GRADE. Likewise, we need to modify the Person1 class defined above to include the EXAM field and create a new Java class to represent the SUBJECT and GRADE fields.

We first create the new class that will represent the nested object. As we discussed before, the class name is irrelevant but the fields' names must match those define in JSON.


package com.albertattard.examples.gson.part1_3;

public class Exam1 {
  private String SUBJECT;
  private double GRADE;

  // Getters and setters are not required for this example.
  // GSON sets the fields directly.

  @Override
  public String toString() {
    return SUBJECT + " - " + GRADE;
  }
}
Now we can modify the Person1 class (Person2 in the following example) and include a new field with the same name as in JSON of type Exam1 as shown next. Instead we created a new class to keep track of the progress made.

package com.albertattard.examples.gson.part1_3;

public class Person2 {

  private String NAME;
  private String LOCATION;
  private Exam1 EXAM; 

  @Override
  public String toString() {
    return NAME + " - " + LOCATION + " (" + EXAM + ")";
  }
}
Note that the changes required are minimal as Gson dynamically discovers (through reflection) the class and its fields. This article does not cover reflection. For more information about reflection please refer to: Reflection in Action.
Finally, let's test the new changes.

package com.albertattard.examples.gson.part1_3;

import java.io.IOException;
import java.io.InputStreamReader;
import java.io.Reader;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;

public class JsonToJava2 {

  public static void main(String[] args) throws IOException {
    Reader reader = new InputStreamReader(JsonToJava2.class
        .getResourceAsStream("Server2.json"));

    Gson gson = new GsonBuilder().create();
    Person2 p = gson.fromJson(reader, Person2.class);
    System.out.println(p);

    reader.close();
  }
}
Instead of using person 1, we're using the new class which include the new exam field.

Conclusion

Even though it may be a new concept, JSON is very simple and straight forward. The Gson API makes it very simple to use and even though not discussed here, it provides a great deal of flexibility.

For more GSON examples please visit the second part (part-2), where we explore more complex examples and discuss how to use the GSON deserializer to take full control over the deserialization process.

Thursday 18 June 2009

PreparedStatement and NULLs in the WHERE Clause

The java.sql.PreparedStatement comes with a method for setting nulls as values for the parameters: setNull(int, int). This method is very tricky and does not work as expected when it comes to the SQL WHERE clause.

This article talks about a common pitfall that may developers fall in and never test. More details about the prepared statement interface can be found at http://java.sun.com/docs/books/tutorial/jdbc/basics/prepared.html

The Problem

Consider the following query SELECT * FROM EMPLOYEES WHERE MANAGED_ID = ?. This query returns all employees for a given manager id. The following code fragment illustrates how this query is used to retrieve all employees for manager id 100.


  PreparedStatement statement = connection.prepareStatement(
      "SELECT * FROM EMPLOYEES WHERE MANAGED_ID = ?");
  statement.setLong(1, 100L);
  ResultSet resultSet = statement.executeQuery();

The above code works well for any manager id. But what if we would like to select all employees that do not have a manager, that is, the manager id column is set to NULL (SQL NULL)? If we do the following we will get nothing back, even if there are employees with no manager assigened.


  PreparedStatement statement = connection.prepareStatement(
      "SELECT * FROM EMPLOYEES WHERE MANAGED_ID = ?");
  statement.setNull(1, Types.INTEGER);
  ResultSet resultSet = statement.executeQuery();

Why? The advantage of the prepared statement is that the statement is compiled once and it is executed many times. This provides performance benefits but comes with a cost. That is, once the statement is compiled, the only things that can change are the parameters' values. The equals sign stays there. This instead of having our SQL statement WHERE MANAGER_ID IS NULL we have WHERE MANAGER_ID = NULL, which does not yield the same results.

The Solution

The solution is very simple. We have to modify the string version of the SQL statement to handle NULLs. Assume we have the following method that returns a list of employees for a given manager id, where a connection is already established and available to the same method.


  public List getEmployeesWithManagerId(Long managerId)
                                        throws SQLException {
    List employees = new ArrayList();
    String select = "SELECT * FROM EMPLOYEES WHERE MANAGER_ID "
                    + (managerId == null?"IS NULL":"= ?");
    PreparedStatement statement = 
        connection.prepareStatement(select);
    if(managerId != null){
      statement.setLong(1, 100L);
    }
    ResultSet resultSet = statement.executeQuery();
    while(resultSet.next()){
      // Get the employee details and add it to the list
    }
    return employees;
  }

Why are we still using the prepared statement? Can we use the select statement instead and concatenate the values? Yes you can but not suggested as the prepared statement also provides protection against SQL injections.

Conclusion

Even though very simple, this is a common bug which can be easily iron out with proper testing. Since the database columns can accept NULLs, refrain from using Java primitives and use their wrappers instead. Note that the result set getXXX methods return the primitive values which can be tricky too. A 0 (zero - the default value for most of the primitives) is returned if the database field was NULL. In that case make sure to use to the method wasNull().

Thursday 4 June 2009

From XML to Java using SAX Parser

XML is a common way to save data especially while in transit between applications. For example, a program used to register students (written by a company) needs extract/load data produced by another program written by another company (may be in a different language). One of the best ways to do this is by having the source program producing an XML file and the destination program loading this file.

This article discusses how the Java SAX parser can be used to load an XML file as a List of Java objects.

The problem

Assume we need to develop a program that read the following simple XML file and loads it as a list of Java objects. The file is called Test.xml.


<students>
  <student name="Albert Attard">
    <class>Java</class>
    <class>Math</class>
    <grade>65</grade>
  </student>
  <student name="Mary Borg">
    <class>English</class>
    <grade>93</grade>
  </student>
  <student name="Joe Vella">
    <class>Math</class>
    <class>English</class>
    <grade>47</grade>
  </student>
  <student name="Paul Galea">
    <class>Math</class>
    <class>Maltese</class>
    <grade>52</grade>
  </student>
</students>

The above XML file contains a list of four students, each student having a name, a list of classes that he/she will attend and their grade. The student name is an attribute of the student tag while the class and grade are inner tags of the student tag.

Loading data from XML file

There are various ways how we can load data from XML files in the Java language, each have cons and prons. Two common ways are DOM and SAX. What are these and why we're using one and not the other? The main difference between the two is that the DOM parser loads the XML file in a tree structure using DOM related classes. Once loaded, we can then create our structure from the DOM tree. On the other hand SAX parser does not load the XML but triggers a series of events thought which we can build our structure. SAX does not load anything into the memory.

The SAX parser

Before proceeding, let's first understand the SAXParser and the required elements. The SAX parser requires a handler to handle the events triggered by the same SAX parser. The handler is a java class that extends the DefaultHandler and provides implementation for some (or many) of the methods in the default handler. Note that the default handler implements a set of interfaces and provides a default implementation (methods doing nothing) for all inherited abstract methods.

SAX Parser

The SAX parser is created using the SAXParserFactory and the XML file is parsed using the provided handler (the default handler in this case) as illustrated in the following example:


import javax.xml.parsers.SAXParser;
import javax.xml.parsers.SAXParserFactory;
import org.xml.sax.helpers.DefaultHandler;

public class Example1 {
  public static void main(String[] args) throws Exception {
    DefaultHandler handler = new DefaultHandler();
    
    SAXParserFactory factory = SAXParserFactory.newInstance();
    SAXParser p = factory.newSAXParser();
    p.parse(Example1.class.getResourceAsStream("Test.xml"), 
            handler);
  }
}

Note that, here I'm using the default package for all examples and the Test.xml file is in the same folder as the class files. If you move the classes into a package make sure to also move the xml file with them or emend the file path accordingly.

Executing the above example will produce nothing as the default handler simply ignores all events triggered by the parser. The SAX parses starts parsing the file and for every tag opened and close it invokes specific methods in the handler and pass the information from the XML file. Let's create a simple handler and some basic methods to helps us understand this better. This handler will be used to load a list of students from the XML file define above (Test.xml).


import org.xml.sax.Attributes;
import org.xml.sax.SAXException;
import org.xml.sax.helpers.DefaultHandler;

public class StudentHandler1 extends DefaultHandler {

  @Override
  public void characters(char[] ch, int start, int length)
                         throws SAXException {
  }

  @Override
  public void endElement(String uri, String localName, 
                         String name) throws SAXException {
  }

  @Override
  public void startDocument() throws SAXException {
  }

  @Override
  public void startElement(String uri, String localName, 
                           String name,Attributes attributes) 
                           throws SAXException {
  }
}

The above class override four methods from the default handler class.

The startDocument method is invoked once when the SAX parser start parsing the XML document. This method has not parameters and can be used to initialise fields (similar to a constructor or initialising block). The default handler also includes the endDocument method which is invoked when the SAX parser is done parsing the file. This method can be used as a destructor method (or the final block) to clean up or wrap up and resources/fields as required.

The startElement method is invoked when the SAX parser encounters an XML tag. For example, this method is invoked when the SAX parser encounters an open tag such as: <students>. Similar, the endElement method is invoked by the parser when the close tag is found (for example: </students>).

Finally, the characters method is invoked when the parser encounters the tag body. The tag body is the text (not XML) within the tags. For example, the characters for the grade XML tag: <grade>85</grade>, is 85. Note that whitespaces are not removed or truncated and must be handled by the handler.

Simple example

Let's start with a simple example that counts the number of students in the XML file (Test.xml). All we need to do here is create an integer field, initialise it to 0 in the start document method and for every student tag we increment this variable. We removed the unnecessary method from the previous handler (StudentHandler1 ) and added the end document method.


import org.xml.sax.Attributes;
import org.xml.sax.SAXException;
import org.xml.sax.helpers.DefaultHandler;

public class StudentHandler2 extends DefaultHandler {

  private int studentsCount;

  @Override
  public void endDocument() throws SAXException {
    System.out.println(studentsCount);
  }

  @Override
  public void startDocument() throws SAXException {
    studentsCount = 0;
  }

  @Override
  public void startElement(String uri, String localName, 
                           String name, Attributes attributes) 
                           throws SAXException {
    if ("student".equalsIgnoreCase(name)) {
      studentsCount++; // Increment statement
    }
  }
}

Note that we enclosed the increment statement within the if statement. The if statement is checking the name of the XML tag. Note that our XML document has four different tags: students, student, class and grade. We only want to increment our counter when the student tag is opened. Also, the if statement is comparing the tag but ignoring case (case insensitive) as XML is not case sensitive and the tags may be in upper case.

Now, using the new handler (StudentHandler2), we can process our XML file (Test.xml) and see how many students we have.


import javax.xml.parsers.SAXParser;
import javax.xml.parsers.SAXParserFactory;

public class Example2 {
  public static void main(String[] args) throws Exception {
    StudentHandler2 handler = new StudentHandler2();

    SAXParserFactory factory = SAXParserFactory.newInstance();
    SAXParser p = factory.newSAXParser();
    p.parse(Example2.class.getResourceAsStream("Test.xml"), 
            handler);
  }
}

The above should produce 4 when executed against the Test.xml file.

As you can see, we didn’t loaded any data from the XML file into the memory. Instead we only performed the require operation (counting the number of student in this case). Using DOM here would be an overkill as all the students would have been loaded into the memory for nothing. We only required a count. I'm not saying that DOM is not good. All I'm saying is that DOM is not the right tool for this job.

Some XML processing

Let's improve our parser and calculate the average grade for all students. In order to calculate the average, we need to first calculate the sum and then divide by the number of students. This is not as simple as it sound as the SAX parser invokes the handler's methods independent from the tag. For example, the startElement method is invoked for every XML open tag. Same applies for all other handler methods. Thus we have to keep track which tag we're handling.

Let's understand this problem first. We need to get the contents between the grade's XML tag. This can be retrieved from the characters method. But this method is executed for every tag (as we established above). So we first need to see which tag is being processed from the start element method. Using a boolean field (referred to as addToTotal in the following example), we set this field to true when the start element method is invoked for the grade tag. Then, we only process characters when this field is set to true. We have to remember to set this field to false once the grade tag is processed. This can be done in the end element method. All this is captured in the following example. Note that changes are shown in bold.


import org.xml.sax.Attributes;
import org.xml.sax.SAXException;
import org.xml.sax.helpers.DefaultHandler;

public class StudentHandler3 extends DefaultHandler {

  private boolean addToTotal;
  private StringBuffer buffer;
  private int studentsCount;
  private int totalGrade;

  @Override
  public void characters(char[] ch, int start, int length)
      throws SAXException {
    if (addToTotal) {
      buffer.append(ch, start, length);
    }
  }

  @Override
  public void endDocument() throws SAXException {
    System.out.println(studentsCount);
    System.out.println(totalGrade * 1.0 / studentsCount);
  }

  @Override
  public void endElement(String uri, String localName, 
                         String name) throws SAXException {
    if ("grade".equalsIgnoreCase(name)) {
      addToTotal = false;
      totalGrade += Integer.parseInt(buffer.toString().trim());
    }
  }

  @Override
  public void startDocument() throws SAXException {
    studentsCount = 0;
    totalGrade = 0;
  }

  @Override
  public void startElement(String uri, String localName, 
                           String name, Attributes attributes) 
                           throws SAXException {
    if ("student".equalsIgnoreCase(name)) {
      studentsCount++;
    } else if ("grade".equalsIgnoreCase(name)) {
      addToTotal = true;
      buffer = new StringBuffer();
    }
  }
}

Why are we using a string buffer in the characters method? The XML tag body (where the grade value is) can be very long and spread across multiple lines. For example, we can have the following:


<grade>
65
</grade>

In this case, the character method will be invoked three times (one for every line). Initially, it will be called with the new-line UNICODE symbol (\n), then with the text 65, and finally with the other new-line UNICODE symbol. Thus we first need to accumulate all content (in this case: \n65\n) and then remove all leading and trailing whitespaces before parsing it into an integer. Note that our example will throw a NumberFormatException is the given grade is not an integer.

Handling XML attributes

The XML student tag also includes an attribute, the student name. We can get this value and from the start element method's Attributes parameter. The attribute parameter holds all attributes that belong to the tag being handled. The SAX parser takes care of populating the attributes when parsing the XML document.


import org.xml.sax.Attributes;
import org.xml.sax.SAXException;
import org.xml.sax.helpers.DefaultHandler;

public class StudentHandler4 extends DefaultHandler {

  @Override
  public void startElement(String uri, String localName, 
                           String name, Attributes attributes) 
                           throws SAXException {
    if ("student".equalsIgnoreCase(name)) {
      String studentName = attributes.getValue("name");
      System.out.println(studentName);
    }
  }
}

The above handler will list all student names one following another. We can enhance this handler and save all students and their grade in a list as illustrated below. Ideally, before we proceed we create a Java class that represents the student. In this case the class only requires two fields, that is, the name and the grade. Note that in our problem we're not handling the student's class XML tag (we're ignoring it).


public class Student1 {

  private int grade;
  private String name;

  // The getters and setters are omitted for brevity

  @Override
  public String toString() {
    return name + " " + grade;
  }
}

Putting it all together

Let's now combine everything together and build a list of student from the XML file. We need to change some of the fields and introduce new ones. For example, we need a list to put all the students in (listOfStudents) and we need a temporary variable to save the student until this is added into the list (student). No changes are highlighted in the following example as there are many changes.


import java.util.List;
import java.util.Vector;
import org.xml.sax.Attributes;
import org.xml.sax.SAXException;
import org.xml.sax.helpers.DefaultHandler;

public class StudentHandler5 extends DefaultHandler {

  private StringBuffer buffer;
  private boolean isStudentGrade;
  private List listOfStudents;
  private Student1 student;

  @Override
  public void characters(char[] ch, int start, int length)
      throws SAXException {
    if (isStudentGrade) {
      buffer.append(ch, start, length);
    }
  }

  @Override
  public void endDocument() throws SAXException {
    System.out.println(listOfStudents);
  }

  @Override
  public void endElement(String uri, String localName, 
                         String name) throws SAXException {
    if ("grade".equalsIgnoreCase(name)) {
      isStudentGrade = false;
      student.setGrade(
        Integer.parseInt(buffer.toString().trim()));
    } else if ("student".equalsIgnoreCase(name)) {
      listOfStudents.add(student);
      student = null;
    }
  }

  @Override
  public void startDocument() throws SAXException {
    listOfStudents = new Vector();
  }

  @Override
  public void startElement(String uri, String localName, 
                           String name, Attributes attributes) 
                           throws SAXException {
    if ("student".equalsIgnoreCase(name)) {
      String studentName = attributes.getValue("name");
      student = new Student1();
      student.setName(studentName);
    } else if ("grade".equalsIgnoreCase(name)) {
      isStudentGrade = true;
      buffer = new StringBuffer();
    }
  }
}

Why the student is added into the list when the student close tag is processed when we could do this at the grade close tag? The reason behind that is if in the future we also add more fields, such as the student's class, we can easily do so with minimal changes to the code.

Conclusion

In this article I covered how to use the SAX parse for simple processing of XML files. The SAX parser is ideal when we need to perform some processing such as count the number of students or calculate the average grade without having to load the entire XML file into the memory.