Wednesday 28 October 2009

Practical Example of Entropy

This article is the fruit of a discussion held between Maura Paterson, Paulo Cardoso and myself about Shannon's entropy in information theory.

Entropy, H(x), can be thought of as a way of measuring how difficult it is to get an answer correct by guessing. In other words, entropy represents the uncertainty value. If the next outcome of the challenge at hand is certain (we all know what it will be), then the entropy is 0 as there is no uncertainty.

The entropy H(x), where the value of x is selected randomly from the set {x1, x2, ..., xn} (of size n) can be calculates using the following formula:

where b is the number of symbols in the target alphabet and p(xi) is the probability mass function of outcome xi. In this article the target alphabet is the binary alphabet which only contains two symbols: {0, 1}

Why are there two ways to calculate entropy? The first formula is the general formula and suites all scenarios, while the second formula (logb(n)) is only used when the elements in the set x have equal probability to be selected (the selection is not biased)

Example 1: If I pick a weekday (Sunday, Monday, Tuesday, Wednesday, Thursday, Friday, Saturday) uniformly at random, then ask you to guess which day I picked, the best you can hope to do is guess randomly, and the entropy is log2(7) = 2.81. In this example we've used the simplest part of the formula (logb(n)) as all elements had the same probability.

Example 2: However, if I pick a weekday using the distribution that I pick Sunday with probability 1/2, and each other day with probability 1/12 then we can use the formula to compute the fact that the entropy of this distribution is 2.29, i.e. the uncertainty is less than in the previous case. This value is calculated as follows:

-( 1/2×log2(1/2)
   + 1/12×log2(1/12) 
   + 1/12×log2(1/12)
   + 1/12×log2(1/12)
   + 1/12×log2(1/12)
   + 1/12×log2(1/12)
   + 1/12×log2(1/12) )
=
 -( 1/2×log2(1/2)     
  + 6/12×log2(1/12) )
= 2.29

This makes sense, because in the last scenario your best strategy is to guess that I've picked Sunday and you will be correct much more often than in the previous case.

Conditional Entropy

If you think in terms of bits, you need (almost) three bits to represent all the weekdays. Example:

000 = Sunday
001 = Monday
...
110 = Saturday
111 is not used.

Example 3: We established that the entropy H(x) of the original distribution was log2(7) = 2.81. Learning that the first bit is 0 gives you extra information, since we now know it can only be Sunday, Monday, Tuesday or Wednesday. In this case the entropy H(x|the first bit of x is 0) is log2(4) = 2 (since there are four equiprobable possibilities remaining). The extra information has reduced the uncertainty.

The entropy H(x|the first bit of x is 1) is log2(3) = 1.58. Thus if x is the variable representing the day of the week and y is the variable representing the first bit of the encoding we have that the conditional entropy H(x|y) = 4/7× log2(4) + 3/7×log2(3) = 4/7× 2 + 3/7×1.58 = 1.822. The first half of the equation: 4/7×2 represents the option when the first bit is 0 while the second half, 3/7×1.58, when the first bit is 1. So we see that in this case H(x) > H(x|y), so we can conclude that learning the first bit of the encoding does give you some information about x (it reduces the uncertainty over the value of x).

In order to be correct you'd need to see three bits, unless the first two bits you saw were 11, in which case you'd know the answer was 110, since we are not using the binary string 111. In fact this depends on the encoding you are using for each of the possibilities (for example, we could take the above encoding and add 10 zeros in front of each triple to get a new, very wasteful, encoding. In this case you'd have to see 13 bits most of the time.) However, the value of the entropy gives a lower bound on the expected number of bits that will be required no matter what encoding is used.

Tuesday 13 October 2009

Practical Example of Simple Java UI Component

In this article we'll discuss how to create a simple Java graphical component such as a javax.swing.JButton (without going into details of event handling) and include it within an application. Our component will display (paint) a flat rectangle of a set colour as illustrated below.

We'll be making use of two classes one representing the simple custom component and the other will be mounting the frame and including our component in it. All classes discussed here are saved under the package "com.albertattard.cgui". You can change this to your own package. In this article we'll only be exploring the creation of simple custom UI component. Many other related topics will be only mentioned here but not discussed. While you are encouraged to explore all the topics that are only mentioned (but not discussed) here, one must be cautious not to overwhelm him/herself in the process.

Custom UI Component

All Java Swing (don't worry if you don't know with Swing means but if you insist here is a link which may answer some of your questions: http://java.sun.com/docs/books/tutorial/uiswing/) graphical components inherit from the javax.swing.JComponent abstract class. Our component is no exception. So let's start by creating a simple class called: FlatRectComponent. This class will be extending the JComponent class mentioned earlier and will override the paintComponent() method as highlighted in the example below.

package com.albertattard.cgui;

import java.awt.Color;
import java.awt.Graphics;
import javax.swing.JComponent;

@SuppressWarnings("serial")
public class FlatRectComponent extends JComponent {
  @Override
  protected void paintComponent(Graphics g) {
    // Clear the previous screen
    super.paintComponent(g);
    // Set the painting colour to magenta
    g.setColor(Color.MAGENTA);
    // Draw a solid rectangle at coordinates x and y set to 
    // 10px with width and height set to 100px
    g.fillRect(10, 10, 100, 100);
  }
}
Why did we override the paintComponent() method? All UI programs are responsible from painting themselves on the screen. For example this article you're reading is an image displayed on your screen (unless you printed it on a piece of paper). The browser downloaded this article as HTML and transformed (or better rendered) it as an image. All other applications do the same. For example a button, is a simple set of paint instructions which makes it look real (as you see it on the screen). Also events, such as mouse hovering, may change the way the button or any other graphical component looks making the graphics interacting with the user inputs.

In the case of Java, all graphical components are responsible of painting themselves by overriding the paintComponent() method and paint on the provided java.awt.Graphics. In the above example, we're simply painting a flat rectangle in magenta. Then this method is called by some other class which displays it on the screen as you see it. You don't have to worry about how or what invokes this method. Just remember that you cannot call it yourself.

Alone this class is not of much use. But since it's inheriting from the JComponent class, we can include it with other Java Swing components such as javax.swing.JFrame.

Including it in a JFrame

Let's create a simple frame which will host our new component. For simplicity, we're not going to discuss Layout Managers. We're going to use the absolute layout where we specify the location and size for every component. An example of our frame class is shown below.

package com.albertattard.cgui;

import javax.swing.JFrame;
import javax.swing.SwingUtilities;

@SuppressWarnings("serial")
public class FrameDemo extends JFrame {

  private static void createFrame() {
    JFrame frame = new FrameDemo();
    frame.setDefaultCloseOperation(EXIT_ON_CLOSE);
    frame.setTitle("Custom UI Demo");
    frame.setSize(600, 400);
    frame.setLocationRelativeTo(null);
    frame.setVisible(true);
  }

  public static void main(String[] args) {
    SwingUtilities.invokeLater(new Runnable() {
      @Override
      public void run() {
        createFrame();
      }
    });
  }

  private FrameDemo() {
    // Set the layout of this frame to absolute layout
    setLayout(null);

    // Create the flat rectangle component and add it to 
    // the frame
    FlatRectComponent comp = new FlatRectComponent();
    comp.setBounds(0, 0, 200, 200);
    add(comp);
  }
}
The above class contains two static methods and a constructor. Let's analyse these further starting from the main() method. The main() method is the first method called by the JVM. This is from where the application starts. This method calls the other static method createFrame() by scheduling a job on the AWT event dispatching thread. This sounds as complex as rocket science but you have nothing to be afraid from. For now all you have to do is copy and paste it. The static method createFrame() creates an instance of our frame class, FrameDemo and set the frame properties, such as title, size and location.

The constructor of this class starts by setting the layout manager to absolute layout. Then it creates an instance of our components and set its size and location accordingly. The setBounds() method takes four parameters, the first two represent the location as x and y coordinate while the latter to represent the size. Finally it adds this component to its container (don't worry about containers for now - this is the fort time I'm asking to accept something as is without having to dig any further). This is a very important step. Many forget to add the components and then wonder why these don't appear in the frame.

How many instances of our component can we include in the frame? As many as you want. In practice there's a limit which is dependent on the amount of memory available to your application, but you don't have to worry about that neither as our application is fairly simple and consume very little memory. Let's amend the previous frame class and include another instance of the FlatRectComponent UI component. All we need to do is create another instance and adding this instance to the frame's container. We also need to set the bounds of the new component such that this don't conflict with the previous one as otherwise we may have a graphical issue. The following code fragment highlights the changes made to the frame class. Only the constructor was changed, the other methods are unchanged and thus not shown here. Also, note that a border was added to the components to visualise the component perimeter.

  private FrameDemo() {
    setLayout(null);

    // The first component
    FlatRectComponent comp1 = new FlatRectComponent();
    comp1.setBounds(0, 0, 200, 200);
    comp1.setBorder(
      BorderFactory.createLineBorder(Color.BLACK, 1));
    add(comp1);

    // The second component
    FlatRectComponent comp2 = new FlatRectComponent();
    comp2.setBounds(250, 50, 200, 200);
    comp2.setBorder(
      BorderFactory.createLineBorder(Color.BLACK, 1));
    add(comp2); 
  }
The program should now display two, identical, components as illustrated below.

Customising the Custom UI Component

As is, our component is not much of use as we cannot customise it. The painComponent() method make use of literals and constants and so our component will always look like that. In order to be able customise it we need to add some states to our custom UI component.

What are these? States in objects are instance fields, or as some call them: class variables. The instance fields are used to save the state of the object. The state of the object is all information about the object. For example our custom UI component displays a rectangle. This rectangle needs to have an x and y coordinates and a size. A rectangle cannot exist without these states (properties). Try to draw a rectangle on a piece of paper yourself without making use of the mentioned states. You'll notice that you cannot as, as soon as you draw it, you have provided the x, y and size states. In this case we need to capture these states in some instance fields. The same applies for the rectangle colour.

We can start by adding an instance of the java.awt.Rectangle class and save our rectangle states (size and location) there. Alternatively, we can create four integer variables which represent the x, y, width and height values for our rectangle. Even though it may sound more complex I suggest we go with the first option as this give us other functionality already available in the Rectangle class, which otherwise we have to program, such as the translate() method.

package com.albertattard.cgui;

import java.awt.Color;
import java.awt.Graphics;
import java.awt.Graphics2D;
import java.awt.Rectangle;
import javax.swing.JComponent;

@SuppressWarnings("serial")
public class FlatRectComponent extends JComponent {

  private Rectangle rect = new Rectangle(10, 10, 100, 100); 

  @Override
  protected void paintComponent(Graphics g) {
    super.paintComponent(g);

    // Make use of the 2D version of the Graphics component
    Graphics2D g2D = (Graphics2D) g; 

    g2D.setColor(Color.MAGENTA);
    g2D.fill(rect); 
  }
}
In the above example we've changed the way the rectangle is painted. Before we used literals, while now we're painting an instance of Rectangle. Also, now we're using a java.awt.Graphics2D component instead of the Graphics component. We're not going to go into much detail about this here as otherwise we'll be ending up writing a book. But in a nutshell, the Graphics2D inherits from the Graphics and adds more functionality (methods) which we can use. The object invoking out paintComponent() method (which we agreed not to discuss here) will be passing an instance of java.awt.Graphics2D to the paintComponent() method. All we're doing is recasting it back to its original form. Don't worry about this for now. All you need to know is that you should cast your Graphics to Graphics2D.

So far we have not yet provided any means (methods) which allow the rectangle instance field, rect, to be changed. That is, there is no way how one can change the x, y and size values for this instance. Let's add four methods, which can be used to set the any of the rectangle states.

package com.albertattard.cgui;

import java.awt.Color;
import java.awt.Graphics;
import java.awt.Graphics2D;
import java.awt.Rectangle;

import javax.swing.JComponent;

@SuppressWarnings("serial")
public class FlatRectComponent extends JComponent {

  private Rectangle rect = new Rectangle(10, 10, 100, 100);

  @Override
  protected void paintComponent(Graphics g) {
    super.paintComponent(g);

    Graphics2D g2D = (Graphics2D) g;

    g2D.setColor(Color.MAGENTA);
    g2D.fill(rect);
  }

  public void setHeight(int height) {
    rect.height = height;
    repaint();
  }

  public void setWidth(int width) {
    rect.width = width;
    repaint();
  }

  public void setX(int x) {
    rect.x = x;
    repaint();
  }

  public void setY(int y) {
    rect.y = y;
    repaint();
  }
}
Note that at the end of every setXXX() method we're calling the repaint() method. All we're doing here is instructing the frame's container (this is the guy which is invoking our paintComponent() method) to repaint this component. The frame's container will in turn call the paintComponent() method for this component and providing the appropriate information. It is important to do so after every change as otherwise your changes will not immediately appear. But these will appear only when the frame completely repaints all its components. Remember that you cannot call the paintComponent() yourself. All you can do is call the repaint() method.

So far we've only provided a means to customise our component. But we have not yet customized it. Let's do that. We can customise our component by calling the setXXX() from the frame class as highlighted bold below. The following code fragment only shows the constructor of the frame class as the other methods are unchanged.

  private FrameDemo() {
    setLayout(null);

    // The first component
    FlatRectComponent comp1 = new FlatRectComponent();
    comp1.setBounds(0, 0, 200, 200);
    comp1.setX(5);
    comp1.setY(5);
    comp1.setWidth(190);
    comp1.setHeight(190); 
    comp1.setBorder(
      BorderFactory.createLineBorder(Color.BLACK, 1));
    add(comp1);

    // The second component
    FlatRectComponent comp2 = new FlatRectComponent();
    comp2.setBounds(250, 50, 200, 200);
    comp2.setX(50);
    comp2.setY(50);
    comp2.setWidth(100);
    comp2.setHeight(100); 
    comp2.setBorder(
      BorderFactory.createLineBorder(Color.BLACK, 1));
    add(comp2);
  }
The above program should produce the following outcome.

Note that now the coloured rectangle size and location can be varied from the setter methods provided. This makes our component customizable. We can also change the paint colour by including an instance field representing the colour and a method in our custom UI component which changes the colour as illustrated below.

// imports not shown here 
public class FlatRectComponent extends JComponent {
  private Color rectColour = Color.MAGENTA; 

  // other field and methods are not show for brevity

  @Override
  protected void paintComponent(Graphics g) {
    super.paintComponent(g);

    Graphics2D g2D = (Graphics2D) g;

    g2D.setColor(rectColour); 
    g2D.fill(rect);
  }

  public void setRectangleColour(Color rectColour) {
    this.rectColour = rectColour;
    repaint();
  }
}
All we need to do now is call this method and change the colour accordingly as highlighted below.

  private FrameDemo() {
    setLayout(null);

    // The first component
    FlatRectComponent comp1 = new FlatRectComponent();
    comp1.setBounds(0, 0, 200, 200);
    comp1.setX(5);
    comp1.setY(5);
    comp1.setWidth(190);
    comp1.setHeight(190);
    comp1.setRectangleColour(Color.GREEN); 
    comp1.setBorder(
      BorderFactory.createLineBorder(Color.BLACK, 1));
    add(comp1);

    // The second component
    FlatRectComponent comp2 = new FlatRectComponent();
    comp2.setBounds(250, 50, 200, 200);
    comp2.setX(50);
    comp2.setY(50);
    comp2.setWidth(100);
    comp2.setHeight(100);
    comp2.setRectangleColour(Color.CYAN); 
    comp2.setBorder(
      BorderFactory.createLineBorder(Color.BLACK, 1));
    add(comp2);
  }

The following screenshot shows the final version of our program.

Conclusion

In this article we've saw how to create a simple custom Java UI component and display a simple flat rectangle. We've also added methods which can be used to customise this component.

Monday 12 October 2009

Generic close method

All resources need to be released once ready from them. This is usually done by invoking a close or dispose method depending on the object. These methods are usually invoked form the finally block following the try and catch blocks to ensure that the resources are released. Also, these methods may throw an exception during the process which we rarely use. But if we don't capture the exception the finally block will fail and not fully complete, failing to release the following resources.

Let say we're copying form one stream to another. Then we need to close both streams at the end of the process as illustrated below.

package com.albertattard.utils;

import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;

public class CopyTwoStreams {

  public static void main(String[] args) throws IOException {
    InputStream in = null;
    OutputStream out = null;

    try {
      // copy code here
    } finally {
      in.close();
      out.close();
    }
  }

}

If an exception is thrown while closing the input stream, the output stream is never closed as the finally block will stop executing.

Handling Exceptions within the finally block

There are several ways how we can handle exceptions within the finally block. One alternative is to make use of a try and catch blocks but that may look bad (from a visual perspective). Alternatively, we can have a method which closes our streams and also absorbing any exceptions that may be thrown during the process.

As from Java 1.5, the java.io.Closable interface was introduce to the Java IO package and most (if not all) classes (and interfaces) in this package implement this interface. As you can see from the JavaDoc, this interface has only one method named close(). The good thing about this interface is that you can define one generic method which handles the close invocation. Following is a simple example:


public static IOException close(Closeable c){
  if(c != null) {
    try {
      c.close();
    } catch(IOException e) {
      return e;
    }
  }
  return null;
}

So now we can change our previous code to make use of this new static method. This method simply attempts the close a non-null closable object and returns null if either the object provided is null or it managed to close the object without any issues. Otherwise, the exception thrown in the process is returned (instead of propagated) to the caller.

package com.albertattard.utils;

import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;

public class CopyTwoStreams {

  public static void main(String[] args) throws IOException {
    InputStream in = null;
    OutputStream out = null;

    try {
      // copy code here
    } finally {
      close(in);
      close(out);
    }
  }

}

Any exceptions thrown during the first close are absorbed and only returned by the new static close() method. These will not disrupt the flow of the finally block and all streams' close() methods are invoked.

Unfortunately the above method cannot be used with classes and interfaces that do not implement or inherit from the Closable interface. Classes and interfaces outside the Java IO package such as the interface java.sql.Connection cannot benefit from such a method.

A generic approach

By using reflection, we can retrieve the close() method and dynamically invoke it. Our generic method can be used with anything that has a method called close which takes no parameters, just like the method defined by the Closeable interface. The advantage of this approach is that we don't have to handle different close objects with different methods which basically do the same thing - simple close the object and absorbing any exceptions thrown in the process.

  public static Exception close(Object o){
  if(o != null) {
    try {
      Class<?> clazz = o.getClass();
      Method closeMethod = 
          clazz.getDeclaredMethod("close", new Class[0]);
      closeMethod.invoke(o, new Object[0]); 
    } catch(Exception e) {
      return e;
    }
  }
  return null;
}

We had to make some changes to our original static close() method to make use of reflection (highlighted in the above example). In a nutshell our Java classes, fields and methods are represented by other Java classes which we can use and invoke. A more comprehensive explanation about reflection can be found in this article Reflection in Action.

Conclusion

This short article illustrated how to write a simple generic method that can be used to invoke the close() method and return any exceptions thrown in the process. This approach makes use of reflection which adds flexibility but also add performance costs. This is the slowest approach for closing an object.