Oops Null Pointer

Java programming related

ExtGWT Cookbook Review in progress

I’m in the process of reviewing a GXT book called ExtGWT Rich Internet Application Cookbook.

Hopefully I’ll pick up a few tips on the way. It will be interesting to compare it to a book a did a technical review for around a year ago: Ext GWT 2.0 Beginners Guide.

XStream 1.3.1 to 1.4.3 ReflectionConverter changes

Recently while upgrading to Java 7 I had to upgrade XStream due to (I think) Oracle changing the name of the JVM or reflection providers. My converters that subclass the ReflectionConverter class started to fail. I had been omitting fields and then manually marshalling them in by overriding the marshall and marshallFields methods.

The 1.3.1 version of the RelectionConverter (in AbstractConverter) passed all non transient fields to the marshallField method. But in 1.4.3 it now checks if the fields should be omitted and will not pass omitted fields through to the marshallField method.

While mine was a relatively unusual case I hope this post can help others out stuck on similar issues.

C# Error Serialising IList

Recently I hit an error trying to Serialize a IList (with the actual type being an ArrayList) in C#. The only error I could get was:

System.InvalidOperationException: There is an error in XML document

which was not particularly helpful. As far as I can tell the System.Xml.Serialization.XmlSerializer can’t serialise IList as the root node, but has no issues if the IList is a child node.

I found one workaround here that uses a IXmlSerializable wrapper for IList.

(I tried to use a z in serialise when it refers to a method, but couldn’t bring myself to break with Australian English for other uses!)

EasyMock Generic(ish) Collection Matcher

I wanted to check that a collection is basically the same as another without knowing the collection type. I can only really check that the same elements are present, like a basic set.

The code below is based on the code here but with generics where possible (the matches method is not generic in EasyMock 2.5):


private static class CollectionMatcher<T> implements IArgumentMatcher {
    private Collection<T> collection;
    private String notFound = "";

    public CollectionMatcher(Collection<T> collection) { this.collection = collection; }

    public static <T> Collection<T> collectionEq(Collection<T> collection) {
        reportMatcher(new CollectionMatcher<T>(collection));
        return null;
    }

    public void appendTo(StringBuffer buffer) {
        buffer.append(notFound).append(" not found in {");
        String comma = "";
        for (Object o : collection) {
            buffer.append(comma).append(o.toString());
            comma = ",";
        }
        buffer.append("}");
    }

    public boolean matches(Object otherCollection) {
        if (!(otherCollection instanceof Collection)) return false;

        Collection<T> other = (Collection<T>) otherCollection;
        if (other.size() != collection.size()) return false; // Optionally check sizes
        for (T o : other) {
            if (!collection.contains(o)) {
                notFound = o.toString();
                return false;
            }
        }
        return true;
    }
}

Jackrabbit auto commit exception

At work I have been using a Jackrabbit (JCR) repository in our core product for a quite a few months. Recently we added features for managing financial publications which entails a bunch of CRUD activities using REST including creating versions and performing pessimistic locking.

With add the new activity occurring in the repository we starting seeing occasional auto commit errors. Our JBDC driver (JTDS) reports the following error on commit:

org.apache.jackrabbit.core.state.ItemStateException: commit() should not be called while in auto-commit mode.

Caused by: java.sql.SQLException: commit() should not be called while in auto-commit mode.

Hmmm.. it occurred in some orders of operation (create, edit, finish, find, edit, cancel) but not in others. Also it failed on the build machine and on other dev box, but I earned the “works on my machine” badge.

We use currently use Jackrabbit without a transaction as it turns auto commit on and off itself and thus needs to be unmanaged (see here for a raised issue). I’ve seen hints of using it via XASessions but have not yet found any clear documentation on how to do this.

We also had Jackrabbit using the database to store everything but the Lucene indexes and it shared the same database (SQL Server 2005 / 2008) and database connection as the rest of the application. It was this last point that was causing the issue – giving Jackrabbit it’s own data source to the same database fixed the issue.

The type of data source did not matter – I tried: no-tx-datasource, local-tx-datasource and xa-datasource.

TL;DR: Jackrabbit needs it’s own data source when sharing a SQL Server database.

I’m using:

  • Jackrabbit 2.4.2 (latest stable)
  • JBoss 4.2.3 (old and friendly)
  • SQL Server 2005 / 2008

Memory and wire size of message protocols

There are many studies of the speed of message protocols like protobuf, JSON, BSON, etc, but little in the way of measuring the memory usage required to get the in memory data out to the client. The simplest approach (and the worst in terms of memory usage) is buffering the whole data structure before sending. This typically requires at least the same amount of memory as the original data.

My data set at hand for testing was a large (82MiB) 2D array of decimal values represented as strings (about 10 decimal places).

The Java generated CORBA serialisation code I started with buffers everything at once in its write method. 82MiB is copied to 82MiB.

JSON mapped using Jackson had similar but slightly better memory usage.

Using an ancient version of  The Mind Electric’s GLUE SOAP toolkit  (don’t ask!), the SOAP wrapped JSON message also buffers the lot into memory and was horribly inefficient in creating the envelope (using 100’s of MiB’s).

*A note on wire size – the SOAP message compressed very well using GZip as the whole message is available.

BSON (using BSON4Jackson) by default requires the first element to be the message size and thus buffers the lot into memory. By disabling pure BSON using the BsonGenerator.Feature.ENABLE_STREAMING setting, streaming code can be used and the memory usage is about a third of the original data size again to send.

I couldn’t get Google’s protocol buffers to be very large data friendly. Strings appear to be unoptimised (it just uses String.getBytes()), so even sending an array “row” at a time did not yield great performance. Sending a field at at time with string size fields and row length prefixes was even worse.

The least memory usage was by sending the data via Jackson’s streaming API. This coupled with no content length header to enable chunking (a HTTP 1.1 feature) had almost no overhead. Sending 82MiBs took about 4KB! There is some clever code in the streaming API as it is exceptionally efficient at streaming to an output stream (in my case in a servlet or via Restlet’s OutputRepresentation class).

You can also use GZip on this and it produces half the wire size, but takes twice the time.

In summary: Plain JSON streamed with Jackson is the clear winner for my data set with it’s tiny memory usage sending data to a stream.

In practice I felt that it was much simpler that this article made it seem (but I’m sending a very simple message here). Here is my code to stream a JSON representation of a 2D string array:


JsonFactory f = new JsonFactory();
JsonGenerator g = f.createJsonGenerator(outputStream);
g.writeStartObject();
g.writeStringField("type", "JsonJacksonStreaming");
g.writeArrayFieldStart("vals");

for (int r = 0; r < a2d.length; r++)
{
    g.writeStartArray();
    for(int c = 0; c < a2d[r].length; c++)
    {
        g.writeString(a2d[r][c]);
    }
    g.writeEndArray();
}
g.writeEndArray();
g.writeEndObject();
g.close();

Note: I didn’t get time to try MessagePack, but I’d like to. Anyone who has tried message pack with large amounts of string-ish data care to comment?

Upgrade GWT 2.0.4 to GWT 2.4.0 with GXT 2.2.5

I just upgraded from GWT 2.0.4 to GWT to 2.4.0 and here are the jars I had to change:

Run time dependencies (WEB_INF/lib):

  • updated gwt-servlet.jar to 2.4.0 version
  • updated GXT.jar (2.2.5) had to be a version compiled for GWT 2.2 and above (previous I used the 2 version)
  • updated gwt-log-26.2 to gwt-log-3.1.6.jar
  • updated guice-2.0.jar to guice-3.0.jar
  • added guice-assistedinject-3.0.jar
  • removed gwt-java-math-1.0.9.jar (andremoved the reference from the module (gwt.xml) file
Build time dependencies:
  • updated all the GWT jars
  • updated gin-1.0.jar to gin-1.5.jar (post GWT 2.2 version)
  • added javax.inject.jar that comes with guice-3.0 (new)

Struts2 file upload max size error

A few lessons I learned from getting the Struts2 file upload feature to handle files larger that 2MB:

1. There are two settings that have to be changed to handle more than 2MB.

The first is the fileUpload interceptor’s maximumSize param and the second is the struts.multipart.maxSize. I set both in the example below in the struts.xml file (the struts.multipart.maxSize can also be set in the struts.properties or web.xml).

<!-- 104857600 - 100MB -->
<constant name="struts.multipart.maxSize" value="104857600" />
<action name="importCSVFile" method="importCSVFile" class="ImportCSVDataAction" >
  <result name="success" type="tiles">importdata</result>
  <result name="error" type="tiles">importdata</result>
  <result name="input" type="tiles">importdata</result>
  <interceptor-ref name="paramsPrepareParamsStack">
    <!-- 52428800 = 50mb -->
    <param name="fileUpload.maximumSize">52428800</param>
  </interceptor-ref>
</action>

Notice that the above code overrides the fileUpload interceptor that is in the paramsPreparParams stack. It’s a nice shorthand way of using the interceptor name as the prefix and the param name as the suffix.

2. When either of the max size limits are exceeded you get a different error message.

  • The struts.multipart.maxSize gives a fairly clear and readable message as an action error
  • The maximumSize param gives a very ugly message as a field error
I think both are not nice enough to present to the user so I catch them both in the validate method and replace with a nicer message:
private static final String MAX_FILE_SIZE = "50MB";
@SuppressWarnings("unchecked")
public void validate()
{
  List<String> uploadErrors = (List<String>)getActionErrors();
  if (getFieldErrors().get("upload") != null)
  {
    uploadErrors.addAll((List<String>)getFieldErrors().get("upload"));
  }
  
  for (String err : uploadErrors)
  {
    if (err.startsWith("File too large") || err.startsWith("the request was rejected because its size"))
    {
      clearErrorsAndMessages();
      addActionError(getText(MSG_KEY_ERROR_FILE_TOO_LARGE, Arrays.asList(MAX_FILE_SIZE)));
      break;
    }
  }
}

3. Note that in the first example of the struts.xml code I have the input result set. The max size errors will return a input result, so if you don’t have this defined you get a servlet error message.

Debugging Javascript in IE6

Quick note here on debugging Javascript in IE6 as I found a few handy tips

  • Use firebug lite – my IE fails if you call firebug twice, so load up you page and then click the bookmarklet
    • My IE6 crashes on the firebug lite page so I use another browser to get the bookmarklet code
  • Using firebug lite you can use console.log(‘a msg’) to log to the console
  • Use .toString() on a function when you want to see the function’s code
  • If you can, get the pretty / uncompressed version of the js lib you are debugging (for me it was Highcharts)

GXT Hover BorderLayout

I created a GXT Borderlayout that will hover out a collapsed panel (as a popup) and hover it back in when you mouse out.

There is also a hook to allow you to collapse and expand the popup panel from the BorderLayout. In my case I collapse the popup when windows are activated so they don’t cover over the popup panel.

I can’t quite get the mouse out to always close the popup – if another window is overlapping the popup and you mouse into it the popup will not close. I only respond to mouse out’s that are outside the bounds of the panel (as many elements in the panel fire mouse out events). I was also collapsing the popup of it was no longer the front most component, but this means you can’t open popup menus from the panel without closing the popup (as they become the front most panel). Let me know if you have any improvements.

Here’s the HoverBorderLayout

/**
 * Copyright 2011 Calibre Financial Technology
 *
 * Licensed under the Apache License, Version 2.0 (the "License"); you may not
 * use this file except in compliance with the License. You may obtain a copy of
 * the License at
 *
 * http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
 * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
 * License for the specific language governing permissions and limitations under
 * the License.
 */
import com.extjs.gxt.ui.client.Style.LayoutRegion;
import com.extjs.gxt.ui.client.event.BaseEvent;
import com.extjs.gxt.ui.client.widget.BoxComponent;
import com.extjs.gxt.ui.client.widget.CollapsePanel;
import com.extjs.gxt.ui.client.widget.Component;
import com.extjs.gxt.ui.client.widget.ComponentHelper;
import com.extjs.gxt.ui.client.widget.ContentPanel;
import com.extjs.gxt.ui.client.widget.layout.BorderLayout;
import com.extjs.gxt.ui.client.widget.layout.BorderLayoutData;

/**
 * Border layout that supports "hovering" of collapsed regions - mouse over to toggle popup, and close popup on mouse out.
 *
 * @author cpritchett
 */
public class HoverBorderLayoutGXT extends BorderLayout {
    public static final int STANDARD_COLLAPSED_WIDTH = 24;
    public static final int MIN_COLLAPSED_WIDTH = 2;

    private boolean hoverEnabled = true;
    private int collapsedWidth = MIN_COLLAPSED_WIDTH;
    private boolean showCollapseTool = false;

    public HoverBorderLayoutGXT() {}

    public HoverBorderLayoutGXT(boolean hoverEnabled) {
        this.hoverEnabled = hoverEnabled;
        if (!hoverEnabled) {
            collapsedWidth = STANDARD_COLLAPSED_WIDTH;
        }
    }

    public HoverBorderLayoutGXT(int collapsedWidth) {
        this.collapsedWidth = collapsedWidth;
    }

    public HoverBorderLayoutGXT(boolean hoverEnabled, int collapsedWidth, boolean showCollapseTool) {
        this.hoverEnabled = hoverEnabled;
        this.collapsedWidth = collapsedWidth;
        this.showCollapseTool = showCollapseTool;
    }

    // As onExpandClick is private in GXT 2.2.4 use JSNI to break encapsulation until this is fixed in 2.2.5.
    private native void onExpandClickWrapper(CollapsePanel cp) /*-{
        this.@com.extjs.gxt.ui.client.widget.layout.BorderLayout::onExpandClick(Lcom/extjs/gxt/ui/client/widget/CollapsePanel;)(cp);
    }-*/;

    @Override
    protected CollapsePanel createCollapsePanel(ContentPanel panel, BorderLayoutData data) {
        CollapsePanel cp;
        if (hoverEnabled) {
            cp = new HoverCollapsePanelGXT(panel, data, showCollapseTool) {
                protected void onExpandButton(BaseEvent be) {
                    if (isExpanded()) {
                        setExpanded(false);
                    }
                    onExpandClickWrapper(this);
                }
            };
        }
        else {
            cp = new CollapsePanel(panel, data) {
                protected void onExpandButton(BaseEvent be) {
                    if (isExpanded()) {
                        setExpanded(false);
                    }
                    onExpandClickWrapper(this);
                }
            };
        }

        BorderLayoutData collapseData = new BorderLayoutData(data.getRegion());
        collapseData.setSize(collapsedWidth);
        collapseData.setMargins(data.getMargins());
        ComponentHelper.setLayoutData(cp, collapseData);
        cp.setData("panel", panel);
        panel.setData("collapse", cp);
        return cp;
    }

    // As getRegionWidget is private in GXT 2.2.4 use JSNI to break encapsulation until this is fixed in 2.2.5.
    private native BoxComponent getRegionWidgetWrapper(LayoutRegion region) /*-{
        return this.@com.extjs.gxt.ui.client.widget.layout.BorderLayout::getRegionWidget(Lcom/extjs/gxt/ui/client/Style$LayoutRegion;)(region);
    }-*/;

    public void setPopupExpanded(LayoutRegion region, boolean expanded)  {
        Component c = getRegionWidgetWrapper(region);
        CollapsePanel cp = null;
        if (c != null && c instanceof CollapsePanel)  {
           cp = ((CollapsePanel)c);
        }
        else if (c != null && c instanceof ContentPanel) {
            cp = (CollapsePanel) ((ContentPanel)c).getData("collapse");
        }

        if (cp != null) {
            if (expanded == false && cp instanceof HoverCollapsePanelGXT) {
                ((HoverCollapsePanelGXT)cp).forceHide();
            }
            else {
                cp.setExpanded(expanded);
            }
        }
    }
}

And here is the supporting classes – the HoverCollapsePanel

/**
 * Copyright 2011 Calibre Financial Technology
 *
 * Licensed under the Apache License, Version 2.0 (the "License"); you may not
 * use this file except in compliance with the License. You may obtain a copy of
 * the License at
 *
 * http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
 * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
 * License for the specific language governing permissions and limitations under
 * the License.
 */
import com.extjs.gxt.ui.client.event.ComponentEvent;
import com.extjs.gxt.ui.client.widget.CollapsePanel;
import com.extjs.gxt.ui.client.widget.ContentPanel;
import com.extjs.gxt.ui.client.widget.layout.BorderLayoutData;
import com.google.gwt.user.client.Element;
import com.google.gwt.user.client.Event;

/**
 * In collapsed mode this supports the hiding and showing of the popup panel when hovering (mouse over / out of the
 * collapse panel and of the popup panel).
 *
 * @author cpritchett
 */
public class HoverCollapsePanelGXT extends CollapsePanel {
    // Tracks if the mouse is currently over the panel to ensure the toggle behaves correctly
    private boolean over = false;
    private boolean showHeader = true;

    /**
     * Creates a new hover supporting collapse panel
     *
     * @param panel the parent content panel
     * @param data the border layout data
     * @param showHeader Show header with expand tool. Set to false when panel size is too small for expand tool.
     */
    public HoverCollapsePanelGXT(ContentPanel panel, BorderLayoutData data, boolean showHeader) {
        super(panel, data);
        this.showHeader = showHeader;

        // Replace the standard Popup with a hover Popup
        popup = new HoverAutoHidePopupGXT()  {
            // Disable auto hide as we want to support clicks in the panel with hiding
            @Override
            protected boolean onAutoHide(Event event) {
                return false;
            }

            @Override
            protected void onMouseOut(ComponentEvent ce) {
                setExpanded(false);

                // On Popup mouseout clear the over flag unless mouseout was to this panel
                // (as the pointer is still "over" this component and we don't want to trigger another expand
                if (!HoverCollapsePanelGXT.this.el().getBounds().contains(ce.getXY())){
                    over = false;
                }
            };
        };
    }

    /**
     * Hides the popup and resets the over flag - call this when responding externals events that hide the navigator
     * (like a window activation). Resetting the over flag allows the next mouse over to respond (trigger an expand)
     */
    public void forceHide() {
        setExpanded(false);
        over = false;
    }

    @Override
    protected void onRender(Element target, int index) {
        super.onRender(target, index);

        // Ignore auto hide when clicking on this panel as the onComponentEvent will handle this.
        popup.getIgnoreList().add(this.getElement());

        if (!showHeader) {
            el().selectNode(".x-panel-header").hide();
        }
    }

    @Override
    public void onComponentEvent(ComponentEvent ce) {
        super.onComponentEvent(ce);

        // As mouseout / mouseover events are triggered multiple times in the header section,
        // store when the mouse really leaves this panel (the X and Y are outside theh bounds)
        boolean within = el().getBounds().contains(ce.getXY());
        boolean withinPopup = popup.isRendered() ? popup.el().getBounds().contains(ce.getXY()) : false;

        if (ce.getType().getEventCode() == Event.ONMOUSEOUT && !within && !withinPopup) {
            over = false;
            setExpanded(false);
        }

        if (ce.getType().getEventCode() == Event.ONMOUSEOVER && !over) {
            setExpanded(!isExpanded());
        }

        // Once the first mouseover is detected prevent other mouseovers from triggering expands
        if (ce.getType().getEventCode() == Event.ONMOUSEOVER && !over) {
            over = true;
        }
    }
}

and the HoverAutoHidePopup

/**
 * Copyright 2011 Calibre Financial Technology
 *
 * Licensed under the Apache License, Version 2.0 (the "License"); you may not
 * use this file except in compliance with the License. You may obtain a copy of
 * the License at
 *
 * http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
 * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
 * License for the specific language governing permissions and limitations under
 * the License.
 */
import com.extjs.gxt.ui.client.event.ComponentEvent;
import com.extjs.gxt.ui.client.event.Events;
import com.extjs.gxt.ui.client.event.Listener;
import com.extjs.gxt.ui.client.widget.Popup;

/**
 * A Popup that hides itself on hover - on a normal mouse out (when the event is fired out side the bounds of the Popup)
 * @author cpritchett
 */
public class HoverAutoHidePopupGXT extends Popup {
    protected void onMouseOut(@SuppressWarnings("unused") ComponentEvent ce) {
        hide();
    }

    public HoverAutoHidePopupGXT() {
        addListener(Events.OnMouseOut, new Listener() {
            public void handleEvent(ComponentEvent be) {
                // Hide popup then mouse moves out of bounds
                if (!HoverAutoHidePopupGXT.this.el().getBounds().contains(be.getXY())) {
                    onMouseOut(be);
                }
            }
        });
    }
}