Oops Null Pointer

Java programming related

Chrome returning null content type on file upload

Chrome (up to at least 8.0.552.215) has issues with content types of files. For me it was if the file did not have an extension.

See here for the old (unfixed issue) and here for the new issue.

Note: I’m using struts 2.0.14 with the FileUploadInterceptor (uses ServletFileUpload and JakataMultipartRequest) and if I upload a file without an extension then struts throws a null pointer exception. If I take the same file and add any extension (e.g. .bin) then it will upload correctly.

Looking at the code for struts 2.2.1 or the current trunk (line 275) it still looks like this is a problem.

Firefox defaults to “application/octet-stream” which, in this case, works.

Anyone else had this issue or has a workaround? Maybe allow the setting of a default content type in the FileUploadInterceptor?

Update: This appears to be fixed in the code base – waiting for word on when it will get into a release.

Advertisements

Ant 1.7 to 1.8 .Net builds

I recently needed to upgrade to Ant 1.8 to use Gmail as a smtp service for the ant MailLogger (mails the results of builds). Ant 1.8 is required as Gmail (among others) requires a STARTTLS to be sent and Ant 1.8 fixes a bug in 1.7.

Ant 1.8 no longer supports .Net tasks as core tasks as they have been moved to the AntLib project. To get the old tasks running,

  • Add the ant-donet-1.0 .jar to the ant lib dir
  • Add the xmlns to the project node
  • Prefix the old tasks with the xmlns
  • Change the case of any commands that require it (e.g. WsdlToDotnet becomes wsdltodotnet)

So:

<project name="aqua">
  <csc debug="true" optimize="true"  targettype="exe">
    ...
  </csc>
</project>

becomes

<project name="aqua" xmlns:dn="antlib:org.apache.ant.dotnet">
  <dn:csc debug="true" optimize="true"  targettype="exe">
    ...
  </dn:csc>
</project>

Creating NUnit reports with Ant

The current project I’m working on is largely Java with a reasonable amount of Microsoft Office connectivity using C#, VB, with COM and JSON as the bridges to the Java server. To get continuous integration rolling I really wanted to end the NUnit GUI based tests and get the NUnit console results built into a HTML report via Ant.

Not having any experience with NAnt (all our .Net building is done via Ant) I needed some way to get Nunit results into JUnit format so I could run the junitreport task on them.

Here are the steps I took:
1) Run the NUnit tests with the nunit-console and create an xml file

2) Use a modified version of Hudson’s NUnit plugin’s nunit-to-junit.xsl file to create a JUnit file per assembly test suite
Hundon’s XSL creates one file for all the tests with a “testsuites” root node. The junitreport task needs one file per testsuite. Hudson has a java file to do this, while I used the XSL 2.0 result-document (with the saxon XSLT engine) to do this. I use Ant’s XSLT task to transform the results

The modified section of the nunit-to-junit.xsl looks like this:

<!-- I need to replace any non file system compatible chars with '_'-->
<xsl:variable name="assemblyName">
  <xsl:analyze-string select="$assembly" regex="[\*/:&lt;li&gt;\?\|;&quot;&apos;]+">
    <xsl:matching-substring>
      <xsl:value-of select="." />
    </xsl:matching-substring>
    <xsl:non-matching-substring>
      <xsl:value-of select="." />
    </xsl:non-matching-substring>
  </xsl:analyze-string>
</xsl:variable>

<xsl:result-document href="{$outputpath}/TEST-{$assemblyName}.xml" format="xml">
  <testsuite name="{$assemblyName}" ...>
   ...

3) Run Ant’s junitreport task on the result and voila: a junit-y looking HTML report for my NUnit tests.

The ant target is:

<target name="create-nunit-test-report" description="NUnit result into junit report">

  <delete dir="${test.nunit.reports.dir}" failonerror="false"/>
  <mkdir dir="${test.nunit.reports.dir}"/>

  <xslt in="${test.nunit.console.file}"
    out="${test.nunit.results.dir}\temp-nunit-to-junit.xml"
    style="${antBuild}\nunit-to-junit.xsl" force="true"
    processor="org.apache.tools.ant.taskdefs.optional.TraXLiaison">

    <outputproperty name="method" value="xml"/>
    <outputproperty name="standalone" value="yes"/>
    <outputproperty name="encoding" value="iso8859_1"/>
    <outputproperty name="indent" value="yes"/>

    <classpath location="${vendor.lib.dir}/xslt/saxon9he.jar" />

    <!--Split file location for results-document XSLT function  -->
    <param name="outputpath" expression="file:///${full.path.test.nunit.results.dir}"/>
  </xslt>

  <junitreport todir="${test.nunit.results.dir}">
    <fileset dir="${test.nunit.results.dir}">
      <include name="TEST-*.xml" />
    </fileset>
    <report todir="${test.nunit.reports.dir}" />
  </junitreport>
</target>

Update 1/12/2010: Ant 1.8.1 has a bug that will cause the saxon jar not to be found and a “Unsupported XSL element” error.
I found the solution here http://stackoverflow.com/questions/919692/how-to-execute-xslt-2-0-with-ant/4211575#4211575
You need to add the following attribute to the xslt task:

processor="org.apache.tools.ant.taskdefs.optional.TraXLiaison"

I see the future – it’s disappointingly like today (Eclipse plugin setup still painful)

I hate setting up plugins in Eclipse. Every time a new version comes out, like a kid hoping for a transformer under the tree at Christmas but getting a pair of socks, I get disappointed.

Helios, as shiny as its new socks are, still makes me click away to install plugin after plugin.

In desperation I turned to Genuitec’s Pulse, and it showed promise, but has enough bugs to make me fall back to plain Eclipse again. It often got confused about a plugin (hey google stop innovating and updating things!) and then you get stuck weeping “I just want to install the GEF”!

The other thing that puts me off pulse is that you have to have a separate “common” directory for plugins. I personally like the fact that plain Eclipse can just be picked up and moved without any installation hassles.

I while ago I tried and few other services like Yoxos which I should probably have another look at (Yoxos 5 is in Beta).

Anyone else just want to get stuff done and have a good solution?

Avoiding the database – Ehcache’s poor fit as a disk only cache

Recently we needed to a quick fix (with a better solution to come) to speed up our application queries – which involve reading and assembling copious amounts of data from the database.

Large parts of this data changes infrequently (once a day) and is shared by multiple queries. So as retrieving from the database is too slow and there is too much data to store in the same jvm as the app we needed an alternative.

The idea was to pull data out of the database via scheduled jobs (say once a day) and put the data is something that was faster that the DB queries.

Ehcache was already used for another purpose, so we tried creating a disk only cache with the following settings:

diskcache.cacheEnabled = false
diskcache.multiDateRange.cacheEnabled = FALSE
diskcache.maxElementsInMemory = 1
diskcache.overflowToDisk.enabled = TRUE

We discovered Ehcache’s disk store has a few caveats:

  • It’s fragile – if it gets corrupted then the cache file and its index are deleted. This is by design as Ehcache prefers safety over persistence
  • It’s not persisted until there is a flush or the cache manager is shut down (you need to register the shut down hook)
  • Raw object serialisation is used, so unthoughtful serialisation policies cause the cache file size to balloon
  • The disk spooling can run faster than the garbage collector, so you can get OutOfMemory errors unless you flush periodically
  • It’s not very fast and performance degrades if you are updating entries that already exist

That last point on performance is probably the biggest death knell for using Ehcache it this way. You can work around the fragility with lots of flushing, but the performance degradation over time is difficult to avoid.

For example, if the update cache code does a get and then a put, then the first query I run on an empty cache takes 2 minutes. The second time the same query is run it takes 20mins!

If this is replaced by a remove and a put then the second time takes 8mins. So if you can get away with it don’t bother updating, just remove and re-add. Note that without the remove the query’s performance is nearly as bad as in the first example.

The real answer is to use something else as a out of jvm store that is faster than the DB like Infinispan or Terracotta. What has worked for you?

In summary:

  • Use custom serialisation to minimise cached object sises
  • Use many smaller caches so the whole cache can be emptied and recreated
  • Use a remove before a put rather than a get and then a put
  • Flush regularly to minimise the disk spool memory usage and ensure the cache is written to file

Update: see here for the an issue on slow caches when the disk store is bigger than the memory store. Login and vote if this issue affects you!

Accessing an Eclipse Server using IP address

A while ago I got stuck after upgrading to Eclipse 3.5 – suddenly I couldn’t access my server using the IP address or host name of the machine, but localhost and 127.0.0.1 worked fine.

For me this was a bigger issue as my host was a virtual machine (vmware) and I wanted to access the server from outside that virtual machine (in this case for testing a GWT app with different versions of IE).

Turns out that prior to Eclipse 3.5 the server launch config didn’t set the –host property automatically. So even if you put localhost as the in the “host name” field in the server configuration, you could still access the server using the hosts IP address.

Eclipse 3.5 now adds a –host property automatically, causing any old configs to now only serve to localhost.

To fix this set the “host name” field (or –host property in the launch config) to 0.0.0.0 (to listen on all interfaces). Back to work again…

GXT Charts cutting off the y-axis labels

I was having issues with GXT charts deeply embedded in a hierarchy (chart -> ContentPanel -> Portlet -> Portal -> TabPanel -> Window) where the y axis labels where truncated for long strings.

Oringinally I was adding spaces to the labels but causes OFC to add about 10-15 extra steps when using a bar chart will all values less than zero.
So instead I add the chart, set the chart model, remove the chart, then add it again before laying out the panel.

Actually this is a bug in the OFC source. Here is the explanation and the fix.

You need to patch and recompile the swf used for the charts – and below I have provided the way to do it and attached the fixed open-flash-chart.swf file itself

This bug in OFC 2 is mentioned in a few other places:here and here

Here are the details of the issue and the fix:

GXT uses a patched version of OFC2: “2 Ichor Patched by DZ”.
This version has a bug that YAxis labels > 1,000,000 get cut of. If you resize the container of the chart then the labels get fixed.

The latest version of OFC (trunk) is missing some DZ patches (e.g. tooltips show if value is “”) and also potentially not the full font range for unicode.

The GXT version of DZ’s OFC is [B]not[/B] the same as the source code of this version from DZ’s website. The version from his site has another bug – the tooltips do not disappear on mouse out.

Here is how to build a version that fixes the tooltip problem and the truncation issue

Building from source: (See also this link)

Download FlashDevelop – install

  • best location is C:\OFC\FlashDevelop otherwise you will have to edit the path in FullFont.xml and PartialFont.xml in the \obj fir of DZ’s OFC source

Get and install flex sdk 3

  • Get Flex sdk v3
  • Unzip flex sdk
  • Add the bin flex bin dir to your path
  • Copy msvcr71.dllto bin dir of flex

Get source code for DZ’s OFChere

Fix bugs!

  • main.as – Move the line this.set_the_stage() to above the this.find_data() line
  • tooltip.as – The hide method is broken – either uncomment the tween line or manually add a call to hideAway
  • edit the build.bat file and change XAxisLabels-NoUnicodeRange.as and XAxisLabels-UnicodeRange.as to use underscores instead of dashes
  • edit the build.bat file and REM out the line that copies the SWF to a xampp dir

Compile and deploy

  • cd to the open-flash-chart main dir (has the main.as file)
  • from the cmd line run build.bat
  • Rename the open-flash-chart-full-embedded-font.swf to open-flash-chart.swf
  • Copy into the war\lib\gxt\charts dir replacing the existing version

Clear you browser cache or else the new swf will not be picked up.

For a link to the bug report and the swf file see here
Email me / comment if you need more info.

Time stamps in ant

Just putting this here so I don’t forget this neat trick to get timestamps in ant

<macrodef name="ShowTimestamp">
  <sequential>
    <tstamp>
      <format property="current.time" pattern="dd/MM/yyyy HH:mm:ss.SSS"/>
    </tstamp>
    <echo message="${current.time}"/>
  </sequential>
</macrodef>

Call with <ShowTimestamp/>

See the original discussion

Creating UML Sequence Diagrams from Debug / Stack traces

I found a good tool to create sequence / class interaction diagrams from Java stack traces or debug stacks using a free Eclipse plugin called AmaterasUML. Install it in Eclipse and you can open a “Sequence diagram from stacktrace” view, paste in a stacktrace and it will generate a nice looking sequence diagram.

With a bit of text manipulation you can modify a debug trace (from right-click “Copy Stack” on the debug window) to look like a stack trace and generate the UML from a debug breakpoint. The steps are:

  • Add “at ” to the start of every line
  • Replace method args with the class name.java:line number
  • Clean up $2$1 type class suffixes

E.g.

EventRegistryImpl.fireEvent(Event) line: 47
BaseFolderNavigator$2.handleEvent(Event) line: 79
CoverageNavigatorViewGXT(BaseObservable).fireEvent(Event) line: 27
BaseNavigatorViewGXT$2$1.execute() line: 149

becomes

at EventRegistryImpl.fireEvent(EventRegistryImpl.java:47)
at BaseFolderNavigator.handleEvent(BaseFolderNavigator.java:79)
at CoverageNavigatorViewGXT(BaseObservable).fireEvent(CoverageNavigatorViewGXT.java:27)
at BaseNavigatorViewGXT.execute(BaseNavigatorViewGXT.java:149)

Below is a gawk command (via UnxUtils) to convert a stack file (x.txt) to a stacktrace

gawk "{str = $0; sub(/^\t/, \"^\", str);match(str, /(.*)\./, arr); sub(/^/, \"^at \", str);sub(/\(.*\./,\".\",str);sub(/\([^\)]*\)/, \"(\", str);sub(/ line: /, arr[1]\".java:\", str);sub(/$/, \")\", str);sub(/\t\)$/, \"\)\", str);print str}" x.txt

Capturing Integration Testing Input/Output with XStream

A co-worker was looking for a way to capture input and results during a debug session for certain scenarios that were occurring at run time. Ideally they were looking for a tool that could be used in Eclipse and capture parameters and return values from frames in the debug session and then be able to reuse those values in jUnit tests.

A quick search couldn’t find any tools that find the bill, but as a quick and dirty method you can use XStream in an Eclipse “Display” view to dump out input and return variables. Then in a jUnit test you unserialise the XML back into objects and tests the inputs against the results.

The XML format gives you two instant advantages:

  • No need to reconstruct the input / result objects from scratch (which can be difficult, especially in complex or legacy systems)
  • Easy tweaking of the the XML to get the results you want

A disadvantage is that if the objects structure changes then re-factoring tools will miss the XML data and the test may no longer work.

To use this technique:

  • Put XStream in your project classpath
  • Show the Display view in eclipse (Widnow->Show View-> Display)
  • Put breakpoints in the code to capture input and result objects
  • At a breakpoint run the following code in the Display view to serialise the objects to XML
(new com.thoughtworks.xstream.XStream()).toXML(someObjectToSerialise)
  • Save the XML to files and load the files into a jUnit test (say with Commons FileUtils.readFileToString())

If you know of any other tools to do this in a more automated fashion then let me know (it would be cool to do this with a few clicks via a plugin – even generate the test classes).