Long Overdue Update

It is the end of July already. The last time I posted was early in May, and that was really just an alibi post as well. For the last two or three months, I’ve felt like an update is long overdue, and some of you, my readers, have let me know about this as well. I’m sorry to have made you wait so long.

Again, I feel like I have to make the comment that I didn’t write because I worked so much, not because there was nothing to write about. Blogging just has a very low priority for me right now.

Perhaps the biggest story is that Corky and I have published a paper together: ConcJUnit: Unit Testing for Concurrent Programs has been accepted at PPPJ 2009. The 7th International Conference on the Principles and Practice of Programming in Java will be held in Calgary, Canada at the end of August.

I’m quite excited about having this paper published, for two reasons: First, despite having worked with Corky for years now, this is actually the first paper we have co-authored. Second, I really only worked on the draft for four days before submitting it. On Monday night, April 20, I was at the Bill Bryson reading downtown, and when I was in line for autographs and checked my email, I saw a message from Corky that the PPPJ publication deadline had been postponed to Friday. He suggested that I submit something on unit testing concurrent programs.

Of course, right at that moment I thought “Four days? How am I supposed to do that?” and it seemed like a huge pain. But I decided I could pull a few things out of my thesis, enhance them a bit and give it a shot. Tuesday and Wednesday I experimented and added another feature to ConcJUnit, the concurrent unit testing framework I had developed, and Thursday and Friday I wrote the paper. It was rather short, a little under five pages, but I was happy with it. Friday night I almost got killed by a road-raged driver, but around 2 AM on Saturday night I submitted the paper (the submission deadlines are often on midnight at the international date line, i.e. 6 AM in Houston).

A few weeks passed while the paper was under review, but on June 5 we were notified that the paper had been accepted. I was in Finland at the time, so it was fortunate that the submission deadline of the camera-ready version had been postponed by a few weeks, so I didn’t have to work in Finland and could enjoy my vacation fully.

The other big story during the last half year was my work on Mint, a multi-stage extension of Java. This has been really interesting work. I was mostly involved in creating the implementation, creating sample programs and benchmarks, and writing some of the non-mathy parts of the paper. I’m still not nearly as quick thinking about many of the multi-stage constructs as some of the other researchers, but I’m getting there.

In the implementation, we started off with the OpenJDK compiler and added brackets (<| |> and <| { } |>), escapes (`) and the separable keyword. Brackets gets turned into a constructor call to a code object, and this code object contains an AST of the source inside the brackets. When a code object is run, we send the AST back into the compiler (at runtime!) and then call the created code.

This project felt a bit like a hack, like self-modifying code. It was great. I got a pretty good tour of the Java compiler too, more than I got to see with my xajavac experiments. We’re having some overhead from the compiler, but once that’s done we can speed up our code by factors of 5 to 20, depending on the benchmark. Explaining all these things in detail is too complicated here, so I’ll refer the interested reader to our technical report.

We first submitted a paper to GPCE 2009, but it got rejected. Initially I found this quite surprising, since I had so many strong collaborators, but it really came down to the wire with the submission deadline, and our paper still had many rough edges. Now we wrote the aforementioned technical report (which isn’t quite finished yet, actually, the appendix is still missing), and we submitted an improved version to POPL 2010. I dearly hope that the hard work of the last half year will pay off and reward us with a publication. I’ll probably continue to work on this for a bit, since I will probably be supported by Walid during the next year.

Something cool, although not significant from a professional point of view happened early this summer: Someone from VDM Verlag, a German academic publisher, contacted me and asked if I would like to publish my MS thesis as a book. It’s available on Amazon and other websites now. I have to stress that this is merely a republication of my MS thesis. There was no additional peer review beyond what my thesis committee did. As such, it should not be viewed as book or monograph, but it’s cool nonetheless:

I republished my MS thesis in book form.
I republished my MS thesis in book form.

Nothing much has happened with DrJava since the last post. We made a few bugfixes and improvements, and we’re planning to push out a new stable version soon.

I’ll close with some very nice reviews I got in my COMP 402 teaching evaluation:

Great course that introduces coding on a well establish code base. Completely different type of programming than that of lower level COMP classes. Taught a lot of practical programming techniques.

The course is good if you want to practice in Java. And it’s awesome that you can do some real work in the development of a real software.

Knows the code base well and good at helping students without doing the work for them. The few lecturers were interesting and well presented. Good professor for this class.

This class was more of independent study. The teacher was ready to help on completing a task whenever you asked him to. I learned a lot about production programming as a whole.

Mathias is a good instructor. He is very helpful when we have some trouble in work. And he always replies email very quickly. He is a nice guy. In addition, he is really great in DrJava.

Thank you for the kind words, and thanks to the School of Engineering, the department chair Joe Warren, and my advisor Corky for giving me this opportunity.

Share
Posted in COMP402, Concurrent Unit Testing, DrJava, Graduate School, Mint, Pictures, Research, Uncategorized | Leave a comment

Print This Post Print This Post  

Tech Report: Multi-stage Programming for Mainstream Languages (TR09-02)

Multi-stage Programming for Mainstream Languages (TR09-02)
(local mirror)

Technical Report TR09-02, Department of Computer Science, Rice University

Multi-stage programming (MSP) provides a disciplined approach to run-time code generation. In the purely functional setting, it has been shown how MSP can be used to reduce the overhead of abstractions, allowing clean, maintainable code without paying performance penalties. Unfortunately, MSP is difficult to combine with imperative features, which are prevalent in mainstream languages. The central difficulty is scope extrusion, wherein free variables can inadvertently be moved outside the scopes of their binders. This paper proposes a new approach to combining MSP with imperative features that occupies a “sweet spot” in the design space in terms of how well useful MSP applications can be expressed and how easy it is for programmers to understand. The key insight is that escapes (or “anti quotes”) must be weakly separable from the rest of the code, i.e. the computational effects occurring inside an escape that are visible outside the escape are guaranteed to not contain code. To demonstrate the feasibility of this approach, we formalize a type system based on Lightweight Java which we prove sound, and we also provide an implementation, called Mint, to validate both the expressivity of the type system and the effect of staging on the performance of Java programs.

Share
Posted in Mint, Publications | Leave a comment

Print This Post Print This Post  

Paper: ConcJUnit: Unit Testing for Concurrent Programs

ConcJUnit: Unit Testing for Concurrent Programs

7th International Conference on the Principles and Practice of Programming in Java (PPPJ 2009)

In test-driven development, tests are written for each program unit
before the code is written, ensuring that the code has a
comprehensive unit testing harness. Unfortunately, unit testing is
much less effective for concurrent programs than for conventional
sequential programs, partly because extant unit testing
frameworks provide little help in addressing the challenges of
testing concurrent code. In this paper, we present ConcJUnit, an
extension of the popular unit testing framework JUnit that
simplifies the task of writing tests for concurrent programs by
handling uncaught exceptions and failed assertions in all threads,
and by detecting child threads that were not forced to terminate
before the main thread ends.

Share
Posted in Publications | Leave a comment

Print This Post Print This Post  

Book: A Framework for Concurrent Unit Testing: Concutest

A Framework for Testing Concurrent Programs: Concutest (Paperback)

Republished MS thesis. VDM Verlag

Incremental, test-driven development is sweeping the software industry, elevating testing from an ancillary activity to an integral part of the programming process. Unfortunately, in our recent experience developing production programs in Java, unit testing has only proven effective in assuring the reliability of code with a single thread of control; it is much less effective in concurrent programs. To facilitate the development of concurrent programs, we have developed: 1. An extension of the JUnit framework that actively supports the developer by treating tests that could silently ignore failures in auxiliary threads as test errors; 2. A lightweight Java annotation language that can be used to specify and check the threading invariants of both existing and new code; 3. A testing framework that can record and analyze the schedules of unit tests, detect deadlocks, and run the tests using modified schedules, increasing the likelihood that concurrency problems are discovered.

Share
Posted in Concurrent Unit Testing, Publications | Leave a comment

Print This Post Print This Post  

New DrJava Beta 20090505

I just released a new beta version of DrJava: drjava-beta-20090505-r4932. And auto-update worked like a charm for me…

Release Notes:

Available for download at http://drjava.org.

DrJava is a lightweight programming environment for Java designed to foster test-driven software development. It includes an intelligent program editor, an interactions pane for evaluating program text, a source level debugger, and a unit testing tool.

In addition to bug fixes, this beta release includes a number of new features introduced after the last stable release:

These features include default imports for the Interactions Pane, inner classes as main class for projects, Javadocs and auto-completion for JUnit, Javadocs and auto-completion for user-defined libraries (“Additional Javadoc URLs”).

An applet viewer is included, and classes that extend Applet can be run using the “Run Document as Applet” menu item (under “Tools”) or using the “applet MyClass” command in the Interactions Pane.

The debugger has been improved and now includes an automatic trace feature to step through the code at a user-determined interval, array element watches in the debugger, and the ability to debug language level files without seeing the Java code that they get translated to. The debugger also retains the imports that have been made before the last step was taken or a breakpoint was hit.

Notable is the ability to close the System.in stream in the Interactions Pane and an automatic update feature that downloads new versions of DrJava automatically.

The most important bug fixes include changes to the language level converter, compatibility with network paths, a debugger deadlock, and color preferences.

Note: Java 1.4 compatibility has been dropped with this release. To use DrJava, you will need Java 5 or newer.

Changes:

New features:

  • The System.in stream can be closed by pressing Ctrl-D when in the input box.
  • Imported classes and packages can persist over breakpoints and debugger steps. To enable or disable this, change the setting in the Debugger tab of the Preferences..
  • Default imports for the Interactions Pane.
  • Inner classes as main class for projects.
  • Javadocs and auto-completion for JUnit (3.8.2)..
  • Javadocs and auto-completion for user-defined libraries (“Additional Javadoc URLs”).
  • Automatic tracing in the debugger..
  • Array element watches in the debugger..
  • Automatic update feature (“Help/Check for New Version”).
  • Applet viewer.
  • Multiple key bindings per action.
  • Improved Jar creation dialog.
  • Find All/Replace All can be limited to a selected section of code

Bug fixes:

  • When the “java MyClass” command is used to start a program in the Interactions Pane, the command is not translated to “new MyClass().main(…)” anymore, but retained as “java MyClass”..
  • If DrJava is unable to restart itself, e.g. if the heap size is too large for the machine, DrJava asks the user if it should reset the configuration and try again..
  • Javadoc 6 is now linked correctly in the “Open Javadoc” feature..
  • Several bug fixes related to unit testing and the debugger..
  • “Open Java API Javadoc for Word Under Cursor…” has been fixed..
  • Better behavior for next/previous word and selection of word by double-clicking in Definitions Pane..
  • Fixed syntax highlighting and interpretation of long literals and hexadecimal literals.
  • Fixed problems with modal dialog boxes when the user switches to another application..
  • Fixed a memory leak..
  • Several language level fixes..
  • Filtered ASCII text when pasting..
  • Fixed network paths in projects..
  • Colors can now be configured again..
  • Documents that have been closed are not re-opened again if they contained failing unit tests..
  • Removed a deadlock when debugging unit tests..
  • Debugging language level files does not open the generated Java code anymore; line numbers in language level files are correct..
  • Closing a file does not remove all breakpoints and bookmarks anymore..
  • Fixed “No Test Cases” in Find/Replace panel.
  • Empty files are not considered “out of sync” anymore
    during JUnit testing
Share
Posted in DrJava | 1 Comment

Print This Post Print This Post  

I love yUML

I haven’t blogged in a long time again, and I don’t know when I’ll have the chance to update you about everything I have been doing. Aside from finishing COMP 402 and bringing DrJava to a state where we can create a new beta version, I have been working on two conference submissions, but since one of them isn’t submitted yet and the other one is currently in peer review, I want to remain silent about them for now.

I ran into a diagram generator website a few days ago, and today it came in very handy to quickly create AST diagrams. I must say I really love yUML.me. You can design diagrams just by writing a link. For example, the diagram below is dynamically created from the link

http://yuml.me/diagram/class/[JCMethodInv o.foo1().foo2()]method->[JCFieldA x.foo2()],
[JCFieldA x.foo2()]selected->[JCMethodInv o.foo1()], [JCMethodInv o.foo1()]method->[JCFieldA x.foo1()],
[JCFieldA x.foo1()]selected->[JCIdent o], [JCFieldA x.foo2()]name->[Name foo2],
[JCFieldA x.foo1()]name->[Name foo1]

yUML Diagram

Anyway, back to work on secret things ;-)

Share
Posted in COMP402, DrJava | Leave a comment

Print This Post Print This Post  

DrJava User Review

I found another very nice review of DrJava by a student taking Csc101 at an unnamed university:

Our teacher let us use Dr. Java as a working environment for our Computer Science 101 class last semester (S.Y. 2008-2009). So far, it is the best programming environment that I’ve used. I have tried Turbo Pascal and Borland Delphi when I was still in high school. Though they are very easy to learn and good for beginners, their functins are quite inefficient in the long run. There are a lot built-in functions in Java that can’t be found in Pascal and Delphi.

Though Java has several programming environments, I find Dr. Java as the easiest one to use since it is so quick to debug the codes. In the ‘Interactions’ pane, I can easily test whether my program is doing what I want it to do. Also, I can easily determine whether a compile-time, run-time or logical error is occurring.

Dr. Java however, is limited to some programs related to recursion or looping. As what my teacher explained, Java could not hold such functions because it already involves filling in memory of the computer. If the program gets heavier as the loop runs all throughout, the process becomes slower.

Despite this, Dr. Java is one useful tool in object-oriented programming (this is so far, our topic in CSc101). There is a display pane where I can access the previous classes that I want to connect with the one I’m working at. Handy at only a small bit size, I have a copy of it in my flash drive. It’s going to be useful especially now that I’m taking BS Computer Engineering course. =)

Rating: 9

Share
Posted in DrJava | Leave a comment

Print This Post Print This Post  

DrJava Automatic Update Feature

I just made a new DrJava development release. You can download
drjava-20090402-r4864 from SourceForge.

The latest feature I’ve added is automatic update, which enhances the “Help/Check for New Version” dialog we’ve had for a while. If DrJava detects that there is a newer version available on SourceForge, then it gives the user the choice of an automatic update or a manual download.

For the automatic update, DrJava determines from where on the user’s computer it was started from and what kind of file it was (*.jar, *.exe or a Mac DrJava.app). It then downloads the corresponding file from the newest release. To actually replace the existing file, DrJava now starts a helper process in the temporary directory and shuts down. The helper copies the downloaded file onto the old file (if possible) and then starts the new version, which deletes the helper files in the temp directory.

By default, this is done once a week, but only beta and stable releases are suggested. The user can change the frequency, though, and also change the dialog to consider development releases.

Of course I would like you all to try this feature out on your computers. However, because the new version will not detect itself as a new version and you probably don’t want to wait until we make the next release, I have made a simulator available that always detects a new version.

Please download and execute that simulator, go to the “Help” menu and select “Check for New Version”. Make sure that the combo box says “Check for: all versions” so you download the latest developer version, not the latest stable version. Then click on “Automatic Update”. DrJava should then display a progress dialog as it downloads, potentially ask you if you want to quit, and then restart, as if nothing had happened. But now
you have the latest version!

Please let me know how this is working on your systems. I am looking forward to your comments and questions.

Other new features:

  • Default imports for the Interactions Pane
  • Inner classes as main class for projects
  • Javadocs and auto-completion for JUnit (3.8.2)
  • Javadocs and auto-completion for user-defined libraries (“Additional Javadoc URLs”)
  • Automatic tracing in the debugger
  • Array element watches in the debugger

Bug fixes in this release:

  • “Open Java API Javadoc for Word Under Cursor…” has been fixed.
  • Better behavior for next/previous word and selection of word by double-clicking in Definitions Pane
  • Fixed syntax highlighting and interpretation of long literals and hexadecimal literals
  • Fixed problems with modal dialog boxes when the user switches to another application.
  • Fixed a memory leak.
  • Several language level fixes.
  • Filtered ASCII text when pasting.
  • Fixed network paths in projects.
  • Colors can now be configured again.
  • Documents that have been closed are not re-opened again if they contained failing unit tests.
  • Removed a deadlock when debugging unit tests.

I have to thank my great team members in my COMP 402 class for getting so much done!

Share
Posted in Uncategorized | Leave a comment

Print This Post Print This Post  

Dumb Email of the Day

We just received the following email on our mailing list:

[DRJAVA-L] Hi, I am a user of JAVA. I found a big bug!

HI,
I am a student in CWRU. Now, I am studying JAVA. Compared to
Program C. I found that java have more changing. however, it is
something no good. Just like the code here:

String a="anjof";
char[] b=a.toCharArray();
b[2]='\\0';
String n=new String(b);
System.out.println(n);

what do you think it will print out? Actually, it print “an of”! I am
very shocked of this? Do the java maker know the difference
between ‘\0’ and ‘ ‘? I think this is a serious problem.

Why are we even getting this email? My response:

Java does not use null-terminated strings like C does. This is the
expected behavior in Java, and in C++ too, if you are using C++
std::string. The behavior is dependent on the language you are using.
Java is not C.

Furthermore, we are the developers of DrJava, not Java itself. Java is
developed by Sun. If you feel like you have discovered a bug in Java,
please contact Sun.

Share
Posted in DrJava | Leave a comment

Print This Post Print This Post  

Improved Javadoc and Auto-Completion Features for DrJava

A few days ago I committed some changes to DrJava, and honestly, I think they’re the best thing since sliced bread (and by “sliced bread”, I mean “Find All” or “Go to File” or perhaps the original “Open Java API Javadoc”).

The new features make the JUnit 3.8.2 Javadoc available for “Open Java API Javadoc” (Shift-F6), “Open Java API Javadoc for Word Under Cursor” (Ctrl-F6 or Apple-F6), and for auto-completion (Ctrl-Shift-Space or Apple-Shift-Space). The URL can be entered in the Javadoc preference
pane. Since the Javadoc for this version is not available at junit.org anymore, I have made them available at
http://www.cs.rice.edu/~javaplt/javadoc/junit3.8.2/, and I use that as default URL. To speed things up and allow DrJava to use the class names for code completion even without internet connection, the index of the classes is included with the drjava.jar file.

There is also a list of URLs (“Additional Javadoc URLs”) where additional Javadoc URLs can be entered and which will also be parsed and used for “Open Java API Javadoc” and auto-completion. Unless these specify file:// URLs on the local hard drive, network access will be required to retrieve the index of the class files, but DrJava fails gracefully if it cannot acquire network access.

To try out this feature, you can add the URL http://drjava.org/javadoc/drjava to the “Additional Javadoc URLs” in
the Javadoc preference pane, and then enter MainFr in the Definitions Pane. Press Ctrl-Shift-Space (or Apple-Shift-Space on Macs) to auto-complete the word, and DrJava will suggest MainFrame and MainFrameTest. Pick one of them and press Enter. DrJava will auto-complete the word. Now press Ctrl-F6 (or Apple-F6 on Macs) with the cursor on the word, and DrJava will open the corresponding Javadoc page.

The auto-completion and Javadoc features have bin in DrJava for nearly two years, by the way, but we have now added the ability to use user-specified Javadoc libraries.

Share
Posted in DrJava | Leave a comment

Print This Post Print This Post  

Offline Memory Profiling

I think I took care of the DefinitionsPane memory leak that we’ve seen occasionally in our tests during the last years (actually, the last 4 years, 9 months, judging by the repository). The NULL_DOCUMENT instance in AbstractDJPane.java had listeners, and those listeners held references to the DefinitionsPane or the OpenDefinitionsDocument.

For those of you who aren’t that familiar with the code base, the NULL_DOCUMENT is an example of the “null object design pattern”. There is the notion of a “current document”, but in some cases, e.g. during
initialization, there may not be a current document. We could set the field for the current document to null, but then we would have to check for null everywhere the field is used. That’s not very robust. Instead, we created an instance of a document, called NULL_DOCUMENT, that isn’t really used as a document at all. It’s just used to signal that there is no current document, but to act graciously if the field is used.

The document still had listeners, though, even though it should really just be a dummy. It should be as much a dummy as null, without the side effects of a NullPointerException. After tracing the problem to
NULL_DOCUMENT (more on how i did that below), I overrode the addXXXListener
methods for NULL_DOCUMENT to be no-ops. Now listeners just cannot be added to this instance. The memory leak was only intermittent, so it’s difficult to say for sure that the problem has been remedied, but I’ve
run 120 repetitions of the DefinitionsPaneTest and not seen the problem again (before the change, I couldn’t run five repetitions without noticing it).

I want to discuss a little bit how I spotted this problem: There are two tools that Java provides, jmap and jhat (really starting with JDK6; unfortunately not supported on the Mac). jmap dumps the entire heap of a JVM into a file. jhat then analyzes that file and provides a HTML interface to browse the results. For more information, look at this article: Finding Memory Leaks in Java Apps.

The biggest problem here was that the problem wasn’t visible all the time, and when we noticed it (because the unit test failed), it was really too late already to execute jmap to retrieve a heap dump. So I wrote a dumpHeap() method that executes jmap automatically, from within the program; that way, I can dump it only if the memory leak has been detected:

/** Dumps the current heap to a file. */
public static File dumpHeap() throws IOException, InterruptedException {
String javaHome = System.getenv("JAVA_HOME");
char SEP = File.separatorChar;

// try jps first
File jps = new File(javaHome+SEP+"bin"+SEP+"jps");
// if that doesn't work, try jps.exe
if (!jps.exists()) jps = new File(javaHome+SEP+"bin"+SEP+"jps.exe");

// execute jps
ProcessBuilder pb = new ProcessBuilder(jps.getAbsolutePath());
LOG.log(java.util.Arrays.toString(pb.command().toArray()));
Process jpsProc = pb.start();
jpsProc.waitFor();
LOG.log("jps returned "+jpsProc.exitValue());

// read the output of jps
BufferedReader br = new BufferedReader(new InputStreamReader(jpsProc.getInputStream()));
Integer pid = null;
String line = null;
while((pid==null) && (line=br.readLine())!=null) {
LOG.log(line);
// find the PID of JUnitTestRunner, i.e. the PID of the current process
if (line.indexOf("JUnitTestRunner")>=0) {
pid = new Integer(line.substring(0,line.indexOf(' ')));
}
}
if (pid==null) throw new FileNotFoundException("Could not detect PID");
LOG.log("PID is "+pid);

// try jmap first
File jmap = new File(javaHome+SEP+"bin"+SEP+"jmap");
// if that doesn't work, try jmap.exe
if (!jmap.exists()) jmap = new File(javaHome+SEP+"bin"+SEP+"jmap.exe");

// execute jmap -heap:format=b PID
pb = new ProcessBuilder(jmap.getAbsolutePath(),
"-heap:format=b",
pid.toString());
LOG.log(java.util.Arrays.toString(pb.command().toArray()));
Process jmapProc = pb.start();
jmapProc.waitFor();
LOG.log("jmap returned "+jmapProc.exitValue());

// read the output of jmap
br = new BufferedReader(new InputStreamReader(jmapProc.getInputStream()));
while((line=br.readLine())!=null) {
LOG.log(line);
}

// rename the file
File dump = new File("heap.bin");
if (!dump.exists()) { throw new FileNotFoundException("heap.bin not found"); }
File newDump = new File("heap-DefinitionsPaneTest-"+pid+"-"+System.currentTimeMillis()+".bin");
dump.renameTo(newDump);
return newDump;
}

Then I still had to associate the instances of DefinitionsPane and OpenDefinitionsDocument with the instances reported in the HTML output. The contents of integer fields is displayed in the output, so I added a field that contains the identity hash code.

When I looked at the reference chains from the root set (which cannot be garbage-collected) to an instance of DefinitionsPane, I noticed that they all end with the following part:

--> class edu.rice.cs.drjava.ui.AbstractDJPane (84 bytes)  (static field NULL_DOCUMENT:)
--> edu.rice.cs.util.text.SwingDocument@0x72b4b840 (78 bytes) (field listenerList:)
--> javax.swing.event.EventListenerList@0x72b4d450 (12 bytes) (field listenerList:)
--> [Ljava.lang.Object;@0x74f52968 (136 bytes) (Element 23 of [Ljava.lang.Object;@0x74f52968:)
--> javax.swing.plaf.basic.BasicTextUI$UpdateHandler@0x74f6b578 (17 bytes) (field this$0:)
--> javax.swing.plaf.basic.BasicTextPaneUI@0x74f6b5b0 (29 bytes) (field editor:)
--> edu.rice.cs.drjava.ui.DefinitionsPane@0x74f67eb8 (559 bytes)

That means that if we can break this part of the chain, the DefinitionsPane can be deallocated. By not allowing listeners, that’s what I have done.

Share
Posted in DrJava | 2 Comments

Print This Post Print This Post  

Hudson

Another big step in the last two months, I believe, was setting up a Hudson server. Hudson is a continuous integration server. It provides a web interface to run tasks such as compiling, creating documentation, and running unit tests. These tasks can also be scheduled to run at certain times, and Hudson can also monitor source control systems such as Subversion.

Right now, I have Hudson running on my office workstation, finland.cs. It is behind the CSnet firewall, so to use it, you need to SSH into finland.cs and forward port 8080, e.g. using:

ssh -N -L 8080:finland.cs:8080 mgricken@finland.cs

Then you can connect to the web interface by pointing your browser at http://localhost:8080.

Right now, DrJava, the PLT Utilities, Java Language Levels and DynamicJava are all built and unit-tested within 10 minutes of a commit. Once a week, Hudson also updates the FindBugs static analysis reports (both as HTML for the DrJava website and XML reports that can be browsed in the Hudson web interface), Clover code coverage reports and the Javadocs for all of these projects.

I have set up the schedule so that most of the scheduled tasks happen during the night or on weekends when it’s less likely that I am using my workstation:

Hr Mon-Fri Sat Sun
0 DrJava
1   DynamicJava FindBugs DrJava FindBugs
2   DynamicJava FindBugs XML DrJava FindBugs XML
3   DynamicJava Clover DrJava Clover
4   DynamicJava Javadoc DrJava Javadoc
5 DynamicJava
6 DrJava
7  
8
9
10
11
12
13
14
15
16
17
18 DrJava
19 PLT / JLL
20   PLT FindBugs JLL FindBugs
21   PLT FindBugs XML JLL FindBugs XML
22   PLT Clover JLL Clover
23   PLT Javadoc JLL Javadoc

DrJava is built and unit-tested at least 21 times a week; DynamicJava, the PLT Utilities and the Java Language Levels are built 7 times a week.

Every time a build fails or gets fixed again, Hudson sends out emails. That’s probably the most visible part so far. Unfortunately, we have one bug that happens quite frequently now: Sometimes some objects aren’t garbage-collected as we expect. This problem has been around as long as I can remember, i.e. at least since the beginning of 2006. It is difficult to fix, because we can’t really determine which references prevent garbage collection, and we don’t notice it outside of the unit tests.

There are also some problems with Hudson getting stuck or not being able to allocate an X server for the GUI parts of the unit tests, causing the builds to fail. In general, however, I feel a lot better, knowing that Hudson is there, catching errors usually within 10 minutes. I’m also glad I don’t have to build and copy Javadoc and the different reports manually anymore.

Hudson exports the test results to a Google Calendar (XML), which made it quite easy for me to check that the builds were still on track.

In the future, it would be nice to have Hudson run on a dedicated machine, so it doesn’t interfere with my work. Perhaps when a new machine comes in, we can use my current workstation solely for Hudson.

Share
Posted in DrJava | Leave a comment

Print This Post Print This Post  

Two-Thousand Nine

Okay, I have neglected my blog for about two months now, and I have received complaints about it, so I will try to make up for that with a summary: I’ve been spending my time mostly on three different subjects: COMP 402, DrJava and Concutest.

The month of January was pretty thoroughly in the hands of COMP 402. In the past, the class called “Production Programming” was numbered COMP 312, and it was traditionally taught by my advisor, Corky, but this semester I am teaching it. I hope I’m doing an adequate job.

During the first four weeks, we discussed concurrency issues, SourceForge, Ant and Subversion, and we have also played around with some dynamic analysis tools, namely Clover for code coverage and YourKit as profiler. After the lecture phase was over, we started working in pairs in the lab, first writing some more unit tests, and then began finishing off the tasks on the lists I had prepared.

By now, we have finished the following tasks:

  1. Test-Run Hourglass Freeze (Bug 2221819)
  2. Focus Jumps to Interactions Pane (Bug 2153310)
  3. Turn on “Hour Glass” When Indenting Code (Bug 687674)
  4. displays internal representation of command-line args (2117750)
  5. Empty java files cause junit test error
  6. Debugging twice throws a DeadlockException (Bug 2106569)
  7. Javadoc deletes directories (Bug 2498253)
  8. Project Main Document Heuristics should not be hard-coded
  9. Menu Item and Customizable Shortcut for Ctrl-D End of Stream
  10. Imports should persist over breakpoints (Feature 1418853)
  11. Allow “end of file” in System.in input (Feature 1470491)

We are still working on these tasks:

  1. Project Main Document Heuristics should not be hard-coded
  2. Default Imports for Interactions Pane
  3. Change Double-Click Selection and Next/Prev Word

We still have a bunch of tasks on the list. The two most important tasks, though, are:

  1. Network paths
  2. New keyboard shortcut system

In general, most of these tasks don’t introduce big features. I have declared that this semester’s development is all under the name of stability.

Okay, it’s getting late. I think I will continue with the summary tomorrow.

Share
Posted in COMP402, DrJava | Leave a comment

Print This Post Print This Post  

DrJava Survey Results, February 9, 2009

Here are the latest results from the DrJava Survey. Since the last time I looked at the survey, developer builds were not relevant, I did not distinguish between developer builds and official releases this time, or look at the different DrJava versions that were used.

An analysis of the data is below, but here are my conclusions:

  • We need to test on Windows Vista. One third of our users now have Vista.
  • We need to test on MacOS Leopard. There are four times as many users with 10.5 than with 10.4.
  • Only 205 users ran Java 1.4 — about one percent. Abandoning Java 1.4 should not cause any problems.
  • Most users (85%) are now working with JDK6.
  • 66 users ran Java 1.7, so we should keep an eye on that.

Operating system:

  • Darwin: 1 (0%)
  • FreeBSD: 4 (0%)
  • OpenVMS: 1 (0%)
  • Linux: 1612 (6%)
  • MacOS, all: 2721 (10%)
    • 10.2.x: 3 (0%)
    • 10.3.x: 20 (0%)
    • 10.4.x: 656 (2%)
    • 10.5.x: 2041 (8%)
  • SunOS: 122 (0%)
  • Windows, all: 22010 (83%)
    • Windows 98: 4 (0%)
    • Windows ME: 3 (0%)
    • Windows NT: 11 (0%)
    • Windows 2000: 132 (0%)
    • Windows 2003: 79 (0%)
    • Windows XP: 13732 (52%)
    • Windows Vista: 8038 (30%)
  • Windows Server 2008: 11 (0%)

Total: 26470 (100%)

Java:

  • 1.4.x: 205 (1%)
  • 1.5.x: 3686 (14%)
  • 1.6.x: 22513 (85%)
  • 1.7.x: 66 (0%)

Total: 26470 (100%)

Response rate (responses / downloads): 26470 / 147676 = 18%

Share
Posted in DrJava | Leave a comment

Print This Post Print This Post  

Merry Christmas

This semester is now over, I have submitted the final grades for the COMP 202 class that I taught this semester. I’m sure I’ll always fondly look back to this opportunity. Now I’m waiting for my grades, of course: for the instructor evaluations.

We had an Othello tournament again, and it was great fun. Some of the games were really close, even though one strategy won all games.

Now I’m working on the lecture schedule and the homework for the COMP 402 – Production Programming class for next semester.

Share
Posted in COMP202, COMP402, DrJava | Leave a comment

Print This Post Print This Post  

Upgrade to WordPress 2.7

I just upgraded to WordPress 2.7. Everything seems to work as usual now. I did have to update the WP-Email, WP-Print and WP-Sticky plugins, and regenerate the permalinks.

Now, of course, the entire administration side of WordPress looks different, and I have to get used to it, but in general I have to say it looks nice. If you notice any errors on this blog, please let me know.

Share
Posted in Uncategorized | Leave a comment

Print This Post Print This Post  

“50 in 50”

“50 in 50”

A presentation by Richard Gabriel and Guy Steele, containing fifty statements of fifty words each from the realm of programming languages. Probably the most entertaining computer science lecture I have ever watched.

Share
Posted in Uncategorized | Leave a comment

Print This Post Print This Post  

ANTLRWorks and OpenJDK Compiler Grammar

Now that my qualifying exam is behind me, I’ve got more time to focus on other things. I have to admit that the exam pretty much occupied all of my thoughts and hogged lots of time, to the degree that I didn’t prepare as well as I used to for my COMP 202 class. I think that has changed again now.

I’m also starting to look at the Java multi-stage programming project again, and I discovered that the OpenJDK people have added a Compiler Grammar project. This project is trying to replace the hand-coded parser for javac by a parser generated from an ANTLR grammar file. I haven’t fully checked how far this has progressed, but it is interesting. Modifying the grammar to allow annotations on statements and expressions should be easier than rewriting the interpreter by hand. It seems like there is a Java 5 grammar that integrates into javac, but of course there is no JSR 308 grammar, so I’d have to develop that first.

When I looked at ANTLR and the Compiler Grammar project, I also noticed an interesting IDE that has been built to make developing grammars easier: ANTLRWorks. It integrates a several checking tools and a debugger and looks tremendously helpful. For example, it can explain how a grammar is ambiguous. In my case, an annotation on a parenthetical expression could be parsed in two different ways, so I’ll have to be careful there.

ANTLR Ambiguous Path Visualization

ANTLR Ambiguous Path Visualization

On MacOS, I’m still struggling a bit to get the Compiler Grammar project and ANTLRWorks running. It seems like Compiler Grammar needs SoyLatte to work, and I have managed to build the Compiler Grammar javac from the command line, but when I try to run it inside ANTLRWorks, then it seems like it is using the default Apple Java version 1.6.0-dp, which doesn’t work. Maybe I’ll take a look at it on Windows first.

I also need to start thinking about the COMP 202 final exam, and about a lecture schedule and programming tasks for COMP 402 next semester.

Share
Posted in COMP202, DrJava, Pictures, Research, Review, xajavac | Leave a comment

Print This Post Print This Post  

Knocking on the Table

I’ve been listening to Deutsche Welle Radio a lot lately, and today they had a quiz on their “Cool” culture show: “When a group of people are sitting around a table in Germany and start knocking on the table, what does that mean?”

This is, of course, easy to answer for me, but I think it’s a really interesting issue. I remember that when I was in the Sekundarstufe I (middle school, grades 7-10), we generally clapped after someone’s talk. Then I went to the US as a high school exchange student in grade 11, and there we clapped. But after I came back for 12th and 13th grade in Sekundarstufe II (high school), students knocked on the table.

I was originally very confused, but I liked it. It somehow seems more dignified than clapping, you can do it longer without having your hands hurt, and it can be done with just one hand. When I then started college and later graduate school in the US, of course I had to learn to applaud again.

I’m still tempted to somehow introduce knocking on the table to the US — I just like it so much better — but I doubt I will succeed. Anyway, after hearing about the quiz above on the radio, I searched a bit on Google, and I found an interesting collection of responses from linguists.

On a not necessarily related note, I passed my Ph.D. qualification exam and have now completed all preliminaries for Ph.D. candidacy. Knock knock knock :)

Share
Posted in Uncategorized | Leave a comment

Print This Post Print This Post  

SourceForge Subversion Problems

Corky and I are having problems on the CSnet machines when we are trying to check source out from Subversion:

% svn co https://drjava.svn.sourceforge.net/svnroot/drjava/trunk/drjava
svn: PROPFIND request failed on '/svnroot/drjava/!svn/vcc/default'
svn: PROPFIND of '/svnroot/drjava/!svn/vcc/default': SSL negotiation failed:
SSL error: decryption failed or bad record mac (https://drjava.svn.sourceforge.net)

Has anyone else encountered this? I can’t find any useful search results. However, when I do a checkout with http instead of https, it works (although I haven’t attempted to commit anything):

% svn co http://drjava.svn.sourceforge.net/svnroot/drjava/trunk/drjava
A    drjava/strip-license
A    drjava/LICENSE
...

Any advice is appreciated. I’m starting to think this is a SourceForge issue again, because I have been finding a handful of SSL issues.

Judging by my experience in the last few months and all the support requests (“We’ve been making some changes, try again \*request closed\*”), SourceForge has turned into a steaming bucket of excrement. Someone please flush.

I do have some new pictures from today’s lecture about parsing and the extended visitor pattern, though. Explaining the extended visitor pattern with all the type parameters is more difficult than I thought. I’ve decided to spend another day on it.

Update

It turned out that an upgrade to the latest OpenSSL and Subversion software on our side took care of the problem. I figured that would be a more sensible first attempt, rather than downgrading to an ancient version of OpenSSL, which is what SourceForge recommended.

Share
Posted in COMP202, DrJava, Pictures, Ramblings | Leave a comment

Print This Post Print This Post