Showing posts with label closures. Show all posts
Showing posts with label closures. Show all posts

Friday, March 7, 2008

ShapeLogic 1.0 with stream based rules released

Here are the release notes for ShapeLogic 1.0

Changes

  • Rule for image processing have been migrated, previously they were implemented as goal driven tasks with sub tasks. Version 1.0 uses lazy streams which are simpler and more powerful.
  • Letter match example now matches all polygons instead of just the first found.
  • When running ShapeLogic as ImageJ plugin, it is now easy for users to define rules for matching in external Java files.
  • New number matcher to demonstrate how to define rules for matching in an external Java file in 130 lines of code.
  • Enabled use of Java 6 Scripting for rule database, which gives the user access to the 25 scripting languages that are supported
  • ShapeLogic now has beta development status
  • Many unit tests added for Lazy Stream library
  • Fixed bugs in Lazy Stream library
  • Fixed bugs in vectorizer

Lazy streams features

  • Lazy streams can be named and defined based on other lazy streams
  • They work similarly to UNIX pipes or calculation Legos
  • They serve as your query construct, you can directly query them
  • A stream can be wrapped around an Iterator
  • A stream can be created from an input stream and a Calculation
  • They can be instantiated lazily, so you can load calculation networks independently
The lazy stream worked very well with the letter match. It started out as a functional construct, but with the named streams they got a more declarative feel to them.

The Calculation in the stream is using the same signature as Neal Grafter's Java 7 closure prototype. This might make integration with Java 7 easier.

User defined rules for number match

The most significant change is that it has become a lot simpler for users to define rules. Here is the start of DigitStreamVectorizer_ the number matchre in 130 lines of code. Notice that there is barely any setup code. It is almost only rules, defined in a very simple format.

public class DigitStreamVectorizer_ extends StreamVectorizer_ {

@Override
public void matchSetup(ImageProcessor ip) {
loadDigitStream();
}

public static void loadDigitStream() {
LoadPolygonStreams.loadStreamsRequiredForLetterMatch();
makeDigitStream();
String[] digits = {"0", "1", "2", "3", "4", "5", "6", "7", "8", "9"};
LoadLetterStreams.makeLetterXOrStream(digits);
}

public static void makeDigitStream() {

rule("0", HOLE_COUNT, "==", 1.);
rule("0", T_JUNCTION_POINT_COUNT, "==", 0.);
rule("0", END_POINT_COUNT, "==", 0.);
rule("0", MULTI_LINE_COUNT, "==", 1.);
rule("0", CURVE_ARCH_COUNT, ">", 0.);
rule("0", HARD_CORNER_COUNT, "==", 0.);
rule("0", SOFT_POINT_COUNT, ">", 0.);

rule("1", HOLE_COUNT, "==", 0.);
rule("1", T_JUNCTION_LEFT_POINT_COUNT, "==", 0.);
rule("1", T_JUNCTION_RIGHT_POINT_COUNT, "==", 0.);
rule("1", END_POINT_BOTTOM_POINT_COUNT, "==", 1.);
rule("1", HORIZONTAL_LINE_COUNT, "==", 0.);
rule("1", VERTICAL_LINE_COUNT, "==", 1.);
rule("1", END_POINT_COUNT, "==", 2.);
rule("1", MULTI_LINE_COUNT, "==", 0.);
rule("1", SOFT_POINT_COUNT, "==", 0.);
rule("1", ASPECT_RATIO, "<", 0.4);

ShapeLogic's 3 different approaches to declarative programming

Here is a chronological listing of ShapeLogic's 3 different approaches declarative logic:
  1. Declarative goal driven logic engine. From ShapeLogic 0.2.
  2. Logic filter language. From ShapeLogic 0.8 The syntax and development of the logic language is better described in Logic language.
  3. Lazy streams. From ShapeLogic 0.9.

Result of different approaches so far

For the letter matching example, the lazy stream approach has been both simpler and more powerful than the goal driven logic engine.

The Artificial Intelligence choice tree is built into the logical structure of goal driven logic engine, so this approach might work well when reasoning under uncertainty.

The logic filter language is used with both lazy streams and goal driven approach.

ShapeLogic is a toolkit, all 3 approaches are available with unit tests.

For now development is focused on the lazy stream approach.

JSR 223 scripting surprise

I had put a lot of energy into making it easy to create a stream from a Scripting snippet in either Groovy, JRuby or JavaScript. This was clearly superior to text snippet rules defined using Apache Commons JEXL under the Declarative goal driven approach, but using the new Stream it turned out not to be necessary. The rules could easily be defined in plain Java. I still think that good integration with streams and Scripting from ShapeLogic might turn out to be useful.

Download ShapeLogic 1.0

-Sami Badawi
http://www.shapelogic.org

Saturday, January 12, 2008

Lazy streams in Java, Groovy, JavaScript, JRuby

This is a follow up to my last blog: Functional constructs Java 7, Groovy, Commons

I have finished coding the first part of the lazy streams for ShapeLogic. They are going to be key for the declarative programming query interface to ShapeLogic. As a proof of concept I wanted to see how easy it would be to implement a lazy stream of Fibonacci numbers. So far I have tested my framework with definitions written in:
  1. Groovy
  2. JavaScript
  3. JRuby
  4. Java
It is essential that lazy streams are are easy to define so they can be put in a flat file or a database. I was very satisfied with the concise definitions of the Fibonacci streams in the different languages:

Lazy Fibonacci stream in Groovy

new FunctionStream("fibo","def fibo_FUNCTION_ = { fibo.get(it-2) + fibo.get(it-1) };",1,1);

fibo is the name the lazy stream has in the context / name space
fibo_FUNCTION_ it is a naming convention that the function that is use to create the stream has this name
1,1 is the first part of the lazy stream

Lazy Fibonacci stream in JRuby

new FunctionStream("fibo","jruby",null,"def fibo_FUNCTION_(it) return $fibo.get(it-2) + $fibo.get(it-1) end",1,1)

Lazy Fibonacci stream in JavaScript

new FunctionStream("fibo","javascript",null,"function fibo_FUNCTION_(it) { return parseInt(fibo.get(it-2) ) + parseInt(fibo.get(it-1))};",1,1);

Scripting using JSR 223

I added a formula language for users to an enterprise Java system a few years back using Antlr and BeanShell. JSR 223 is dramatically easier to work with, but there are still some problems.

The only scripting language that works out of the box with Java 6 is JavaScript. To get others to work you have to download jsr223-engines.zip from:
https://scripting.dev.java.net/servlets/ProjectDocumentList
For each language you want to use there is a engine jar file.
E.g. jruby-engine.jar.
They have to be on your path. So does the jar file implementing the language.

JSR 223 and Maven 2 problems

JSR 223 does not work well with Maven 2. The engine jar file does not reside in the Maven repository so you have to separately install them into you local Maven repository. Here is the command to install JRuby:
~/bin/maven-2.0.8/bin/mvn install:install-file -Dfile=jruby-engine.jar -DgroupId=org.jruby -DartifactId=jruby-engine -Dversion=1.0.1 -Dpackaging=jar

Scripting languages currently under JSR 223

beanshell, browserjs, ejs, freemarker, groovy, jacl, jaskell, java, javascript, jawk, jelly, jep, jexl, jruby, jst, judo, juel, jython, ognl, pnuts, scheme, sleep, velocity, xpath, xslt.
These languages should in theory work with ShapeLogic, without any additional code.

Other implementations of lazy streams

  1. BaseStream: That is the abstract base class with most of the lazy stream functionality
  2. IteratorStream: Generates elements using Java Iterator
  3. TransformerStream: Generates elements using Java interface
  4. FunctionStream: Generates elements using JRS 223 as described above

Next step for functional constructs for ShapeLogic

  1. Filter that is easy to define in external text using scripting
  2. Transformer transforming one lazy stream to the next
  3. Query interface from where a result can be retrieved
-Sami Badawi

Wednesday, January 9, 2008

Functional constructs Java 7, Groovy, Commons

What functional constructs to use for ShapeLogic

I have started to code the lazy streams for ShapeLogic. They are going to be key for the query interface to ShapeLogic. I need some functional constructs to work with these. The top sources candidates are:
  1. Apache Commons
  2. Groovy
  3. Java 7
  4. Hand coding
None of these are a perfect fit. This post list some of the advantages and disadvantages of these.

Apache Commons functional constructs

I am currently using Apache Commons JEXL for user expressions.
  1. Apache Commons do not use templates
  2. Apache Commons Functor are still in the sandbox and not actively developed
  3. The code is not uniform
  4. You cannot make user define functions in Apache Commons JEXL

Groovy functional constructs

I need to use a scripting language, to define user functions.
  1. Groovy contains all of Java's constructs
  2. Groovy comes with good functional constructs
  3. I like the Groovy syntax for using these
  4. I cannot use these expressions directly since Groovy is not a lazy language

Java 7 functional constructs

Java 7 comes with good functional constructs.
  1. There are a lot of interest for functional constructs in Java 7
  2. I would rather use Java 7 than compete with it
  3. Java 7 still seem to be pretty far away
  4. It is not sure that closures will make it into Java 7

Thoughts on immutable constructs

I will probably make the convention that a lazy stream like the Fibonacci numbers are immutable, but not enforce this by making a LISP list.

Scala envy

Now I suffer from Scala language envy. These Scala language features would come in handy now:
  1. Lazy streams
  2. List comprehension, with same syntax for lists, iterators and streams
  3. First class function
  4. Closures
  5. Lazy calculation
  6. Uniform access to functions and lists
I could of cause try to include the scala.jar and directly use the Scala constructs in ShapeLogic, but there lay the road to madness.

Hand coding functional constructs

This might not be too much work, but I would rather not create yet another implementation of functional constructs. Apache Commons Functor never took off, despite some good press. That is probably an indication that Java was not that well suited for elegant functional constructs when this library was made, and maybe still isn't.

Current work plan

  1. I will start by implementing a lazy Fibonacci stream
  2. Then I will move the polygon finder to a lazy stream, before it could only find one polygon
I will try to get ShapeLogic 0.9 out soon, even if it is a very limited version.

If you know of any libraries that would fit my need and not be too heavyweight, please let me know.

Two Fibonacci implementations

The Haskell language has an incredibly elegant lazy stream implementation:
fibs :: [Int]
fibs = 1 : 1 : [ a + b | (a, b) <- zip fibs (tail fibs)]

Drools Fibonacci implemented. This is not a good fit for my computer vision project.

-Sami Badawi