Computer vision C++ libraries review

I am trying to create an easy to use, minimalistic C++ cross-platform computer vision system, with a non-restrictive license. My biggest challenge was to chose the best libraries and to get them to work together; this took some investigation and experimenting. This posting is a brief description of my findings.

This is what ShapeLogic C++ currently looks like:


Windows


Linux

In order to construct ShapeLogic C++, I had to make choices within the following categories:

• Computer vision and image processing libraries
• GUI libraries
  
• Unit test systems
• Build systems
• Compilers under Windows
  
• C++ IDEs under UNIX
  

ImageJ, the Java open source image processing tool, is the inspiration for the first part of my work: it is very simple to learn, use and program in. This is a follow up to my last posting: Computer Vision C++ vs Java. The result of my work is released as an open source project ShapeLogic C++, under the MIT license.

Computer vision and image processing libraries

The candidates I considered were:

• GIL, Generic Image Library
  
• OpenCV
• VXL

GIL, Generic Image Library

GIL, Generic Image Library by Adobe.

Pros

• Very non intrusive, only based on header files
  
• Puts a wrapper around most image format
• You can write a algorithm once and it will work for most image types
• Part of Boost since 1.35
  

Cons

• Does not come with a lot of image processing algorithms
  

OpenCV

OpenCV, Open Computer Vision by Intel.

Pros

• Very simple
• Works with both C and C++
• Very broad range of algorithms
• Complex algorithms: face detection, convexity defects
• Very popular
  

Cons

• You have to use OpenCV's IplImage
• IplImage byte order is BGR instead of the normal RGB
  

VXL, Vision X Library

VXL a combination of 2 big older vision libraries TargetJR and IUE

Pros

• Well tested technology
  
• Simpler build process using cmake
  
• Uses modern programming techniques: classes, template and STL
  
• It has a lot of functionality
  
• Simple to get started with

Cons

• It is not using normal STL, but in order to work on different compiler it had to make its own version with different names.
• Class structure is somewhat complex.
  

Choice computer vision library for ShapeLogic C++
OpenCV for existing image processing and vision algorithms. GIL for writing new algorithms.

Cross platform GUI

The candidates I considered were:

• GIMP plugin
• GTK+, GIMP toolkit
• FLTK
• HighGui from OpenCV
• PhotoShop plugin
• wxWidget

Run ShapeLogic as a GIMP plugin

Pros

• GIMP is the main cross platform OSS image editing programs.
  
• It is in wide use.
  
• Has a lot of powerful features including scripting functionality in Scheme and Python.
  
• From a user perspective this would be an excellent choice.

Cons

• GIMP is GPL, but you could have a wrapper around plugins in order to access them as GIMP plugins.
• The plugin works with tiles, which gives good performance, but does not fit well with either the way OpenCV or GIL are processing images.

GTK+, GIMP Toolkit

Pros

• GTK+ is a great looking and very powerful framework that works on: Windows, Linux, Mac, a.o.

Cons

• It is written in C and has a homegrown object system, which is not type safe.

GTKMM C++ wrapper around GTK+

Pros

• CTKMM is a great looking and very powerful framework, that works on: Windows, Linux, Mac, a.o.
  
• It feels natural to program in for a C++ programmer.

Cons

• The class hierarchy is somewhat deep since it is built on top of GTK.

FLTK, Fast light toolkit

Pros

• FLTK is very lightweight.
  
• Very clean C++, you actually have a main().
  
• Native C++ build on top of X11 or Windows.
• Fluid, a simple GUI builder
  

Cons

• Not as many widgets.
  
• Dated look.

HighGui from OpenCV

Pros

• Very lightweight.
  
• There is some functionality for displaying images, video and an event handler for mouse events.

Cons

• Does not come with a menu system.

Run ShapeLogic as a PhotoShop plugin

Pros

• PhotoShop is the main image editing programs.
  
• It is in wide use, has a lot of powerful features including macros.
  
• From a user perspective this would be an excellent choice.

Cons

• The PhotoShop SDK is not freely available, you have to apply to get it.
• The plugin does not fit well with either the way OpenCV or GIL are processing images.

wxWidgets

Pros

• wxWidgets is a full featured GUI toolkit, built on top of native toolkits: Win32, Mac OS X, GTK+, X11, Motif, WinCE and more.
• Looks good and modern.
  
• Big community.
• Several GUI builders.
  

Cons

• The programming style is close to Windows MFC programming.
  
• There are many layers.

Choice
This was a hard choice and I went back and forth between FLTK and wxWidgets, but went with FLTK. All the GUI code is separate from the image processing code, so if I wanted to change from FLTK to another toolkit later it should not be too dramatic.

C++ unit test frameworks

There are a lot of different choices and no clear leader. Some of the candidates were:

• Boost.test
• CppUnit
  
• Google C++ Testing Framework
  

Boost.test

Pros

• Boost.test is part of the Boost library.
  
• Powerful with a lot of options.
  

Cons

• You have to manually set up test suites.
  
• It is somewhat heavy.
  
• The documentation is extensive but not easy to read.
  

CppUnit

Pros

• CppUnit is following a standard unit testing convention XUnit.
• Integration with Eclipse CDT.
  

Cons

• You have to manually set up test suites.
  
• It is an extra library to install.

Google C++ Testing Frameworks

Pros

• Google test is following a standard unit testing convention XUnit.
  
• Strong focus on simplicity.
• Documentation is short and easy to read.
  

Cons

• It is an extra library to install.

Choice of C++ unit test framework for ShapeLogic C++

I spent quite a bit of time reading the Boost Test documentation, finally I tried Google C++ Testing Framework and got it working very fast.

Build system

The candidates I considered were:

• Boost build
  
• Make

Boost build

Pros

• Boost build is part of Boost.
  
• Clean design, made as a Make replacement.
• Works on most platforms and with most compilers.
• The scripts are pretty short.
  

Cons

• There is a learning curve.
  

Make

Pros

• Make is the standard for build on C++.
• Widely used.
• Works with Eclipse, MSVC, NetBeans.
• Short scripts.
  

Cons

• It has gotten messy over time.
  
• Shell script dependency.
  
• There is too much magic for my taste.
  

Choice of build system for ShapeLogic C++

I chose to go with Boost Build because it has a cleaner design, but Make looks very competitive when looking over the pros and cons.

Compilers under Windows

In order to compile Boost you need a pretty modern and standard compliant compiler. The candidates that I looked at are:

• Cygwin GCC
  
• MinGW GCC
• MSVC Microsoft Visual C++
  

Cygwin GCC

Pros

• Cygwin GCC is close to GCC under UNIX
  

Cons

• Uses emulation of UNIX system call.
  
• You can only use it to build GPL compatible application.
  

MinGW GCC

Pros

• MingGW integrates well with Eclipse CDT.
• Works more natively with Windows.
• Most libraries build fine with MinGW.
  

Cons

• It was supposed to be able to build FLTK, but I tried a few times and could not get it to work.
• In order to run Make files you also have to install MSYS, which is a minimal shell.
  

MSVC, Microsoft Visual C++

Pros

• MSVC is a high quality compiler.
  
• Most used compiler under Windows.
  
• There is a free Express version.
  

Cons

• There seems to be some restrictions of the Express version that I did not quite understand.
  

Choice of compiler under Windows for ShapeLogic C++

MSVC.

C++ IDEs under UNIX

The candidates I considered were:

• Eclipse 3.4
  
• Emacs / Xemacs
• NetBeans 6.1
  

Eclipse 3.4

Pros

• Eclipse 3.4 CDT has a good debugger.
• Easy to jump from classes to files defining the classes.
  

Cons

• Not nearly as good as Eclipse for Java.
• Unstable under Linux AMD64.

Emacs

Pros

• Emacs is powerful tools that runs in a terminal.
• Takes up less resources.
• Not dependent on Java.
  

Cons

• The Java bases IDE have more features.
• Demands more knowledge to use.
  

NetBeans 6.1

Pros

• NetBeans 6.1 seem a little lighter than Eclipse.

Cons

• It is made to work with Make files, and ShapeLogic C++ is using Boost Build / Bjam.
  

Choice of IDE under UNIX for ShapeLogic C++

Eclipse.

Summary of libraries and tools used

• Boost the C++ library
• Generic Image Library for writing new image processing code
• OpenCV for existing computer vision algorithms
• FLTK, Fast Light Toolkit lightweight cross platform GUI
• Google C++ Testing Framework
• Boost.build v2 for command line based build system

Status of ShapeLogic C++

ShapeLogic C++ 0.4 is the first alpha release. It can do some useful work, but it still mainly an example application.

Currently has

• Comes with some image processing operation
• Comes with 3 brushes
• It is pretty simple to program an image processing algorithm
  

Missing

• Drawing is currently slow and there is only one pen size
• None of the ShapeLogic Java algorithms have been ported yet
• Documentation is poor

Hardest problems

• Learning how FLTK works
• Building a cross platform C++ build script covering several libraries

None of these problems will effect ShapeLogic users.

Porting computer vision code from Java to C++

Before I started porting ShapeLogic from Java, I thought that C++ was moving towards becoming a legacy language. What I have learned from this work is that C++ has advanced substantially since 2002, when I last used it professionally. C++ still seems competitive, at least in computer vision, and according to my old video game colleagues also in games, where I though that C# might have taken a lead by now. Both C++ and Java have substantial advantages.

C++

• OpenCV has a lot of vision algorithms, e.g. face recognition
• C++ is faster than Java
  
• Better for video processing
  
• Programs are shorter
• Generic programming working on primitive types
  
• You can make build script that build under both Windows and UNIX

Java / ImageJ

• ImageJ has more open source algorithms for medical image processing
  
• Better support for medical image files formats under ImageJ
• IDEs are better under Java
• Build process is simpler than C++
  
• Simpler language
• Better support for parallel processing
• A lot easier to dynamically load plugins
  

The next step is to port my framework for declarative programming -- it is based on lazy streams -- and port the Color Particle Analyzer. C++ / Boost have good support for functional programming techniques: Boost.Bind and Boost.Lambda, and the Phoenix library has just been accepted into Boost. When complete, I will do another posting about how it went.

-Sami Badawi
http://www.shapelogic.org

Computer Vision C++ vs Java review

In 2007 I created an open source computer vision project, ShapeLogic, built in Java to work with ImageJ. This setup has been very easy to work with and very productive. Bjarne Stroustrup the creator of C++ gave an interview about the new features in the C++0x standard and TR1. C++ now has a lot of innovating programming constructs e.g. template meta programming, lambda functions, concepts and traits. When I found out that "axiom" is going to be a keyword in C++ my inner mathematician demanded that I take a second look at C++ in connection with computer vision.

This post is a review of my personal past experience with computer vision in C++ and Java. I did my masters thesis in computer vision in the early 90ies, but I ended up working in other fields: video games, Internet and finance, which only left a little time to do vision in my free time. While both C++ and Java were good choices for professional vision programmers, several of the approaches I chose caused me to run out of steam. I also tried to do computer vision with functional, declarative and hybrid languages e.g. Oz, Scheme and Scala but will not cover that here.

Borland C++ early 90ies

C++ did not have STL or any other standard library so I used Borland's OWL library for images and for the application. I used C++ templates, classes with multiple inheritance, RTTI just to set up basic container functionality. There were a few books that has some free C or C++ source code for image processing and vision, but they did not spawn a user community. I did not really get to do anything interesting.

JAI, Java Advanced Imaging late 90ies

I was very excited when Java came around, this was the language to cure all programming ailments. Now they had added a library that could be used for vision and a lot of big companies were sponsoring JAI. It turned out to be a very complex framework with a deep class hierarchy, I spent a lot of time reading the manual trying to find out how to get access to image pixels. I gave up using it and the framework never gained much popularity.

VXL, C++, STL, Boost, Python, GCC, Linux around 2000

Open source software, OSS had started to become prominent. There were 2 OSS libraries:

• OpenCV (Open Computer Vision), wich was still in alpha.
• VXL (Vision X Library) wich was a merge of 2 big non OSS libs TargetJR and IUE.

VXL finally got into beta and I tried to combine it with Python for more high level processing.

Tools needed for build and GUI

• VXL does builds using CMake to create Make files
• Boost uses BJam to do builds
  
• Python bindings using Pyste from Boost
• VXL used FLTK and OpenGL as a GUI

Problems encountered

• It was hard to get the different build systems, CMake, Bjam and Make, to work together
  
• GCC 3.1 and 3.2 core dumped when compiling certain Boost classes
• Python bindings worked for simple C++ classes, but not for the nested template classes in VXL
  
• It was hard to debug the template programs
  
• Emacs was not really as easy to use as Visual Studio
  
• Bad drivers for OpenGL on Linux
  

I actually got some examples set up, but spent more time fighting with the tool stack than doing vision work.

ImageJ in Java around 2004

A colleague showed me a visualization tool he had worked on and said that he did it in around 1 month. I barely believed him, but tried the underlying framework, ImageJ. To my big surprise I was up and running and doing real work in a few hours. ImageJ just got things right. It was built using pure Java by one man, Wayne Rasband. It is very easy to work with and very modular, so a lot of people have made plugins and there is a vibrant development community. When I started working on ShapeLogic that was the best choice.

OpenCV, GIL Generic Image Library, Boost and Eclipse in C++ 2008

In the light of advance in the C++ language and tools, I have decided to try it again.
C++ image libraries choices

• OpenCV, Open Computer Vision
  
• Openframeworks built on top of OpenCV
  
• GIL Generic Image Library
• VXL

I chose to start with OpenCV made by Intel and GIL made by Adobe but a part of Boost since 1.35.

C++ IDE tried

• Eclipse 3.4
  
• NetBeans 6.1
  

Eclipse worked better for me, it has its own build system so you do not have to mess with Make files.

C++ cross platform GUIs

• FLTK Fast Light Toolkit
• wxWidgets
• HighGui from OpenCV

Not sure which one will be best for my purpose.

First attempt
I tried Boost, OpenCV and GIL and got them up and running under both Linux and Windows in a few hours. Eclipse CDT C++ IDE works great.

Porting ShapeLogic algorithms to C++ version

My plan is to port some algorithms from ShapeLogic from Java to C++. ShapeLogic is a toolkit for declarative programming, specialized for vision. In principle you should be able to make a list of rules for categorizing say the shape of a particle in a particle analyzer. You put them in a database or a flat file and the same rules should work for C++ and Java version of ShapeLogic. In practice this might not work out.

Advantages of C++ and Java

This is a loose first assessment.
Constructs used in ShapeLogic that are missing or less convenient in C++

• Uniform cross platform GUI
• Dynamic cross platform libraries
• HashTable
• Reflection
• Garbage collection
• Antlr for parsing logic language
  

Advantages of C++ over Java in vision

• Substantially higher speed
  
• Better handling of video
• Used more frequently for computer vision programming
  
• Good tracking and face recognition algorithms in OpenCV
  

For me, Java has been very good for doing medical image processing algorithms. I have heard conflicting evidence about whether it is feasible for doing computer vision on video using Java. Video handling in Java has been bad up to now, this is supposed to be fixed with the new JavaFX. Shadow Monsters is a computer vision based art piece taking video footage of silhouette of the viewer and adding monsters to them, I saw it on display at Museum of Modern Art. It was programmed using Processing, which is a Java based image processing tool for artists. I discussed the issue with a computer programmer / artist who said that he had tried to do a motion algorithm in Processing and had to port it to C++ based Openframeworks since Java was too slow. After being discouraged by my prior attempts to do vision in C++, I am very happy to see the dramatic developments in C++ and see if it is suitable for a simple port of ShapeLogic algorithms. The result of this C++ port will be covered in 2 postings:

• Computer vision C++ libraries review
• Declarative framework and Particle Analyzer Java to C++ port
  

-Sami Badawi
http://www.shapelogic.org

ShapeLogic 1.2 with color particle analyzer released

Here are the release notes for ShapeLogic 1.2

Changes

• Particle analyzer working directly on color and gray scale images without manual user intervention
• Both particle counter and particle analyzer now take parameters and print reports about each particle's color, area, standard deviation to result table
• Color replacer replaces one color within a tolerance with another color. Parameter input dialog with preview check box
• Organize plugins and macros under ShapeLogic? and ShapeLogicOld? menus, until 1.1 they where all placed under shapelogic menu
• ShapeLogic still has beta development status

The particle analyzer in ShapeLogic v 1.2 has gone through limited
testing and seems to work well. There is still a bug in the edge
tracer.

Using particle analyzer as a ImageJ plugin

The particle analyzer was tested on the particle sample images from ImageJ embryos.jpg

To run it from ImageJ select "Color Particle Analyzer" in the ShapeLogic? menu:

First a particle count dialog is displayed:

Here is the result of running the non-customized particle analyzer on it. This is written to a result table that can be exported to Excel:

The categories for the particles are only examples, it is easy to setup different rules for categorizing particles. In ShapeLogic? 1.3 there will be custom rules to recognize specific cells.

ShapeLogic? 1.2 also contains the second version of a color particle counter. It also prints a smaller report of the particle's properties.

Plans for next release

The next release, ShapeLogic v 1.3, will be a more mature particle analyzer which will come with custom rules to recognize specific cells.

Seeking particle images

In order to create these rules, I am looking for images of particles on a relatively uniform background. Please let me know if you have sample images that I could work from, preferably standard images like the embryo sample image that comes with ImageJ.

Possible future plans for particle analyzer

• Create rule for recognizing cells using neural networks or machine learning techniques
  
• Be able to handle a background that is not uniform, and cell organelles
• Incorporate reasoning under uncertainty using the lazy stream library
  
• Find overlapping particles and distinguish them as separate

Download ShapeLogic 1.2

-Sami Badawi
http://www.shapelogic.org

ShapeLogic 1.1 with particle counter released

Here are the release notes for ShapeLogic 1.1

Changes

• Particle counter working directly on color and gray scale images without manual user intervention
• Particle counter finds average color, standard deviation, area and location for each particle
• Framework to build more advanced particle counters and particle analyzers
• Color clustering using K-mean algorithm
• Background color finder
• Extend all the image processing algorithms in ShapeLogic to work in both ImageJ and in plain Java
• Better support for NetBeans
• ShapeLogic still has beta development status
• 29000 lines of Java code
• 440 unit test that all works on local machine

The particle counter in ShapeLogic v 1.1 has gone through limited testing, and seems to work well though tweaking the parameters is still a bit clumsy. Users looking for a mature particle counter should probably wait for ShapeLogic v 1.2.

Test on sample images from ImageJ

The particle counter was tested on the particle images from ImageJ:

embryos.jpg. The un-tweaked particle counter in ShapeLogic 1.1 found

particle count = 9

embryos.jpg contains 6 particles and a few shadows.

After changing the parameter setting it found

particle count = 5

which is the correct value since ShapeLogic 1.1 cannot split overlapping particles.

Direction

This is the first use of ShapeLogic in medical image processing. The next few releases should also be focused on the particle counter, making it more robust and automatic. This is the plan for ShapeLogic 1.2:

• More testing and tweaking of particle counter
• Make it easier to set parameters for particle counter in a macro or a configuration file
• Print report about each particle's color, area, standard deviation to a file
• Different implementations of particle counters
• Vectorize particles by tracing the edge
• Filter particles based geometric properties of the edge using the same techniques as letter matcher
  

Download ShapeLogic 1.1

-Sami Badawi
http://www.shapelogic.org

ShapeLogic 1.0 with stream based rules released

Here are the release notes for ShapeLogic 1.0

Changes

• Rule for image processing have been migrated, previously they were implemented as goal driven tasks with sub tasks. Version 1.0 uses lazy streams which are simpler and more powerful.
• Letter match example now matches all polygons instead of just the first found.
• When running ShapeLogic as ImageJ plugin, it is now easy for users to define rules for matching in external Java files.
• New number matcher to demonstrate how to define rules for matching in an external Java file in 130 lines of code.
• Enabled use of Java 6 Scripting for rule database, which gives the user access to the 25 scripting languages that are supported
• ShapeLogic now has beta development status
• Many unit tests added for Lazy Stream library
• Fixed bugs in Lazy Stream library
• Fixed bugs in vectorizer

Lazy streams features

• Lazy streams can be named and defined based on other lazy streams
  
• They work similarly to UNIX pipes or calculation Legos
• They serve as your query construct, you can directly query them
• A stream can be wrapped around an Iterator
• A stream can be created from an input stream and a Calculation
  
• They can be instantiated lazily, so you can load calculation networks independently
  

The lazy stream worked very well with the letter match. It started out as a functional construct, but with the named streams they got a more declarative feel to them.

The Calculation in the stream is using the same signature as Neal Grafter's Java 7 closure prototype. This might make integration with Java 7 easier.

User defined rules for number match

The most significant change is that it has become a lot simpler for users to define rules. Here is the start of DigitStreamVectorizer_ the number matchre in 130 lines of code. Notice that there is barely any setup code. It is almost only rules, defined in a very simple format.

public class DigitStreamVectorizer_ extends StreamVectorizer_ {

@Override
public void matchSetup(ImageProcessor ip) {
 loadDigitStream();
}

public static void loadDigitStream() {
 LoadPolygonStreams.loadStreamsRequiredForLetterMatch();
 makeDigitStream();
 String[] digits = {"0", "1", "2", "3", "4", "5", "6", "7", "8", "9"};
LoadLetterStreams.makeLetterXOrStream(digits);
}

public static void makeDigitStream() {

 rule("0", HOLE_COUNT, "==", 1.);
 rule("0", T_JUNCTION_POINT_COUNT, "==", 0.);
 rule("0", END_POINT_COUNT, "==", 0.);
 rule("0", MULTI_LINE_COUNT, "==", 1.);
 rule("0", CURVE_ARCH_COUNT, ">", 0.);
 rule("0", HARD_CORNER_COUNT, "==", 0.);
 rule("0", SOFT_POINT_COUNT, ">", 0.);

 rule("1", HOLE_COUNT, "==", 0.);
 rule("1", T_JUNCTION_LEFT_POINT_COUNT, "==", 0.);
 rule("1", T_JUNCTION_RIGHT_POINT_COUNT, "==", 0.);
 rule("1", END_POINT_BOTTOM_POINT_COUNT, "==", 1.);
 rule("1", HORIZONTAL_LINE_COUNT, "==", 0.);
 rule("1", VERTICAL_LINE_COUNT, "==", 1.);
 rule("1", END_POINT_COUNT, "==", 2.);
 rule("1", MULTI_LINE_COUNT, "==", 0.);
 rule("1", SOFT_POINT_COUNT, "==", 0.);
 rule("1", ASPECT_RATIO, "<", 0.4);  

ShapeLogic's 3 different approaches to declarative programming

Here is a chronological listing of ShapeLogic's 3 different approaches declarative logic:

1. Declarative goal driven logic engine. From ShapeLogic 0.2.
2. Logic filter language. From ShapeLogic 0.8 The syntax and development of the logic language is better described in Logic language.
3. Lazy streams. From ShapeLogic 0.9.

Result of different approaches so far

For the letter matching example, the lazy stream approach has been both simpler and more powerful than the goal driven logic engine.

The Artificial Intelligence choice tree is built into the logical structure of goal driven logic engine, so this approach might work well when reasoning under uncertainty.

The logic filter language is used with both lazy streams and goal driven approach.

ShapeLogic is a toolkit, all 3 approaches are available with unit tests.

For now development is focused on the lazy stream approach.

JSR 223 scripting surprise

I had put a lot of energy into making it easy to create a stream from a Scripting snippet in either Groovy, JRuby or JavaScript. This was clearly superior to text snippet rules defined using Apache Commons JEXL under the Declarative goal driven approach, but using the new Stream it turned out not to be necessary. The rules could easily be defined in plain Java. I still think that good integration with streams and Scripting from ShapeLogic might turn out to be useful.

Download ShapeLogic 1.0

-Sami Badawi
http://www.shapelogic.org

Lazy streams in mathematics and vision

ShapeLogic 0.9 contains new functional and declarative constructs

1. Lazy streams
2. Simplified and expanded lazy calculations

The functional and declarative construct of ShapeLogic can be used independently of the image processing code. It only requires a 200 KB jar file to use this in other application, and there is no dependency of third party jars.

ShapeLogic 0.9 letter recognition example is still using a different system for declarative programming. More on that later in this post.

To test lazy streams in pure form I tried them on the 10 first mathematical problems in Project Euler. I think that ShapeLogic streams provided for simple solutions to the mathematical problems.
However they are more verbose that say solutions in the Scala language:
http://scala-blogs.org/2007/12/project-euler-fun-in-scala.html

Solutions to a few of the Project Euler mathematical problems

Project Euler is a list of 178, mathematical problems, that can be solved by computers.

1 Add all the natural numbers below 1000 that are multiples of 3 or 5

NaturalNumberStream naturalNumberStream = new NaturalNumberStream(1,999);
ListFilterStream<Integer> filter = new BaseListFilterStream<Integer>(naturalNumberStream) {
public boolean evaluate(Integer object) {return object % 3 == 0 || object % 5 == 0;}
};
SumAccumulator accumulator = new SumAccumulator(filter);
System.out.println(accumulator.getValue());

2 Find the sum of all the even-valued terms in the Fibonacci sequence which do not exceed one million

BaseListStream1<Object,Integer>fibonacci = new BaseListStream1<Object,Integer>(){
{ _list.add(1); _list.add(2);}
public Integer invoke(Object input, int index) {return get(index-2) + get(index-1);}
};
ListFilterStream<Integer> filter = new BaseListFilterStream<Integer>(fibonacci) {
public boolean evaluate(Integer object) { return object % 2 == 0; }
};
SumAccumulator accumulator = new SumAccumulator(filter) {
{_inputElement = 0;}
public boolean hasNext(){ return _inputElement <= theNumber; }
};
System.out.println(accumulator.getValue());

More solutions to Project Euler

Here are the next 8 solutions:
10 first solutions to Project Euler in ShapeLogic
10 first solutions to Project Euler as Java unit tests

Comparison between streams and old declarative constructs

In ShapeLogic 0.8 a cornerstone in declarative programming was a goal driven system of tasks with sub tasks. The letter recognition example reads in a rule database and translated it into goals/tasks with sub tasks. Somewhat like the programming language Prolog.
In case of uncertainty the different choices would live in an artificial intelligence choice tree that is tied to the tree of tasks and sub tasks.

Integrating old declarative constructs with lazy streams

Based on what is needed in the current letter match example. It seems like the stream based approach is simpler and able to handle all the problem that the goal driven approach could.
I expect the stream approach to supersede the goal approach.
The rule database will be read and translated to streams. So the rule for the letter A would be translated into a filter that could filter a stream of polygons into a stream of polygons that represent the letter A.

Also currently the rules now are using JEXL to translate a text rule into something executable.
Example:
Rule for letter A: polygon.holeCount == 1.
In ShapeLogic 1.0, you should be able to use Java 6 Scripting, JSR 223, to be able to define these rules, in one of the 25 supported scripting languages. In ShapeLogic 0.9: Groovy, JRuby, and JavaScript was tested with streams, but not with the rule databases.

Using streams for concurrent programming

Streams can also support parallel or concurrent programming, which is important with the CPU intensive operations in image processing and computer vision. Especially with the advent of cheap multi processor machines.

Example: Find polygons in a stack of images

You define a lazy data stream for this and set a stream property
randomAccess = true
This indicates that individual elements can be calculated independently. The factory creating the stream could create a parallel version of the stream and assign each operation its own thread. Note that the result would be a stream of polygons for each image.

-Sami Badawi

ShapeLogic 0.9 with lazy stream library released

Here are the release notes, I will soon describe the changes in more details.

This is the first release where ShapeLogic is moving beyond current parameters as a plugin library for ImageJ, currently only used in a letter recognition example. The improved system will be for declarative programming where the user can define rules in either a database or flat file. The focus will still be on image processing and computer vision, but the system will be more broadly applicable. There has been no new work on image processing or letter recognition in this release. ShapeLogic 1.0 will combine these new changes with the current image processing code.

Changes

• Introduce new functional, declarative and query constructs to Java
• Implement lazy streams like Haskell, Scala or Scheme
• These functional constructs are very lightweight and you only need one 200KB jar file to use it in other applications
• Test streams by solving the first 10 mathematical problems from Project Euler
• Enabled Java 6 Scripting for evaluating expressions.
  
• Tested with Groovy, JRuby, JavaScript, but should work with other supported Scripting languages, currently that are 25 of these. This makes it possible for users to add rule, formulas and queries in real time using text format. They can interact with a running Java application, which can be useful in science, finance or web applications.

-Sami Badawi

ShapeLogic for general declarative programming

ShapeLogic 0.9 is going to be the first release where users define rule sets read from either databases or flat files. I have decided to broaden the scope of the project.

First I hoped that I would be able to make minor adjustment to ShapeLogic's letter match example so that it would read the rules from an external source and gradually expand. But since backward compatibility is not a problem yet, I came to the conclusion that it was better to try to construct a solid foundation for declarative programming. While it will be primarily geared toward computer vision problem solving, the system will have other, broader, applications as well.

Do we need another system for declarative programming?

SQL rules supreme in the field of simple data, but once you go outside this field there are many different directions that you can take.

The declarative logic programming system that I had highest expectations for was CYC. It is a large, hand coded knowledge base trying to capture common sense, using many different inference techniques. Strangely, it never got a big following even after they released part of the engine as open source.

I hope to find a sweet spot where a simple system will be able to do some real work and be simple to learn. My work is geared toward a domain where there are a limited number of objects with somewhat constant features.

A few of the changes in ShapeLogic 0.9

Lazy data streams

Implement lazy data streams like Scala or Scheme.

Decouple annotation

Make the annotation of shapes more loosely coupled to classes in ShapeLogic. I was not super happy with the first solution that I came up with of how to annotate point, lines and polygon.
If ShapeLogic should be used for more general problem this is completely unacceptable.

Better interpreter

I order to save user defined rule in a database or a flat file, I need to be able to compile or interpret this code. This code in the rules should mainly be fairly straightforward.

The top contenders are:

• Java 6 Scripting interface, that should give access to different scripting languages for the JDK


• Java 6 Compiler interface, the problem is that Java is pretty verbose


• Groovy intepreter


• Do the parsing in ShapeLogic and write a little interpreter


• Stick with JEXL for now, and make the move in a later release

-Sami Badawi

Language and tutorials for the external rule base

Language

I have spent a couple of weeks thinking about how to organize the external rule database for ShapeLogic, and have done more reading about Java 6 Scripting. That should be adequate for my need, but I am reluctant to add more dependencies to ShapeLogic than absolutely necessary.

I am now debating what scripting language would be best for Java 6 Scripting:
Groovy: Comes out as maybe the strongest contender, but I tried Groovy 2 times before only to find out it was not ready for prime time yet, but maybe with version 1.5 it is finally there.
BeanShell 2: It has been in beta for over 2 years and does not seem to be in active development.
Jyton: I have been a big fan of Python for almost 10 years now, but the white space indention does not work so well with code stored in a database or flat file.
JavaScript/Rhino: I like it and people know it, but it would be better if it was a language that was using native Java types.

Tutorial

After seeing a 20 minute screen cast for Ruby on Rail by David Heinemeier Hansson, my new test for if a programming library or language is worth spending time on is if it has a 20 minute screen cast, where they can do something non-trivial. I do not adhere to this rigorously.
Given that I have a thicker Danish accent than David Hansson, I have been looking for other options.
One of my friends Joe Orr, see Joe's blog 3DTree Notebook, has created a very interesting alternative to screen casts called Screenbook Maker, it is a program that takes screen shots of a demonstration and adds text to it, to turn it into a tutorial, which is searchable.
Joe has promised me to help make a Screenbook tutorial when the external rule database is released.

-Sami Badawi

Declarative programming using Java 6 Scripting

I am working on moving the declarative programming in ShapeLogic into an external rule database now.

Currently the rules in ShapeLogic are parsed from strings using Apache Commons JEXL library.
So the letter A would have a rule saying:
polygon.holeCount == 1

This is not trivial since a variable say polygon.holeCount could have different values in different contexts.
E.g. if there was a choice of 2 different thresholds levels, then in one part of the choice tree we could have
polygon.holeCount == 1 and in another we could have
polygon.holeCount == 2.

I am considering changing from JEXL to using the Java 6 Scripting instead.
JEXL has not been released for over 1 year, and it is a little awkward to handle static fields and functions.
It might also be better to let the user chose what scripting language they want to use.
Currently there languages should be available for scripting: JavaScript, BeanShell, Jython, Groovy and JRuby.

One issue is that I cannot just use variable binding in a global scripting context.
In my example from above if the variable polygon.holeCount does not exist in the top context, I will have to make sure that it is taken from the right context. This was relatively easy in JEXL since a context here mainly is just a map you store your key values pairs in, I am not sure if this is a problem when you are dealing with a whole dynamic scripting language. I am also a little concerned about performance.

I might make a release of ShapeLogic 0.9 where you just can select another rule database stored in a flat file or a database, but using the current system, in order not to drag the next release out too long. This should allow the users to define rules for matching a separate alphabet, say the Greek.
But it is far from what I want ShapeLogic to be able to do.

-Sami Badawi

Declarative and Object Oriented programming

One of the main objectives for ShapeLogic is to make a good hybrid of Declarative programming and Object Oriented programming. This is not specific to computer vision, but is a general problem. This is a daunting task, and many people have tried and the state of the art still leaves a lot to be desired.

I have started to work on the first release of ShapeLogic with an external rule based engine, I have not worked through all the problems yet. I think that this is a case of evolutionary programming where you have to try out and approach, not knowing if it will lead to anything useful. Hopefully I will have ShapeLogic 0.9 ready pretty soon.

I think that the key is to keep it simple and keep the syntax easy to work with. Let me just give my 2 cents on a few project that combined Declarative programming and Object Oriented programming.

Approaches that impressed me

Prova is a Java Prolog hybrid. I was very impressed by the simplicity and how well it managed to integrate queries with normal database access. Unfortunately I do not think that Prolog is applicable to the approach to computer vision, that I am pursuing in ShapeLogic now.

List Comprehension in Python and Haskell. It is somewhat limited, but it is very convenient to work with.

Microsoft LINQ, I think that it is great that you can use the same simple syntax to query databases, XML and collections.

Hibernate and ORM tools: While I do not think that the dust has settled yet as to how feasible they are for production system with large databases. I think they are very promising. This was the reason that I included Hibernate in ShapeLogic, despite it not being used much yet.

Promising approaches that I found hard to work with

Drools: An open source RETE engine for the Java JVM. It comes with a lot of cool features, but I thought that the example program setting a rule up to calculate Fibonacci numbers was too complicated.

OWL: Works with XML / RDF. It is a standard. It comes with good open source tools, but it just seems too heavy weight for my purpose.

-Sami Badawi