Monday, November 17, 2008

Computer vision C++ libraries review

I am trying to create an easy to use, minimalistic C++ cross-platform computer vision system, with a non-restrictive license. My biggest challenge was to chose the best libraries and to get them to work together; this took some investigation and experimenting. This posting is a brief description of my findings.

This is what ShapeLogic C++ currently looks like:


Windows


Linux

In order to construct ShapeLogic C++, I had to make choices within the following categories:
  • Computer vision and image processing libraries
  • GUI libraries
  • Unit test systems
  • Build systems
  • Compilers under Windows
  • C++ IDEs under UNIX
ImageJ, the Java open source image processing tool, is the inspiration for the first part of my work: it is very simple to learn, use and program in. This is a follow up to my last posting: Computer Vision C++ vs Java. The result of my work is released as an open source project ShapeLogic C++, under the MIT license.

Computer vision and image processing libraries

The candidates I considered were:
  • GIL, Generic Image Library
  • OpenCV
  • VXL

GIL, Generic Image Library

GIL, Generic Image Library by Adobe.

Pros
  • Very non intrusive, only based on header files
  • Puts a wrapper around most image format
  • You can write a algorithm once and it will work for most image types
  • Part of Boost since 1.35
Cons
  • Does not come with a lot of image processing algorithms

OpenCV

OpenCV, Open Computer Vision by Intel.

Pros
  • Very simple
  • Works with both C and C++
  • Very broad range of algorithms
  • Complex algorithms: face detection, convexity defects
  • Very popular
Cons
  • You have to use OpenCV's IplImage
  • IplImage byte order is BGR instead of the normal RGB

VXL, Vision X Library

VXL a combination of 2 big older vision libraries TargetJR and IUE

Pros
  • Well tested technology
  • Simpler build process using cmake
  • Uses modern programming techniques: classes, template and STL
  • It has a lot of functionality
  • Simple to get started with
Cons
  • It is not using normal STL, but in order to work on different compiler it had to make its own version with different names.
  • Class structure is somewhat complex.
Choice computer vision library for ShapeLogic C++
OpenCV for existing image processing and vision algorithms. GIL for writing new algorithms.

Cross platform GUI

The candidates I considered were:
  • GIMP plugin
  • GTK+, GIMP toolkit
  • FLTK
  • HighGui from OpenCV
  • PhotoShop plugin
  • wxWidget

Run ShapeLogic as a GIMP plugin

Pros
  • GIMP is the main cross platform OSS image editing programs.
  • It is in wide use.
  • Has a lot of powerful features including scripting functionality in Scheme and Python.
  • From a user perspective this would be an excellent choice.
Cons
  • GIMP is GPL, but you could have a wrapper around plugins in order to access them as GIMP plugins.
  • The plugin works with tiles, which gives good performance, but does not fit well with either the way OpenCV or GIL are processing images.

GTK+, GIMP Toolkit

Pros
  • GTK+ is a great looking and very powerful framework that works on: Windows, Linux, Mac, a.o.
Cons
  • It is written in C and has a homegrown object system, which is not type safe.

GTKMM C++ wrapper around GTK+

Pros
  • CTKMM is a great looking and very powerful framework, that works on: Windows, Linux, Mac, a.o.
  • It feels natural to program in for a C++ programmer.
Cons
  • The class hierarchy is somewhat deep since it is built on top of GTK.

FLTK, Fast light toolkit

Pros
  • FLTK is very lightweight.
  • Very clean C++, you actually have a main().
  • Native C++ build on top of X11 or Windows.
  • Fluid, a simple GUI builder
Cons
  • Not as many widgets.
  • Dated look.

HighGui from OpenCV

Pros
  • Very lightweight.
  • There is some functionality for displaying images, video and an event handler for mouse events.
Cons
  • Does not come with a menu system.

Run ShapeLogic as a PhotoShop plugin

Pros
  • PhotoShop is the main image editing programs.
  • It is in wide use, has a lot of powerful features including macros.
  • From a user perspective this would be an excellent choice.
Cons
  • The PhotoShop SDK is not freely available, you have to apply to get it.
  • The plugin does not fit well with either the way OpenCV or GIL are processing images.

wxWidgets

Pros
  • wxWidgets is a full featured GUI toolkit, built on top of native toolkits: Win32, Mac OS X, GTK+, X11, Motif, WinCE and more.
  • Looks good and modern.
  • Big community.
  • Several GUI builders.
Cons
  • The programming style is close to Windows MFC programming.
  • There are many layers.
Choice
This was a hard choice and I went back and forth between FLTK and wxWidgets, but went with FLTK. All the GUI code is separate from the image processing code, so if I wanted to change from FLTK to another toolkit later it should not be too dramatic.

C++ unit test frameworks

There are a lot of different choices and no clear leader. Some of the candidates were:
  • Boost.test
  • CppUnit
  • Google C++ Testing Framework

Boost.test

Pros
  • Boost.test is part of the Boost library.
  • Powerful with a lot of options.
Cons
  • You have to manually set up test suites.
  • It is somewhat heavy.
  • The documentation is extensive but not easy to read.

CppUnit

Pros
  • CppUnit is following a standard unit testing convention XUnit.
  • Integration with Eclipse CDT.
Cons
  • You have to manually set up test suites.
  • It is an extra library to install.

Google C++ Testing Frameworks

Pros
  • Google test is following a standard unit testing convention XUnit.
  • Strong focus on simplicity.
  • Documentation is short and easy to read.
Cons
  • It is an extra library to install.

Choice of C++ unit test framework for ShapeLogic C++

I spent quite a bit of time reading the Boost Test documentation, finally I tried Google C++ Testing Framework and got it working very fast.

Build system

The candidates I considered were:
  • Boost build
  • Make

Boost build

Pros
  • Boost build is part of Boost.
  • Clean design, made as a Make replacement.
  • Works on most platforms and with most compilers.
  • The scripts are pretty short.
Cons
  • There is a learning curve.

Make

Pros
  • Make is the standard for build on C++.
  • Widely used.
  • Works with Eclipse, MSVC, NetBeans.
  • Short scripts.
Cons
  • It has gotten messy over time.
  • Shell script dependency.
  • There is too much magic for my taste.

Choice of build system for ShapeLogic C++

I chose to go with Boost Build because it has a cleaner design, but Make looks very competitive when looking over the pros and cons.

Compilers under Windows

In order to compile Boost you need a pretty modern and standard compliant compiler. The candidates that I looked at are:
  • Cygwin GCC
  • MinGW GCC
  • MSVC Microsoft Visual C++

Cygwin GCC

Pros
  • Cygwin GCC is close to GCC under UNIX
Cons
  • Uses emulation of UNIX system call.
  • You can only use it to build GPL compatible application.

MinGW GCC

Pros
  • MingGW integrates well with Eclipse CDT.
  • Works more natively with Windows.
  • Most libraries build fine with MinGW.
Cons
  • It was supposed to be able to build FLTK, but I tried a few times and could not get it to work.
  • In order to run Make files you also have to install MSYS, which is a minimal shell.

MSVC, Microsoft Visual C++

Pros
  • MSVC is a high quality compiler.
  • Most used compiler under Windows.
  • There is a free Express version.
Cons
  • There seems to be some restrictions of the Express version that I did not quite understand.

Choice of compiler under Windows for ShapeLogic C++

MSVC.

C++ IDEs under UNIX

The candidates I considered were:
  • Eclipse 3.4
  • Emacs / Xemacs
  • NetBeans 6.1

Eclipse 3.4

Pros
  • Eclipse 3.4 CDT has a good debugger.
  • Easy to jump from classes to files defining the classes.
Cons
  • Not nearly as good as Eclipse for Java.
  • Unstable under Linux AMD64.

Emacs

Pros
  • Emacs is powerful tools that runs in a terminal.
  • Takes up less resources.
  • Not dependent on Java.
Cons
  • The Java bases IDE have more features.
  • Demands more knowledge to use.

NetBeans 6.1

Pros
Cons
  • It is made to work with Make files, and ShapeLogic C++ is using Boost Build / Bjam.

Choice of IDE under UNIX for ShapeLogic C++

Eclipse.

Summary of libraries and tools used

Status of ShapeLogic C++

ShapeLogic C++ 0.4 is the first alpha release. It can do some useful work, but it still mainly an example application.

Currently has
  • Comes with some image processing operation
  • Comes with 3 brushes
  • It is pretty simple to program an image processing algorithm
Missing
  • Drawing is currently slow and there is only one pen size
  • None of the ShapeLogic Java algorithms have been ported yet
  • Documentation is poor

Hardest problems

  • Learning how FLTK works
  • Building a cross platform C++ build script covering several libraries
None of these problems will effect ShapeLogic users.

Porting computer vision code from Java to C++

Before I started porting ShapeLogic from Java, I thought that C++ was moving towards becoming a legacy language. What I have learned from this work is that C++ has advanced substantially since 2002, when I last used it professionally. C++ still seems competitive, at least in computer vision, and according to my old video game colleagues also in games, where I though that C# might have taken a lead by now. Both C++ and Java have substantial advantages.

C++
  • OpenCV has a lot of vision algorithms, e.g. face recognition
  • C++ is faster than Java
  • Better for video processing
  • Programs are shorter
  • Generic programming working on primitive types
  • You can make build script that build under both Windows and UNIX
Java / ImageJ
  • ImageJ has more open source algorithms for medical image processing
  • Better support for medical image files formats under ImageJ
  • IDEs are better under Java
  • Build process is simpler than C++
  • Simpler language
  • Better support for parallel processing
  • A lot easier to dynamically load plugins
The next step is to port my framework for declarative programming -- it is based on lazy streams -- and port the Color Particle Analyzer. C++ / Boost have good support for functional programming techniques: Boost.Bind and Boost.Lambda, and the Phoenix library has just been accepted into Boost. When complete, I will do another posting about how it went.

-Sami Badawi
http://www.shapelogic.org