Computer vision C++ libraries review

I am trying to create an easy to use, minimalistic C++ cross-platform computer vision system, with a non-restrictive license. My biggest challenge was to chose the best libraries and to get them to work together; this took some investigation and experimenting. This posting is a brief description of my findings.

This is what ShapeLogic C++ currently looks like:


Windows


Linux

In order to construct ShapeLogic C++, I had to make choices within the following categories:

• Computer vision and image processing libraries
• GUI libraries
  
• Unit test systems
• Build systems
• Compilers under Windows
  
• C++ IDEs under UNIX
  

ImageJ, the Java open source image processing tool, is the inspiration for the first part of my work: it is very simple to learn, use and program in. This is a follow up to my last posting: Computer Vision C++ vs Java. The result of my work is released as an open source project ShapeLogic C++, under the MIT license.

Computer vision and image processing libraries

The candidates I considered were:

• GIL, Generic Image Library
  
• OpenCV
• VXL

GIL, Generic Image Library

GIL, Generic Image Library by Adobe.

Pros

• Very non intrusive, only based on header files
  
• Puts a wrapper around most image format
• You can write a algorithm once and it will work for most image types
• Part of Boost since 1.35
  

Cons

• Does not come with a lot of image processing algorithms
  

OpenCV

OpenCV, Open Computer Vision by Intel.

Pros

• Very simple
• Works with both C and C++
• Very broad range of algorithms
• Complex algorithms: face detection, convexity defects
• Very popular
  

Cons

• You have to use OpenCV's IplImage
• IplImage byte order is BGR instead of the normal RGB
  

VXL, Vision X Library

VXL a combination of 2 big older vision libraries TargetJR and IUE

Pros

• Well tested technology
  
• Simpler build process using cmake
  
• Uses modern programming techniques: classes, template and STL
  
• It has a lot of functionality
  
• Simple to get started with

Cons

• It is not using normal STL, but in order to work on different compiler it had to make its own version with different names.
• Class structure is somewhat complex.
  

Choice computer vision library for ShapeLogic C++
OpenCV for existing image processing and vision algorithms. GIL for writing new algorithms.

Cross platform GUI

The candidates I considered were:

• GIMP plugin
• GTK+, GIMP toolkit
• FLTK
• HighGui from OpenCV
• PhotoShop plugin
• wxWidget

Run ShapeLogic as a GIMP plugin

Pros

• GIMP is the main cross platform OSS image editing programs.
  
• It is in wide use.
  
• Has a lot of powerful features including scripting functionality in Scheme and Python.
  
• From a user perspective this would be an excellent choice.

Cons

• GIMP is GPL, but you could have a wrapper around plugins in order to access them as GIMP plugins.
• The plugin works with tiles, which gives good performance, but does not fit well with either the way OpenCV or GIL are processing images.

GTK+, GIMP Toolkit

Pros

• GTK+ is a great looking and very powerful framework that works on: Windows, Linux, Mac, a.o.

Cons

• It is written in C and has a homegrown object system, which is not type safe.

GTKMM C++ wrapper around GTK+

Pros

• CTKMM is a great looking and very powerful framework, that works on: Windows, Linux, Mac, a.o.
  
• It feels natural to program in for a C++ programmer.

Cons

• The class hierarchy is somewhat deep since it is built on top of GTK.

FLTK, Fast light toolkit

Pros

• FLTK is very lightweight.
  
• Very clean C++, you actually have a main().
  
• Native C++ build on top of X11 or Windows.
• Fluid, a simple GUI builder
  

Cons

• Not as many widgets.
  
• Dated look.

HighGui from OpenCV

Pros

• Very lightweight.
  
• There is some functionality for displaying images, video and an event handler for mouse events.

Cons

• Does not come with a menu system.

Run ShapeLogic as a PhotoShop plugin

Pros

• PhotoShop is the main image editing programs.
  
• It is in wide use, has a lot of powerful features including macros.
  
• From a user perspective this would be an excellent choice.

Cons

• The PhotoShop SDK is not freely available, you have to apply to get it.
• The plugin does not fit well with either the way OpenCV or GIL are processing images.

wxWidgets

Pros

• wxWidgets is a full featured GUI toolkit, built on top of native toolkits: Win32, Mac OS X, GTK+, X11, Motif, WinCE and more.
• Looks good and modern.
  
• Big community.
• Several GUI builders.
  

Cons

• The programming style is close to Windows MFC programming.
  
• There are many layers.

Choice
This was a hard choice and I went back and forth between FLTK and wxWidgets, but went with FLTK. All the GUI code is separate from the image processing code, so if I wanted to change from FLTK to another toolkit later it should not be too dramatic.

C++ unit test frameworks

There are a lot of different choices and no clear leader. Some of the candidates were:

• Boost.test
• CppUnit
  
• Google C++ Testing Framework
  

Boost.test

Pros

• Boost.test is part of the Boost library.
  
• Powerful with a lot of options.
  

Cons

• You have to manually set up test suites.
  
• It is somewhat heavy.
  
• The documentation is extensive but not easy to read.
  

CppUnit

Pros

• CppUnit is following a standard unit testing convention XUnit.
• Integration with Eclipse CDT.
  

Cons

• You have to manually set up test suites.
  
• It is an extra library to install.

Google C++ Testing Frameworks

Pros

• Google test is following a standard unit testing convention XUnit.
  
• Strong focus on simplicity.
• Documentation is short and easy to read.
  

Cons

• It is an extra library to install.

Choice of C++ unit test framework for ShapeLogic C++

I spent quite a bit of time reading the Boost Test documentation, finally I tried Google C++ Testing Framework and got it working very fast.

Build system

The candidates I considered were:

• Boost build
  
• Make

Boost build

Pros

• Boost build is part of Boost.
  
• Clean design, made as a Make replacement.
• Works on most platforms and with most compilers.
• The scripts are pretty short.
  

Cons

• There is a learning curve.
  

Make

Pros

• Make is the standard for build on C++.
• Widely used.
• Works with Eclipse, MSVC, NetBeans.
• Short scripts.
  

Cons

• It has gotten messy over time.
  
• Shell script dependency.
  
• There is too much magic for my taste.
  

Choice of build system for ShapeLogic C++

I chose to go with Boost Build because it has a cleaner design, but Make looks very competitive when looking over the pros and cons.

Compilers under Windows

In order to compile Boost you need a pretty modern and standard compliant compiler. The candidates that I looked at are:

• Cygwin GCC
  
• MinGW GCC
• MSVC Microsoft Visual C++
  

Cygwin GCC

Pros

• Cygwin GCC is close to GCC under UNIX
  

Cons

• Uses emulation of UNIX system call.
  
• You can only use it to build GPL compatible application.
  

MinGW GCC

Pros

• MingGW integrates well with Eclipse CDT.
• Works more natively with Windows.
• Most libraries build fine with MinGW.
  

Cons

• It was supposed to be able to build FLTK, but I tried a few times and could not get it to work.
• In order to run Make files you also have to install MSYS, which is a minimal shell.
  

MSVC, Microsoft Visual C++

Pros

• MSVC is a high quality compiler.
  
• Most used compiler under Windows.
  
• There is a free Express version.
  

Cons

• There seems to be some restrictions of the Express version that I did not quite understand.
  

Choice of compiler under Windows for ShapeLogic C++

MSVC.

C++ IDEs under UNIX

The candidates I considered were:

• Eclipse 3.4
  
• Emacs / Xemacs
• NetBeans 6.1
  

Eclipse 3.4

Pros

• Eclipse 3.4 CDT has a good debugger.
• Easy to jump from classes to files defining the classes.
  

Cons

• Not nearly as good as Eclipse for Java.
• Unstable under Linux AMD64.

Emacs

Pros

• Emacs is powerful tools that runs in a terminal.
• Takes up less resources.
• Not dependent on Java.
  

Cons

• The Java bases IDE have more features.
• Demands more knowledge to use.
  

NetBeans 6.1

Pros

• NetBeans 6.1 seem a little lighter than Eclipse.

Cons

• It is made to work with Make files, and ShapeLogic C++ is using Boost Build / Bjam.
  

Choice of IDE under UNIX for ShapeLogic C++

Eclipse.

Summary of libraries and tools used

• Boost the C++ library
• Generic Image Library for writing new image processing code
• OpenCV for existing computer vision algorithms
• FLTK, Fast Light Toolkit lightweight cross platform GUI
• Google C++ Testing Framework
• Boost.build v2 for command line based build system

Status of ShapeLogic C++

ShapeLogic C++ 0.4 is the first alpha release. It can do some useful work, but it still mainly an example application.

Currently has

• Comes with some image processing operation
• Comes with 3 brushes
• It is pretty simple to program an image processing algorithm
  

Missing

• Drawing is currently slow and there is only one pen size
• None of the ShapeLogic Java algorithms have been ported yet
• Documentation is poor

Hardest problems

• Learning how FLTK works
• Building a cross platform C++ build script covering several libraries

None of these problems will effect ShapeLogic users.

Porting computer vision code from Java to C++

Before I started porting ShapeLogic from Java, I thought that C++ was moving towards becoming a legacy language. What I have learned from this work is that C++ has advanced substantially since 2002, when I last used it professionally. C++ still seems competitive, at least in computer vision, and according to my old video game colleagues also in games, where I though that C# might have taken a lead by now. Both C++ and Java have substantial advantages.

C++

• OpenCV has a lot of vision algorithms, e.g. face recognition
• C++ is faster than Java
  
• Better for video processing
  
• Programs are shorter
• Generic programming working on primitive types
  
• You can make build script that build under both Windows and UNIX

Java / ImageJ

• ImageJ has more open source algorithms for medical image processing
  
• Better support for medical image files formats under ImageJ
• IDEs are better under Java
• Build process is simpler than C++
  
• Simpler language
• Better support for parallel processing
• A lot easier to dynamically load plugins
  

The next step is to port my framework for declarative programming -- it is based on lazy streams -- and port the Color Particle Analyzer. C++ / Boost have good support for functional programming techniques: Boost.Bind and Boost.Lambda, and the Phoenix library has just been accepted into Boost. When complete, I will do another posting about how it went.

-Sami Badawi
http://www.shapelogic.org