Ying Tong Iddle I Po

Update: It appears the wordpress/LJ cross poster can’t handle indentation. I suggest you just download the full source from the link at the bottom.

Update 2: Fixed it (whit a bit of a hack).

This time we’ll learn how to use one of Firtree’s most powerful features: kernels. Firtree includes a little C-like language which can be used to specify image processing operations. In essence, it is a function that is called once per output pixel and is asked to compute the colour of that pixel. The kernel can, itself, make use of samplers just like a rendering engine.

Under the covers Firtree compiles all of your kernel functions (and it’s the samplers it uses) into one optimised machine code routine.

Firstly, we’ll write a Python function which can help us compile kernels. It takes a string containing some kernel source and returns a kernel and a sampler for that kernel. It also prints out an error log if you made a mistake in the syntax:

 Python | 
 copy code |
?
 
01
<pre>def compile_kernel(source):
02
        """
03
        A simple function which compiles a kernel, checks that the
04
        compilation succeeded and returns a pair containing the kernel and a
05
        sampler for it. 
06
        """
07
 
08
        kernel = ft.Kernel()
09
        kernel.compile_from_source(source)
10
        if not kernel.get_compile_status():
11
                print("Error compiling kernel:")
12
                print("\n".join(kernel.get_compile_log()))
13
                return
14
 
15
        kernel_sampler = ft.KernelSampler()
16
        kernel_sampler.set_kernel(kernel)
17
 
18
        return (kernel, kernel_sampler)</pre>

Now let’s actually create a kernel. I’m going to use the example of a desaturate kernel, essentially converting a colour image into a black and white one:

 Python | 
 copy code |
?
 
01
<pre># Create a desaturate kernel.
02
(desat, desat_sampler) = compile_kernel("""
03
        kernel vec4 desaturate(sampler src)
04
        {
05
                vec4 src_colour = unpremultiply( sample(src, samplerCoord(src)) );
06
                float luminance = dot(src_colour, vec4(0.299,0.587,0.114,0));
07
                return premultiply(
08
                        vec4(luminance,luminance,luminance,src_colour.a)
09
                );
10
        }
11
""")</pre>

The kernel language itself should be familiar to anyone who has programmed in C, GLSL or CoreImage. I’ll explain some of the functions we use:

• [un]premultiply – Firtree uses a pre-multiplied representation internally. That is to say that the red, green and blue components of a colour are pre-multiplied by the alpha value. These functions can be used to undo and redo this operation.
• sample – Call a sampler. It takes two parameters: one is the sampler to sample from and the second is a 2d vector specifying where to do so.
• samplerCoord – Samplers have associated with them a co-ordinate transform. This function returns the co-ordinate in the sampler’s co-ordinate system of the current pixel.
• dot – Perform a dot-product between two vectors. In this case, it takes the right combination of red, green and blue components to return a luminance value.

We now need to tell the kernel what sampler to use for ’src’. We do this with one line of Python:

 Python | 
 copy code |
?
 
1
# Wire the lena sampler into the desaturate kernel.
2
desat['src'] = lena_sampler

Finally we need to tell the CPU renderer to use the desat_sampler sampler instead of the Lena one:

 Python | 
 copy code |
?
 
1
<pre># Use the engine to render the output.
2
engine.set_sampler(desat_sampler)
3
engine.render_into_cairo_surface(
4
        lena_sampler.get_extent(),         # what area of the input to render
5
        output_surface                     # into what
6
        )</pre>

The output image is what we wanted, a desaturated version of Lena.

If one wanted to check that Firtree is indeed doing some work behind the scenes, we can get it to print out the compiled assembler for the function. Simply add the following line:

 Python | 
 copy code |
?
 
1
print(ft.debug_dump_cpu_renderer_asm(engine, ft.FORMAT_RGBA32))

Compating the resulting output to the kernel language input clearly shows how much easier it is writing image processing kernels in Firtree! :)

Next time, we’ll see how to chain kernels together and let Firtree worry about the details.

The full source code is available.

I’m going to write a set of blog posts all about how to use Firtree from Python. This post is about the simplest thing that you can do with Firtree, load an image and write it back out again.

To get Firtree on Ubuntu Karmic, you can add the Firtree PPA to your system and install the python-firtree package.

Firtree is based around the concept of a sampler. A sampler in essence knows how to get the colour of a pixel given it’s location. The location is specified as a 2d vector of floats and the colour is a 4d vector of floats. The colour is made up of the red, green, blue and alpha components scaled into the rage zero to one.

Our first example will load our input, the ubiquitous Lena, into a Cairo surface and create a sampler which knows how to get data out of that surface:

 Python | 
 copy code |
?
 
1
import cairo
2
import pyfirtree as ft
3
 
4
# Firstly, load the lena image
5
lena_surface = cairo.ImageSurface.create_from_png('lena.png')
6
 
7
# Create a sampler for the surface
8
lena_sampler = ft.CairoSurfaceSampler()
9
lena_sampler.set_cairo_surface(lena_surface)

So far, so easy. Firtree also has the concept of a renderer which knows how to run over each pixel in an output, ask the sampler for the appropriate pixel colour and write it out. Firtree ships with a CPU based renderer which uses LLVM to compile your pipeline down into efficient code and run it over all the CPUs in your machine. Let’s make use of that:

 Python | 
 copy code |
?
 
01
# Create an output surface similar to the input
02
output_surface = cairo.ImageSurface(
03
	cairo.FORMAT_ARGB32,
04
	lena_surface.get_width(),
05
	lena_surface.get_height() )
06
 
07
# Create a CPU render engine.
08
engine = ft.CpuRenderer()
09
 
10
# Use the engine to write the input to the output.
11
engine.set_sampler(lena_sampler)
12
engine.render_into_cairo_surface(
13
	lena_sampler.get_extent(), 	# what area of the input to render
14
	output_surface 			# into what
15
	)
16
 
17
# Write the output
18
output_surface.write_to_png('output.png')

And that is it, you have written some code that loads an input file and writes it back out to another file. Next time you’ll learn how to make use of the main feature of Firtree: image processing kernels.

The source code for this example is available.