Ying Tong Iddle I Po

Right, people. I would like to propose that this has now completely jumped the shark.

The current compiler in Firtree is based on the GLSL reference compiler released by 3DLabs and the GLSL backend is very much tied into it. This was useful for getting Firtree up and running but it is sub optimal. Firstly it is hard to extend the compiler and secondly it is hard to write a new backend.

So, on the back burner, I’ve been playing with a more traditional compiler → intermediate representation (IR) → pluggable backend model using the wonderful LLVM. Using LLVM has a number of advantages, not least that it comes with a good vector-aware x86 backend already and it has a number of common optmisations which can be applied directly to the IR.

The (so far unintegrated) compiler is currently living in a bzr branch on Launchpad. It comes with a little debug tool called, rather boringly, compilekernel. To see the advantages that come with LLVM, consider the following simple kernel:

$ cat test/testkernel.knl
kernel vec4 desaturateKernel(sampler src)
{
  __color a = sample(src, samplerCoord(src));
  vec4 rgb2int = vec4(0.299, 0.587, 0.114, 0.0);
  float intensity = dot(rgb2int, unpremultiply(a));
  return premultiply(vec4(intensity * vec3(1,1,1), a.a));
}

Compiling with no LLVM optimisation and no GLSL optimisation (more on this later) leads to the following:

$ ./compiletest -print=glsl -no-opt-llvm -no-opt-glsl test/testkernel.knl
// Definition of desaturateKernel_s.
vec4 desaturateKernel_s ( int src )
{
    int paramtmp;
    paramtmp = src;
    vec4 a;
    int tmp = paramtmp;
    int tmp1 = paramtmp;
    vec2 tmp2 = samplerCoord_s( tmp1 );
    vec4 tmp3 = sample_sv2( tmp, tmp2 );
    a = tmp3;
    vec4 rgb2int;
    vec4 tmp4 = vec4(0.0,0.0,0.0,0.0);
    tmp4.x = 0.299;
    vec4 tmp5 = tmp4;
/* ... and much more ... */
    tmp29.y = tmp25;
    vec4 tmp30 = tmp29;
    tmp30.z = tmp26;
    vec4 tmp31 = tmp30;
    tmp31.w = tmp27;
    vec4 tmp32 = premultiply_v4( tmp31 );
    return tmp32;
}

Pretty disgusting and, unfortunately, representative of the code that comes out of the current compiler. We’re basically relying on the vendor’s GLSL compiler to cover all of our sins[1]. Activating the LLVM optimiser immediately makes things a little better:

$ ./compiletest -print=glsl -opt-llvm -no-opt-glsl test/testkernel.knl
// Definition of desaturateKernel_s.
vec4 desaturateKernel_s ( int src )
{
    vec2 tmp2 = samplerCoord_s( src );
    vec4 tmp3 = sample_sv2( src, tmp2 );
    vec4 tmp10 = unpremultiply_v4( tmp3 );
    float tmp11 = dot_v4v4( vec4( 0.299, 0.587, 0.114, 0 ), tmp10 );
    vec3 tmp19 = vec3(0.0,0.0,0.0);
    tmp19.x = tmp11;
    vec3 tmp20 = tmp19;
    tmp20.y = tmp11;
    vec3 tmp21 = tmp20;
    tmp21.z = tmp11;
    vec3 tmp22 = (tmp21) * (vec3( 1, 1, 1 ));
    float tmp24 = tmp22.x;
    float tmp25 = tmp22.y;
    float tmp26 = tmp22.z;
    vec4 tmp28 = vec4(0.0,0.0,0.0,0.0);
    tmp28.x = tmp24;
    vec4 tmp29 = tmp28;
    tmp29.y = tmp25;
    vec4 tmp30 = tmp29;
    tmp30.z = tmp26;
    vec4 tmp31 = vec4( tmp30.x, tmp30.y, tmp30.z, tmp3.w );
    vec4 tmp32 = premultiply_v4( tmp31 );
    return tmp32;
}

This is still not as good as it might be. Luckily the LLVM IR turns out to be a double win here. Since the IR is in SSA form with the usage count of each instruction output tracked, we can trivially do some nice inlining directly in the GLSL backend:

$ ./compiletest -print=glsl -opt-llvm -opt-glsl test/testkernel.knl
// Definition of desaturateKernel_s.
vec4 desaturateKernel_s ( int src )
{
    vec4 tmp3 = sample_sv2( src, samplerCoord_s( src ) );
    float tmp11 = dot_v4v4( vec4( 0.299, 0.587, 0.114, 0 ), unpremultiply_v4( tmp3 ) );
    vec3 tmp22 = (vec3(tmp11,tmp11,tmp11)) * (vec3( 1, 1, 1 ));
    return premultiply_v4( vec4( tmp22.x, tmp22.y, tmp22.z, tmp3.w ) );
}

Which is far closer to some GLSL which isn’t going to run like a dog. Overall I’m pretty interested to see what sort of speed-ups the optimising compiler brings.

[1] Backtraces from crashes on OS X reveal that the Intel driver is also using the GLSL reference compiler. Thus, it is highly unlikely it will be optimising our sins away.

This weekend was a disaster NaNo-wise. I got up on Saturday full of good intentions, sat down and… nothing. Nothing would come out. I started to force a few sentences and I suddenly realised ‘I have no idea what I’m doing’.

The problem, of course, stemmed from a lack of planning. I had carefully mind mapped and planned the central themes of the book, the beginning, the characters, their introduction and final goals. What I hadn’t planned in sufficient detail was how I was going to tell the middle of the story. I’m unsure of both my characters’ motivations and the time line of events.

Not knowing exactly what was going on meant moving forward was like mentally fighting through treacle. I had encountered the ‘writers’ block’. I had found something similar when writing my thesis. About half way through I suddenly lost track of the overall message by concentrating on the details.

I was a bit depressed on Saturday because of this and had a bit of a moan to Saf. Luckily she had the, obvious but welcome, suggestion of just stopping, sitting down and thinking about it. Work out what was going to happen, to whom and when. Do it away from the computer, away from distractions and without any sense of urgency.

On Sunday morning I had a nice long bath and sorted things out. I worked out where I had to get to to write the end of the book and resolved to just write over the middle and get there. I also came to terms with writing crap. The middle of the book, as I wrote in the previous post, is crap but that’s OK. I know that I can sort it. I just need to get the damn thing feature complete :). Once it is out, re-writing and re-plotting the middle will become a pleasure rather than a chore because I’ll have a full plot set out, in my head and on paper.

Current word count: 29,010

Time for bed methinks. I’m only 1k short of 30k but it is time that I got some sleep. Writing went better today with lots of chapters which I had abandoned re-visited and fleshed out. I’m still hating everything I’m writing at the moment but that is because I’m in the messy ‘middle bit’ where all the plot lines interleave. I’m not writing anything particularly clever or particularly funny but I’m getting the plot down. Later on, when editing, I can come back and redo most of the bits I’m writing at the moment.

The thing which I’m finding incredibly annoying is the ‘clunkiness’ of it all. What I liked about the beginning of the book is how things from one chapter would foreshadow things in another. In the middle I have currently, things just ‘happen’ without any prior justification. On the other hand that is the sort of thing that can be sorted out later. Also I need to finalise some of the characters a bit better. They’re not really gelling as an ensemble at the moment.

So that is what is wrong. What is good? Well, the plot is moving, albeit slowly, through the middle of the book and out into the light at the end of the tunnel. Soon we should get everyone to the little village which will be the location for the end of the book and the finishing post will be in sight (both in the story and for the book).

Currently, given the plot progress, I’m estimating that the first draft of the novel will be around 60k. This will in all likelihood increase with editing since all the scenes need more description and grounding as well as the insertion of the foreshadowing. There is also a huge chunk of exposition prose which needs trimming into bits before too long.

Overall thought I’m quite pleased. I had a flip through the book as it stands in the bath tonight and, aside from being poorly paced and poorly written, the actual story is quite engaging. I’m hopeful this turd can be polished into a nugget of nutty delight :).

Current word count: 20,424

Argh! Real life interferes. Yesterday I quite literally did not have five unallocated minutes in which to NaNo and today I’ve just come back from visiting a school in the South of England where I was part of an Oxbridge application day. I managed to find a little time in between lectures to increase the word count but not by much.

The half way mark is tantalisingly in my sights now, a scant 4,500 words need to come out to be within striking distance. I’m finding it hard to work in short spurts though—I work better in long runs where I can get into my flow. Consequently, not only are there not many new words today, but I also view them as rather poor and lazy.

This weekend, though, I should get an almost uninterrupted 48 hours to NaNo so we’ll see if I can’t eat into things a little. Were I to stick to my initial writing rate I’d be on 30k now!

Current word count: 18,399

A ‘bitty’ day today. I added some words this morning after I woke up before work and have been beavering away in the hour between work and improv practice to get the count up. Not sure if I’m going to hit today’s target (20,000) but I’ll see what I can do between practice and the show tonight.

Hopefully, some friends of ours are having a ‘celebrate the (end/start) of the world’ party tonight which I might go along to. It’ll be post midnight so, by that time, I’ll have the spectre of a target of 25,000 waving in front of my eyes. I sincerely hope that by tomorrow I’ll be sitting on that many words. It is half way after all!

Pleasingly today, at around 18,000 words, the actual plot of the novel started to be revealed. Now I can stop starting stories and actually tell them. After the halfway point they’ll start meshing back together again as well (one hopes!).

Current word count: 15,034

Phew! Today was a bit of a struggle to get another 5,000 out. I luckily had a meeting this morning in Cafe Nero so I decided to turn up a little early and get some novelling done. Get me! Three days in and I’m sitting in a coffee shop writing a novel on my Macbook. I should start wearing berets or something!

I re-visited the Chapter of Doom from day 2 and got it out fairly nicely. I then, of course, immediately replaced it with another Chapter of Doom meaning that my novel still contains the phrase ‘Bleurgh! Will return to this later’, just in a different place.

More worryingly I’ve developed a style of having lots of short chapters. 15,000 words in and I’m just starting Chapter 10. At this rate the novel’ll have 40 chapters! I suspect that once I decide on the order I want to tell the story in, most of these will be merged. Indeed the word count per chapter is slowly edging up from 1,500 to nearer 2,500.

I’ve also found the process quite… cathartic. Most of the characters’ internal dialogues generally explore, or expose, some deep seated hatred I have about something. Getting it out on paper has exorcised a few demons.

I also had the first, rather embarrassingly unprofessional, moment today of laughing at one of my own jokes. I very rarely laugh at my own, or indeed anyone else’s, jokes when I’m concentrating on writing since I’m in some sense over analysing. Today, something just tickled me though. Re-reading it tonight, however, it didn’t seem nearly as funny.

So, tomorrow is another day and, one hopes, one with enough time to get some writing done. Unfortunately I’m busy all tomorrow evening with an improv show. Perhaps I’ll find time in the breaks to type a little.

End of day 2. Current word count: 10,919

So nearly 11,000! I’ve reached a stopping point where I need to have a think and, more importantly, sleep. I’ve already had a disaster chapter. It started, got four paragraphs in and I decided it was so awful, there was nothing left but to declare it hopeless and move on. Always forward, never back. It sits there now in my source control challenging me to come back and fix it.

I’ll be strong though. It can sit there until its livelier brethren are finished though. Pleasingly it is low on plot so could even be left out if necessary.

Less pleasing is that I’m running out of the story that I was firm on. I’m venturing into Here Be Dragons territory with every word. Let us hope that the Dragons leave me be for the moment and instead concentrate their fiery wrath on burning away all the plot holes.

Word count after day 1: 5068

NaNoWriMo started yesterday and, in the way of such things, I had managed to make myself busy with other things I had foolishly booked for that day. Despite the interruptions, I managed to get a respectable amount down.

There is a story that is told about the Bond film The World is Not Enough; the writers approached Barbara Broccoli asking for advice about writing a Bond screen play and she replied: “the number one rule is that Bond only ever goes forward; he never re-visits somewhere he has been.”

As I understand it, there is a similar ‘rule’ for NaNo. No matter how many typos and grammer errors you see above the cursor, no matter how much it hurts to do so, never go back and edit.

This I’ve found the hardest rule. The current draft is resplendent with poor turns of phrase, clumsy sentences and outright errors but I’m leaving them there until I edit the book.

Why this defensive blog post? Well, it is an attempt to excuse the current quality of the book. Why? Who is going to read it? Well, the drafts will be available from my shiny new nano site until the end of November. This should hopefully force me to actually write the damn thing!