The LilyPond Report #11

Welcome to this eleventh issue of the LilyPond Report!

This week we’ll see what is going on on the development mailing list; we will also talk about Free Culture, silly contests, paper sizes and cross-compiling, and have a look at the upcoming next stable LilyPond version! As always, you can post your comments at the bottom of the page, or even register and contribute to the LilyPond Report’s next issues.

 This Week’s Desultory Editorial

Well, getting back to normal was not easy after last week’s huge and pandemoniac special LilyPond Report, that took me nearly a month to make.

After having read it, Trevor Bača asked me to keep an eye on my website’s statistics:

probably you’ve just created an important starting point in the research of algorithmic music and compouter-aided composition ... and the traffic sources to the page will probably bear that out some weeks from now.

There was indeed some kind of an early bump in the stats (which, as I already told you, is always flattering); however I was struck by the lack of comments left below the “special” issue (except for Graham Percival, who raised an interesting point). Maybe many readers, expecting to find just another issue of the usual column, didn’t have the courage or the time to read it, or felt too unfamiliar with the subject.

So did I myself, by the way; my point was precisely to imagine some kind of a bridge that could allow “normal” people like me to discover some esoteric stuff... I could have published this as a feuilleton through many installments of the Report; well, perhaps this will be possible for future investigations on other subjects...

During the weeks I spent being away (on the planet of computer-generated musics), many interesting things kept happening in the LilyPond community.

I couldn’t keep track of everything, but I do remember a few things, so in this issue I’ll try to give you some kind of a “previously on LilyPond” for those who’d have missed some episodes on the lilypond-devel list.

 News from the Free World

Speaking of bridges, this week an interesting and promising one was created at the Creative Commons organization.

There’s a good chance that you already know the Creative Commons licenses, that provide “normal” people (again) to easily choose which freedoms they’re ready to allow for their works; this user-friendliness has led CC licenses to be extensively used in all domains, from computers to art. Well, meet this new label: Approved for Free Cultural Works.

We’ve just added the seal you see at right to Creative Commons licenses that qualify as Free Culture Licenses according to the Definition of Free Cultural Works — Attribution and Attribution-ShareAlike. Public domain is not a license, but is an acceptable copyright status for free cultural works according to the Definition.

[...] Of course you don’t have to agree with the definition of freedom used by the free software movement and the Definition of Free Cultural Works, and even if you do agree, there may be reasons for using a more restrictive license in some cases. However, this seal and approval signals an important delineation between less and more restrictive licenses, one that creators and users of content should be aware of.

This label relies on a definition of “free cultural works” proposed by the website freedomdefined.org, and based on the Free Software Definition from Richard Stallman’s GNU project. It’s a spectacular move, since until now Lawrence Lessig’s Creative Commons and Stallman’s Free Software Foundation had quite some differences.

However, this move was not really welcomed by some adepts of the CC licenses: among their main concerns, they fear this label might divide CC-licenses users between “good” and “evil”, depending on whether the license they choose has the label or not. Quoting some mails I’ve read:

I used to see the different CC licenses as delimiting different _spaces_ of freedom. Now I feel like a _scale_ of freedom has been created, with this label at the top, and all non-approved licenses at the bottom.

Look at this website, freedomdefined.org: “Freedom defined”! Is it me or is this even worse than everything Orwell has ever written?

Hm, I would have been surprised if the Free community had suddenly ceased to be teared apart by debates and conflicts. This label, I think, is merely a reference, and a quite useful one for small projects or organizations in need of a legal frame: look, for instance, at the Mutopia project and its licenses; wouldn’t it be much simpler (and legitimate) to simply use this label, instead of always having to specify “this license is allowed, but this one isn’t, oh, wait, actually maybe it should”, etc.?

Another piece of news: what makes a project a “good project”? Documentation, Usability, Support? Is LilyPond what you would call a “good project”?

This week, Oscar van Eijk has noticed that SourceForge.net was launching an event called “2008 Community Choice Award”.

I’m involved in several projects (not Lilypond; here I’m just a user ), and IMHO LilyPond is an example project in usability, documentation and support, so I nominated LilyPond in the category "Best Project".

That looks really nice — I don’t like SourceForge at all, but I do like such events for the community. Therefore I have put a link at the bottom of this page; feel free to click on it if you happen to have a SourceForge account, and if you approve Oscar’s choice. You only have a few days left!

 What’s up with LilyPond? (1)

On May 30., Han-Wen Nienhuys, leader of the Lilypond project, stated:

The status is that we have 97 defects left, of which one regression, and one marked high. After fixing these, I’ll go through the list once more to pick the low-hanging fruit. Then, from a bugs perspective, we are ready to release 2.12.

A new stable LilyPond release? That would be a big event. Of course, if you are a bit familiar with the way LilyPond evolves, you know that the latest “development versions” (numbered 2.11.xx) already contain most of the features that are to be expected in 2.12. No Big Suprise, so.

But what are these features more precisely, and how will they make LilyPond 2.12 even better than 2.10? Here’s a non-exhaustive list:

 New (mostly vertical) spacing code, that solves many more collisions

 Better support for microtonal music

 Better regression test system

 New glyphs

 New text markup commands

 New lines and spanners code

 Better page breaking algorithms

 Much, much better MusicXML conversion

 Better code for ties

 New dynamics engraving code

However, the most impressive change is to be found, of course, in the Documentation itself, that is being completely rewritten thanks to Graham Percival’s “Grand Documentation Project”. Note that LilyPond 2.12 will also make much easier to translate both the program and its documentation in other languages.

Speaking of documentation, Graham immediately answered that he’d rather avoid to make an official release before the GDP is more advanced:

In summary, you’d have to be crazy to be seriously contemplating a release in the near future.

He added that this could be an opportunity to have a “beta” release state, as requested by some LilyPonders.

 The Bug of the Week

When can a program be regarded as “stable” enough? You never know for sure that no annoying bug remains to be unveiled. Here’s a short story I’d like to tell you, as the current BugMeister of LilyPond.

On May the 2nd, Daniel Tonda cross-posted a long mail to the bug and the -devel mailing lists. The subject of the mail was a bit verbose, the example he provided was not a minimal example; long story short, his mail didn’t meet our bug report guidelines and I didn’t have the time to deal with it immediately: therefore I bookmarked it and somehow forgot it afterwards.

On May the 3rd, another bug was reported by Stan Sanderson, and added to the tracker by Graham. Whereas Stan had noticed that all tablature-related regression tests were broken, Graham’s report only mentioned the banjo tablature test. Therefore, I confess I didn’t pay much attention to it.

While reviewing my bookmarked bug reports, yesterday I re-discovered Daniel’s report, compiled his example, discovered that the stems were indeed missing. I stripped down his example to a minimal report and finally added it to the tracker.

Then Stan made me realize that it was the same bug that the one already in the tracker.

This was a major regression, and Han-Wen noticed it and fixed it immediately. It would have been really bad to release a “stable” version without supporting stems in tablatures, indeed.

The bottom line is
1) individuals fail. We all miss things, forget things, hide things.
2) the community is a wonderful way to compensate these faults, and make software live happily ever after. In this story, everyone, from Daniel to Han-Wen, has played a complementary role and, in the end, provided a balance to (mostly) my mistakes.

And this just feels great.

 What’s up with LilyPond? (2)

On a different tone, on May 26. Han-Wen also raised a debate about whether more non-programmer people should be allowed to take part in LilyPond development or not:

I have the feeling that the number of people that need commit access is getting larger than it needs be - we keep adding people that only do relatively small tasks.

[...] If we have lots of people pushing to the master branch, a bad push by a single person will affect all contributors.

I am in no place to judge this, being myself a non-programmer contributor with write permission to the source code (certainly the clumsiest and the most potentially harmful of all contributors, by the way). Therefore, I felt too personally concerned by what was at stake in this discussion (and not in a pleasant way) to take part in it.

However, when I try to have a neutral point of view on the subject, it seems to me that we deeply need fresh blood — and that means new people.

If these people could be skilled programmers with an extensive knowledge of Lily’s secrets, that would be ideal; but I have this hope (this utopia, you might say) that even non-programmers can, in time, become experienced enough to do minor but sensible programming tasks: fix bugs, implement small features, etc.; I’m thinking about valuable persons such as John Mandereau, or more recently Neil Puttock.

Of course, most of these contributors mostly deal with documentation stuff, that could very well be developed on a separate area (a dedicated branch, or a fork, or whatever). John offered to keep track of such a system, to regularly merge the updated documentation into the main development branch. The discussion hasn’t gone much further yet.

 The Contribution of the Week

As I’m about to publish this Report, Michael Pozhidaev, a new user, has just introduced himself on the list:

I am blind programmer from Russia and music takes a very important part in my life. Lilypond is just brilliant software for me. Now I can make my own music publications by myself. Great thanks to everybody, who made it possible!

Welcome Michael!

As a matter of fact, LilyPond is great for people with specific needs, and its text-based interface is particularly convenient for blind people. On the very last days of 2007, another blind person from China, Hu Haipeng, discovered LilyPond and has kept using it ever since.

Lilypond is written using ASCII, so it’s the best way for blind to compose or read music without sighted people’s help. I don’t know whether there are many other blind people in European and American countries using Lilypond, but for me, this is a good thing.

This led to an interesting (though unsuccessful) discussion on the mailing list, since some of the documentation resources (particularly ornaments) were mostly available as undescribed pictures.

Anyway. This week, Haipeng took a really cool initiative: since some people had been discussing about various paper sizes for scores, he decided to contribute to LilyPond’s development by implementing every paper sizes you can think of as pre-defined options in LilyPond:

I’ve also suffered from the paper size, and don’t know how to handle them when writing oversized large orchestral scores like Mahler and Schoenberg did. Thanks to someone, who pointed out the Wikipedia [1]. I’m attaching my paper.scm, with most of the sizes copied from the page. Some may perhaps never be used in music output, but I added them as a preparation for accidental use. Hope it will be useful!

Now, thanks to Haipeng, you can feel free to use any paper size from various countries, such as

#(set-default-paper-size "government-letter") or
#(set-default-paper-size "monarch")

 What’s up with LilyPond? (3)

On the lilypond-devel list again, John noticed an annoyance when trying to build LilyPond with the latest version GCC compiler. An impressive technical discussion followed, that proved that this bug wasn’t because of LilyPond, not even because of GCC, but... because of the Intel x86 processor in your computer! According to Joe Neeman,

The summary is that you cannot expect sane behaviour with the double data type using gcc (and apparently most other C compilers) on x86. In particular, doubles may be truncated from 80 bits to 64 bits at any time. Therefore, you can compare two (80-bit) doubles and find that they are not equal only for them to become equal on the very next line.

This allowed me to have a look at the GCC bug tracker... And understand how happy I was to be in charge of LilyPond’s tracker instead of this one

 The Snippet of the Week

A couple months ago, we briefly mentioned a nice feature that allows a voice to quote another voice. However, some limitations remain in this function.

According to Gilles Thibault, a French user whom you may have met also on the English-speaking list:

the \addQuote function has three limitations:
 it isn’t transpose-able ( \transpose c d \quoteDuring #“clarinet” s2. has no effect, for instance )
 it filters events: expressive marks and dynamics are not taken into account ( see \set Staff.quotedEventTypes )
 You cannot specify from where you want to extract the music.

Actually, it was not written to extract music from an existing expression.

So, here’s a solution provided by Gilles:
Extracting unmodified fragments of a music expression

Nice work!

 What’s up with LilyPond? (4)

When you want to download and install LilyPond, you can choose about any version you want depending on your computer and your operating system. How is it possible? Do you think LilyPond authors have a bunch of different computers, running different operating systems?

Er, indeed they probably do, but all LilyPond versions are actually built on a single computer, running a single operating system (GNU/Linux), thanks to an unified build process developed specially for LilyPond, called GUB.

Grand Unified Builder (GUB) is the mini packaging system that we developed for building LilyPond binaries. It cross-compiles several packages and assembles them into a single package.

A month ago, Mark Hanlon, contributor of an interesting Music Notation project, asked on the list if it was possible to build binaries for Windows and Macs in Linux.

John answered: Of course, this is exactly how released binaries are built: binaries are cross-compiled on a GNU/Linux system for all platforms (although it’s a bit more problematic for MacOS 10.5).

21 days later... Mark: I am using the latest gub, started from scratch on Sunday night, first ever gub build!

Han-Wen: You realize that you are probably the first or second person beyond me & Jan to build a binary in GUB. Congrats!

Indeed; building LilyPond for linux is already an involved task, but building the whole set of GUB versions can be really challenging. For instance, have a look at the new technical discussion about compiling flags that it raised...

 The Quote of the Week

On its very first issues, the LilyPond Report taught you how to curse, Graham’s style.

Well, recently Francisco showed us how to beg, genuine LilyPond style:

From my humble position, I beg you please, please,
\repeat unfold 10 { please, } to make possible that translated docs are updateable after release.

Lily-talk: a new trend of geekness is born

And this concludes the eleventh issue of The LilyPond Report.

Cheers,
Valentin Villenave