Further Developments Latex and Lilypond now with Arara and International Unicode Text via XeLatex

The title might be a white lie, because while I did get arara working, in combination with a bash script, to automate the steps to render the hymnal.

Good news. One of the arara developers helped me solve the problem. Stack overflow:

The solution ends up being quiet simple. The only limitation is that we are not outputting the lilypond-book content to a subdirectory as originally planned.

At the top of the music.lytex file (after configuring arara as directed in the referenced post), add the following two directives:

% arara: lilypond: {options: "--pdf"}
% arara: pdflatex: { files: [ 'hymnal.tex' ] }

Now make a file called render.sh containing something like the following:

arara music.lytex
pdflatex music.tex
find . -maxdepth 1 -type d -name '??' -exec rm -rf {} \;
rm *.aux *.bcf *.dep *.out *.run.xml *.toc tmpx*
open music.pdf

And render and view the resultant pdf file by calling it with bash:

bash render.sh

The middle three lines in the bash script:

pdflatex music.tex

This is because pdflatex needs to be run a second time in order to generate the table of contents.

find . -maxdepth 1 -type d -name '??' -exec rm -rf {} \;

What this line does is remove all of the temporary directories created by the lilypond-book command, which would, if generating into a subdirectory (with the --output=out) flag, be neatly contained in there.

rm *.aux *.bcf *.dep *.out *.run.xml *.toc tmpx*

All the other temporary files that might interfere with subsequent renderings.

Thanks, arara crew. Long live the arara bird and Brazilian hospitality!

Note that two files needed to be created on my system in order for this to work. The araraconfig.yaml file located in my home directory:

    # Config file to use texmfhome as search path
    # # author: Mike iLL
    # # requires arara 3.0+
    - /Users/mikekilmer/Library/texmf/scripts/arara/rules

    - extension: lytex
      pattern: ^(\s)*%\s+

And a single rule, lilypond.yaml, located in the above referenced directory:

    # Mainfile rule for arara
    # author: Marco Daniel
    # requires arara 3.0+
    identifier: lilypond
    name: Lilypond
    command:  lilypond-book @{format} @{options} @{output} "@{file}"
    - identifier: format
      flag:  --format=@{parameters.latex-programm}
      default:  --format=latex
    - identifier: options
      flag:  @{parameters.options}
    - identifier: output
      flag:  --output=@{parameters.output}

But then as the text developed, I realized that some Sanscrit were cropping up. And pdflatex only deals with 256 characters, which is the beginning of the utf8 universal character set. Fortunately I found this post on the TeX StackExchange site, offering the following flag to tell lillypond-book to use XeLatex instead of pdflatex:


And a fine example of use of Sanscrit and various Hindi scripts in a LaTeX document:

% This is a Unicode file.
\usepackage{multicol} % just to get narrow columns on one page
\usepackage{polyglossia} % the multilingual support package
% for XeLaTeX - includes Sanskrit.
% Next, from the polyglossia manual:
\setdefaultlanguage{english} % this is mostly going to be English text, with
\setotherlanguage{sanskrit} % some Sanskrit embedded in it.
% These will call appropriate hyphenation.
\usepackage{xltxtra} % standard for nearly all XeLaTeX documents
\defaultfontfeatures{Mapping=tex-text} % ditto
\setmainfont{Gandhari Unicode} %could be any Unicode font

% Now define some Devanagari fonts:
% At least *one* font family must be called \sanskritfont or \devanagarifont,
% so that XeTeX loads all hyphenation and other stuff for Sanskrit. 
% Once the Sanskrit ``intelligence'' is loaded, it can be invoked at
% other places as needed using \selectlanguage{sanskrit}
% John Smith's Nakula, input using Velthuis transliteration
\sanskritfont [Script=Devanagari,Mapping=velthuis-sanskrit]{Nakula}
% John Smith's Sahadeva, input using Velthuis transliteration
\sahadevafont [Script=Devanagari,Mapping=velthuis-sanskrit]{Sahadeva}
% John's Sahadeva, input using scholarly romanisation in Unicode
\sahadevaunicodefont [Script=Devanagari,Mapping=RomDev]{Sahadeva}
% Microsoft's Mangal font (ugh!), input using standard romanisation in Unicode.
\mangalfont [Script=Devanagari,Mapping=RomDev]{Mangal}
% Somdev's RomDev.map is used above to get the mapping
% from Unicode -> Devanāgarī. Zdenek Wagner's velthuis-sanskrit.map
% is used to get the Velthuis->Devanāgarī mapping. These are the files
% that XeTeX uses to make all the conjunct consonants without needing
% any external preprocessor (like the old devnag.c program).
% % Set up the font commands:
\newcommand{\velt}[1]{{{\selectlanguage{sanskrit}\sanskritfont #1}}}
\newcommand{\sahauni}[1]{{{\selectlanguage{sanskrit}\sahadevaunicodefont #1}}}
\newcommand{\mangaluni}[1]{{{\selectlanguage{sanskrit}\mangalfont #1}}}
% \textssanskrit, above, is a Polyglossia command that gets Sanskrit hyphenation right.
% ... and here we go!
\begin{multicols}{2} % narrow cols to force plenty of hyphenation
\large % ditto.
With Xe\LaTeX\ it's easy to typeset Velthuis encoded Devanagari like the following example, without using a preprocessor:
\velt{sugataan sasutaan sadharmakaayaan pra.nipatyaadarato 'khilaa.m"sca vandyaan|
sugataatmajasa.mvaraavataara.m kathayi.syaami yathaagama.m samaasaat||} Bodhicaryāvatāra 1,1.
NB: automatic hyphenation.
A different Devanāgarī font:
\saha{sugataan sasutaan sadharmakaayaan pra.nipatyaadarato 'khilaa.m"sca vandyaan|
sugataatmajasa.mvaraavataara.m kathayi.syaami yathaagama.m samaasaat||} Bodhicaryāvatāra 1,1.
Another sentence: \saha{ratnojjvalastambhamanorame.su muktaamayodbhaasivitaanake.su|
svacchojjvalasphaa.tikaku.t.time.su sungandhi.su snaanag.rhe.su te.su||} 2,10.
Now, thanks to Somdev's RomDev.map, we can input in Unicode, using standard scholarly transliteration, and get Devanāgarī generated for us automatically:
\sahauni{āsīdrājā nalo nāma vīrsenasuto balī||\par }
Plain Unicode input, no tricks:
āsīdrājā nalo nāma vīrsenasuto balī||
Plain Unicode romanisation input, no tricks:
\mangaluni{āsīdrājā nalo nāma vīrsenasuto balī||\par }
Plain Unicode Devanāgarī input, no tricks:
{\mangalfont आसीद्राजा नलो नाम वीरसेनसुतो बली|\par}

English and Devanāgarī are both doing okay. The only thing that isn't hyphenating well yet is Sanskrit in roman transliteration.

Other nice stuff becomes easy. E.g., define a command \verb|\example| that prints a romanised word in Nāgarī, and then repeats it in romanisation, in parentheses:


Input: \verb|\example{ekadhā}|

Output: \example{ekadhā}
%that's all folks! 

Once those required fonts were added to the system (all found for free via Search Engine), we had our Sanscrit (and Bengali).

So now I can run the following bash script to render:

    lilypond-book --latex-program=xelatex --pdf hymnal.lytex 
    xelatex hymnal.tex
    xelatex hymnal.tex
    find . -maxdepth 1 -type d -name '??' -exec rm -rf {} \;
    rm *.aux *.bcf *.dep *.out *.run.xml *.toc tmpx* snippet-map* snippet-names*
    open hymnal.pdf

But I would like to also get xelatex working with arara…

That was easy:

    % arara: lilypond: {options: "--latex-program=xelatex --pdf"}
    % arara: xelatex: { files: [ 'hymnal.tex' ] }

The raw files on GitHub.


Automatically creating an Index of terms took a little while to figure out. But it’s actually pretty simple.

We want the package:
\usepackage{imakeidx} % index

Then we need to call the \makeindex command within the preamble.

Wherever we want it within the book we add the command: \printindex, and within the document, for an item we want to include in the index we add:


To include images I think they need to be PDFs, supposedly Inkscape can transform SVGs to PDF:

inkscape -D -z --file=AllOnePose.svg --export-pdf=poses/AllOnePose.pdf --export-latex

Now we can use Arabic via ArabXeTeX:


لَا إِلٰهَ إِلَّا الله مُحَمَّدٌ رَسُولُ الله

I had to install the Amiri font by Khaled Hosny.

Latex and Lilypond Hymnal Volume Two

So this time there are two new elements to the Hymnal.

  1. It will be a smaller size, 5.5″ x 8″
  2. It will blend text and music snippets more than the first book.

It’s fairly easy to make a single sheet or even songbook a custom size:

#(set! paper-alist (cons '("my size" . (cons (* 5.5 in) (* 8 in))) paper-alist))

\paper {
  #(set-paper-size "my size")

And this can be included in a text document:



Documents for \verb+lilypond-book+ may freely mix music and text.
For example,

\relative c' {
  c2 e2 \tuplet 3/2 { f8 a b } a2 e4

Options are put in brackets.

  c'4 f16

Larger examples can be put into a separate file, and introduced with


(If needed, replace @file{screech-and-boink.ly} by any @file{.ly} file
you put in the same directory as this file.)


Supposedly there’s a tutorial here, but the link is broken. There is the main Open LilyLib site.

This repository looks like it’s exactly what I need, but it’s a few years without update and not terribly well documented. It also appears to be unfinished.

I cloned to repository and am trying to compile one of the LateX files in it’s documentation directory:

\usepackage{musicexamplesStyles} gives the error that the code is requiring the package, musicexamplesStyles, when what is available is musicexamplesStyles (singular). So I updated that code to: \usepackage{musicexamplesStyle}.

Then it was looking for a file called musicexamples.sty. I see one called musicexamplesStyles.sty in the directory with the tex file so copying it to musicexamples.sty. Now am getting an error that it’s looking for a native Linux Font: Linux Libertine 0, which is not found on this OS X machine. Neither can “Linus Biolinum 0”, “Inconsolata”.

There’s a program that came with LaTeX called Tex Live, that doesn’t seem to be working, but there’s also a program in there called FixMacTex, which I am running now, and supposedly will fix some issues.

Wait. The musicexampleStyles docs say

The main requirement to make use of the present material is to make the \package{musicexamples} package available to your \LaTeX{} distribution.
If you have obtained \package{musicexamples} as an archived download archive you may just unpack it into a folder inside your \dir{texmf/tex/latex} directory.
Depending on your (operating and \LaTeX) system you may have to run \env{texhash} afterwards.
If you aren't sure what this means, please consult your \LaTeX{} documentation or a book.

Seems to be an issue with TexLive, which I’m guessing is like a package manager for TeX/LaTeX. Looks like my version of LaTeX is out of date so dowlnoading the 2 GigaByte newer version. In the meantime I do have texhash installed. It’s a small database that keeps track of all the locations of various TeX files.

which texhash

Will try running it.

$ texhash
texhash: /usr/local/texlive/2014/texmf-config: directory not writable. Skipping...
texhash: /usr/local/texlive/2014/texmf-dist: directory not writable. Skipping...
texhash: /usr/local/texlive/2014/texmf-var: directory not writable. Skipping...
texhash: /usr/local/texlive/texmf-local: directory not writable. Skipping...
texhash: Done.
mikekilmer@jamaaladeen-2:~/Documents/Vault/Hymnal/musicexamples/documentation$ sudo !!
sudo texhash
texhash: Updating /usr/local/texlive/2014/texmf-config/ls-R... 
texhash: Updating /usr/local/texlive/2014/texmf-dist/ls-R... 
texhash: Updating /usr/local/texlive/2014/texmf-var/ls-R... 
texhash: Updating /usr/local/texlive/texmf-local/ls-R... 
texhash: Done.

Hmmm. Same errors. Maybe this repository won’t be the best route at the moment.

This looks promising: https://github.com/openlilylib/snippets

Not sure, but the LaTeX booklet posts I’m seeing refer to sizes A4, A5, A6. This posting seems to offer some useful information:

\usepackage{lipsum}  % this package is for creating filler text

\title{The booklet}






and if the above was saved as filler.tex:


\title{The booklet}




So if that’s going to be the method for now, the only big piece of the puzzle is determining margin and/or page sizes on the lilypond-book coding.

While searching for that, I came across this post inf which someone asks if it’s possible to run lilypond and LaTeX as one a single command. Apparently a library called arara is the key, which seems to require a Java runtime environment, Downloading Java JDE now.

Meanwhile the latest LaTeX is 20% downloaded. JDE installed and now:

  __ _ _ __ __ _ _ __ __ _ 
 / _` | '__/ _` | '__/ _` |
| (_| | | | (_| | | | (_| |
 \__,_|_|  \__,_|_|  \__,_|

arara 3.0 - The cool TeX automation tool
Copyright (c) 2012, Paulo Roberto Massa Cereda
All rights reserved.

usage: arara [file [--log] [--verbose] [--timeout N] [--language L] |
             --help | --version]
 -h,--help             print the help message
 -l,--log              generate a log output
 -L,--language <arg>   set the application language
 -t,--timeout <arg>    set the execution timeout (in milliseconds)
 -V,--version          print the application version
 -v,--verbose          print the command output

I read the really fun (really!) manual and finally posted on Stack Exchange, eventually sending an email to the awesome developer of Arara (Paulo, noted above) and got it worked out. The problem was simply that I was using the .yml (shortened) extension, while Arara (version 3.0) is looking for the full .yaml extension.

Getting closer and closer to the (simple) formula that will get this Hymnal properly formatted with some help from this Stack Exchange answer.

The current test scripts look like this:

First just a simple test A5 TeX document (test.tex):



Documents for \verb+lilypond-book+ may freely mix music and text.
For example,

(If needed, replace @file{screech-and-boink.ly} by any @file{.ly} file
you put in the same directory as this file.)


Good. Our song file (song.ly):

\version "2.19.45"

\header {
  title = "Gayatri Mantra"
  author = "anonymous"
  composer = "Music by Rivka and Mike iLL"
  tagline = "Copyright Rivka and Mike iLL Kilmer Creative Commons Attribution-NonCommercial BMI - Engraving by Lilypond"

#(set! paper-alist (cons '("hymnal" . (cons (* 5.5 in) (* 8 in))) paper-alist))

\paper {
  #(set-paper-size "hymnal")
  ragged-last-bottom = ##f
  print-page-number = ##f

melody = \relative c'' {
  \clef treble
  \key d \major
  \time 4/4
  \set Score.voltaSpannerDuration = #(ly:make-moment 4/4)
    \new Voice = "words" {
        g4( a) a2 | g8( a4) a8 a4 a | g8( a4.) a4 a | g8( a4) a4. cis4 | % Om bur ... maha
        g8( a4) a8 a4 a | g8( a4) a8 a4 a | g8( a4.) a4 r | a8( bes)~ bes2. | % Om sawaha... sat yam
        a4 a8-> r8 \times 2/3 { a4 a8 } g4 | a bes g2 | a4 \times 2/3 { a4 a8 } a4 g | a bes g2 | % Om tat savitur ... dimahi
        \times 2/3 { g4( a) a } \times 2/3 { a a a } | \times 2/3 { a a a } a4 r \bar "|." % Dhi yo yo... prachodhayat

text =  \lyricmode {
    Om bhur | Om bhu -- ga -- ha  | Om swa -- ha | Om ma -- ha |
    Om ja -- na -- ha | Om ta -- pa -- ha | Om sat | yam |
    Om tat sa -- vi -- tur | bar -- vi -- nam | Bhar -- go  de -- va -- sya | Dhi -- ma -- hi |
    Dhi -- yo yo -- na -- ha | pra -- cho -- dha -- yat |

twochords = \chordmode { a1 | g:m | }

harmonies = {
    \twochords \twochords
    \twochords \twochords
    \twochords \twochords

\score {
    \new ChordNames {
      \set chordChanges = ##t
    \new Staff \with { \magnifyStaff #5/7 } {
        \new Voice = "one" { \melody }
    \new Lyrics \lyricsto "words" \text
  \layout { 
   % #(layout-set-staff-size 14)
  \midi { }

Being referenced by song.lytex:



Documents for \verb+lilypond-book+ may freely mix music and text.
For example,

\relative c' {
  c2 e2 \tuplet 3/2 { f8 a b } a2 e4

Options are put in brackets.

  c'4 f16

Larger examples can be put into a separate file, and introduced with


(If needed, replace @file{screech-and-boink.ly} by any @file{.ly} file
you put in the same directory as this file.)


The terminal commands are:

lilypond-book --output=out --pdf song.lytex 

This creates a directory called out, inside of which we need to run the command:

pdflatex song.tex

Then finally open song.pdf. Still some margin issues. Then Need to work on using Arara in the workflow.

Close! Removing the following paper settings from the included song.ly file helped. Now just need to reduce the margins in the outer document.

#(set! paper-alist (cons '("hymnal" . (cons (* 5.5 in) (* 8 in))) paper-alist))

\paper {
  #(set-paper-size "hymnal")
  ragged-last-bottom = ##f
  print-page-number = ##f

That’s it! Just needed to update following setting (second line, in this case) in song.lytex:


Meanwhile, Mom did proofreading on the first Hymnal, and offered to help on this one too, but I realized that probably the main reason lilypond creates midi files is so you can use your own ears to proof the writing. How do you play midi files on OS X? Quicktime Player 7 (and old version) seems to be the answer.

To make Quicktime 7 open the files from the command line I add this to ~/.bashrc (then run source ~/.bashrc):

#play midi files using Quicktime 7
alias midi="open -a /Applications/Utilities/QuickTime\ Player\ 7.app"

Of note ~/.bash_profile contains the following directive:

if [ -f ~/.bashrc ]; then
   source ~/.bashrc

Now I need to set a tempo in lilypond, I guess ’cause DAMN is that file playing slowly.

Simple answer: \tempo 4 = 125.

At the moment the process for viewing updates:

rm -rf out
lilypond-book --output=out --pdf hymnal.lytex 
cd out && pdflatex hymnal.tex
pdflatex hymnal.tex # have to run it a second time to generate the Table of Contents.
open hymnal.pdf
cd ../

Fun Times with Luddites and Zero Tracks on QuodLibet

Wow. That was in-ter-es-ting!

So yes – a couple of the “older” folks were a little wary of moving from cassette to digital format, which put me in the awkward position of defending this archiving project. Kind of the opposite of Paul Bunyan – with the new technology fighting to justify it’s value.

Last week, Quodlibet was finding Zero (of over 17,000) tracks. So there I am unable to play ANY music. Fortunately the old system is still in place, but I’m sure my ears were looking a bit red around the tips.

I thought it might have been an issue with the external drive either in terms of data, format or drive type.

After way too much time experimenting I decided to try looking into the Qoudlibet database. Is it postgreSQL? No. Actually it’s not a database at all. QL uses a “pickled list of AudioFile instances”. “Pickling” is the Python (language) word for a compressed file.

As per the FAQ, you’re supposed to be able to examine this “pickled” songs list:

Python 2.6.2 (r262:71600, Jun  4 2009, 15:54:27)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import quodlibet
>>> import cPickle
>>> with open(".quodlibet/songs", 'r') as songsfile:
...     songs = cPickle.load(songsfile)

(Note import quodlibet only worked for me when running the Python interactive interpreter from within the same directory as the main quodlibet.py file. I think there’s that directory could be added to the Python path, but haven’t looked into it yet.)

But the above was returning an error:

Traceback (most recent call last):
  File "/Users/mikekilmer/quodlibet/quodlibet/quodlibet/plugins/events.py", line 104, in __invoke
  File "/Users/mikekilmer/quodlibet/quodlibet/quodlibet/ext/events/viewlyrics.py", line 81, in plugin_on_song_started
    if not lyrics:
UnboundLocalError: local variable 'lyrics' referenced before assignment

Apparently in the current version of quodlibet you have to also initialize quodlibet: quodlibet.init().

Still no luck. More errors. Back to the Quodlibet Development Group on Google.

In the meantime I tried removing all but one of the directories from the Library. I tried replacing the /.quodlibet/songs cPickle file.

The songs pickle file resides in a directory named .quodlibet which is in your “home directory” and on Unix and Mac OSX this is referenced with the tilde: ~/. It also contains the playlists, lists, config and a few other files and directories. So I backed up ~/.quodlibet to ~/QLbackups and reinstalled the application. Working again. Yay!

Finally I removed the entire ~/.quodlibet directory (backing it up elsewhere), and through swapping components in and out between a fresh install’s ~/.quodlibet directory and the previous one, I discovered that it was two lines under the [browsers] heading in the ~/.quodlibet/config file that were causing the issue:

albums =
background = ketriketriketriketri # (this is the [whitespace ommited] name of one of the tracks from the collection)

Back in business. But wait. All of our playlist files are empty! Fortunately I back up the computer on our QNAP Network Attach Storage system using Time machine. So I was able to open the ~/.quodlibet direcory in the finder (by entering it in the Go>Go to folder field in the Mac finder) and then open TIme Machine, go back a few weeks and “restore” it.

Now we’re back in biz! Sweet.

Adventures in QuodLibet

The vision of Steve Jobs and Apple products are beautiful and beautifully intuitive, but the magnitude of the company and it’s market share combined with it’s Closed Source approach create a bit of a Bog Brother feeling at times. As far as all of that goes, Quodlibet is pretty much the opposite. Open Source. Written in Python which is a classic Open Source programming language. Anyone in the world can make additions to the main code base or create plugins to add or modify functionality. The public is welcome to look through the list of issues, tackle one or more and submit it to the applications maintainers for merging into the code base.

One thing I’d like to see is a more intuitive and useful interface for seeing and entering specific fields for each track – for example: description, comments, dance notes. You can download what’s called a bundle, which is basically a configured “application”, with an icon you click on and run as with your typical mail or internet browser program. You can “clone” the source code and build your own bundle, or you can do a combination of the two and install (download) the (pre-compiled) “bundle”, but also clone the code base and run the downloaded code base with the downloaded bundle. NOTE: my experience is with Mac/OS X and may differ for someone on a PC.

The way you run the bundle as development environment is by having the bundle run the cloned source code. On this computer you open the terminal program that resides in Applications/Utilities. It’s just a black and white window that puts you fairly directly in touch with the file system and features that run behind all the pretty windows of the GUI, or graphic user interface that some people pronounce, “gooey”. So you “move” into the Applications directory by typing cd /Applications. If you were to type ls you’d see a listing of all the applications that are in that directory (folder).

In the OS X Terminal
In the OS X Terminal

Then type something along the lines of the following:

./QuodLibet.app/Contents/MacOS/run /Users/mikilmer/quodlibet/quodlibet/quodlibet.py

Where the first part is ./ telling your computer this is a script to run, then QuodLibet.app/Contents/MacOS/run is the path to the part of the Quodlibet bundle that runs the application. It’s a “bash” (born again shell) script that looks something like this:

Screen Shot 2016-02-01 at 10.00.46 AM

The last part of the line to run QuodLibet as a development environment, /Users/mikilmer/quodlibet/quodlibet/quodlibet.py is the path to the cloned codebase. I cloned it directly into my “home directory” located at Users/mikilmer.

I can test that I’m running from the cloned code base by adding a little bit of code to the main Quodlibet script referenced above:

def write():

    print('Creating new text file')
    name = raw_input('Enter name of text file: ')+'.txt'  # Name of text file coerced with +.txt

        file = open(name,'w')   # Trying to create a new file or open one

        print('Something went wrong! Can\'t tell what?')
        sys.exit(0) # quit Python


And when I run the command above the terminal prints “Creating new text file”, prompts to “Enter name of text file”: Mad Mike iLL, and creates a file called “Mad Mike iLL.txt” (But where?), then opens the player as normal. Ah. there’s the Mad Mike iLL.txt file in the /Applications directory where we actually ran the script from (as opposed to the directory the script/file is in). So we can rm Mad\ Mike\ iLL.txt which removes it from existence. Those back slashes “escape” the spaces to that rm doesn’t think it’s getting more than one argument. We don’t actually desire this “function” in the player so with the magic of Git we’ll revert back to the main code base: git reset --hard HEAD. (If I’m going to be making changes, I’ll want to add another “remote” branch to the git tree to which I am allowed to “push” code changes. This would be a “fork” of the main code base. And we want it to be synced so we always have the latest version, or at least access to it.

To easily open QL this way from wherever in the directly system by simply typing quod in the terminal, I can create an alias:

alias quod="bash /Applications/QuodLibet.app/Contents/MacOS/run /Users/mikekilmer/quodlibet/quodlibet/quodlibet.py"

So now we know that we can play with the Quodlibet code. Let’s see what’s going on with these “plugins” that add various features.

We’d like to have a feature that auto generates a list of each song we have done in a given session. To make a plugin you can create a file in ~/.quodlibet/plugins/. We’ll call ours generatelist.py, and I think it will be of the class EventPlugin. The EventPlugin base class. Hmmm. There are a bunch of EventPlugin subclasses (plugins) (found using grep -rnw '/path/to/quodlibet/'- e "plugin_on_song_ended")
that create an instance method called plugin_on_song_ended and by copying one of them I can see:

from quodlibet.plugins.events import EventPlugin

class GenerateList(EventPlugin):

    PLUGIN_ID = "generatelist"

    PLUGIN_NAME = _("Generate List")

    PLUGIN_DESC = _("Automatically add to playlist for current date."

                "Well for now let's just print song name out.")

    def plugin_on_song_ended(self, song, skipped):

        if song is not None:

            print song

That code printed out an “object reference” to the terminal, at the ending of a track playing. The “object reference” is just a line of code with object name and a hash of numbers. Then I could print out dir(song) which listed all the attributes and methods of the object (I think), one of which was _song. Printing song._song provided this:

    {comment': u'', 'composer': u'delete', 'genre': u'Line', 'tracknumber': u'04', '~mountpoint': '/Volumes/Pensacola International Folk Dance Music Archive', '~#laststarted': 1454363749, '~#skipcount': 2, 'album': u'33-12-07 Macedonian Folk Dances', '~#playcount': 7, '~#bitrate': 128, 'artist': u'Krivo Zensko Oro', '~#length': 153, 'title': u'Krivo Zensko Oro', '~#rating': 1.0, '~#added': 1448030939, '~#lastplayed': 1454274254, '~#mtime': 1100040376.0, '~filename': '/Volumes/Pensacola International Folk Dance Music Archive/SteveCollins/DVD 03/CIFD 33s part 6/Macedonian Folk Dances [LPY 50985]/04 Krivo Zensko Oro.mp3', '~#filesize': 2457600}

So one idea would be to write the values of song._song['title'] for each track that plays through to a text file with current days date, creating a new file if one doesn’t exist. That might be a good first step. Next step would be adding the song to a playlist. Wait – it looks like playlists are simply text files anyway, stored in ~/.quodlibet/playlists.

Success! This plugin creates a playlist of all the tracks that have played through the end on any given day:

    This plugin creates a playlist of all the tracks that have played through the end on any given day.

    TODO: Add an enable disable button in GUI?

    from quodlibet.plugins.events import EventPlugin
    import time
    import os
    from os.path import expanduser

    class GenerateList(EventPlugin):
        PLUGIN_ID = "generatelist"
        PLUGIN_NAME = _("Generate List")
        PLUGIN_DESC = _("Automatically add to playlist for current date."
                        "Well for now let's just print song name out.")

        def plugin_on_song_ended(self, song, skipped):
            if song is not None:
                    print song._song['~filename']

        def write_to_playlist(self, filename):
            today = time.strftime("%F")
            home = expanduser("~")
            name = os.path.join(home, '.quodlibet/playlists', today)
                file = open(name, 'a')
                print("Could not write file name out")


More later…

International Folk Dance Archive

Earier this year we received another grant from the Puffin Foundation West for a project to develop the music database we use with Pensacola International Folk Dancers.

We\’ve been using a combination of cassette tapes and a fairly un-maintained iTunes music collection and we had a few objectives for the project.

  1. Digitize cassette collection
  2. Archive some of our collective knowledge about the songs and dances
  3. Ensure that all members of the group have access to the music
  4. I need to check if this was part of the initial grant proposal, but we are also learning to play some of the repertoire as well as transcribing it and codifying with lilypond software.

\"Ciulandra\"There have been many steps in this process. The first thing I did was to speak with Larry Weiner, who is considered to be an expert in the field of International Folk Dance and specifically it\’s archiving. One of the things we discussed were potential archival frameworks. Larry\’s occupation for may years was as a database administrator and he had at one point built, from scratch so to speak, a music archiving system. He advised me against spending so much time developing a system that the archiving itself suffers. He also taught me the term, \”Analysis Paralysis\”, which comes into play because there are so many potential parameters to consider in terms of what information to archive.

Each \”track\” may be named by the dance it accompanies, may have multiple spellings in multiple languages. For each \”track\” there may be a specific archivist to whom it is attributed. There are performers, various music collections (albums) it may be sourced from. There may be images, ratings, composer, etc. So one thing we would need to consider is \”how much information, and which information do we want to archive for each track?\”

Then there is the question of how the tracks will be associated into categories and collections, which ties in with the content of the stored track data.

I\’ll take a moment to mention here that this music is absolutely awesome and inspiring. So raw. So earthy. So tactile. Currently listening to Aino Kchume, Assyrina folk dance, Recorded in 1962 by the Shemiram Assyrian Folklore Groupe in Tehran, Iran. From the album \”Assyrian Folk Dances\”, Folkcraft LP-4. Why don\’t we consider the Iranian culture when considering war with them? But that\’s another (related) topic.

My next step was to explore options for a music archiving platform that would fulfill our needs. Larry recommended I join the East European Folklife Center mailing list, which I believe I was already on. I believe it was through them that I discovered the Wikipedia (yay croud-sourced information and Open Source) comparison of audio player software. It needed to run on an Apple (OSX) computer, which is what the group (and Mad haPPy) have, and needed to be flexible in terms of track information configuration. QuodLibet not only meets those qualifications, but it has longevity (released in 2004), is still being maintained and supported, is open source and is written in Python, which is a computer language I know (to some degree, anyway). Perfect.


Now that our platform was determined (barring unforeseen glitches), it was time to look at our current digital collection and the first thing that needed doing was to get rid of the many many duplicate tracks. This would have been a really grueling manual task, but a cinch with a small python script, which compares the actual digital content of each file and moves all duplicates out into a separate directory.

Now we have a relatively clean version of our digital collection. But we need to get the cassettes digitized. It turns out that the cassettes were recorded from actual vinyl records some of our members have. But most of them are 4 and a half hours drive from here in Tuscaloosa, Alabama. It doesn\’t make a ton of sense to digitize a cassette recording of vinyl when the vinyl is accessible. Driving 9 hours and humping literally hundreds of pounds of vinyl records, while it sounds fun, would be expensive and time-consuming, and being a lazy poet, I looked for a way to avoid it. Guess what? Larry Weiner tells me that there is a group in the Washington DC area that already has a digital database of international folk dance music that we may be able to get our hands on, and lo and behold there\’s a guy who is quite close with a few members of our group that has a copy of it… somewhere.

Steve, who has the copy of the archive, lives a few hours away also and isn\’t quite sure where it is. It takes us months to finally get the music into my hands. But here are like ten thousand old, rare international folk dance music tracks. I\’ve already begun the process of transferring our cassette collection to digital, but let\’s pause and check the database for a previously digitized version of each track first.


Well this database is also full of duplicates, including many tracks which include the word \”delete\” in their name. Let\’s see what our dupFinder.py scripts thinks:

python dupFinder.py -t FOLDER_WITH_10000_TRACKS

The track names fly down the terminal screen for a long time. (There\’s a folder called NJ Lame – hey what you saying about New Jersey?) This collection of music comes with small document that contains the acronym CIFD. Does this stand for Charlottesville International Folk Dancers? Wow. That script took like ten minutes to run. Running du -sh FOLDER_WITH_10000_TRACKS on the command line reveals that it is 46 Gigbytes!

Let\’s run the script for real:

python dupFinder.py FOLDER_WITH_10000_TRACKS

Back in 20 minutes… or after a visit to the dentist… Dentist rocks! $15 fee? Sweet!

Our script appears to have removed 15 Gigabytes of duplicate data from the directory, leaving us at 31Gig. The directory of duplicates that was created contains 7.4G worth of data, which probably means that 7.6G were duplicates of duplicates. This was all done in our QNAP backup server, so let\’s now run the script on the Seagate portable 1 terabyte system. Same result. Good.

It appears that there are tracks that ended up in the duplicate/removed track directories that don\’t exist in the main directories, so perhaps our scripts needs to be revisited. In the meantime, we can just search the \”removed\” directories for tracks that can\’t be found in the main collection to see if they exist there. If not – we\’ll import them from cassette or vinyl.

\"playerwindow\"In the meantime we can begin using the music collection and organizing the archive using the Quodlibet player. For each track we can ALT-CLICK the track to \”edit tags\” and add comments and description tags and content. Now we can click in the HEADER section of the main players track list and add columns to display the comments and description for each track. We can create numerous playlists and manage their tracks, and we can configure Quodlibet to search within specified playlists.


This achieved in one of two ways: ~playlists is a flag, which accepts an argument which can be a single or set of playlists. For example: ~playlists=Standards or ~playlists=|(\'Standards\',\'Repertoire\'). This flag can either be placed in the track browser input bar or as a \”Global Filter\” in the preferences.

On our list of future Quodlibet TODO (learn)/feature requests are

  • sub-categorize playlists
  • generate playlists based on plays to automate archiving of selections per session
  • display specified track tags more eloquently
  • Move or copy database/song info collection to portable drive

Listening to our massive collection (over 20,000 tracks) of International Folk Dance music and trying not to be overwhelmed by our ignorance. There’s a lot to learn about music, dance, culture and managing an archive of music. The members of Pensacola International Folk Dancers are giddy with the new influx of inspired energy, not to mention the ease with which we can now make use of the music collection. What used to be a matter of searching through old cassettes and hoping they would play in the old cassette player is now as simple as naming a dance and either playing it or selecting a track from an array of potential versions.  The group has increased regular attendance and as well as session length, thanks in part to the support of Puffin West and as always we Puffins are grateful.

Thanks for the tips

Wanna throw some thanks out there:

Really cool blog that helped me assemble Hymnal.

Chris who turned us on to lilypond engraving library.

Seth Tobocman did the cover art.

These folks sponsored the project personally:

Version:1.0 StartHTML:0000000155 EndHTML:0000001876 StartFragment:0000000475 EndFragment:0000001859

Shavaun Pizar
George Blitch
Geraldine T. Vaurigaud
Boo Reiners
Carla Murray
Alice K Genese
Norman Vazquez
Sylvie Harris
Jennie Spanos
Adrian Fallwell
The Perrin Family
Hans and Marina Frei
Alex Sniderman
Scott Anthony
Stephen A Stetson
Leah N Pietrusiak
Jason Daniels
Georgeanna Presnell


And for this project we were also honored with another Puffin Foundation West grant.

Description of R We Done Yet?

This is the original description for R We Done Yet?:

Original Seth Tobocman Design
Seth Tobocman Design

Are We Done Yet? A two person theatre show using (international) folk music and spiritual concepts from around the world to ask a question. As a species, can we be done killing for profit? By embracing the never-ending campaign toward joy, peace, justice and freedom we gain insight into the inexhaustible value of life.

Themes include health, sexuality, commitment, education, confusion, frustration, birth, death and psychic development. The show comines music, ceremony and stand up philosophy to fearlessly honor the unique, sensual mysteries each of us is born to exemplify.