The anatomy of a MIDI feature detector

contents abstract intro web ACOI detector query retrieval conclusions References

Automatic indexing for musical data is an inherently difficult problem. Existing systems mostly rely on hand-crafted solutions, geared towards a particular group of users, such as for example composers of film music [MM]. In this section, we will look at a simple feature detector for MIDI encoded musical data. It provides a skeleton for future experimentation.

slide: MIDI features

The hierarchical information structure that we consider is depicted in slide midi-structure. It contains only a limited number of basic properties and must be extended with information along the lines of the musical ontology proposed in [AI]. However, the detector presented here provides a skeleton solution that accomodates an extension with arbitrary predicates over the musical structure in a transparent manner.

The grammar given below corresponds in an obvious way with the structure depicted in slide midi-structure.


  
  detector song;  to get the filename
  detector lyrics;  extracts lyrics
  detector melody;  extracts melody
  detector check;   to walk the tree
  
  atom str name;
  atom str text;
  atom str note;  
  
  midi: song;
  
  song: file lyrics melody check;
  
  file: name;
  
  lyrics: text*;
  melody: note*;

slide: A simple feature grammar for MIDI files

The start symbol is a song. The detector that is associated with song reads in a MIDI file. The musical information contained in the MIDI file is then stored as a collection of Prolog facts. This translation is very direct. In effect the MIDI file header information is stored, and events are recorded as facts, as illustrated below for a note_on and note_off event.


  event('twinkle',2,time=384, note_on:[chan=2,pitch=72,vol=111]).
  event('twinkle',2,time=768, note_off:[chan=2,pitch=72,vol=100]).

After translating the MIDI file into a Prolog format, the other detectors will be invoked, that is the composer, lyrics and melody detector, to extract the information related to these properties.

To extract relevant fragments of the melody we use the melody detector, of which a partial listing is given below.

  int melodyDetector(tree *pt, list *tks ){
  char buf[1024]; char* _result;
  void* q = _query;
  int idq = 0; 
  
    idq = query_eval(q,"X:melody(X)");
    while ((_result = query_result(q,idq)) ) {
           putAtom(tks,"note",_result);
           }
    return SUCCESS;
  }

slide: The melody detector

The embedded logic component is given the query X:melody(X), which results in the notes that constitute the (relevant fragment of the) melody. These notes are then added to the tokenstream. A similar detector is available for the lyrics.

Parsing a given MIDI file, for example twinkle.mid, results in updating the Monet database as indicated below.

  V1 := newoid();
    midi_song.insert(oid(V0),oid(V1));
  V2 := newoid();
      song_file.insert(oid(V1),oid(V2));
        file_name.insert(oid(V2),"twinkle");
      song_lyrics.insert(oid(V1),oid(V2));
        lyrics_text.insert(oid(V2),"e");
        lyrics_text.insert(oid(V2),"per-");
        lyrics_text.insert(oid(V2),"sonne");
        lyrics_text.insert(oid(V2),"Moi");
        lyrics_text.insert(oid(V2),"je");
        lyrics_text.insert(oid(V2),"dis");
        lyrics_text.insert(oid(V2),"que");
        lyrics_text.insert(oid(V2),"les");
        lyrics_text.insert(oid(V2),"bon-");
        lyrics_text.insert(oid(V2),"bons");
        lyrics_text.insert(oid(V2),"Val-");
        lyrics_text.insert(oid(V2),"ent");
      song_melody.insert(oid(V1),oid(V2));
        melody_note.insert(oid(V2),"a-2");
        melody_note.insert(oid(V2),"a-2");
        melody_note.insert(oid(V2),"g-2");
        melody_note.insert(oid(V2),"g-2");
        melody_note.insert(oid(V2),"f-2");
        melody_note.insert(oid(V2),"f-2");
        melody_note.insert(oid(V2),"e-2");
        melody_note.insert(oid(V2),"e-2");
        melody_note.insert(oid(V2),"d-2");
        melody_note.insert(oid(V2),"d-2");
        melody_note.insert(oid(V2),"e-2");
        melody_note.insert(oid(V2),"c-2");

slide: Update of Monet database

The updates clearly reflect the structure of the musical information object that corresponds to the properties defined in the grammar.

Implementation status

Currently, we have a running prototype of the MIDI feature detector. It uses an adapted version of public domain MIDI processing software. The embedded logic component is part of the hush framework. It uses an object extension of Prolog that allows for the definition of native objects to interface with the MIDI processing software. A description of the logic component is beyond the scope of this paper, but will be provided in [OO]. The logic component, however, allows for the definition of arbitrary predicates to extract the musical information, such as the melody and the lyrics. As stated before, the current detector must be regarded as a skeleton implementation that provides the basis for further experimentation.