Do Re Mi Fa Sci Lab

Ohhh Activity 9....
This is uhh. hmm
This activity took me quite a long time. But this post is going to be very short.

I really worked hard on another piece of music...
Oh right ugh...
In this activity we're gonna make Scilab play us music by processing the image of a music sheet.
So anyway as I was saying. There was this quite complicated music sheet that is very special to me. I really wanted to use that. In fact I spent the long weekend before November 2 devoted to that and make a pun about how I poured all of my soul into this activity in celebration of All Souls Day and stuff.

Unfortunately for me, I did not get the piece playing quite right. And no, I'm not willing to post that half baked result. I mean sure I do like the output of that one (I got LSS quite a few times because of it) but, as I said that music piece is special to me so I just can't bring myself to post it until it's perfect. I plan to finish that during this Christmas break so yeah... stay tuned.

This is now all just for the sake of completion.

Suppose we have a piece of sheet music. For simplicity I will only show three 3/4 measures.

Step 1. Binarize the image

Since I binarized this in MS Paint, some features are lost. Thankfully, those missing features are supposed to be removed later on anyway.

Step 2. Remove staff and "flagpoles" (those things that attach notes to their flags)
Using morphological operations (see Activity 8)
I Dilated the image using a thin horizontal line. Thus removing the flagpoles (and all black features that are thin horizontally)

Then I Dilated the image again, now using a thing vertical line. Thus removing the staves (and all vertically thin black features)

Step 3. Extract notes
Notes are in general (or always?) circular. I obtained the notes by using the OPEN operator using a circular Structure Element. Due to the nature of extraction, this step can be done right after Step 1 and the results will be the same.

Doesn't OPEN get rid of the black features?
Right. I opted to first take the complement of the image (reverse black and white) due to the nature of Blob Analysis that will be used in the next steps

Step 4. Get note positions

Using the Scilab IPD Toolbox function, SearchBlobs and AnalyzeBlobs, the centroid of the notes can easily be obtained. Based on the y-position it is easy to identify the pitch (frequency) of the note to be played. While based on the x-position, we can identify the order with which to play them.

Step 5. Consider the flags.

Now that we know the position of each note, we can scan for it's surroundings. Shown here in red is the window with which I scanned the environment of the note.

I know this example is weird because they are all eight notes anyway. By simply counting the amount of blobs within the windowed region, it is easy to identify if the note is a quarter note, eight note, or even a sixteenth note. Each flag becomes a separate blob since we removed the 'flagpole'

Well, enjoy the music.

Here is a "raw" rendition.



And this one is using both an ASDR envelope and some "harmonics" effect
  

---

Oh right. The music is L's Theme from Death Note.
Arranged by Olivia Jelks
Obtained from Musecore.com

Hmm I guess I could say Mission Accomplished once more.
That's a $10/10!$ for me then.