MIDI data visualization using Maestro Midi Player Tool Kit

Description

Maestro Midi Player Tool Kit (Maestro MPTK) is a Unity asset that allows you to add MIDI music to your application. With Maestro, you can play MIDI files, change speed or pitch, program specific actions when certain notes are played in the music, control play/pause, and more.

This quickstart tutorial will cover downloading and setting up the free version of Maestro Midi Player Tool Kit and discuss its potentials.

Under "Scene Example" is a walkthrough of how to use scripts to manipulate

Now you have all you need to play a midi file in Unity!

To demonstrate basic use (MIDI file will play on game startup), make sure

• there is an audio listener in your scene

• Play At Startup box is checked

Now when you start the game, you will be able to hear your MIDI file playback.

Let's now explore basic scripting for the MPTK

Scripts Example: Using MPTK to create an audio pitch/volume visualizer

This example will guide you in creating a Unity midi audio pitch/volume visualizer. We will be using MPTK scripts to manipulate 9 cylinders's heights in our scene, where each cylinder represents the pitch (based on which octave the note is in) and the height represents volume.

Setting up the scene

Make sure you have followed the setup instructions above about adding the MPTK to your project. You should have the MPTK asset imported in your project and the MidiFilePlayer prefab in your scene.

Before we jump into scripting, we need to make sure we have some gameobjects in our scene to manipulate.

1. Add a cylinder to your scene (In the hierarchy, right click > 3D Objects > Cylinder). Reset its position and set its Y-scale to 0.1, and rename it to "octaveO"

2. Duplicate it 8 times so you have a total of 9 cylinders. Rename them all to follow the pattern "octave1", "octave2", ... "octave8"

Now let's get into scripting!

Basic Scripting

Note: official API/documentation for Maestro MPTK can be found here (https://mptkapi.paxstellar.com). Scroll to see full script from this tutorial.

This section will go over creating a script so you can manipulate the midi data.

1. If you do not have a script yet, create a new script (in the projects tab, right click > Create > C# Script)

2. In your script, add the import statement

using MidiPlayerTK;

to the top of your script.

3. Inside your class, declare the private variable

private MidiFilePlayer midiFilePlayer;

4. In your Start function, assign midiFilePlayer to the Midi File Player in your hierarchy using FindObjectOfType:

midiFilePlayer = FindObjectOfType<midiFilePlayer>();

5. In order to trigger events based on MIDI notes, we need to add an event listener method to the midi file player. To do that using scripts, add this to Start:

midiFilePlayer.OnEventNotesMidi.AddListener(NoteActions);

where NoteActions is your function that performs actions. Let's write that method now.

NoteActions Function

The listener takes in a list of MPTKEvents, with each MPTKEvent holding information about one note.

Useful Maestro API methods that we will use are:

• MidiPlayerTK.MPTKEvent.Value - returns the value (pitch) of the note as an integer where 60=C5, 61=C5#, ..., 72=C6, ... .

• MidiPlayerTK.HelperNoteLabel.LabelFromMidi - given the value of a note, returns the label of the note as a string (eg. C4, D2, etc.)

  • The label of a note is in the format "<letter name><octave number><# if the note is a sharp>"

• MidiPlayerTK.MPTKEvent.Velocity - returns the velocity (volume) of the note as an integer between 0 and 127

• MidiPlayerTK.MPTKEvent.Duration - returns the duration of the note in milliseconds as a long

Using these, we can write:

public void NoteActions(List<MPTKEvent> mptkEvents) {

foreach(MPTKEvent note in mptkEvents) {

if (note.Command == MPTKCommand.NoteOn) { // if the note is being played

int noteValue = note.Value; // get the note value

string noteLabel = HelperNoteLabel.LabelFromMidi(noteValue); // get the note label

char noteOctave = noteLabel[1]; // get the octave of the note

GameObject octaveModel = GameObject.Find("octave" + noteOctave); // get the correct octave gameobject

float volume = note.Velocity; // get the note velocity

long duration = note.Duration; // get the note duration

StartCoroutine(OctaveHeightChanger(octaveModel, duration, volume));

}

}

}

/// <summary>

/// this coroutine changes the octave gameobject's height for the duration

/// + half a second so short notes can be visible to the visualizer

/// </summary>

IEnumerator OctaveHeightChanger(GameObject octaveModel, long duration, float volume) {

octaveModel.transform.localScale = new Vector3(1f, volume / 10f, 1f);

yield return new WaitForSeconds(duration/1000 + 0.5f);

octaveModel.transform.localScale = new Vector3(1f, 0.1f, 1f);

}

Make sure you drag your script onto any gameobject in your hierarchy (eg. a new empty object) and press play to see it in action!