MIDI data visualization using Maestro Midi Player Tool Kit

Description

Maestro Midi Player Tool Kit (Maestro MPTK) is a Unity asset that allows you to add MIDI music to your application. With Maestro, you can play MIDI files, change speed or pitch, program specific actions when certain notes are played in the music, control play/pause, and more.

This quickstart tutorial will cover downloading and setting up the free version of Maestro Midi Player Tool Kit and discuss its potentials.

Under "Scene Example" is a walkthrough of how to use scripts to manipulate 

Scripts Example: Using MPTK to create an audio pitch/volume visualizer

This example will guide you in creating a Unity midi audio pitch/volume visualizer. We will be using MPTK scripts to manipulate 9 cylinders's heights in our scene, where each cylinder represents the pitch (based on which octave the note is in) and the height represents volume.

Setting up the scene

Make sure you have followed the setup instructions above about adding the MPTK to your project. You should have the MPTK asset imported in your project and the MidiFilePlayer prefab in your scene.

Before we jump into scripting, we need to make sure we have some gameobjects in our scene to manipulate.

1. Add a cylinder to your scene (In the hierarchy, right click > 3D Objects > Cylinder). Reset its position and set its Y-scale to 0.1, and rename it to "octaveO"

2. Duplicate it 8 times so you have a total of 9 cylinders. Rename them all to follow the pattern "octave1", "octave2", ... "octave8"

Now let's get into scripting!

Basic Scripting

Note: official API/documentation for Maestro MPTK can be found here (https://mptkapi.paxstellar.com). Scroll to see full script from this tutorial.

This section will go over creating a script so you can manipulate the midi data.

1. If you do not have a script yet, create a new script (in the projects tab, right click > Create > C# Script)

2. In your script, add the import statement

using MidiPlayerTK;

to the top of your script.

3. Inside your class, declare the private variable 

private MidiFilePlayer midiFilePlayer;

4. In your Start function, assign midiFilePlayer to the Midi File Player in your hierarchy using FindObjectOfType:

midiFilePlayer = FindObjectOfType<midiFilePlayer>(); 

5. In order to trigger events based on MIDI notes, we need to add an event listener method to the midi file player. To do that using scripts, add this to Start:

midiFilePlayer.OnEventNotesMidi.AddListener(NoteActions);

where NoteActions is your function that performs actions. Let's write that method now.

NoteActions Function

The listener takes in a list of MPTKEvents, with each MPTKEvent holding information about one note.

Useful Maestro API methods that we will use are:

• MidiPlayerTK.MPTKEvent.Value - returns the value (pitch) of the note as an integer where  60=C5, 61=C5#, ..., 72=C6, ... .

• MidiPlayerTK.HelperNoteLabel.LabelFromMidi - given the value of a note, returns the label of the note as a string (eg. C4, D2, etc.)

  • The label of a note is in the format "<letter name><octave number><# if the note is a sharp>"

• MidiPlayerTK.MPTKEvent.Velocity - returns the velocity (volume) of the note as an integer between 0 and 127

• MidiPlayerTK.MPTKEvent.Duration - returns the duration of the note in milliseconds as a long

Using these, we can write:

public void NoteActions(List<MPTKEvent> mptkEvents) {

   foreach(MPTKEvent note in mptkEvents) {

       if (note.Command == MPTKCommand.NoteOn) { // if the note is being played

           int noteValue = note.Value; // get the note value

           string noteLabel = HelperNoteLabel.LabelFromMidi(noteValue); // get the note label

           char noteOctave = noteLabel[1]; // get the octave of the note

           GameObject octaveModel = GameObject.Find("octave" + noteOctave); // get the correct octave gameobject

           float volume = note.Velocity; // get the note velocity

           long duration = note.Duration; // get the note duration

           StartCoroutine(OctaveHeightChanger(octaveModel, duration, volume));

       }

   }

}

/// <summary>

/// this coroutine changes the octave gameobject's height for the duration

/// + half a second so short notes can be visible to the visualizer

/// </summary>

IEnumerator OctaveHeightChanger(GameObject octaveModel, long duration, float volume) {

   octaveModel.transform.localScale = new Vector3(1f, volume / 10f, 1f);

   yield return new WaitForSeconds(duration/1000 + 0.5f);

   octaveModel.transform.localScale = new Vector3(1f, 0.1f, 1f);

}

Make sure you drag your script onto any gameobject in your hierarchy (eg. a new empty object) and press play to see it in action!

Switch Song with meta building block

Part 0: Install Meta Building Block

• Follow this tutorial carefully:

  • ➔ Meta Building Blocks In UNITY

• Make sure the Meta SDK and Meta Building Blocks are correctly installed in your Unity project!

Part 1: Organize Your Visualization Setup

• Create an empty GameObject in your scene.

• Name it something like "MidiPlay 1 Visual".

• Drag all your related objects (e.g., octave1, octave2, ..., octave8) under this GameObject.

Part 2: Duplicate for Each Song

• Duplicate your MidiFilePlayer and MidiPlay Visual:

  • Right-click > Duplicate.

  • Name them properly:

    • "MidiFilePlayer 1", "MidiPlay 1 Visual" (original)

    • "MidiFilePlayer 2", "MidiPlay 2 Visual" (copy)

  • Assign a different MIDI to each MidiFilePlayer if you want different songs.

Part 5: Implement Restart Function

• When you switch between songs, you must reset playback properly. Add this script to your controller or a new manager script:

   public MidiFilePlayer midiFilePlayer;

   private Dictionary<int, Queue<GameObject>> trackCubes = new Dictionary<int, Queue<GameObject>>();

   private Dictionary<int, float> trackXOffsets = new Dictionary<int, float>();

   public void Restart()

   {

       if (midiFilePlayer != null)

       {

           midiFilePlayer.MPTK_Stop();

       }

       ClearAllVisuals();

       if (midiFilePlayer != null)

       {

           midiFilePlayer.MPTK_Play();

       }

   }

   // ⚡ Revise this function according to your visualization setup (e.g., cylinders, cubes, or lines)

   public void ClearAllVisuals()

   {

       foreach (var trackPair in trackCubes)

       {

           foreach (GameObject cube in trackPair.Value)

           {

               if (cube != null)

                   Destroy(cube);

           }

       }

       trackCubes.Clear();

       trackXOffsets.Clear();

   }

Important:

If you're using cylinders instead of dynamic cubes, adjust ClearAllVisuals() to reset cylinder heights instead of destroying cubes.

   public void ClearAllVisuals()

   {

       for (int i = 0; i <= 8; i++) {

           GameObject cylinder = GameObject.Find("octave" + i);

           if (cylinder != null)

               cylinder.transform.localScale = new Vector3(1f, 0.1f, 1f); // Reset height

       }

   }

Part 6: Hook Restart to Controller Button

• In Unity Inspector:

  • On your controller button action (click "+" to add an event),

  • Drag the object with your MidiPlayerManager script into the slot,

  • Select MidiPlayerManager.Restart().