MIDI data visualization using Maestro Midi Player Tool Kit

Page created by Beatrice Hoang, May 2022

Description

Maestro Midi Player Tool Kit (Maestro MPTK) is a Unity asset that allows you to add MIDI music to your application. With Maestro, you can play MIDI files, change speed or pitch, program specific actions when certain notes are played in the music, control play/pause, and more.

This quickstart tutorial will cover downloading and setting up the free version of Maestro Midi Player Tool Kit and discuss its potentials.

Under "Scene Example" is a walkthrough of how to use scripts to manipulate

Download

  • You can download the asset from the Unity asset store here:

https://assetstore.unity.com/packages/tools/audio/maestro-midi-player-tool-kit-free-107994#description

Once you have clicked "Add to My Assets,"

  • go to your opened project in Unity and navigate to the Package Manager (in Unity's top menu: Window > Package Manager).

  • Under Packages: My Assets, search "Maestro." Select the MPTK package and click the Import button. Import all files into your project.

Once the package has finished importing, you should see "MPTK" in Unity's top menu. You're now ready to play and manipulate MIDI files in your project!


Adding the MPTK to your environment

Add the MidiFilePlayer prefab to your scene by going to MPTK > Add Prefab MidiFilePlayer

Click on the MidiFilePlayer prefab in your scene to reveal it in the inspector. You should see something like this:

The "Select Midi" attribute allows you to choose a Midi file to play. By default, MPTK has many Midi files to choose from.

Upload your own midi file

  • Go to MPTK > Midi File Setup

  • In the box that pops up, you can select either "Add a Midi File" to add 1 midi file, or "Add From Folder" to add multiple midi files within one folder.

  • Now when you return to the inspector, you should see your files as dropdown options under "Select Midi."

Now you have all you need to play a midi file in Unity!

To demonstrate basic use (MIDI file will play on game startup), make sure

  • there is an audio listener in your scene

  • Play At Startup box is checked

Now when you start the game, you will be able to hear your MIDI file playback.


Let's now explore basic scripting for the MPTK

Scripts Example: Using MPTK to create an audio pitch/volume visualizer

This example will guide you in creating a Unity midi audio pitch/volume visualizer. We will be using MPTK scripts to manipulate 9 cylinders's heights in our scene, where each cylinder represents the pitch (based on which octave the note is in) and the height represents volume.

Setting up the scene

Make sure you have followed the setup instructions above about adding the MPTK to your project. You should have the MPTK asset imported in your project and the MidiFilePlayer prefab in your scene.

Before we jump into scripting, we need to make sure we have some gameobjects in our scene to manipulate.

  1. Add a cylinder to your scene (In the hierarchy, right click > 3D Objects > Cylinder). Reset its position and set its Y-scale to 0.1, and rename it to "octaveO"

  2. Duplicate it 8 times so you have a total of 9 cylinders. Rename them all to follow the pattern "octave1", "octave2", ... "octave8"

Now let's get into scripting!

Basic Scripting

Note: official API/documentation for Maestro MPTK can be found here (https://mptkapi.paxstellar.com). Scroll to see full script from this tutorial.

This section will go over creating a script so you can manipulate the midi data.

  1. If you do not have a script yet, create a new script (in the projects tab, right click > Create > C# Script)

  2. In your script, add the import statement

using MidiPlayerTK;

to the top of your script.

  1. Inside your class, declare the private variable

private MidiFilePlayer midiFilePlayer;

  1. In your Start function, assign midiFilePlayer to the Midi File Player in your hierarchy using FindObjectOfType:

midiFilePlayer = FindObjectOfType<midiFilePlayer>();

  1. In order to trigger events based on MIDI notes, we need to add an event listener method to the midi file player. To do that using scripts, add this to Start:

midiFilePlayer.OnEventNotesMidi.AddListener(NoteActions);

where NoteActions is your function that performs actions. Let's write that method now.

NoteActions Function

The listener takes in a list of MPTKEvents, with each MPTKEvent holding information about one note.

Useful Maestro API methods that we will use are:


Using these, we can write:


public void NoteActions(List<MPTKEvent> mptkEvents) {

foreach(MPTKEvent note in mptkEvents) {

if (note.Command == MPTKCommand.NoteOn) { // if the note is being played

int noteValue = note.Value; // get the note value

string noteLabel = HelperNoteLabel.LabelFromMidi(noteValue); // get the note label

char noteOctave = noteLabel[1]; // get the octave of the note

GameObject octaveModel = GameObject.Find("octave" + noteOctave); // get the correct octave gameobject

float volume = note.Velocity; // get the note velocity

long duration = note.Duration; // get the note duration

StartCoroutine(OctaveHeightChanger(octaveModel, duration, volume));

}

}

}


/// <summary>

/// this coroutine changes the octave gameobject's height for the duration

/// + half a second so short notes can be visible to the visualizer

/// </summary>

IEnumerator OctaveHeightChanger(GameObject octaveModel, long duration, float volume) {

octaveModel.transform.localScale = new Vector3(1f, volume / 10f, 1f);

yield return new WaitForSeconds(duration/1000 + 0.5f);

octaveModel.transform.localScale = new Vector3(1f, 0.1f, 1f);

}

Make sure you drag your script onto any gameobject in your hierarchy (eg. a new empty object) and press play to see it in action!

Full Script

using System.Collections;

using System.Collections.Generic;

using UnityEngine;

using MidiPlayerTK;


public class ExampleMidiControl : MonoBehaviour

{

private MidiFilePlayer midiFilePlayer;


// Start is called before the first frame update

void Start()

{

// get midi player object

midiFilePlayer = FindObjectOfType<MidiFilePlayer>();

midiFilePlayer.OnEventNotesMidi.AddListener(NoteActions);

}


public void NoteActions(List<MPTKEvent> mptkEvents) {

foreach(MPTKEvent note in mptkEvents) {

if (note.Command == MPTKCommand.NoteOn) { // if the note is being played

int noteValue = note.Value; // get the note value

string noteLabel = HelperNoteLabel.LabelFromMidi(noteValue); // get the note label

char noteOctave = noteLabel[1]; // get the octave of the note

GameObject octaveModel = GameObject.Find("octave" + noteOctave); // get the correct octave gameobject

float volume = note.Velocity; // get the note velocity

long duration = note.Duration; // get the note duration

StartCoroutine(OctaveHeightChanger(octaveModel, duration, volume));

}

}

}


/// <summary>

/// this coroutine changes the octave gameobject's height for the duration

/// + half a second so short notes can be visible to the visualizer

/// </summary>

IEnumerator OctaveHeightChanger(GameObject octaveModel, long duration, float volume) {

octaveModel.transform.localScale = new Vector3(1f, volume / 10f, 1f);

yield return new WaitForSeconds(duration/1000 + 0.5f);

octaveModel.transform.localScale = new Vector3(1f, 0.1f, 1f);

}

}