Announcements

Special Announcement Update:


You asked, we listened! Please note that User Reviews are back! The Home page has returned to an editorial focus, with an emphasis on Articles and Industry News. The drop down menus are gone and simpler navigation has been implemented, and the forum landing page structure is going back to a more familiar layout. This transition is being performed "live", so some things may be moving around temporarily as we complete the process. Thanks for your patience as we implement these changes that our Community Members have asked for.


- Your Harmony Central Team
Anderton

Backing Up Projects with MIDI Sys Ex

by Community Manager ‎04-08-2012 02:06 PM - edited ‎01-16-2013 03:06 PM

Backing up projects is easy—until external hardware gets involved. Fortunately, there's a solution

 

by Craig Anderton

 

By now, you’ve probably figured out that backing up is important. With today’s “in the box” recordings, backing up is easy: if all the parameters, data, and audio are a part of the project, then backing up the project saves all the data. Easy.

 

But despite the proliferation of in the box recording, outboard gear didn’t just disappear—plenty of projects still use hardware synths or external signal processors.  And when it’s time to back up, how do you save this external data as part of a DAW project?

 

Fortunately, the developers of the MIDI spec anticipated that instruments could have non-standard data apart from the usual MIDI commands for notes, controllers, etc.—such as patch data. So they created a special MIDI data type called system exclusive, or sys ex. This data string starts with a unique header identifying a particular manufacturer, followed by the data, and ending with an end-of-message command. If you send this string of data to a MIDI device and that device recognizes the header, it will “listen” to the data; otherwise, the data is irrelevant. Sys ex data can provide a “snapshot” of the synth’s contents including programs, combis/multis, effects settings, and the like, as well as preset data in signal processors.

 

Instruments and processors send sys ex as a “data dump,” which can almost always be initiated from the device itself (Fig. 1; the device might also be able to send a data dump in response to a request from a program like an editor/librarian, or from within a DAW that’s capable of “querying” particular MIDI gear). As this is MIDI data, it can be recorded into a DAW’s MIDI track.

 

Send MIDI.jpg

Fig. 1: Data is being sent from a Yamaha Motif XS6 by implementing a data dump. This exits the MIDI out port, and goes to your DAW's MIDI interface.

 

SAVING MIDI DATA TO A DAW

 

To save this data, the basic procedure is to patch a cable from the device’s MIDI out to your computer interface’s MIDI in, initiate a dump at the device, and record the data into a MIDI track in your host. To reload the data, patch your computer interface’s MIDI out to your device’s MIDI in. The data from the track plays into the device, restoring its settings.

 

Or at least, that’s how it works in theory. In reality, some issues can complicate this process. Here are the main ones.

 

Size. Large sys ex dumps may bog down your sequencer if it can’t accept the rate at which data flows in. And even if it can store this data, on playback the MIDI data stream can “clog” if you’re trying to send huge amounts of data to the MIDI out port. Fortunately, most gear can send varying types of sys ex—a single patch, all patches, etc. For example to change only a synthesizer patch in the middle of a song, dump that single patch into the instrument’s track. It will likely be only a few kilobytes, and your sequencer should be able to burp it back into your synth easily.

 

An alternative way to re-create the intended sound would be to insert a program change command into a MIDI track instead of sys ex, but that assumes the keyboard will contain the same patches as when you inserted the program change command. By sending sys ex, you’re sending the actual patch data, not just calling up a particular memory slot.

 

Misset preferences. Most hosts disable sys ex recording because unlike note data, it’s not something you record all the time. So, check for a “MIDI filter” dialog box and make sure the host can record sys ex (Fig. 2). Sys ex input might also be disabled on your outboard device; check that too.

 

Fig1_MOTU.png

Fig. 2: Like most programs, MOTU’s Digital Performer defaults to not recording sys ex data. Going Setup > Set Input Filter brings up a dialog box where you can enable sys ex recording (click to enlarge).

 

Track assignments. Ideally, you’d include the patch data needed for an outboard device at the beginning of the MIDI track driving it, but that would re-send the sys ex every time you start the song. A more efficient approach is to record all needed “setup” sys ex, one data file after another, into a single track. When you start a song, solo that track, and as the sequence plays it will sequentially load up all your devices with the appropriate patch information. Then, un-solo the sys ex track and mute it.

 

Variations among sequencer protocols. Not all sequencers work the same way. For example, MOTU Digital Performer treats sys ex like any other MIDI data: Record it into a track, then play it back. You can even edit the sys ex data if you’re handy with hexadecimal. Cakewalk Sonar limits sys ex recording to 255 bytes, which is optimized for short messages like control setting changes. However, Sonar also has a sys ex librarian that’s saved with each project, where you can store up to 8,192 banks of sys ex data (Fig. 3). You can insert a track event that triggers sending a bank’s contents through the MIDI out, thus accomplishing the same result as playing sys ex from a track.

 

YamahaSysEx.png

Fig. 3: The background screen is Sonar's Sys Ex view, which shows the banks of Sys Ex. Two Banks contain patches from Yamaha's Motif XS6 synthesizer. The Receive System Exclusive screen has "You start dump on instrument" but if you're saving sys ex from one of the supported instruments, you can initiate the dump from within Sonar. The topmost screen shows that sys ex (2,074 bytes) has been received for Bank 3, which hasn't been named yet (click to enlarge).

 

Get into the habit of saving sys ex with a project; next time you re-load the project and play back the sys ex, your outboard hardware should be exactly as you left it. Sys ex can be an important part of your backup protocol—take advantage of it.