All About Computers and Audio Latency

Meet the ghost in your machine

By Craig Anderton

Musicians are used to an instant response: Hit a string, hit a key, strike a drum, or blow into a wind instrument, and you hear a sound. This is true even if you’re going through a string of analog processors.

But if you play through a digital signal processor, like a digital multieffects, there will be a very slight delay called latency—so small that you probably won’t notice it, but it’s there. Converting an analog signal to digital takes about 600 microseconds at 44.1kHz; converting back into analog takes approximately the same amount, for a “round trip” latency of about 1.2 milliseconds. There may also be a slight delay due to processing time within the processor.

Because sound travels at about 1 foot (30 cm) per millisecond, the delay of doing analog/digital/analog conversion is about the same as if you moved a little over a foot away from a speaker, which isn’t a problem. However, with computers, there’s much more going on. In addition to converting your “analog world” signal to digital data, pieces of software called drivers have the job of taking the data generated by an analog-to-digital converter and inserting it into the computer’s data stream. Furthermore, the computer introduces delays as well. Even the most powerful processor can do only so many millions of calculations per second; when it’s busy scanning its keyboard and mouse, checking its ports, moving data in and out of RAM, sending out video data, and more, you can understand why it sometimes has a hard time keeping up.

As a result, the computer places some of the incoming audio from your guitar, voice, keyboard, or other signal source in a buffer, which is like a “savings account” for your input signal. When the computer is so busy elsewhere that it can’t deal with audio, it makes a “withdrawal” from the buffer instead so it can go deal with other things. The larger the buffer, the less likely the computer will run out of audio data when it needs it. But a larger buffer also means that your instrument’s signal is being diverted for a longer period of time before being processed by the computer, which increases latency. When the computer goes to retrieve some audio and there’s nothing in the buffer, audio performance suffers in a variety of ways: You may hear stuttering, crackling, “dropouts” where there is no audio, or worse case, the program might crash.

The practical result of latency is that if you listen to what’s you’re playing after it goes through the computer, you’ll feel like you’re playing through a delay line, set for processed sound only. If the delay is under 5 ms, you probably won’t care too much. But some systems can exhibit latencies of tens or even hundreds of milliseconds, which can be extremely annoying.

Because you want the best possible “feel” when playing your instrument through a computer, let’s investigate how to obtain the lowest possible latency, and what tradeoffs will allow for this.

MINIMIZING LATENCY

The first step in minimizing delay is the most expensive one: Upgrading your processor. When software synthesizers were first introduced, latencies in the hundreds of milliseconds were common. With today’s multi-core processors and a quality audio interface, it’s possible to obtain latencies well under 10 ms (and often less) at a 44.1kHz sampling rate.

The second step toward lower latency involves using the best possible drivers, as more efficient drivers reduce latency. Steinberg devised the first low-latency driver protocol specifically for audio, called ASIO (Advanced Streaming Input Output). This tied in closely with the CPU, bypassing various layers of both Mac and Windows operating systems. At that time the Mac used Sound Manager, and Windows used a variety of protocols, all of which were equally unsuited to musical needs. Audio interfaces that supported ASIO were essential for serious musical applications.

Eventually Apple and Microsoft realized the importance of low latency response and introduced new protocols. Microsoft’s WDM and WASAPI in exclusive mode were far better than their previous efforts; starting with OS X Apple gave us Core Audio, which was tied in even more closely with low-level operating system elements. Either of these protocols can perform as well as ASIO. However for Windows, ASIO is so common and so much effort is put into developing ASIO drivers that most musicians select ASIO drivers for their interfaces.

So we should just use the lowest latency possible, yes? Well, that’s not always obtainable, because lower latencies stress out your computer more. This is why most audio interfaces give you a choice of latency settings (Fig. 1), so you can trade off between lowest latency and computer performance. Note that latency is given either in milliseconds or samples; while milliseconds is more intuitive, the reality is that you set latency based on what works best (which we’ll describe later, as well as the meaning behind the numbers). The numbers themselves aren’t that significant other than indicating “more” or “less.”

Fig. 1: Roland’s VS-700 hardware is being set to 64 samples of latency in Cakewalk Sonar.

If all your computer has to do is run something like a guitar amp simulator in stand-alone mode, then you can select really low latency. But if you’re running a complex digital audio recording program and playing back lots of tracks or using virtual software synthesizers, you may need to set the latency higher.

So, taking all this into account, here are some tips on how to get the best combination of low latency and high performance.

• If you have a multi-core-based computer, check whether your host recording program supports multi-core processor operation. If available, you’ll find this under preferences (newer programs are often “multiprocessor aware” so this option isn’t needed). This will increase performance and reduce latency.

• With Windows, download your audio interface’s latest drivers. Check the manufacturer’s web site periodically to see if new drivers are available, but set a System Restore point before installing them—just in case the new driver has some bug or incompatibility with your system. Macs typically don’t need drivers as the audio interfaces hook directly into the CoreAudio services (Fig. 2), but there may be updated “control panel” software for your interface that provides greater functionality, such as letting you choose from a wider number of sample rates.

Fig. 2: MOTU’s Digital Performer is being set up to work with a Core Audio device from Avid.

• Make sure you choose the right audio driver protocol for your audio interface. For example, with Windows computers, a sound card might offer several possible driver protocols like ASIO, DirectX, MME, emulated ASIO, etc. Most audio interfaces include an ASIO driver written specifically for the audio interface, and that’s the one you want to use. Typically, it will include the manufacturer’s name.

• There’s a “sweet spot” for latency. Too high, and the system will seem unresponsive; too low, and you’ll experience performance issues. I usually err on the side of being conservative rather than pushing the computer too hard.

• Avoid placing too much stress on your computer’s CPU. For example, the “track freeze” function in various recording programs lets you premix the sound of a software synthesizer to a hard disk track, which requires less power from your CPU than running the software synthesizer itself.

MEASURING LATENCY

So far, we’ve mostly talked about latency in terms of milliseconds. However, some manufacturers specify it in samples. This isn’t quitebas easy to understand, but it’s not hard to translate samples to milliseconds. This involves getting into some math, so if the following makes your brain explode, just remember the #1 rule of latency: Use the lowest setting that gives reliable audio operation. In other words, if the latency is expressed in milliseconds, use the lowest setting that works. If it’s specified in samples, you still use the lowest setting that works. Okay, on to the math.

With a 44.1kHz sampling rate for digital audio (the rate used by CDs and many recording projects), there are 44,100 samples taken per second. Therefore, each sample is 1/44,100th of a second long, or about 0.023 ms. (If any math wizards happen to be reading this, the exact value is 0.022675736961451247165532879818594 ms. Now you know!)

So, if an audio interface has a latency of 256 samples, at 44.1 kHz that means a delay of 256 X 0.023 ms, which is about 5.8 ms. 128 samples of delay would be about 2.9 ms. At a sample rate of 88.2 kHz, each sample lasts half as long as a sample at 44.1 kHz, so each sample would be about 0.0125 ms. Thus, a delay of 256 samples at 88.2 kHz would be around 2.9 ms.

From this, it might seem that you’d want to record at higher sample rates to minimize latency, and that’s sort of true. But again, there’s a tradeoff because high sample rates stress out your computer more. So you might indeed have lower latency, but only be able to run, for example, half the number of plug-ins you normally can.

SNEAKY LATENCY ISSUES

Audio interfaces are supposed to report their latency back to the host program, so it can get a readout of the latency and compensate for this during the recording process. Think about it: If you’re playing along with drums and hear a sound 6 ms late, and then it takes 6 ms for what you play to get recorded into your computer, then what you play will be delayed by 12 ms compared to what you’re listening to. If the program knows this, it can compensate during the playback process so that overdubbed parts “line up” with the original track.

However, different interfaces have different ways to report latency. You might assume that a sound card with a latency of 5.8 milliseconds is outperforming one with a listed latency of 11.6 ms. But that’s not necessarily true, because one might list the latency a signal experiences going into the computer (“one-way latency”), while another might give the “round-trip” latency—the input and output latency. Or, it might give both readings.

Furthermore, these readings are not always accurate. Some audio interfaces do not report latency accurately, and might be off by even hundreds of samples. So, understand that if an audio interface claims that its latency is lower than another model, but you sense more of a delay with the “lower latency” audio interface, it very well might not be lower.

WHAT ABOUT “DIRECT MONITORING”?

You may have heard about an audio interface feature called “direct monitoring,” which supposedly reduces latency to nothing, so what you hear as you monitor is essentially in real time. However, it does this by monitoring the signal going into the computer and letting you listen to that, essentially bypassing the computer (Fig. 3).

Fig. 3: TASCAM’s UH-7000 interface has a mixer applet with a monitor mix slider (upper right). This lets you choose whether to listen to the input, the computer output, or a combination of the two.

While that works well for many instruments, suppose you’re playing guitar through amp simulation plug-in running on your computer. If you don’t listen to what’s coming out of your computer, you won’t hear what the amp simulator is doing. As a result, if you use an audio interface with the option to enable direct monitoring, you’ll need to decide when it’s appropriate to use it.

THE VIRTUES OF USING HEADPHONES

One tip about minimizing latency is that if you’re listening to monitor speakers and your ears are about 3 feet (1 meter) away, you’ve just added another 3 ms of latency. Monitoring through headphones will remove that latency, leaving only the latency caused by using the audio interface and computer.

MAC VS. WINDOWS

Note that there is a significant difference between current Mac and Windows machines. Core Audio is a complete audio sub-system that already includes drivers most audio interfaces can access. Therefore, as mentioned earlier, it is usually not necessary to load drivers when hooking an audio interface up to the Mac. With Windows, audio interfaces generally include custom drivers you need to install, that are often on a CD-ROM included with the interface. However, it’s always a good idea to check the manufacturer’s web site for updates—even if you bought a product the day it hit the stores. With driver software playing such a crucial part in performance, you want the most recent version.

With Windows, it’s also very important to follow any driver installation instructions exactly. For example, some audio interfaces require that you install the driver software first, then connect the interface to your system. Others require that you hook up the hardware first, then install the software. Pay attention to the instructions!

THE FUTURE AND THE PRESENT

Over the last 10 years or so, latency has become less and less of a problem. Today’s systems can obtain very low latency figures, and this will continue to improve.

But if you experience significant latencies with a modern computer, then there’s something wrong. Check audio options, drivers, and settings for your host program until you find out what’s causing the problem.

Craig Anderton is Editor Emeritus of Harmony Central. He has played on, mixed, or produced over 20 major label releases (as well as mastered over a hundred tracks for various musicians), and written over a thousand articles for magazines like Guitar Player, Keyboard, Sound on Sound (UK), and Sound + Recording (Germany). He has also lectured on technology and the arts in 38 states, 10 countries, and three languages.