Rant - software industry

[orangefunk]

It would be great if all the major software developers all got together and made a standard Music OS which worked on PC hardware..

Ahh.. we can only but dream...

I'm a programmer/researcher and even in the programming world its all getting messed up with IDEs/MDD/Languages/PlugIns/Build/Configuration stuff... its become a nightmare to develop because it seems everyone is interested in long tool chains... but if one of those tools evolves it can cause all kinds of horrible plugin dependency problems... even this week an update to the IDE meant I had to update everything else... and even then I found that the programming language I was using wouldn't work with the new IDE... then I got trapped in a cyclical dependency where somethign wouldn't install because something else wasn't installed and vice versa... grrr..

I know of at least two huge companies I've been involved in have had all kinds of problems... for example one TV company can't find the bug which for no reason resets periodically every TV product in their range. Yet they still sell these TVs for the last 3-4 years in every country... probably because they believe the customer probably wouldn't notice too much... (i.e. used to Windows failing and accept it)

A friend of mine was telling me that the test manager at the mobile phone company she works for, uses a rival competitors mobile phone becuase he finds them more reliable and less buggy.

I know some companies involved with medical software have actually gone back to basics as the problems they were having with reuse and component based development were getting in the way actually creating the software...

Its gonna get a lot worse before it gets better I fear...

[wades_keys]

It won't change because John Q Public will be happy as long as they can use instant messenger, send mail, write a letter and download music and movies.

 

Part of that change is knowlege; people need to be INFORMED that the line of bull they have been fed is just that.

 

It's not a question of more RAM, and faster processors.

 

It's time to question, what has the MS monopoly done for ME lately?

[P321]

We all live with it, just a natural consequence of software development? Were told its too complex to get right etc. Ive just absolutely had enough. If it was a car - can you imaging the deaths, law suits etc - crash has a whole different meaning there.

No need to imagine.. Although you're talking computer code, you can draw parallels with other things. EG Science still can't fully tell you why one person will react to a drug one way, and someone else another. As a result prescription medicine, when used as prescribed, kills or contributes to the deaths of tens of thousands of people per year. Why do we put up with it? Because, overall, it makes most peoples lives better. That doesn't mean there isn't a massive downside and thousands of people don't lose their lives to the chemical lottery every year.

So it's about weighing any benefits against the negatives. For all the quirks of a computer, it's a lot easier for people to fit a 60GB sample library in their house than a symphony orchestra Or load multiple instances of a synth rather than buy several pieces of hardware.

You can also do things which lessen your complaints. That means finding the things that work and sticking with them if you value predictability more than new features or "improvements". The classic example is NASA with their space program. For years they would stick to old slow CPU's because the bugs in them were well known, and their behaviour somewhat predictable.

If you start with a basic DAW and plugin setup, and use it for a few weeks, and it proves stable, if you start adding plugins somewhat slowly you'll usually spot which ones will give you more problems. Likewise, you need to build from a solid base with solid drivers and hardware. If you really value stability you need to start looking outside of the consumer stuff and more towards products geared at servers.. things like ECC ram, or devices to stop small dips in the power reaching the computer PSU.

Im tired of all this best effort bull{censored}

But the reality is that software is full of errors, even when it appears to work solidly. Even software running on things like a Boeing or Airbus jet, in which dozens of people are paid to spend years of their life just looking for errors, contains errors. It's the nature of the beast. That's not saying a lot of companies don't do half arsed jobs, because clearly they do.

it just means that testing programs have to be better, and vendors way more open and accountable for what they ship and what they have tested

Then vote with your feet and your cash. A lot of people have mentioned how responsive someone like the Reaper developer is. He made his money selling Winamp to AOL, so he's doing Reaper as much out of his desire to create a good music creation solution as to be a commercial enterprise. I'm not a user myself, but it's nice to see there's still developers around listening to end users more than some corporate management type of set up that leads to a lot of the entirely valid complaints you raise..

[P321]

Until someone can get windows out of the picture entirely and come up with a type of real-time kernel for music apps, it will always be this way.

No need to reinvent the wheel. There are already several alternatives which could be used, like perhaps QNX. But, if audio app support on Linux is still kinda crappy, don't expect support to come rushing for anything else..

 

Some companies ARE moving Windows out of the way and doing their own custom OS thing, building on the open source operating systems which already exist.. like Korg with the Oasys, and I think EWQL are building some sort of 64-Bit Linux sample playback application/system. But check the prices for such "fixed" hardware alternatives.. (fixed.. IE it knows what components to expect and doesn't have to worry so much about what printer you might plug in)

 

Also, whilst a realtime OS could potentially deliver advantages, the reality is that - even with all its faults - an OS like Windows can deliver sub 3ms latency today on plugins with a card that has good drivers. That's several ms FASTER than some dedicated digital hardware could manage in the 1990's. I think the Wavestation had about 10ms latency when generating audio output from the keyboards scanner detecting the push of a key.

 

Statements like the following..

 

 

Yet, no noticeable latency from key on to audible sound......

 

 

..seem to suggest it's not possible to get minimal latency and reliable timing now. Except it is.

 

Probably the biggest issue these days is something like USB being used for MIDI on multiple devices, a task which it is not suited for. It'll do it, but may well provide worse timing than devices which are years old with slower processors. We're using a protocol in a way which wasn't really intended. Something like PCIE is perfectly capable (with a good chipset) of giving rock solid timing and low latency for audio though. So your statements are only true in certain circumstances and are a somewhat inaccurate representation of computers and audio in 2007.

 

 

Again, isn't it sad that 4 Mhz cpu's could do what our multi-core maxed out PC's can't?

 

No, because they can given the right combination of hardware and software. If you want specialized devices, which can squeeze more performance, you pay specialized prices. Feel free to go to the world of $2,000 synths. I'll stick with a mass market device that delivers $99 plugins, but with an awareness that such devices are made for "average consumers", and so require "cautionary buying" if you're going to do anything outside of printing a letter reliably. It's not like there weren't any 4Mhz devices around that weren't unreliable either. if they were more reliable it's only because with increased complexity comes increased chances of things going wrong. Although the balance may never be as good on a general purpose device, as something dedicated to the task, it's not that bad these days, and sometimes we even take steps in the right direction. EG XP is still more reliable than Win95 when it comes to crashes which will reboot the whole system. When that happens on XP it's almost always hardware related these days.. bad drivers etc.

 

 

Or put it this way, you will never have any idea of the true power of the hardware in that box.

You do, because there are scientific benchmarks which measure it, and you can run them from a command line only OS if you're really determined to remove the OS from the equation as much as possible. It doesn't really reflect real world usage anyway, because there are good and crappy coders, and the best coders can make a plugin sound really good and still deliver low CPU usage. So the "true power in the box" is something that can never really be quantified fully because they're only as good as the code running through them on multiple levels.

 

The reality is that you can do more than you ever could for far less money. Just because we don't live in an ideal world, where the OS and all its apps run hand crafted machine code, doesn't make that any less true. Yes, you can always get more reliable audio and more done with less cycles, and yes general usage operating systems like OSX or Windows get in the way of that. But what you're saying suggests that many people aren't getting reliable low latency performance out of existing solutions. They are.

 

As bullet proof I'd like? No. But that's mainly an issue with the DAW I'm using. I see next to no reliability issues with my OS and drivers. There are more simpler apps that are more stable (but with less features). EG I don't trust Kontakt as a sampler, because it can cause crashes but I still usually use it because I like it better than several other sampler plugins which are much more reliable. If someone comes along and makes something as full featured or better, and delivers stability, I'd switch to it in a second.

[wades_keys]

EG I don't trust Kontakt as a sampler, because it can cause crashes but I still usually use it because I like it better than several other sampler plugins which are much more reliable. If someone comes along and makes something as full featured or better, and delivers stability, I'd switch to it in a second.

 

Yup and this is what the pros are dealing with too.

 

You didn't hear of Synclaviers and Fairlights offering this abyssmal performance...

[wades_keys]

The beancounters assume that we're going to be happy beta testers because of the pressure and "it can always be fixed afterwards", plus, hey, we're used to it with personal computers, right?

 

THis is because the beancounters are marketing towards home consumers only, to the detriment of professionals.

 

There are pros who would gladly shell out hundreds of thousands of dollars, for a workable comuter based music solution like the Synclavier.

 

Yet no one will make such a thing; why should they, when they can sell their crappy softsynths for 99 bucks a pop to Joe bedroom musician?

[stikygum]

The secret is NOT to jump immediately into a new piece of software / operating system / driver. Do your computers a favor and get them a lot of the fastest RAM and do not use them for internet.

 

 

Don't use the internet with your music computer... why? I'm interested in your comment because I've been having having unexpected errors for using the internet on my Mac, and it will close my internet windows. I don't know why. My friend told me to get rid of all the icons I have on my desktop. He said that it slows your computer down and it takes more power to have them on my desktop.

 

I really need to clean out the crap on my computer. I have a 1.8ghz Macbook with 2 gigs of Ram, but only a 60gig HD. I have about 25 gigs of HD space left because of all the crap. But I bought an external Firewire HD, 300 gigs, and am using this for audio work in Logic.

[Yoozer]

it's not likely you'll get some hack doing that type of damage.

How do I loathe the hack? Let me count the ways!

JX10 having an abysmal OS that can't handle Sysex. The AMT8 being quite a bit of a disaster (serial worked better than USB and the guy who wrote the drivers did the firmware too). Yamaha TX16 - a completely new OS (Typhoon 2000) was made for that thing and it worked better than the original.

Those types of horror stories are in inverse proportion to the skill level of the programmer.

But we can't influence a company to hire competent programmers; if we don't buy, they can't hire better people. If we do buy the stuff and they have someone incompetent, then we've already given them our money. We as musicians might suffer stuff they haven't caught in testing; but as I said, we get no guarantee and we get no way for actual feedback. I only know one company who did this right, which was Elektron, while developing the Machinedrum.

There are pros who would gladly shell out hundreds of thousands of dollars, for a workable comuter based music solution like the Synclavier.

Those pros have already shelled out the cash for ProTools. While it's not tightly integrated, it's got higher standards. You have to be really, really good to make 'm shell out that money again.

[P321]

THe problem is, that even the pros who have the money to spend, cannot get these "specialized devices", at any price. The Fairlight and Synclavier were the last 2 specialized devices, for pro musicians.

Most pro musicians don't make anywhere near the amount of money that they could afford something like a Synclavier when it came out new. The market for these types of things was only as large as a few thousand people worldwide even back then. And that's when you were talking about a device that basically had no major alternatives, other than maybe the Fairlight and eventually the Emulator.

The problem for manufacturers is that today the PC is the alternative, and a low cost one at that. Even a low end box can churn out more computing power than an $8,000 Oasys keyboard, and would certainly decimate all these $2000-$3,000 type workstations and VA boxes with their Motorola DSP's. Once Intel/AMD get a proper architecture in place, to take advantage of multiple cores, things like TDM are likely to be history too. Indeed, the biggest problem might be that there's so much power it means coders are being much more lazy at optimising the code than they probably would be on an old DSP with limited resources.

Another thing is that modern PC's (built to a decent standard) are generally more reliable than even dedicated electronics from 20-30 years ago. Perhaps more "replace than repair" compared to early electronics, but generally more reliable (I'm talking about the hardware remember ) It's to the point that many people run machines non-stop for YEARS without major problems. Consider all the servers in the world that make the internet possible and how most measure uptime in weeks and months, some even in years.

Since a very expensive dedicated solution can't beat the reliability of even cheap PC hardware (except to use the highest rated MTBF components they can find), everything comes down to the software they run. If that's the case then they may as well use PC parts to begin with. The only reason most companies don't do exactly that is simple - As the Korg Oasys showed, the markups that exist in the world of music hardware are somewhat outrageous. It's largely (but not totally) down to the size of the market involved, which means, if they stick a PC inside a music keyboard, you come out with some silly prices on the other side. This will improve as time goes on to the point where it's highly likely you'll see more things like the Oasys, if only because realism sells far more than synthesis ever did in the synth world, and (modelling aside) that requires streaming large samples.

Another irony is that, even with an expensive closed system like the Oasys, no other hardware in the picture, no 3rd party software to worry about, and an extremely basic sequencer compared to something like Logic or Cubase, the Oasys still crashes! Maybe not nearly as often as more open platforms using VST/AU plugins. But yet it still can. So just because you do that, you may not see the amount of benefits you expect. I'm not really sure what you'd want to see manufacturers build, or who you think the market is for massively restricted, expensive, closed, systems dedicated to do only a few tasks well? Other than maybe the 5000 or so people who've bought an Oasys, and that's when it's priced nowhere near the price of a Synclavier new. It's also somewhat ironic that most Oasys users will probably use it sequenced with a PC too. So why not just make it a plugin, and build a control surface for it, unless you're effectively building an $8,000 dongle?

As I already said, with large streaming sample libraries, the PC will become common in so-called workstations once they can get the prices low enough to allow for the outrageous mark-ups. When it does it's likely they'll use some form of customized open source OS. Some might call that progress. I'm not so sure. I'd rather they stopped trying to compete with PC hardware (because they simply can't) and instead concentrated on the things they can perhaps do better - building the software and giving us expressive hardware controls with which to manipulate the sounds. No need to massively inflate the costs by duplicating hardware already built by others for far cheaper.

I would also say that, until we live in a perfect world, the programmers of sequencer packages should concentrate on building massively better crash recovery techniques too, so that it really doesn't matter too badly if there's a crash. For live situations the "big acts" are already using multi PC systems anyway, where it doesn't matter so much if one falls over. Also if, due to live usage, you're more likely to be in extreme conditions or bumping stuff around a lot, get rid of the moving parts in the system.. fanless, solid state storage etc.. much less likely to fail than moving parts in general. Which takes us back to the fact that the main issue for reliability is software. At OS level something like Windows tends to be very reliable. Vista has also added code in order to ensure audio is more glitch resistant. It can't guarantee it in the same way a realtime OS can, but it's pretty much good enough for even pro users.

It mainly comes down to the 3rd party developers.. those who write the drivers and the software, doing their jobs well. Once you introduce that, and things akin to VST/AU, there's not a whole lot of reason to believe that something based on a realtime OS would necessarily be any more reliable unless massively restricting freedom. I'd rather live in an open world than be choked by closed, expensive systems.. But I do still hope that something like audio on Linux becomes more viable, and sequencers really should offer better methods for pointing the finger at "bad" plugins, and recovering from errors they or other software make.

[mildbill]

'Non-Techy' question here:

would it be possible to have a pc with an OS that could be set up (simply) to perform specific tasks, and disregard extraneous conditions?

In other words - you boot up your pc and a menu appears that has options like 'DAW', 'Internet', 'Business Apps', etc.

If you click on DAW, the processor dedicates itself to only those tasks, and shuts out other types of in/output.



My first 'music computer' - the Yamaha CX5M, sort of worked that way, except when you stated it up, you inserted a cartridge that contained the area you wanted to concentrate on.

Seems like a do-able, simple solution to me, but what do I know?

[.Powell.]

the invention of dongles makes it worse

[P321]

would it be possible to have a pc with an OS that could be set up (simply) to perform specific tasks, and disregard extraneous conditions? In other words - you boot up your pc and a menu appears that has options like 'DAW', 'Internet', 'Business Apps', etc. If you click on DAW, the processor dedicates itself to only those tasks, and shuts out other types of in/output.

 

 

Yes it's possible. But lack of profit/demand means you're not going to see it from big companies like Apple/Microsoft, and they'd also need all the things in place which basically detect hardware, run the GUI, etc. Once you're at that stage you're at the "why bother" stage. You're looking at the main advantage being a smaller memory and disk footprint, rather than much CPU being saved. You can also do some of that yourself right now, by shutting down services you don't require. An operating system like XP will function normally for DAW duties with 10 (maybe less) services active, and they don't really steal a lot of cycles by themselves anyway, so cutting it down to 3 or 4 wouldn't result in that much gain, especially when considering we're about to enter a world of CPU's with 8 cores shortly.

 

A more viable way to do that is via Linux, where you can easily somewhat customize the kernel to your needs. removing and adding support for only the things you know you'll require. That's the advantage of open source, and it's what the Oasys basically uses. It's basically a software package which runs on top of that. However, things are never as simple as that. EG Many Linux games give lower framerates under Linux than Windows, even though their Linux distro may be more streamlined in theory.

 

There are a lot of noises being made about integrating specific features into the main processor in the future though.. in order to give further improvements to things like graphics speed (A large part of the reason why AMD bought ATi). That may well extend down to selling a specific CPU that's more specialized at delivering fast performance for audio plugins, or video encoding, or gaming etc.

[Mintbeetle]

The Fairlight and Synclavier were the last 2 specialized devices, for pro musicians. The OASYS as wonderful as it is, is not in the same league as the Synclavier, relatively speaking. People still use their Synclaviers, not out of nostalgia, but because today, even 20 years later, there are things the Synclavier can do that cannot be easily done otherwise.

Like what? I'm not challenging you, just interested in knowing what makes the Synclavier so unique.

[orangefunk]

The main thing about the Synclav was the 100MHz sample rate... I think thats still unique even now... but overall I can't see anything else special about it compared to modern day synths/soft synths...

My impression of the Synclav in 1980s was that it bought you a few years of exclusivity... I remember listening to Eddie Jobsons fantastic Theme of Secrets CD in 1985 marvelling at those gorgeous synth washes with "sparkley bits in them" (technical term I know) only to hear all that common place a few years later with the Korg M1/D50/O1W/Wavestation and so on..

there was also a resynthesis mode and some kind of FM. Though from all accounts the FM was pretty basic compared to Yamahas implementation... no idea about the resynthesis...

[Khazul]

Embedded programmers are the cream of the crop; it's not likely you'll get some hack doing that type of damage. This code must be reliable, and as bug free as possible, because once it is in the wild, it cannot reasonably be updated.

 

 

Actually - no they are not.

 

Embedded programmers (myself when I was younger and others I worked with) where technically brilliant. They would come up with amazing efficient algorythms to solve the some particularly nasty problems. And algorthymically their code was nothing short of blrilliant - however - robust? No - not at all.

 

In short they were excellent at finding a solution to a complex coding problems, but completely lacked the disclipline required for writing robust code and working on deep testing of that code.

 

When I was young - I was a guilty as any of them for that. It took alot of years and experience to realise the value of writing robust and safe, testable code over the value of making it work brilliantly under what we call 'happy path'.

 

Most leading edge code these days is written by similar young geniuses - and they all quite simply lack the experience to adequately design for the wider issues that go towards ensuring reliabilty. IU dont think universities are doing anything to change this - at least not from recent interviews with potential hires.

[Yoozer]

would it be possible to have a pc with an OS that could be set up (simply) to perform specific tasks, and disregard extraneous conditions?

 

 

Yes, but then you'd have to compile the source manually, leaving out everything you don't need. This means having access to the source, being allowed to modify it, and having no parts that depend on stuff you plan on leaving out.

 

You might've heard of the adage that 90% of the users use 10% of the options. The problem is that those 90% all use a slightly different 10%, so "lite" versions aren't that useful.

 

Truth to be told, I'd be happy already with a system for MIDI that'd work properly - like a Midex or AMT, only without the bull{censored} that it'll only work on a particular sequencer, and with truly parallel processing; I won't mind a fixed delay on sorting and processing.

 

Disregard even that, just replace MIDI with something better. I don't think we'll see that in the age of Blu-Ray vs. HD-DVD though.

[XorAxAx]

If you need the ability to " immediately respond, in a predetermined and predictable way, to external events", then you need an RTOS. Period, end of story.

 

This is the truth.

 

Most issues with softsynths would be solved by an RTOS specifically designed to turn a PC into a hardware synthesizer. Imagine a small, tight OS with all components specifically written to be the perfect environment to install and run commercial music software with minimal latency, maximum performance, and minimal bugs.

[wades_keys]

Actually - no they are not. Embedded programmers (myself when I was younger and others I worked with) where technically brilliant. They would come up with amazing efficient algorythms to solve the some particularly nasty problems. And algorthymically their code was nothing short of blrilliant - however - robust? No - not at all.

 

I concede your point - the robustness of the code is proportional to the quality of the testing that occurs.

 

Agreed 100% (my step-brother is a nationally recognized expert on software testing; I've got his book right here...)

 

Poor software testing is an epidemic in the industry...

[wades_keys]

This is the truth. Most issues with softsynths would be solved by an RTOS specifically designed to turn a PC into a hardware synthesizer. Imagine a small, tight OS with all components specifically written to be the perfect environment to install and run commercial music software with minimal latency, maximum performance, and minimal bugs.

 

 

This is 100% what I am ranting about; all these other solutions, are not solutions at all, but bull{censored} workarounds.

 

A dedicated RTOS that could host say VSTI's or whatever the flavor of the day is; would be a freaking GODSEND.

[XorAxAx]

 

I was an embedded software developer before I sold out and started developing boring business apps in .NET (for way more money).

 

 

Testing? Who needs it when the bugs are driven out by the pure force of the programmer's ego?

 

 

 

 

(Actually, the embedded work was way more fun, and I'd love to go back to it if the right job came along.)

[Khazul]

I was an embedded software developer before I sold out and started developing boring business apps in .NET (for way more money). Testing? Who needs it when the bugs are driven out by the pure force of the programmer's ego? (Actually, the embedded work was way more fun, and I'd love to go back to it if the right job came along.)

Heh - thats exactly what I ended up doing. {censored} - do I regret it - was lots of fun working with lots of talented people, now its just boring aspie no-life IT mongs...

Now looking to move back to real development again before I die of boredom.

[mildbill]

'embedded software' - OK, so what kind of stuff did you guys do that for?

Is that the term that you use for the software in keyboards? (and other kinds of non-computer hardware).

[Khazul]

'embedded software' - OK, so what kind of stuff did you guys do that for? Is that the term that you use for the software in keyboards? (and other kinds of non-computer hardware).

 

 

All sorts - from custom mechanical contol systems to mobile phones. Usually did alot of the core kernel work (threading and task management logic for hard real time OSs) and later networking, comms, GSM etc. I probably have a significant chunk of code in most phones sold today.

[XorAxAx]

'embedded software' - OK, so what kind of stuff did you guys do that for? Is that the term that you use for the software in keyboards? (and other kinds of non-computer hardware).

 

I did embedded work at a firm that produced industrial security systems. Most of my work there was in developing a new set of sensor and relay boards that would all share a very long serial communications bus. I was the software side; another guy was designing the boards.

 

The work included:

-- Banged out bug fixes for the previous guy's 8051-based board.

-- Wrote a driver for the serial port hardware on embedded DOS.

-- Developed the protocol for the new serial bus.

-- Created a custom build framework for the FreeBSD operating system. This allowed us to strip down the OS to almost nothing, inject our application code, and package it all in a flash ROM image.

-- Developed a software watchdog daemon for FreeBSD. If any of our app's processes locked up, it would reset the unit.

-- Wrote the firmware (on our FreeBSD platform) for the bus controller unit. It had to be capable of autonomously taking over core operations of its part of the security system in case the system's master computer went down.

 

I haven't done any of the really cool stuff, like NASA satellites or a synthesizer.

 

Yes, synth keyboards are embedded, but a special subtype of embedded. Things like set-top boxes and MP3 players are yet another subtype.

[mildbill]

Do either of you know what you'd have to learn to write something like an editing program for a synth?

How do you open a line of communication between a synth and a computer?