A Question of Data

[Dogoth]

This probably belongs in the recording forum but I know a couple of you guys are more computer savvy than I am so here is my question anyway.

 

I'm building a new studio DAW (Steinberg Nuendo is the program I know best). My last machine had a 4 drive stripped raid for quick access to multiple tracks, bits, bytes and loops. The new machine will have an SSD instead. What I envision is taking a couple of the older drives and making a 2 drive stripped raid and then taking that volume and making it a mirror of that SSD data drive (sort of a raid within a raid). QUESTION: Will this slow down the SSD while the physical disk drives write as well OR will the slower volume "sync" at a slower rate in the background. FWIW this isn't really necessary as I backup regularly to a server where I keep all of my........well.......everything .

 

It'd be nice to have a real time backup of a session at any time though (Not sure the SSD will like all of the many read/writes needed for sliced and diced audio and may live a short life - we'll see).

 

If you know anything about this, let me know if it's worth it or would be a PITA performance hit.

 

Thanks in advance for your replies

[Luckenbacher]

Personally, I don't know if I would bother trying to mirror the SSD.

 

I'd probably just use that as my disk for the OS and sample libraries, and then record to the older disks, either raided or not. The throughput on most of this stuff is so large compared to audio, that IME it's not worth doing much that is very fancy for an audio only machine. IMO, the various raid setups I have seen are iffy about being able to actually recover stuff, though it was helpful to have the faster disks when I was doing more video editing.

 

I can't give you any good data about your data, but that's just what I'd do. I don't think that there would be any upside to it.

 

Also, I don't think that I'd worry about the life on the SSD... I haven't been seeing many problems, and I don't know of folks who are, at least in my very limited experiences. But they are very fast, which is great for running programs off them.

 

The backups (and making them happen automatically somehow) is what I spend my efforts on... I have a NAS that I rsync my development machine to, but I'd like to figure out how to get it to automatically upload to aws glacier.

[Dogoth]

Thanks for the reply. I sort of figured as much. I'd love to setup a periodic backup (instead of a mirror) but that machine isn't on often (only if I'm doing a session or other editing work). Possibly I can setup a timer to turn it on at some odd hour of the morning and backup the important files (if any) to another network drive that IS always on. Probably a better solution anyway.

[Pro Sound Guy]

Yes OS on SSD and why not more SSD's?

Take the plunge.

[RoadRanger]

If I was doing anything with important irreplaceable data I'd be using an offsite backup drive (or three) and/or cloud services. In fact, what coding I still do is hooked into Google Drive. But audio requires MUCH more space than the free cloud services provide.

 

I also have a hosting account with Godaddy that provides me with 150GB - I've used that to backup video in the past although they do state that the space is for online content only.

[wesg]

This depends a great deal on the volume manager, available free RAM, and frequency of writes, but ....... in general, adding significantly slower disks to a mirror will slow things down.

 

SSDs are so cheap now, just mirror them for that super-cool automatic backup. Unless you're on a mac, in which case, Time Machine, yum yum.

 

Luckenbacker: tunnel rsync over ssh to your AWS instance.

 

Wes

[Luckenbacher]

Luckenbacker: tunnel rsync over ssh to your AWS instance.

 

I do exactly that between my dev machine and my local nas.

 

As I understand it (feel free to correct me) glacier is a pain in the ass cause it isn't a machine but a service, so there is no instance to tunnel into per se, an it's not a file store that you can rsync.

 

I think that you are "supposed" to create a job with a cli tool, then upload parts to that job, then close the job with a checksum. Or at least, thats what the couple of sh scripts I have messed with try and do... I've had no luck with that.

 

I have an application on my synology nas, and it works okay as long as I am not trying to do more than a couple hundred MB in a go, otherwise it times out and dies.

[wesg]

I just spent 10 seconds reading about Glacier. Ugh. I just thought it was another VM offering. How much control/flexibility do you have? If you can run arbitrary shell scripts, you could (in theory ) kick off an ssh job which logs into your machine and kicks off an rsync job.

 

If you can't do that, I wonder if you can contort their REST API into doing zsync (rsync over HTTP)

 

I'm a master at bending UNIX tools into doing things they were never meant to do. It was necessary in the early 90s, now it's mostly just fun.

 

*googling*

 

http://blog.tkassembled.com/326/creating-long-term-backups-with-amazon-glacier-on-linux/ looks interesting. You could always ditch the idea of using rsync and straight up use tar. Just like in the bad old days, when you requested a tape archive and a little old lady had to fetch the tape from the library shelf and load it before you could untar it. The first day I got to use a tape robot was pretty exciting. The big StorageTek silos back in the day were a hoot. We had one that could literally decapitate an unwary operator.

 

Anyhow - remember that tar and ufsdump/ufsrestore both support differential archiving. It's not as IO-compact as rsync, but it's better than total snapshots. Easiest way with GNU tar is probably --newer (keep a timestamp file around). ufsdump/ufsrestore support the concept of dump levels

 

[Luckenbacher]

Well, I like the cli (and some day I may even get to where I can awk without looking stuff up). I can really do whatever... my NAS runs some flavor of nix and will let me run sh scripts, and my dev machine is os x.

 

But I ended up using my Synology's Glacier tool via its web gui. I checked my AWS console, and it looks like sometime in the middle of last month it finally uploaded all 2.7GB of my test archive, so I guess I'll just setup that as a job for the rest.

 

I can usually do okay if I am working with whatever... I know PHP and CURL well enough that I can do a lot of things even if my sh skills are rudamentary. But this is just too much for me to really worry about... it's a complex, multi-step api that frequently times out, and I ain't got time to mess with it.

 

My hope is that I will never want to pull any of this off glacier.... it is only there to backup some physical discs that I don't ever even look through often, and I'll only need it if my local backups and my offsite backups simultaneously die.

[RoadRanger]

Does anyone know what Glacier uses for offline storage? I suppose it's some type of super-density tape but I haven't kept up on the state-of-the-art there...

[Luckenbacher]

Does anyone know what Glacier uses for offline storage? I suppose it's some type of super-density tape but I haven't kept up on the state-of-the-art there...

 

Not really. Most folks assume that it is high density tape storage, but I had read something about it possibly bing optical... but no one knows as far as I know.