RAID for Backup

[ameerat42]

Well, I've been doing some reading about RAID* disk drive arrays, and have concluded that for all my
various data, it might be better than my present manually backing up to 2 or 3 external drives

It includes mainly backing up of image/video files and various document files, both of which I accumulate
about 100~500 MB a week (or so).

Anyway, from reading about it, I reckon that 2 drives in a RAID 1 arrangement would do the trick

What I would like to hear about is any arrangements that other members use here (Thanks!) and
some of your reasons.

BTW, I don't know when this will start. I've got the task of sorting our my present "RAID" method...
Random Accumulation of Irregular Data.

* RAID is usually supposed to denote either one of "Random Array of Independent/Inexpensive Disks".
But, all RAID methods seem to have some interdependence of disk drives , and of what use is a
description of "Inexpensive" in this sense? So, it seems to be more of a catchphrase than
anything else.

[EMGonzo]

I used to use a 4 bay NAS using RAID 5, of course one of the drives died, then when I replaced it and was rebuilding the data another drive failed. Yup, lost everything.

I now use a Synology DS1813+ 8 bay NAS. It's set up using RAID 6. It can have 2 drives die at the same time and I won't lose any data. I thought this was overkill but after my previous experience it's not.

I also back up my photo's to an external drive as well as online storage.

I've had too many experiences with dying hard drives and equipment to just rely on one backup solution.

[ameerat42]

Ta, eMGonzo. How insightful and... wrrying that it still happens at that level.

I'll check up on what Raid 6 means

[arthurking83]

I won't do RAID again. I understand the ease of it .. sounds convenient, etc, etc.
But sometimes the idealistic expectations of marketing don't translate perfectly into a reality.

The main issue I have with RAID is that the data is automatically backed up and you have pretty much zero control of that backed up data.
99.9% of the time it works and is seamless, and no worries.
But if you ever find yourself in the situation that you have been backing up corrupted data .. having multiple copies of that corrupted data across multiple sites .. doesn't bring back the older and non corrupted data you wish you had instead.

Of course you can have a versioning setup too .. but then where does it stop? Do you really want an 8 bay array with versions of backups going back for eons too?

More simple:
keep one back up on one(or more) drives .. keep older backups on one or more drives. every so often check the integrity of the data you want to backup, and it's a simple process of backing up every so often.

This is what I've done now for a good 6 or 7 years .. no raids, just 4 drives in a NAS, one ext drive on my PC, mainly for speed, but also as a bit of a filter.

Using a good quality sync program that allows flexibility of use helps too(for that look, into FreefileSync )

So in a similar situation as you want to do:
I backup my photos to a PC connected drive: if I feel this one needs backing up(ie all data is to be backed up) .. it then gets backed up to one drive(lets call it Y drive) on the NAS.
Then every so often(not too regularly) I then backup this Y drive on the NAS to another similar drive on the NAS, we'll call it Z drive, and only after a confirmation that all files on the Y drive are good.
ie. not corrupted in any way or maybe an older version of a file(eg. image file) of one elsewhere.

For image file integrity checking(mainly for raw files), I prefer to use XnViewMP for it's speedy process.

If you don't mind spending a few $s and want a more modern useful device .. I can't recommend strongly enough to get yourself a NAS box, 2 or 4 up to you preference and budget.

One of the biggest issues with RAID setups is that it's all great when it works well, and when you do have an issue(unlikely but possible) .. the rebuild time makes it vulnerable to problems. Some can take a couple of days to rebuild themselves.
The issue of possible corruption, or drive failure may be low, but as EMGonzo seems to have discovered .. it's not impossible to have happen.

Feel free to PM me about a possible answer to your question if you like.

[ameerat42]

Ta, AK. I only just started to read up on NAS. Now I will continue.
It sounds like my present system is not so bad after all...

[arthurking83]

PM sent.

Online storage is another option as gonzo mentioned, but 2Tb of data is a lot to upload to a NAS .. let alone a cloud! ... in an Aussie NBN environment.
If we had gigabit WAN options to play with .. yeah, it'd take a day or so to download back to the local environment ... uploading modern 200Mb raw files tho ...

offline/cloud type backups only a reality for small jpg file types really.

Some NAS solutions have personal cloud systems too. I don't use this feature, not really the security risk(I reckon fairly low anyhow) .. more the speed of such transfers.

say, if you were on a trip and wanted to back up data on a daily/weekly basis, instead of transferring to the cloud services readily available, these cloud capable NAS boxes, always connected to the web .. you backup your files whilst on holiday directly to your local NAS(ie. at home).
Data speed is, again, the main issue there.

[ameerat42]

Ta, AK.

I am not (yet) much of a cloud fan. - Just as necessary, and very little too.

Still researching. It looks like a NAS is the general path.

[markdphotography]

I agree with all that has been said about RAID. Had 2 bad experiences both similar to Arthur and EMGonzo.

Personally I have not gone back to it.

Myoption would be to stick to your manual system and sync monthly. I use Goodsync and have 4 x 8TB drive 1 x for recent images - 1 for Archived images and 2 for copies of those 2 drives.

All my work real estate images are sored on the cloud server but I also keep a copy for local access using the above method.

The other issue to rember is to replace drives regular, do not use SSD drives and be prepared to loose some images.

None of this tkakes into account a catastophic occurance like a fire or storm/flood and this is whe cloud and offsite backup come into their own.

Always remeber nothing is fool proof, technology is wonderful when it is working but a PIA when it is not, rember the good times because the bad times do linger for a while

[ameerat42]

Ta, Mark.

Would you please expand on the point "...do not use SSD drives..."

[markdphotography]

Ta, Mark.

Would you please expand on the point "...do not use SSD drives..."

I just updated my PC use an SSD drive for my OS but back this up to a spindle /spinning drive once all the applications are loaded.

Over the years since being introduced, I have had a few SSD drives fail and find that they are more expensive and prone to fail if used for storage or backup.

[ameerat42]

Ta for that. I will do a bit of research on the matter.

For my own experience, I've had a 2-SSD lappy since 2014. It's been going well... so far.

[markdphotography]

Ta for that. I will do a bit of research on the matter.

For my own experience, I've had a 2-SSD lappy since 2014. It's been going well... so far.

Just had a 1TB go feral "RAW" on a laptop purchased in December 2018 that does not get much use.

RAW means the hard drive partition is not formatted with file systems including FAT12, FAT16, FAT32 and NTFS/NTFS5. When any of your drive partitions becomes RAW, you will not be able to access the data in it. There are many symptoms when your hard drive partition became RAW.

Cheers

Mark

[ameerat42]

Can such a drive be re-formatted?

- - - Updated - - -

I've found this: https:/www.crucial.com/support/articles-faq-ssd/ssd-shows-raw-partition,
which is "crucially" useful as I have a couple of Crucial SSDs

[markdphotography]

Can such a drive be re-formatted?

- - - Updated - - -

I've found this: https:/www.crucial.com/support/articles-faq-ssd/ssd-shows-raw-partition,
which is "crucially" useful as I have a couple of Crucial SSDs

I spent a few hours the other night looking at options on the web (and there are many). All involved a free download and pay for recovery. As I used apps like this in the past without success and may never need it again during the subscription period, I gave it to my friendly PC repair shop.

They have just messaged me to call in so will know more later today.

Cheers

Mark

[torro]

I’ve done pretty much the same with photo backups on three different external hard drives, though it’s fiddly I feel assured about images being safe.

[markdphotography]

I spent a few hours the other night looking at options on the web (and there are many). All involved a free download and pay for recovery. As I used apps like this in the past without success and may never need it again during the subscription period, I gave it to my friendly PC repair shop.

They have just messaged me to call in so will know more later today.

Cheers

Mark

Drive is DEAD - data recovered but not required.

It was a spindle and not an SSD - just over 2 years old, had little use and was not covered by HDD warranty as it was in Dell laptop and only covered by Dell warranty not 3 year HDD warranty.

Lucky I have a backup drive to replace it.

Mark

[ameerat42]

Wht can one sy?

- - - Updated - - -

Just for interest, how much did they charge - assuming it was for one drive only?
I have an external HDD that I've been thinking of taking for recovery
There's a wide range of charges (from a casual search).

[markdphotography]

Just noticed I had not replied to this.

The data was not important to me as it is only a backup device.

He did manage to recover some data which I reviewed and did not want.

The drive was dead and had a lot of dead sectors so was not worth reusing.

The cost was $25 - lunch money for a coule of days for the techie.

Cheers

Mark

[ameerat42]

Ta, Mark. That is certainly cheaper than I've been quoted.

I might yet get this drive recovered, though.

[landyvlad]

A lot of those (smaller) places quote according to how desperate they think you are for the data.
The larger places are always expensive - though they often also have better labs, equipment and processes and success rate is much more likely to be higher.

Succesful recovery is the exception rather than the rule, generally speaking.

Far better to never have to worry and to have a good backup or two elsewhere.

Which reminds me I must check my old CD's and see if those files are still any good



In regard to a few other things mentioned above.

- SSDs are perfect for drives where you are running programs such as operating systems, editing, etc anything that needs a fat read/write speed. They are WAY faster than traditional drives - and of course silent.
- SSDs are NOT a good choice fore data backup. They are, after all, a type of flash memory - would you rely on backing up everything to a USB stick? Of course not.

RAID - never think of the I as inexpensive disks. Cheaper = likely higher failure rate. (Not, of course, that any brands are immune, it can happen at any time.)
For my NAS I use WD Red drives. These are optimised for backup - prioritising stability and accuracy over read/write times.

Most off-the-shelf NAS systems include RAID of some sort.

A failed synology drive is much more difficult to retrieve data from as they use a proprietary format.

I had a synology once - it's certainly by far the easiest system to use.

(I have built a specific NAS box, using TrueNas Core (previously known as FreeNAS) with 6 x 3TB Wd red drives in a raid 6, giving me a maximum of 12Tb of storage, with 2 drive redundancy.

Here's a RAID calculator is you are interested: Free RAID Calculator - Caclulate RAID Array Capacity and Fault Tolerance. (raid-calculator.com)