| Author |
Message |
Xecaquan
Guest
|
 Posted:
Thu Dec 08, 2005 5:15 pm Post subject:
Indefinite Storage |
|
|
I had a question, I am now required to store my email backups
indefinitely and looking for a solution. I am looking for a product
that allows for permanent storage of about 300-500GB, without taking a
ton of media and time. Scalability to at least 1 Terra Bytes, I don't
know of any optical storage that has that kind of capacity. Currently
I am using SDLT600 to store office site, and regardless of marketing
hype these magnetic tapes are not stable out to their stated life span
of 20-30 years. Ideally any solution should be compatible with
Netbackup as well, for strictly a manageability ease. Thanks for your
time. |
|
| Back to top |
|
 |
Faeandar
Guest
|
| Posted:
Fri Dec 09, 2005 1:16 am Post subject:
Re: Indefinite Storage |
|
|
Well, first off I seriously doubt NetBackup will be around in 20-30
years, let alone indefinitely.
Second, for the amount of data you're talking about I would personally
go with just disk. Not even raid is necessary.
Backup/copy the data you need to keep indefinitely to 1 or 2 SATA
drives of 500GB. Back them up and send those tapes offsite as part of
your normal backup routine; these tapes can easily be recycled once
another set or two are offsite.
As the next gen of drives becomes available migrate the data to those
and recycle the old drives.
Simple, safe, and extremely flexible.
~F |
|
| Back to top |
|
|
Douglas Siebert
Guest
|
| Posted:
Fri Dec 09, 2005 2:55 am Post subject:
Re: Indefinite Storage |
|
|
For such a small amount of storage, what's wrong with using disk? You can
buy individual disks up to 500GB today, and scalability to "at least 1TB"
would be easy since I'm sure they'll sell disks that big in the not too
far future, or you split it up across multiple disks. Obviously you would
make at least two copies, stored in separate locations. If you are paranoid
make three or more copies, using different brands of disks (in case some
problem affects one vendor) and make sure the checksum matches between at
least two of them to verify your data.
Of course disks can't be counted on to survive 20-30 years and finding a
compatible interface in 2035 might take a bit of work, so you'd want to
copy the disks to something newer every few years. If you want to use
Netbackup that's fine, but have it write in some widely available open
source standard such as TAR or ZIP so that being able to read it in 2035
isn't a problem. Don't even think of using commercial software such as
Netbackup, Ghost, Microsoft's tools, etc. because once that vendor
obsoletes and replaces that product, or worse goes out of business the
exact data format and code to read it may no longer be possible to obtain.
--
Douglas Siebert dsiebert@excisethis.khamsin.net
Give a man a match, and he'll be warm for a minute. Set him on fire, and
he'll be warm for the rest of his life. |
|
| Back to top |
|
|
Chevalier des bois
Guest
|
| Posted:
Fri Dec 09, 2005 5:16 pm Post subject:
Re: Indefinite Storage |
|
|
Disk is not the subject here, no serious magnetic retention times are
specified by vendors so you will be obliged to rewrite the date from
time to time.
What is the specified magnetic retention time for tape ?
Chances are that in 5 years the data on your precious tapes won't be
there anymore, so 20-30 years ...
For media that are specified for such time frame (To 40 years max), look
at dvdroms that are specified for such time frame and a dvd library.
As far as I know it is the only way.
Archiving the os and the backup software is not a bad idea though. |
|
| Back to top |
|
|
Xecaquan
Guest
|
| Posted:
Fri Dec 09, 2005 5:16 pm Post subject:
Re: Indefinite Storage |
|
|
Yes, I understand Netbackup will not be around in 20-30 years.
However, using hard drives would require me to have at least 2 500gig
drives per week. Then replace the, at minimal, 312+ drives every 3ish
years as technology changes. Currently the cost for 500gb drives are
around $320 each, the cost for SDLT600 tapes is $85, so no benefit in
cost reduction. However, keeping a server with the hard drive
interface would be easier than ensuring the compatible robotic library
is kept alive.
As I said this is a new directive to storage all email everyday for an
undefined time frame, and so I am just looking for any solutions
possible. |
|
| Back to top |
|
|
Rob Turk
Guest
|
| Posted:
Sat Dec 10, 2005 1:16 am Post subject:
Re: Indefinite Storage |
|
|
| Quote: | The problem does not deal with backup software but with tape.
Disk is not the subject here, no serious magnetic retention times are
specified by vendors so you will be obliged to rewrite the date from time
to time.
What is the specified magnetic retention time for tape ?
Chances are that in 5 years the data on your precious tapes won't be there
anymore, so 20-30 years ...
For media that are specified for such time frame (To 40 years max), look
at dvdroms that are specified for such time frame and a dvd library.
As far as I know it is the only way.
Archiving the os and the backup software is not a bad idea though.
|
Do you have any data to substantiate that tapes will not retain their data?
I have plenty of tapes that are perfectly readable after 15 years. Not sure
what to do with my data from back then on my old EXB-8200 2GB tapes, but
they can be read just fine. Proper care and a decent tape technology will
work quite well. (that does not include the dinky
floppy-tape/Travan/Onstream kind of drives).
My experience with CD/DVD is to the contrary. Good mass-produced audio CD's
last a long time, but the writable variant fails without exception after 2-5
years. They all develop bad area's. This is due to the substrate that is
being used for writable CD/DVD media. It's not stable over time. The bad
thing is that newer CD's appear to break down faster than the
first-generation 2x and 4x writeable CD's. I guess that's because it must be
cheap to produce...
Rob |
|
| Back to top |
|
|
Faeandar
Guest
|
| Posted:
Sat Dec 10, 2005 1:16 am Post subject:
Re: Indefinite Storage |
|
|
| Quote: | Yes, I understand Netbackup will not be around in 20-30 years.
However, using hard drives would require me to have at least 2 500gig
drives per week. Then replace the, at minimal, 312+ drives every 3ish
years as technology changes. Currently the cost for 500gb drives are
around $320 each, the cost for SDLT600 tapes is $85, so no benefit in
cost reduction. However, keeping a server with the hard drive
interface would be easier than ensuring the compatible robotic library
is kept alive.
As I said this is a new directive to storage all email everyday for an
undefined time frame, and so I am just looking for any solutions
possible.
|
You misunderstood what I was saying.
"Backup/copy the data you need to keep indefinitely to 1 or 2 SATA
drives of 500GB. Back them up and send those tapes offsite as part of
your normal backup routine;"
Now unless you're saying you need to do that amount PER WEEK the above
would be, imo, the simplest and safest way to keep that data around.
~F |
|
| Back to top |
|
|
Chevalier des bois
Guest
|
| Posted:
Sat Dec 10, 2005 9:15 am Post subject:
Re: Indefinite Storage |
|
|
No regular CD/DVD but DVDROM.
This is the only choice for these who have legal retentention of more
than ten years.
All other solutions are unsure and of course no professionnal. |
|
| Back to top |
|
|
Faeandar
Guest
|
| Posted:
Sat Dec 10, 2005 9:15 am Post subject:
Re: Indefinite Storage |
|
|
Not exactly true. DVD(ROM) constitutes non-modifiable media (worm
class) but is not certified for permanent or "long term" storage.
Long term being 7+ years.
Certain types of tape, on the other hand, are considered viable for up
to 15 years. I'm not aware of any specific certification for tape
either.
Optical storage is currently considered the most viable long term
storage available today.
All of these are based on 'offline' storage of course.
~F |
|
| Back to top |
|
|
jay maechtlen
Guest
|
| Posted:
Mon Dec 12, 2005 1:15 am Post subject:
Re: Indefinite Storage |
|
|
Xecaquan wrote:
| Quote: | Yes, I understand Netbackup will not be around in 20-30 years.
However, using hard drives would require me to have at least 2 500gig
drives per week. Then replace the, at minimal, 312+ drives every 3ish
years as technology changes. Currently the cost for 500gb drives are
around $320 each, the cost for SDLT600 tapes is $85, so no benefit in
cost reduction. However, keeping a server with the hard drive
interface would be easier than ensuring the compatible robotic library
is kept alive.
As I said this is a new directive to storage all email everyday for an
undefined time frame, and so I am just looking for any solutions
possible.
Your company/organization really needs to look at what's in the emails. |
To paraphrase from a records management list:
"email is not a record type- it is a medium by which a record may be
carried or transmitted."
Go check out recmgmt-l or equivalent.
In the meantime, if the boss says 'archive', we say 'ok, how long?'
cheers
Jay |
|
| Back to top |
|
|
jlsue
Guest
|
| Posted:
Tue Dec 13, 2005 1:16 am Post subject:
Re: Indefinite Storage |
|
|
On 8 Dec 2005 09:11:54 -0800, "Xecaquan" <Junk@lucidearth.com> wrote:
| Quote: | I had a question, I am now required to store my email backups
indefinitely and looking for a solution. I am looking for a product
that allows for permanent storage of about 300-500GB, without taking a
ton of media and time. Scalability to at least 1 Terra Bytes, I don't
know of any optical storage that has that kind of capacity. Currently
I am using SDLT600 to store office site, and regardless of marketing
hype these magnetic tapes are not stable out to their stated life span
of 20-30 years. Ideally any solution should be compatible with
Netbackup as well, for strictly a manageability ease. Thanks for your
time.
|
Let me just say that this is a very, very complicated request.
I've read through many of the replies here and there is quite a lot of
misinformation out there.
The first key thing to remember is that record retention is NOT a
technical (i.e., IT) problem. It is a business and legal problem. A
full study of the risks and benefits of the different options must be
provided to upper management so that they can understand what kind of
investment they really need to consider. The company's legal council
must develop policies that not only are simple enough to follow, but
are actually implementable in technology (lest your company end in
court and having to admit that they have policies that they know they
aren't following).
The second key thing to remember is that B&R is Backup and Recovery.
Note that it is *not* Backup and Reference. In other words, backups
are most certainly not the correct soluton to your records retention
needs. Let me say that again: Do not rely on backups for records
retention. What may seem to be a fairly inexpensive investment today
will come back to bite you horribly in the future when you have to
restore 9,000 tapes, most of which are on media which none of your
current drives can read. I have worked with companies who needed to
do this. $3.5million (USD) is not an unheard-of price.
And, lest you think that it's cheaper to just pay the fines from the
courts rather than pay the $3.5mil, please note that the fines are
only part of the problem. Courts are increasingly forcing record
holders to obtain the information anyway, and to bear the full cost -
in addition to levying heavy fines on the holders - sometimes even
paying computer forensic companies to restore deleted files. And then
there's the adverse jury instructions: courts are telling juries that
they are safe to assume that the documents would harm the record
holders, which ends up being far more expensive in the verdict phase.
And there are many, many new government regulations that cover nearly
all businesses today. Fall afoul of some of those, and your executive
management may be spending time in the state-run hotel industry (i.e.,
prisons).
Third key thing to rememger: Email most definitely IS considered a
coporate record. You really need to have all your ducks in a row,
processes in place, etc., to try to demonstrate otherwise. But recent
court action has shown that email, in it's raw electronic form, is
considered a business record. If you can't produce it - and produce
it quickly too, I might add - you are tempting some very severe and
harsh retribution from the courts.
There are archiving solutions which utilize less-expensive spinning
drive solutions, fully index the words in emails and attachments for
fast retrieval, and provide much simpler migration strategy as storage
technology changes.
NOTE: In the interests of full disclosure, I work for HP Consulting
and Integration in the Storage Practice. However, in my consulting
capacity I mostly spend time researching the legal landscape and doing
presentions and discussions with executives in various different
industries.
--- jls
The preceding message was personal opinion only.
I do not speak in any authorized capacity for anyone,
and certainly not my employer. |
|
| Back to top |
|
|
Curious George
Guest
|
| Posted:
Tue Dec 13, 2005 5:16 pm Post subject:
Re: Indefinite Storage |
|
|
On Mon, 12 Dec 2005 20:03:23 GMT, jlsue <jeffls-delete@sbcglobal.net>
wrote:
| Quote: | On 8 Dec 2005 09:11:54 -0800, "Xecaquan" <Junk@lucidearth.com> wrote:
I had a question, I am now required to store my email backups
indefinitely and looking for a solution. I am looking for a product
that allows for permanent storage of about 300-500GB, without taking a
ton of media and time. Scalability to at least 1 Terra Bytes, I don't
know of any optical storage that has that kind of capacity. Currently
I am using SDLT600 to store office site, and regardless of marketing
hype these magnetic tapes are not stable out to their stated life span
of 20-30 years. Ideally any solution should be compatible with
Netbackup as well, for strictly a manageability ease. Thanks for your
time.
Let me just say that this is a very, very complicated request.
I've read through many of the replies here and there is quite a lot of
misinformation out there.
The first key thing to remember is that record retention is NOT a
technical (i.e., IT) problem. It is a business and legal problem. A
full study of the risks and benefits of the different options must be
provided to upper management so that they can understand what kind of
investment they really need to consider. The company's legal council
must develop policies that not only are simple enough to follow, but
are actually implementable in technology (lest your company end in
court and having to admit that they have policies that they know they
aren't following).
|
<snip>
while an interesting overview of the legal landscape, what would you
suggest the OP present as possible solutions to upper management? The
OP, after all asked:
"I am looking for a product that allows for permanent storage of about
300-500GB, without taking a ton of media and time. Scalability to at
least 1 Terra Bytes..."
Surely you're not proposing a response to the OP as "dump all the
possibilities on management's lap and walk away believing no one will
ask or expect anything further of you. After all, since there are
legal & business ramifications to the technical (IT) decision it's a
legal & business problem - not a technical (IT) one" |
|
| Back to top |
|
|
Anton Rang
Guest
|
| Posted:
Wed Dec 14, 2005 5:15 pm Post subject:
Re: Indefinite Storage |
|
|
Chevalier des bois <fla.fla@wanadoo.fr> writes:
| Quote: | Five years max specified from tape vendors, after, it's luck.
|
http://www.sony.net/Products/Media/DataMedia/products/SuperDLT/
"Archival Life: 30 years"
http://www.storagetek.com/products/product_page47.html
"Archival Life: 15-30 years"
http://www.thic.org/pdf/Apr98/stk.mleonhardt.pdf
"With special storage and handling ... magnetic tape ...
can reliably store information for 30 years or more."
There are certainly plenty of low-cost tape formats which aren't so
stable, but tape vendors know what they're doing...and more to the
point, there are a lot of sites with experience in maintaining tape
archives for many years.
If you do need to maintain the data for more than 20-30 years, you'd
better plan to migrate it to new technologies anyway. I suspect that
MO disks written today may be readable in 100 years ... if you could
find either a drive, or an appropriate microscope. ;-) Remember the
phase-change disks in the mid-1980s? Seen a phase-change drive lately?
-- Anton
(Disclaimer: I work for Sun, now a tape vendor. I don't speak for them.) |
|
| Back to top |
|
|
Anton Rang
Guest
|
| Posted:
Wed Dec 14, 2005 5:15 pm Post subject:
Re: Indefinite Storage |
|
|
Curious George <cg@email.net> writes:
| Quote: | while an interesting overview of the legal landscape, what would you
suggest the OP present as possible solutions to upper management? The
OP, after all asked:
"I am looking for a product that allows for permanent storage of about
300-500GB, without taking a ton of media and time. Scalability to at
least 1 Terra Bytes..."
|
Well, I'm not in the storage practice though I am in the industry.
But I'd certainly suggest that a company generating 500+ GB of *new
and unique* email per week is exceptional! This sounds like the
suggestion is to basically copy each user's mail file every week.
That's a very inefficient way to store the data.
I (not speaking for my employer, of course) would suggest
investigating email management software ala Lotus Domino, or a mail
server which can otherwise detect and manage duplicates ... and has
the ability to spool out only new data for archiving.
I suspect all of the major IT consulting firms have dealt with this
problem before. Sun, for instance, has implemented their "Infinite
Mailbox" solution for several clients. This integrates Lotus Domino
with SAM-FS to archive email to tape.
| Quote: | Surely you're not proposing a response to the OP as "dump all the
possibilities on management's lap and walk away believing no one will
ask or expect anything further of you."
|
Not to put words into his mouth, but I'd suggest a response more along
the lines of "suggest to management that we work with a consulting
firm with experience in both the legal and technical aspects of
records management." It's certainly possible to implement all of this
yourself, but is it a wise investment of time and resources?
Anton |
|
| Back to top |
|
|
carmelomcc
Guest
|
| Posted:
Wed Dec 14, 2005 11:43 pm Post subject:
Re: Indefinite Storage |
|
|
Just outsource your email to a vender and make it a requirement. Do
not worry about it your self. If you have a requirement to keep the
data that long it will out last you. It is better to rely on a vender
this way it will scale. Companies like Sungard do this all the time. |
|
| Back to top |
|
|
|
|
|
|
FAQ
Memberlist
Register
Profile
Log in to check your private messages
Log in