| Author |
Message |
Lazer
Guest
|
 Posted:
Tue Dec 07, 2004 1:50 am Post subject:
15TB SCSI storage solution. Any idea? |
|
|
Hi everyone!
Does anyone knows what do I have to use to store 15TB of data without using
a storage server (i.e. I need an off-the-shelf solution, ready to be
installed and configured under Linux whit the minimum effort).
I was thinking about some sort of disk array, also because i need to
configure it in RAID 1 mode, but I have no experience, and don't know what
to choose.
Thanks in advance!
Paolo.
_________________________________________
Usenet Zone Free Binaries Usenet Server
More than 120,000 groups
Unlimited download
http://www.usenetzone.com to open account |
|
| Back to top |
|
 |
Yura Pismerov
Guest
|
| Posted:
Tue Dec 07, 2004 10:09 pm Post subject:
Re: 15TB SCSI storage solution. Any idea? |
|
|
Ever considered multiple dual-channel SCSI cards and a bunch of intelligent (with onboard brain to maintain hardware level volumes)
disk arrays under LVM ?
I never tried this myself though, so it is rather a hypothetical setup that might work.
I imagine you would need really good server for that so your PCI bus would not be a bottleneck.
Lazer wrote:
| Quote: | Hi everyone!
Does anyone knows what do I have to use to store 15TB of data without using
a storage server (i.e. I need an off-the-shelf solution, ready to be
installed and configured under Linux whit the minimum effort).
I was thinking about some sort of disk array, also because i need to
configure it in RAID 1 mode, but I have no experience, and don't know what
to choose.
Thanks in advance!
Paolo.
_________________________________________
Usenet Zone Free Binaries Usenet Server
More than 120,000 groups
Unlimited download
http://www.usenetzone.com to open account
|
--
Yuri Pismerov, Sr. System Administrator,
TUCOWS.COM INC. (416) 535-0123 ext. 1352 |
|
| Back to top |
|
|
Morten Reistad
Guest
|
| Posted:
Fri Dec 10, 2004 5:50 am Post subject:
Ideas for multimedia low-cost solution sought |
|
|
A frient threw a problem at me that I want to present to a bigger audience.
He is in the audio/video business. They record lots of audio (and some video)
in digital formats, and use some very expen$ive gear to actually store the data.
He wonders if standard systems can be built from off-the-shelf components that
can equal the performance of these systems; so he can offload less critical work.
The worst case scenario implies streaming of around 18 MB per second for a few hours.
It should be free of SPOF's, but is not extremely critical [2]. If they are cheap enough
he can duplicate them for redundancy. He needs around 4T of actual data storage for this
to be useful. [1]
I went home and did some stress tests on a simple 4-way raid5 using stock 7200 rpm ATA 180G
disks, and found a sustainable long-term bandwidth (OpenBSD, Raidframe, 1Ghz old Via/C3)
of around 5 MB/sec on write, and around three times that on read. This was via NFS over
a gigabit ethernet.
Simple math tell me that this should be possible with a raid-over-raid. If we make a
software raid on top of hardware raids we may get to the bandwidth, but be stuck on the
raid5 write penalty. We could do stripe-on-raids or raids-on-stripes; but here I am a
little short on theory on what I would recommend. I also foresee interference problems
between stripe sizes if raids/mirrors are run on top of each other.
Also, this should ideally be accessible from Macs and Windows machines. Is such
perfomance (18 MB/SEC) over NFS or SMB even possible? ( I do 5(write) and 15(read) on pretty
low-end hardware without much problems).
Setting up disks to do a lower-scale test is doable.
[1] Actually, having one or two of these systems could alliviate a congested backup window
situation on the existing hardware.
[2] Not used for live performances. Think "Live TV". Could be used as backups for such performances
though.
-- mrr |
|
| Back to top |
|
|
Tom Petrocelli
Guest
|
| Posted:
Fri Dec 10, 2004 5:03 pm Post subject:
Re: Ideas for multimedia low-cost solution sought |
|
|
The demands of digital video and audio are such that the entire system
needs to be specifically tailored to it. Companies like SGI, Pinnacle,
and Avid actual use off the shelf components for their storage systems
but fine tune them to D A/V. In most cases, they add special software to
better manage the streaming data.
Your friend also has to consider the support of these systems. Plunking
in a homegrown solution means not getting support from the tools vendor.
There are a number of resellers that can put a system together as an
aftermarket addition to the major vendors and then support it. That
would seem a better choice.
Tom Petrocelli
"Never send a monster to the work of an evil scientist" - Evil Scientist
while chasing Bugs Bunny
On 12/9/2004 7:50 PM, Morten Reistad wrote:
| Quote: | A frient threw a problem at me that I want to present to a bigger audience.
He is in the audio/video business. They record lots of audio (and some video)
in digital formats, and use some very expen$ive gear to actually store the data.
He wonders if standard systems can be built from off-the-shelf components that
can equal the performance of these systems; so he can offload less critical work.
The worst case scenario implies streaming of around 18 MB per second for a few hours.
It should be free of SPOF's, but is not extremely critical [2]. If they are cheap enough
he can duplicate them for redundancy. He needs around 4T of actual data storage for this
to be useful. [1]
I went home and did some stress tests on a simple 4-way raid5 using stock 7200 rpm ATA 180G
disks, and found a sustainable long-term bandwidth (OpenBSD, Raidframe, 1Ghz old Via/C3)
of around 5 MB/sec on write, and around three times that on read. This was via NFS over
a gigabit ethernet.
Simple math tell me that this should be possible with a raid-over-raid. If we make a
software raid on top of hardware raids we may get to the bandwidth, but be stuck on the
raid5 write penalty. We could do stripe-on-raids or raids-on-stripes; but here I am a
little short on theory on what I would recommend. I also foresee interference problems
between stripe sizes if raids/mirrors are run on top of each other.
Also, this should ideally be accessible from Macs and Windows machines. Is such
perfomance (18 MB/SEC) over NFS or SMB even possible? ( I do 5(write) and 15(read) on pretty
low-end hardware without much problems).
Setting up disks to do a lower-scale test is doable.
[1] Actually, having one or two of these systems could alliviate a congested backup window
situation on the existing hardware.
[2] Not used for live performances. Think "Live TV". Could be used as backups for such performances
though.
-- mrr
|
|
|
| Back to top |
|
|
Maxim S. Shatskih
Guest
|
| Posted:
Sat Dec 11, 2004 1:09 am Post subject:
Re: Ideas for multimedia low-cost solution sought |
|
|
| Quote: | disks, and found a sustainable long-term bandwidth (OpenBSD, Raidframe, 1Ghz
old Via/C3)
of around 5 MB/sec on write, and around three times that on read. This was
via NFS over
a gigabit ethernet.
|
Looks like this is very slow. I would compare several OSes, filesystems and
network protocols. NFS or SMB, FreeBSD/Linux/Windows, FFS/ext2/ext3/FAT/NTFS
and so on.
--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
maxim@storagecraft.com
http://www.storagecraft.com |
|
| Back to top |
|
|
Morten Reistad
Guest
|
| Posted:
Sat Dec 11, 2004 5:00 am Post subject:
Re: Ideas for multimedia low-cost solution sought |
|
|
Maxim S. Shatskih wrote:
| Quote: | disks, and found a sustainable long-term bandwidth (OpenBSD, Raidframe, 1Ghz
old Via/C3)
of around 5 MB/sec on write, and around three times that on read. This was
via NFS over
a gigabit ethernet.
Looks like this is very slow. I would compare several OSes, filesystems and
network protocols. NFS or SMB, FreeBSD/Linux/Windows, FFS/ext2/ext3/FAT/NTFS
and so on.
That is just my point. A thrown-together pretty lame software-only |
solution manages around a third to a quarter of the actual requirement.
A decently designed system should be able to cope pretty nicely.
Now, my question is about getting some pointers BEFORE we embark
on a goose-chase. For something like this I would expect to have to use
a journalling file system like XFS or JFS (possibly ext3, but I don't
know how far that scales well). I would expect to give a lot of tuning
effort to NFS; and I am questioning whether this is at all possible in
SMB. These two are showstoppers, and I would appreciate feedback on the
performance of these. These file system issues I know well from other
projects.
I am also sending a query about guidelines about the effects when
stacking raids.
Have googled, but haven't found anything useful, so I appreciate
insights from this group.
Tom Petrocelli wrote :
| Quote: | Your friend also has to consider the support of these systems.
Plunking >in a homegrown solution means not getting support from the |
tools >vendor. There are a number of resellers that can put a system
together >as an aftermarket addition to the major vendors and then
support it. >That would seem a better choice.
You seem to have a lot more faith in tools vendors than I do. I am here
to get some feedback on the feasibility of the solution before they use
any work on looking for a low(er) cost solution.
A fully digitized record label (which this is) does an astounding amount
of data shuffling; and pay through the nose for systems to do this. In
lots of places this isn't really warranted.
Even a pretty homegrown version can be quite valuable to them, as it can
be used for taking nightly backups; and the cheap system can then
generate tape media in business hours. It can also be used for doing
ad-hoc masters, simple off-line edits, proofs of concept etc without
having to contend for studio time.
(Studio time is around $400/hr). |
|
| Back to top |
|
|
|
|
|
|
FAQ
Memberlist
Register
Profile
Log in to check your private messages
Log in