| Author |
Message |
daben
Guest
|
 Posted:
Mon Oct 10, 2005 12:15 am Post subject:
Looking to build storage of about 3TB -- recommendations? |
|
|
Hi All
For my research I need large storage for satellite data files. I am
looking to build storage for about 1TB redundant array and >2TB
volatile array. I have looked around at snap servers or other SAN
solutions. I also have looked at scsi solutions but don't think that
scsi is economicable. What I am proposing is to build 2 arrays of SATA
drives with the following setup:
1 3u supermicro chasis with 15 hotswappable sata bays
2 3ware RAID controller (for ex 9550SX-8LP)
12 WD SATA HDS (400GB)
RAID ARRAY 1: RAID 5, 5 drives total, 1 hot spare == 1.2TB
RAID ARRAY 2: RAID 0 (or none), 7 drives total == 2.8TB
This will all be hooked to dual xeon chips with and internal boot/sys
mount. Will probably use win2k3 server due to software requirements.
The second array is for temporary storage and can crash and we will not
loose much except the need to download again. The first array needs to
be fault tolerant. We envision swapping out drives in array 2 as
larger drives become available.
My questions are:
+ Can anyone propose reasons NOT to do this?
+ Are the better solutions?
+ NOTE: We anticipate this will cost $5k-$6k -- the other solutins are
MUCH more that I have seen
thanks
daben |
|
| Back to top |
|
 |
Al Dykes
Guest
|
| Posted:
Mon Oct 10, 2005 4:16 pm Post subject:
Re: Looking to build storage of about 3TB -- recommendations |
|
|
In article <1128947603.447478.176960@o13g2000cwo.googlegroups.com>,
Spindle <arthedge@gmail.com> wrote:
| Quote: |
daben wrote:
Hi All
For my research I need large storage for satellite data files. I am
looking to build storage for about 1TB redundant array and >2TB
volatile array. I have looked around at snap servers or other SAN
solutions. I also have looked at scsi solutions but don't think that
scsi is economicable. What I am proposing is to build 2 arrays of SATA
drives with the following setup:
1 3u supermicro chasis with 15 hotswappable sata bays
2 3ware RAID controller (for ex 9550SX-8LP)
12 WD SATA HDS (400GB)
RAID ARRAY 1: RAID 5, 5 drives total, 1 hot spare == 1.2TB
RAID ARRAY 2: RAID 0 (or none), 7 drives total == 2.8TB
This will all be hooked to dual xeon chips with and internal boot/sys
mount. Will probably use win2k3 server due to software requirements.
The second array is for temporary storage and can crash and we will not
loose much except the need to download again. The first array needs to
be fault tolerant. We envision swapping out drives in array 2 as
larger drives become available.
My questions are:
+ Can anyone propose reasons NOT to do this?
+ Are the better solutions?
+ NOTE: We anticipate this will cost $5k-$6k -- the other solutins are
MUCH more that I have seen
thanks
daben
Nor sure I understand how you're going to connect the server(s) to the
array. Fibre channel could add significantly to the cost. iSCSI could
create a 1Gb/sec bottleneck, file serving as in CIFS would probably be
even slower, but you don't mention performance reqs.
|
Is there a comodity raid product that allows me to expand the array by
poping larger drives in, one at a time and leting the raid recovery
work?
What's the status of Infiniband?
http://en.wikipedia.org/wiki/Infiniband
--
a d y k e s @ p a n i x . c o m
Don't blame me. I voted for Gore. |
|
| Back to top |
|
|
daben
Guest
|
| Posted:
Mon Oct 10, 2005 4:16 pm Post subject:
Re: Looking to build storage of about 3TB -- recommendations |
|
|
This storage is for server that it is connected to .. it would not be
network storage. In other words, all the processintg and data access
will be on the machine. It will not be a file server in the normal
sense. This is kinda a special case of just needing a lotof storage
for local processing.
Back up will be accomplished with a lot of tapes :). |
|
| Back to top |
|
|
Spindle
Guest
|
| Posted:
Mon Oct 10, 2005 4:16 pm Post subject:
Re: Looking to build storage of about 3TB -- recommendations |
|
|
daben wrote:
| Quote: | Hi All
For my research I need large storage for satellite data files. I am
looking to build storage for about 1TB redundant array and >2TB
volatile array. I have looked around at snap servers or other SAN
solutions. I also have looked at scsi solutions but don't think that
scsi is economicable. What I am proposing is to build 2 arrays of SATA
drives with the following setup:
1 3u supermicro chasis with 15 hotswappable sata bays
2 3ware RAID controller (for ex 9550SX-8LP)
12 WD SATA HDS (400GB)
RAID ARRAY 1: RAID 5, 5 drives total, 1 hot spare == 1.2TB
RAID ARRAY 2: RAID 0 (or none), 7 drives total == 2.8TB
This will all be hooked to dual xeon chips with and internal boot/sys
mount. Will probably use win2k3 server due to software requirements.
The second array is for temporary storage and can crash and we will not
loose much except the need to download again. The first array needs to
be fault tolerant. We envision swapping out drives in array 2 as
larger drives become available.
My questions are:
+ Can anyone propose reasons NOT to do this?
+ Are the better solutions?
+ NOTE: We anticipate this will cost $5k-$6k -- the other solutins are
MUCH more that I have seen
thanks
daben
|
Nor sure I understand how you're going to connect the server(s) to the
array. Fibre channel could add significantly to the cost. iSCSI could
create a 1Gb/sec bottleneck, file serving as in CIFS would probably be
even slower, but you don't mention performance reqs.
Also, how are you going to back up array 1?
Another point of concern is that recovery from a disk error on array 1
could take hours or days. I would go with raid 6 and with RAID enhanced
SATA drives (WD and Seagate have them). |
|
| Back to top |
|
|
Al Dykes
Guest
|
| Posted:
Mon Oct 10, 2005 10:58 pm Post subject:
Re: Looking to build storage of about 3TB -- recommendations |
|
|
In article <1128960466.487265.93070@g44g2000cwa.googlegroups.com>,
daben <dabenpb@yahoo.com> wrote:
| Quote: | This storage is for server that it is connected to .. it would not be
network storage. In other words, all the processintg and data access
will be on the machine. It will not be a file server in the normal
sense. This is kinda a special case of just needing a lotof storage
for local processing.
Back up will be accomplished with a lot of tapes :).
|
It's possible you can do what you want with dfs, part of the server
ntfs file system if you are loooking to RAID just to aggregate disks
it might be acceptable.
--
a d y k e s @ p a n i x . c o m
Don't blame me. I voted for Gore. |
|
| Back to top |
|
|
Steve Cousins
Guest
|
| Posted:
Tue Oct 11, 2005 11:34 pm Post subject:
Re: Looking to build storage of about 3TB -- recommendations |
|
|
daben wrote:
[quote]Hi All
For my research I need large storage for satellite data files. I am
looking to build storage for about 1TB redundant array and >2TB
volatile array. I have looked around at snap servers or other SAN
solutions. I also have looked at scsi solutions but don't think that
scsi is economicable. What I am proposing is to build 2 arrays of SATA
drives with the following setup:
1 3u supermicro chasis with 15 hotswappable sata bays
2 3ware RAID controller (for ex 9550SX-8LP)
12 WD SATA HDS (400GB)
RAID ARRAY 1: RAID 5, 5 drives total, 1 hot spare == 1.2TB
RAID ARRAY 2: RAID 0 (or none), 7 drives total == 2.8TB
This will all be hooked to dual xeon chips with and internal boot/sys
mount. Will probably use win2k3 server due to software requirements.
The second array is for temporary storage and can crash and we will not
loose much except the need to download again. The first array needs to
be fault tolerant. We envision swapping out drives in array 2 as
larger drives become available.
My questions are:
+ Can anyone propose reasons NOT to do this?
+ Are the better solutions?
+ NOTE: We anticipate this will cost $5k-$6k -- the other solutins are
MUCH more that I have seen
Hi Daben,[/quote]
I have created similar servers with similar equipment (3Ware,
Supermicro,Xeon's and Opterons). My preference is Opterons due to
better performance for the money, at least for our ocean models but as
far as the storage goes what you have should work very well. The only
other concern might be the power supply size for that many drives. I
have 11 drives with dual opterons in a Supermicro 933 case and the
triple-redundant 600 watt PS. It works fine.
Good luck,
Steve |
|
| Back to top |
|
|
daben
Guest
|
| Posted:
Wed Oct 12, 2005 8:15 am Post subject:
Re: Looking to build storage of about 3TB -- recommendations |
|
|
Hi Steve
Thanks for your input. Our latest thinking is to go with 2+ servers.
We too have found that the AMD chips are far faster than the Xeon chips
but I have some doubts about the AMD chipsets and their reliability
over long periods of time. Maybe you can comment on this. Therefore
our plan is as follows:
+ Storage server as above with the Xeon chips (we already have them
from a previous purchase).
+ 1U dual AMD Opteron 275 server with 2GB RAM, 1 sys disk
We would anticipate the ability to add more 1U servers to increase our
computing power over time. I don't think that Blade servers or fiber
is right for our applications or budgets.
Comments?
thanks
daben |
|
| Back to top |
|
|
Zak
Guest
|
| Posted:
Thu Oct 13, 2005 1:17 pm Post subject:
Re: Looking to build storage of about 3TB -- recommendations |
|
|
Al Dykes wrote:
| Quote: | Is there a comodity raid product that allows me to expand the array by
poping larger drives in, one at a time and leting the raid recovery
work?
|
Infortrend has that. But it is a dangerous method: drive failure may
kill you.
That said these are rather affordable so why not buy two...
Thomas |
|
| Back to top |
|
|
Steve Cousins
Guest
|
| Posted:
Thu Oct 13, 2005 9:08 pm Post subject:
Re: Looking to build storage of about 3TB -- recommendations |
|
|
daben wrote:
| Quote: | Hi Steve
Thanks for your input. Our latest thinking is to go with 2+ servers.
We too have found that the AMD chips are far faster than the Xeon chips
but I have some doubts about the AMD chipsets and their reliability
over long periods of time. Maybe you can comment on this. Therefore
our plan is as follows:
We have had some for close to two years now and haven't had an issue. I |
think if you go with a reputable brand you should be fine. We have been
using Tyan with good results although I am glad that I didn't start
using Opterons a little earlier because there were some problems with
some early Tyan boards. At this point I think things are pretty well
established and safe.
| Quote: | + Storage server as above with the Xeon chips (we already have them
from a previous purchase).
+ 1U dual AMD Opteron 275 server with 2GB RAM, 1 sys disk
We would anticipate the ability to add more 1U servers to increase our
computing power over time. I don't think that Blade servers or fiber
is right for our applications or budgets.
This is similar to what I have done. I started out with a 2 TB storage |
server with two 1.6 Ghz Opterons. We use it to run models on too with
good results. It does fine doing both storage and computing. Then we
started adding 1U dual-Opterons and when we needed more storage we
bought a 6.4 TB RAID array and connected it to one of the 1U compute
servers via SCSI. All volumes are NFS mounted to all machines and
performance is acceptable with Gigabit ethernet, even without Jumbo
frames. I originally wanted to set up a SAN with GFS or GPFS instead of
NFS but the FibreChannel hardware was too expensive. As we add compute
servers maybe the NFS performance will become an issue but I'll deal
with that as it comes.
Good luck,
Steve |
|
| Back to top |
|
|
Guest
|
| Posted:
Thu Oct 13, 2005 10:02 pm Post subject:
Re: Looking to build storage of about 3TB -- recommendations |
|
|
Be a little careful here on the Opteron vs. Xeon speed comparisons. In
raw compute speeds the Opteron is definately faster. But the Intel
chipsets support a much higher performance in I/O throughput.
So consider your application. If your system has a PCI based RAID
controller, and multiple Gbit or 10gig NICs, then you could be easily
saturating the I/O bandwidth of an Opteron based platform. If you are
doing a OS based RAID, then you probably need all the CPU speed you can
get, so the Opteron will yield the best performance. A hardware based
RAID solution won't need that kind of CPU power, you will be
bottlenecked on the disk subsystem and the CPU will be idle a lot of
the time.
I don't know what your complete appliaction entails, but keep in mind
that CPU speed is always the best thing to optimize.
-Jim
daben wrote:
| Quote: | Hi Steve
Thanks for your input. Our latest thinking is to go with 2+ servers.
We too have found that the AMD chips are far faster than the Xeon chips
but I have some doubts about the AMD chipsets and their reliability
over long periods of time. Maybe you can comment on this. Therefore
our plan is as follows:
+ Storage server as above with the Xeon chips (we already have them
from a previous purchase).
+ 1U dual AMD Opteron 275 server with 2GB RAM, 1 sys disk
We would anticipate the ability to add more 1U servers to increase our
computing power over time. I don't think that Blade servers or fiber
is right for our applications or budgets.
Comments?
thanks
daben |
|
|
| Back to top |
|
|
daben
Guest
|
| Posted:
Fri Oct 14, 2005 12:16 am Post subject:
Re: Looking to build storage of about 3TB -- recommendations |
|
|
Jim
The appllications to run on the Opteron chips will actually be all
loaded into memory before run. Tun times are to be about 6hrs so the
IO will be a very small portion of that.
We are doing HW based RAID with 3Ware controllers.
daben |
|
| Back to top |
|
|
flux
Guest
|
| Posted:
Sun Oct 16, 2005 8:15 am Post subject:
Re: Looking to build storage of about 3TB -- recommendations |
|
|
In article <1129221599.323718.278400@g47g2000cwa.googlegroups.com>,
jimwall2000@yahoo.com wrote:
| Quote: | Be a little careful here on the Opteron vs. Xeon speed comparisons. In
raw compute speeds the Opteron is definately faster. But the Intel
chipsets support a much higher performance in I/O throughput.
|
Do you by chance have a reference for this? |
|
| Back to top |
|
|
flux
Guest
|
| Posted:
Sun Oct 16, 2005 8:15 am Post subject:
Re: Looking to build storage of about 3TB -- recommendations |
|
|
In article <1128890239.363615.207280@z14g2000cwz.googlegroups.com>,
"daben" <dabenpb@yahoo.com> wrote:
| Quote: | Hi All
For my research I need large storage for satellite data files. I am
looking to build storage for about 1TB redundant array and >2TB
volatile array. I have looked around at snap servers or other SAN
solutions. I also have looked at scsi solutions but don't think that
scsi is economicable. What I am proposing is to build 2 arrays of SATA
drives with the following setup:
1 3u supermicro chasis with 15 hotswappable sata bays
2 3ware RAID controller (for ex 9550SX-8LP)
12 WD SATA HDS (400GB)
RAID ARRAY 1: RAID 5, 5 drives total, 1 hot spare == 1.2TB
RAID ARRAY 2: RAID 0 (or none), 7 drives total == 2.8TB
This will all be hooked to dual xeon chips with and internal boot/sys
mount. Will probably use win2k3 server due to software requirements.
The second array is for temporary storage and can crash and we will not
loose much except the need to download again. The first array needs to
be fault tolerant. We envision swapping out drives in array 2 as
larger drives become available.
My questions are:
+ Can anyone propose reasons NOT to do this?
|
I have seen, for whatever odd reason, a high rate of failure of the SATA
cables. I have also seen backplane failure in these types of boxes.
Ironically, as some will see it, the SATA drives themselves seem to be
quite reliable.
| Quote: | + Are the better solutions?
|
A self-contained controller is probably simpler and presumably less
likely to fail.
| Quote: | + NOTE: We anticipate this will cost $5k-$6k -- the other solutins are
MUCH more that I have seen
|
Promise VTrak is not expensive and actually seems to be reliable. It
also has decent management software.
(Someone else mentioned RAID 6; that's the one thing the VTrak lacks.) |
|
| Back to top |
|
|
David Magda
Guest
|
| Posted:
Mon Oct 17, 2005 5:19 am Post subject:
Re: Looking to build storage of about 3TB -- recommendations |
|
|
flux <support@fluxsoft.com> writes:
| Quote: | Promise VTrak is not expensive and actually seems to be reliable. It
also has decent management software.
|
The experience I've had with Promise cards is that they're mostly
software RAID (a lot happens in the driver). This is under Linux.
I'm not familiar with the VTrak specifically though.
I've heard many good things about 3ware.
--
David Magda <dmagda at ee.ryerson.ca>
Because the innovator has for enemies all those who have done well under
the old conditions, and lukewarm defenders in those who may do well
under the new. -- Niccolo Machiavelli, _The Prince_, Chapter VI |
|
| Back to top |
|
|
flux
Guest
|
| Posted:
Mon Oct 17, 2005 8:15 am Post subject:
Re: Looking to build storage of about 3TB -- recommendations |
|
|
In article <m27jcddmh7.fsf@gandalf.magda.ca>,
David Magda <dmagda+trace050401@ee.ryerson.ca> wrote:
| Quote: | flux <support@fluxsoft.com> writes:
Promise VTrak is not expensive and actually seems to be reliable. It
also has decent management software.
The experience I've had with Promise cards is that they're mostly
software RAID (a lot happens in the driver). This is under Linux.
|
The VTrak is a completely self-contained hardware RAID.
| Quote: | I'm not familiar with the VTrak specifically though.
I've heard many good things about 3ware.
|
Ironically, 3ware may be problematic under Linux. I've occasionally
experienced "Out of IOMMU" space errors and such an error generally
spits junk data on the RAID filesystems, which is not good. I don't
know, however, if the source of the problem is the 3ware driver (is it
really "hardware" RAID?), the Linux kernel, the card itself, or
something interaction with the motherboard (Arima HDAMA) or the (Wintec)
memory. But since I don't see this problem on the LSI/mptfusion
card/driver in the same box, I'm inclined to believe it's the 3ware
driver or the card itself. |
|
| Back to top |
|
|
|
|
|
|
FAQ
Memberlist
Register
Profile
Log in to check your private messages
Log in