| Author |
Message |
Guest
|
 Posted:
Wed Dec 07, 2005 5:16 pm Post subject:
storage solution |
|
|
Hello,
I am planning to experiment with an online service technology for file
sharing something similar to sendthisfile.com or putfile.com where
there is a huge storage requirement and bandwidth
Any ideas of a cost effective storage solution (in Linux or MS) which
could be implemented from a home-office. Cost is an important factor
since I am just experimenting with the technology right now.
I would not like to go for shared hosting since I need an hands on
experience.
Precisely,
i) What hardware/architecture is needed for storage (about 2+ TB), web
server with say daily 25 Gb bandwidth of download.
ii) Also, what bandwidth of Internet backbone is needed (in Mbps????).
Leased Line?
"HOW MUCH WOULD ALL THIS COST ?"
Thanks and Regards,
Chival |
|
| Back to top |
|
 |
Nik Simpson
Guest
|
| Posted:
Thu Dec 08, 2005 1:16 am Post subject:
Re: storage solution |
|
|
| Quote: |
i) What hardware/architecture is needed for storage (about 2+ TB), web
server with say daily 25 Gb bandwidth of download.
|
Tha'ts ~300Kbit/s, you'd be hard pressed to find any storage that *couldn't*
sustain that without breaking a sweat. So I'd go for SATA drives as big as
you can get plus perhaps some sort of RAID. Cheapest solution might be to
build yourself a PC with onboard SATA RAID.
| Quote: |
ii) Also, what bandwidth of Internet backbone is needed (in Mbps????).
Leased Line?
|
Again, you aren't talking about much if the load is evenly balanced
throughout the day, 300Kbit's is something a good DSL line can handle and
certainly wouldn't be a problem for a cable modem. Even if the demand is
"bursty" it's still not likely to need more than a cable modem with
500K-1Mbit of upstream bandwidth.
| Quote: |
"HOW MUCH WOULD ALL THIS COST ?"
|
2-2.5K tops for this type of application would be my guess.
--
Nik Simpson |
|
| Back to top |
|
|
Guest
|
| Posted:
Thu Dec 08, 2005 9:15 am Post subject:
Re: storage solution |
|
|
Many thanks Nik.
I have come accross many web references which recommend a SAN if
storage is in TB's and likely to exceed
How scalable is SATA RAID?
Thanks again.
Nik Simpson wrote:
| Quote: |
i) What hardware/architecture is needed for storage (about 2+ TB), web
server with say daily 25 Gb bandwidth of download.
Tha'ts ~300Kbit/s, you'd be hard pressed to find any storage that *couldn't*
sustain that without breaking a sweat. So I'd go for SATA drives as big as
you can get plus perhaps some sort of RAID. Cheapest solution might be to
build yourself a PC with onboard SATA RAID.
ii) Also, what bandwidth of Internet backbone is needed (in Mbps????).
Leased Line?
Again, you aren't talking about much if the load is evenly balanced
throughout the day, 300Kbit's is something a good DSL line can handle and
certainly wouldn't be a problem for a cable modem. Even if the demand is
"bursty" it's still not likely to need more than a cable modem with
500K-1Mbit of upstream bandwidth.
"HOW MUCH WOULD ALL THIS COST ?"
2-2.5K tops for this type of application would be my guess.
--
Nik Simpson |
|
|
| Back to top |
|
|
Miroslav Pragl
Guest
|
| Posted:
Thu Dec 08, 2005 4:44 pm Post subject:
Re: storage solution |
|
|
SATA RAID: i'd be afraid of CPU load as SATA RAIDs are usually software
raids (performed by driver, not hardware). Doublecheck the chipset used!
Miroslav
>> build yourself a PC with onboard SATA RAID. |
|
| Back to top |
|
|
Nik Simpson
Guest
|
| Posted:
Thu Dec 08, 2005 5:15 pm Post subject:
Re: storage solution |
|
|
"Miroslav Pragl" <lists.subscriber@pragl.cz> wrote in message
news:dn92rd$256r$2@ns.felk.cvut.cz...
| Quote: | SATA RAID: i'd be afraid of CPU load as SATA RAIDs are usually software
raids (performed by driver, not hardware). Doublecheck the chipset used!
|
1. Any half decent system will have plenty of CPU power to spare. Think
about it, your typical PCI RAID card with an aging 3-500MHz Strong-Arm CPU
or similar vs a 2+GHz Pentium 4 or Athlon, tell me which is going to have
more CPU cycles free to handle RAID overhead?
2. The sort of bandwidth requirements of the original poster make this a
moot point, average throughput pof 300Kbits/s is nothing
--
Nik Simpson |
|
| Back to top |
|
|
Guest
|
| Posted:
Thu Dec 08, 2005 11:43 pm Post subject:
Re: storage solution |
|
|
In article <1134019993.342816.156230@g43g2000cwa.googlegroups.com>,
<chivalc@yahoo.com> wrote:
| Quote: | I have come accross many web references which recommend a SAN if
storage is in TB's and likely to exceed
How scalable is SATA RAID?
|
This rule made a lot of sense when enterprise-class disks were 9GB and
18GB. In those days (which are only a few years ago!) a 1TB storage
system required say 70 or 200 disks (depending for example on the
level of redundancy, the lower end is RAID5 4+1 with 18GB disks
requiring 70 disks, the upper end is RAID1 with 9GB disks requiring
224 disks). A system with that many disks should not be built by
direct-attaching disks to a host, as the administration, management,
cabling etc. are all hard problems. With that many disks, regular
maintenance is required, as you expect a disk to fail every few weeks
or months.
Today, 500GB SATA disks are available easily on the open market. This
means that building a 1TB storage system with redundancy requires
between 3 and 4 disks (3 for 2+1 RAID5, 4 for RAID1). This easily
fits inside any decent server case. With SATA, cabling 4 disks is
trivial (even I could do it before breakfast, and I'm not a hardware
hacker). And setting up and administring a RAID controller (whether
software or hardware) for 3 or 4 disks takes less than 1/2 hour, and
would require no more than a cup of coffee to get going. And with
this few disks, failures are going to be rare events (every few
years), so they can be dealt with by keeping one spare disk in the
cabinet, and doing a manual hardware swap when needed.
I think it is VERY VERY important to express the complexity of a
system not in terms of GB/TB/PB (nor in terms of MIPS and FLOPS), but
in terms of the number of "moving parts": How many disk drives, how
many CPUs (or how many independent hsots with operating systems, as
SMPs and multicore-CPUs are quute similar to single-CPU systems), how
many network switches, how many host names and IP addresses, and so
on. So today a 1TB system has 3-4 spindles, which does not require a
SAN.
--
The address in the header is invalid for obvious reasons. Please
reconstruct the address from the information below (look for _).
Ralph Becker-Szendy _firstname_@lr_dot_los-gatos_dot_ca.us |
|
| Back to top |
|
|
Guest
|
| Posted:
Fri Dec 09, 2005 5:16 pm Post subject:
Re: storage solution |
|
|
Thanks.
Any ideas how this would work out if a remote backup (outside city
limits) is needed.
- Sid
_firstname_@lr_dot_los-gatos_dot_ca.us wrote:
| Quote: | In article <1134019993.342816.156230@g43g2000cwa.googlegroups.com>,
chivalc@yahoo.com> wrote:
I have come accross many web references which recommend a SAN if
storage is in TB's and likely to exceed
How scalable is SATA RAID?
This rule made a lot of sense when enterprise-class disks were 9GB and
18GB. In those days (which are only a few years ago!) a 1TB storage
system required say 70 or 200 disks (depending for example on the
level of redundancy, the lower end is RAID5 4+1 with 18GB disks
requiring 70 disks, the upper end is RAID1 with 9GB disks requiring
224 disks). A system with that many disks should not be built by
direct-attaching disks to a host, as the administration, management,
cabling etc. are all hard problems. With that many disks, regular
maintenance is required, as you expect a disk to fail every few weeks
or months.
Today, 500GB SATA disks are available easily on the open market. This
means that building a 1TB storage system with redundancy requires
between 3 and 4 disks (3 for 2+1 RAID5, 4 for RAID1). This easily
fits inside any decent server case. With SATA, cabling 4 disks is
trivial (even I could do it before breakfast, and I'm not a hardware
hacker). And setting up and administring a RAID controller (whether
software or hardware) for 3 or 4 disks takes less than 1/2 hour, and
would require no more than a cup of coffee to get going. And with
this few disks, failures are going to be rare events (every few
years), so they can be dealt with by keeping one spare disk in the
cabinet, and doing a manual hardware swap when needed.
I think it is VERY VERY important to express the complexity of a
system not in terms of GB/TB/PB (nor in terms of MIPS and FLOPS), but
in terms of the number of "moving parts": How many disk drives, how
many CPUs (or how many independent hsots with operating systems, as
SMPs and multicore-CPUs are quute similar to single-CPU systems), how
many network switches, how many host names and IP addresses, and so
on. So today a 1TB system has 3-4 spindles, which does not require a
SAN.
--
The address in the header is invalid for obvious reasons. Please
reconstruct the address from the information below (look for _).
Ralph Becker-Szendy _firstname_@lr_dot_los-gatos_dot_ca.us |
|
|
| Back to top |
|
|
Arne Joris
Guest
|
| Posted:
Sat Dec 10, 2005 1:16 am Post subject:
Re: storage solution |
|
|
Now you're gettng into very murky waters...
Do you need instant replication of your data or a nightly backup ?
Hundreds of kilometers or just one ? Does the data need to be
application coherent so you can restart apps from it in case of a
disaster ?
Whereas the storage can be cheap because it has 24 hours to receive 25
GB and disks can easily sustain that rate, the line you'll need to get
all this data out in a backup window < 24 hours will be very expensive.
Say you need a nightly backup while your webserver is closed. You have
25 GB to move in, say 8 hours; that's slightly under one meg per
second. A T1 line is not going to cut it unless you can apply some
extreme compression to your data.
Doing a tape backup and driving the tape to your destination will be
the cheapest method. |
|
| Back to top |
|
|
|
|
|
|
FAQ
Memberlist
Register
Profile
Log in to check your private messages
Log in