| Author |
Message |
Joe Seigh
Guest
|
 Posted:
Sun Dec 11, 2005 5:15 pm Post subject:
Re: Sun's Niagara is out |
|
|
Bill Todd wrote:
| Quote: | Joe Seigh wrote:
...
So the Niagara seems to be a good match for Google.
Quite possibly.
Except Google does
mainly
distributed processing, not shared memory multi-threading, so it
doesn't really
need a multicore chip, just lots of cheap low power cpus.
It seems unlikely that Google runs single-threaded on each of its many
processors: since it does much more data-moving (from disk to network)
than data-processing, that would leave each processor idle most of the
time.
They're essentially running multiple i/o bound single threads which aren't |
sharing memory. So running SMP doesn't buy them anything. They communicate
by sockets with everything, though you might argue that IPC sockets are
more economical than separate hardware based sockets.
--
Joe Seigh
When you get lemons, you make lemonade.
When you get hardware, you make software. |
|
| Back to top |
|
 |
Joe Seigh
Guest
|
| Posted:
Sun Dec 11, 2005 5:15 pm Post subject:
Re: Sun's Niagara is out |
|
|
Casper H.S. Dik wrote:
| Quote: | Bill Todd <billtodd@metrocast.net> writes:
That leaves the BL25p as the sole entry in your list which might manage
to squeak in below the Niagara system in price.
But does it come below in power consumption?
I guess it's not just market speak. The size of your power bill |
does matter. :) Google thinks so.
http://news.com.com/Power+could+cost+more+than+servers%2C+Google+warns/2100-1010_3-5988090.html
Cringly reports that Google is siting next to power plants (except he claims
it's because the disk drives eat power)
http://www.pbs.org/cringely/pulpit/pulpit20051020.html
And The Inquirer has awarded Intel "the Society of Electricity Generation
Corporations award for increasing electricity demand"
http://www.theinquirer.net/?article=28266
So the Niagara seems to be a good match for Google. Except Google does mainly
distributed processing, not shared memory multi-threading, so it doesn't really
need a multicore chip, just lots of cheap low power cpus.
--
Joe Seigh
When you get lemons, you make lemonade.
When you get hardware, you make software. |
|
| Back to top |
|
|
Bill Todd
Guest
|
| Posted:
Sun Dec 11, 2005 5:15 pm Post subject:
Re: Sun's Niagara is out |
|
|
Joe Seigh wrote:
....
| Quote: | So the Niagara seems to be a good match for Google.
|
Quite possibly.
Except Google does
| Quote: | mainly
distributed processing, not shared memory multi-threading, so it doesn't
really
need a multicore chip, just lots of cheap low power cpus.
|
It seems unlikely that Google runs single-threaded on each of its many
processors: since it does much more data-moving (from disk to network)
than data-processing, that would leave each processor idle most of the time.
So Niagara might provide significantly better performance per Watt for
their server farm, and significantly reduce its floor-space requirements
as well - as long as its bandwidth didn't turn out to be the limiting
factor (such that it wound up being idle most of the time).
- bill |
|
| Back to top |
|
|
Douglas Siebert
Guest
|
| Posted:
Sun Dec 11, 2005 9:51 pm Post subject:
Re: Sun's Niagara is out |
|
|
lindahl@pbm.com (Greg Lindahl) writes:
| Quote: | Try working out how much you pay for power now. A typical number is
less than 10% of the cost of the hardware, per year. So over a 3-year
lifetime, it's not bigger than the cost of the servers. And while it
would be nice to lower that number, it doesn't drive TCO, so Sun is
going to have a hard time selling Niagra to new customers, unless said
new customers are out of machine room capacity.
|
Or said new customers are like many who pay per square foot for data center
space. Fitting your equipment into one rack instead of two would pay for
the equipment in many locations.
Might even work out for companies that run their own datacenters and have
plenty of space, depending on how/whether they do chargebacks between the
different business units.
--
Douglas Siebert dsiebert@excisethis.khamsin.net
Give a man a match, and he'll be warm for a minute. Set him on fire, and
he'll be warm for the rest of his life. |
|
| Back to top |
|
|
Bill Todd
Guest
|
| Posted:
Sun Dec 11, 2005 11:37 pm Post subject:
Re: Sun's Niagara is out |
|
|
Joe Seigh wrote:
....
| Quote: | They're essentially running multiple i/o bound single threads which aren't
sharing memory. So running SMP doesn't buy them anything.
|
I'm not sure you understand.
If Google is indeed concerned about processor power dissipation, then
almost by definition they've designed their system (which they've
apparently designed to the hilt) such that each processor is working
very close to as hard as it can - i.e., running *enough* I/O-bound
largely-independent software threads to saturate it, or nearly so.
That means that unless Niagara is bandwidth-limited to the extent that
it *can't* be similarly saturated, it would be a win for them (and it
might well still be a win even if it could only be, say, 50% loaded,
given how much less power it consumes for a given workload than anything
save possibly dual-core AMD64s). Hell, given the relatively small
number of instructions each thread executes between I/O events, just the
saving in context-switching overhead by doing the multi-threading in
hardware should be significant.
(That's assuming that in fact they *are* using multiple software threads
rather than a wholly-asynchronous approach that divides use of a single
software thread per core dynamically among a large number of clients,
though even there there's *some* overhead when contexts are switched.)
- bill |
|
| Back to top |
|
|
Joe Seigh
Guest
|
| Posted:
Mon Dec 12, 2005 12:54 am Post subject:
Re: Sun's Niagara is out |
|
|
Bill Todd wrote:
| Quote: | Joe Seigh wrote:
...
They're essentially running multiple i/o bound single threads which
aren't
sharing memory. So running SMP doesn't buy them anything.
I'm not sure you understand.
If Google is indeed concerned about processor power dissipation, then
almost by definition they've designed their system (which they've
apparently designed to the hilt) such that each processor is working
very close to as hard as it can - i.e., running *enough* I/O-bound
largely-independent software threads to saturate it, or nearly so.
That means that unless Niagara is bandwidth-limited to the extent that
it *can't* be similarly saturated, it would be a win for them (and it
might well still be a win even if it could only be, say, 50% loaded,
given how much less power it consumes for a given workload than anything
save possibly dual-core AMD64s). Hell, given the relatively small
number of instructions each thread executes between I/O events, just the
saving in context-switching overhead by doing the multi-threading in
hardware should be significant.
|
No, I understand. The difference between a SMP system and just a bunch
of single processor systems running multiple threads is the lack of a
coherent shared memory mechanism. Distributed programming means the
threads are communicating via i/o not by memory so memory coherency
isn't needed strictly speaking.
--
Joe Seigh
When you get lemons, you make lemonade.
When you get hardware, you make software. |
|
| Back to top |
|
|
Greg Lindahl
Guest
|
| Posted:
Mon Dec 12, 2005 1:14 am Post subject:
Re: Sun's Niagara is out |
|
|
In article <huWdnfjqtoON2AHenZ2dnUVZ_vmdnZ2d@comcast.com>,
Joe Seigh <jseigh_01@xemaps.com> wrote:
If you actually read the article, the statement is about power in the
future if nothing is done about it.
Try working out how much you pay for power now. A typical number is
less than 10% of the cost of the hardware, per year. So over a 3-year
lifetime, it's not bigger than the cost of the servers. And while it
would be nice to lower that number, it doesn't drive TCO, so Sun is
going to have a hard time selling Niagra to new customers, unless said
new customers are out of machine room capacity.
-- g |
|
| Back to top |
|
|
Chris Thomasson
Guest
|
| Posted:
Mon Dec 12, 2005 1:15 am Post subject:
Re: Sun's Niagara is out |
|
|
| Quote: | Scalability is a measure of how much improvement in performance you
get when you add extra processors (assuming you have enough threads
to exploit the processors and aren't overwhelmingly i/o bound). You
don't automatically get 2x performance when you double the number of
processors.
For the most part it is caused by contention on locks by the threads
which goes up exponentially with the number of threads/processors.
You can reduce your locking granularity but eventually even that won't
help due to the exponential nature of the contention.
|
Yes. However, IMO, even non-contended locks are expensive and can adversely
affect performance and scalability. As you know, the atomic operations and
membars involved come with a whole lot of overhead. So, even if an
applications locking scheme rarely hits any contention, its still exposing
its threads to fairly high amounts of overhead. We have created some novel
solutions to the problem, but it seems as though most programmers simply do
not understand them. For instance, I am still waiting for some kind of a
response from David Abrahams to this post:
http://groups.google.com/group/comp.lang.c++.moderated/msg/e8ed4f5f000c336a
in this news-thread:
http://groups.google.com/group/comp.lang.c++.moderated/browse_frm/thread/65c9c2673682d4cf/81aeb2429f1c19f9?q=STL+zero-overhead&rnum=1#81aeb2429f1c19f9
Oh well... I will post the demo applications anyway!
;)
--
Chris Thomasson
http://home.comcast.net/~vzoom/
Virtually Zero-Overhead Object Management (VZOOM) |
|
| Back to top |
|
|
CJT
Guest
|
| Posted:
Mon Dec 12, 2005 1:15 am Post subject:
Re: Sun's Niagara is out |
|
|
Peter Matthias wrote:
| Quote: | Andi Kleen wrote:
I think for a fair power comparison you would need to measure systems
running the benchmark.
I think one should not compare /power/ but /energy/ consumption for
benchmarks.
Peter
|
As long as the systems are fairly well matched in speed, that'll be
no different.
--
The e-mail address in our reply-to line is reversed in an attempt to
minimize spam. Our true address is of the form che...@prodigy.net. |
|
| Back to top |
|
|
Bill Todd
Guest
|
| Posted:
Mon Dec 12, 2005 9:15 am Post subject:
Re: Sun's Niagara is out |
|
|
Joe Seigh wrote:
| Quote: | Bill Todd wrote:
Joe Seigh wrote:
...
They're essentially running multiple i/o bound single threads which
aren't
sharing memory. So running SMP doesn't buy them anything.
I'm not sure you understand.
If Google is indeed concerned about processor power dissipation, then
almost by definition they've designed their system (which they've
apparently designed to the hilt) such that each processor is working
very close to as hard as it can - i.e., running *enough* I/O-bound
largely-independent software threads to saturate it, or nearly so.
That means that unless Niagara is bandwidth-limited to the extent that
it *can't* be similarly saturated, it would be a win for them (and it
might well still be a win even if it could only be, say, 50% loaded,
given how much less power it consumes for a given workload than
anything save possibly dual-core AMD64s). Hell, given the relatively
small number of instructions each thread executes between I/O events,
just the saving in context-switching overhead by doing the
multi-threading in hardware should be significant.
No, I understand.
|
Actually, from your comment below it is now clear that you don't.
The difference between a SMP system and just a bunch
| Quote: | of single processor systems running multiple threads is the lack of a
coherent shared memory mechanism. Distributed programming means the
threads are communicating via i/o not by memory so memory coherency
isn't needed strictly speaking.
|
SMP, schmeshmp. It *doesn't matter a whit* that memory coherency is
irrelevant to inter-thread communication in Google: the issue is how
much work can be done for a given amount of energy, and since the
environment is sufficiently multi-threaded to make full use of Niagara
(and sufficiently light on processing requirements that Niagara should
be able to handle it) unless Niagara hits some *other* limit (such as
single-core bandwidth) it can likely process Google-style threads
considerably more efficiently than most conventional MPUs could.
To put it another way, if they need N single-processor Pentium systems
currently, there's a good chance that they'd need less than (perhaps
*considerably* less than) 0.5N Niagara systems to handle the workload.
- bill |
|
| Back to top |
|
|
Joe Seigh
Guest
|
| Posted:
Mon Dec 12, 2005 5:15 pm Post subject:
Re: Sun's Niagara is out |
|
|
Bill Todd wrote:
| Quote: | Joe Seigh wrote:
No, I understand.
Actually, from your comment below it is now clear that you don't.
The difference between a SMP system and just a bunch
of single processor systems running multiple threads is the lack of a
coherent shared memory mechanism. Distributed programming means the
threads are communicating via i/o not by memory so memory coherency
isn't needed strictly speaking.
SMP, schmeshmp. It *doesn't matter a whit* that memory coherency is
irrelevant to inter-thread communication in Google: the issue is how
much work can be done for a given amount of energy, and since the
environment is sufficiently multi-threaded to make full use of Niagara
(and sufficiently light on processing requirements that Niagara should
be able to handle it) unless Niagara hits some *other* limit (such as
single-core bandwidth) it can likely process Google-style threads
considerably more efficiently than most conventional MPUs could.
To put it another way, if they need N single-processor Pentium systems
currently, there's a good chance that they'd need less than (perhaps
*considerably* less than) 0.5N Niagara systems to handle the workload.
|
In order to go the commodity route, Google went with distributed programming
which doesn't require or really benefit from SMP. To them it's just a bunch
of single processors. So if Google looks at Niagara, it's strictly from a
cost per processor, cost being a number of factors, i.e. money, energy, space,
infrastrucure, amount of work a processor can do, etc...
I agree that energy would be an important factor. That's why I mentioned it
in the first place.
And saying Google doesn't need SMP isn't the same as saying they don't need
or use multi-threading (or whatever mechanism they use to fully utilize the
processor for i/o bound tasks). If Sun came out with a significantly cheaper
Niagara processor which did not have coherent memory, Google would likely
go with that if they had to decide between the two. Google doesn't need
SMP so it's not a factor in that decision.
--
Joe Seigh
When you get lemons, you make lemonade.
When you get hardware, you make software. |
|
| Back to top |
|
|
Douglas Siebert
Guest
|
| Posted:
Mon Dec 12, 2005 8:43 pm Post subject:
Re: Sun's Niagara is out |
|
|
Joe Seigh <jseigh_01@xemaps.com> writes:
| Quote: | In order to go the commodity route, Google went with distributed programming
which doesn't require or really benefit from SMP. To them it's just a bunch
of single processors. So if Google looks at Niagara, it's strictly from a
cost per processor, cost being a number of factors, i.e. money, energy, space,
infrastrucure, amount of work a processor can do, etc...
I agree that energy would be an important factor. That's why I mentioned it
in the first place.
And saying Google doesn't need SMP isn't the same as saying they don't need
or use multi-threading (or whatever mechanism they use to fully utilize the
processor for i/o bound tasks). If Sun came out with a significantly cheaper
Niagara processor which did not have coherent memory, Google would likely
go with that if they had to decide between the two. Google doesn't need
SMP so it's not a factor in that decision.
|
But the fact that a single processor can run 32 processes simultaneously
is probably a big thing in Niagara's favor from google's perspective,
versus current Intel and AMD CPUs which can run only 2. Granted those
processes will get a lot more work done, but 32 slower processors is more
valuable for certain tasks. For memory bound tasks (like their index
searches, since the indexes are entirely RAM resident from what I understand
of their setup) it wouldn't be useful, because you are limited by your
memory bandwith. But for more disk or network intensive work Niagara may
be a big win for them.
I think it is a safe bet Google has a bunch of Niagara boxes in house for
testing, and Sun would probably cut them a fabulous deal on pricing just
to gain the publicity of selling to Google, since they're positioning
Niagara as ideal for web related tasks and Google is the king of the web.
--
Douglas Siebert dsiebert@excisethis.khamsin.net
Give a man a match, and he'll be warm for a minute. Set him on fire, and
he'll be warm for the rest of his life. |
|
| Back to top |
|
|
Stephen Fuld
Guest
|
| Posted:
Tue Dec 13, 2005 9:15 am Post subject:
Re: Sun's Niagara is out |
|
|
"Douglas Siebert" <dsiebert@excisethis.khamsin.net> wrote in message
news:dnknd9$frq$1@narsil.avalon.net...
Discussion of whether Niagara would be usefull for Google
| Quote: | For memory bound tasks (like their index
searches, since the indexes are entirely RAM resident from what I
understand
of their setup) it wouldn't be useful, because you are limited by your
memory bandwith.
|
Then wouldn't the multiple independent servers actually be better (not the
same, but better) as you could get far more memory bandwidth from say 8
totally independent memory systems each attached to one x86 system than with
one memory system attached to a single Niagara processor?
--
- Stephen Fuld
e-mail address disguised to prevent spam |
|
| Back to top |
|
|
Russell Crook - Computer
Guest
|
| Posted:
Tue Dec 13, 2005 5:15 pm Post subject:
Re: Sun's Niagara is out |
|
|
Stephen Fuld wrote:
| Quote: | "Douglas Siebert" <dsiebert@excisethis.khamsin.net> wrote in message
news:dnknd9$frq$1@narsil.avalon.net...
Discussion of whether Niagara would be usefull for Google
For memory bound tasks (like their index
searches, since the indexes are entirely RAM resident from what I
understand
of their setup) it wouldn't be useful, because you are limited by your
memory bandwith.
Then wouldn't the multiple independent servers actually be better (not the
same, but better) as you could get far more memory bandwidth from say 8
totally independent memory systems each attached to one x86 system than with
one memory system attached to a single Niagara processor?
|
Minor correction: The T1 chip has four DDR2 memory controllers.
(I do not know what the interplay is between memory latency and
memory bandwidth is on Google's search, there seems to be a
presumption above that the index seach is bandwidth constrained
as opposed to latency constrained. Is there any documentation on
this (just curious).
Russell
> |
|
| Back to top |
|
|
Bill Todd
Guest
|
| Posted:
Wed Dec 14, 2005 1:15 am Post subject:
Re: Sun's Niagara is out |
|
|
Russell Crook - Computer Systems - System Engineer wrote:
| Quote: | Stephen Fuld wrote:
"Douglas Siebert" <dsiebert@excisethis.khamsin.net> wrote in message
news:dnknd9$frq$1@narsil.avalon.net...
Discussion of whether Niagara would be usefull for Google
For memory bound tasks (like their index
searches, since the indexes are entirely RAM resident from what I
understand
of their setup) it wouldn't be useful, because you are limited by your
memory bandwith.
Then wouldn't the multiple independent servers actually be better (not
the same, but better) as you could get far more memory bandwidth from
say 8 totally independent memory systems each attached to one x86
system than with one memory system attached to a single Niagara
processor?
|
I'm having difficulty seeing how the above differs from what Doug
stated: he seemed to be suggesting (as was I) that Niagara would be a
win for those tasks primarily involving traffic-cop-style data movement
from storage to the network (for which my vague recollection is Google
uses different servers than it uses for index processing).
| Quote: |
Minor correction: The T1 chip has four DDR2 memory controllers.
(I do not know what the interplay is between memory latency and
memory bandwidth is on Google's search, there seems to be a
presumption above that the index seach is bandwidth constrained
as opposed to latency constrained.
|
A third possibility is that it is (or rather would on Niagara become)
processor-limited - e.g., a conventional b-tree scans its leaves
sequentially, and if they're large and effectively pre-fetched that can
eat up a noticeable amount of processing power (which Google may have
cheerfully given up using unoptimized approaches if they had a lot of
processing power to spare with faster single processors than Niagara
offers).
- bill |
|
| Back to top |
|
|
|
|
|
|
FAQ
Memberlist
Register
Profile
Log in to check your private messages
Log in