| Author |
Message |
Dan Koren
Guest
|
 Posted:
Thu Dec 30, 2004 1:36 am Post subject:
Re: Athlon cache question. |
|
|
Sorry to top post. I do not want to force people to
read your entire post before they get to the point.
I don't mean to spoil your fun, but there is a lot
of empirical evidence that frequency of reference
based replacement strategies tend to outperform
pure LRU (whether local or global).
dk
"Anne & Lynn Wheeler" <lynn@garlic.com> wrote in message
news:m3oegd6jh5.fsf@lhwlinux.garlic.com...
| Quote: | "Dan Koren" <dankoren@yahoo.com> writes:
IBM researched cache architectures
pretty much to death during the '70s.
Of course younger designers tend to
ignore the work of earlier generations.
slightly related is replacement strategies ... my recollection
is that at the same asilomar/sigops meeting where i432 people
presented ... including lots of comments about patching complex
operating system features that had been dropped into silicon
... recent refs:
http://www.garlic.com/~lynn/2004q.html#60 Will multicore CPUs have
identical cores?
http://www.garlic.com/~lynn/2004q.html#64 Will multicore CPUs have
identical cores?
.. jim introduced me to a co-worker at tandem that was having
trouble getting his stanford phd ... so it must have been after
jim had left sjr for tandem .... random sjr/systemr posts
http://www.garlic.com/~lynn/subtopic.html#systemr
the problem was that the thesis was basically on global LRU
replacement strategy and stanford was getting a lot of pushback from
strong advocate of local LRU replacement.
in the late 60s there was some academic literature on local LRU
replacement and working sets. at that time i was an undergraduate and
doing lots of operating system changes ... including having come up
with this global LRU idea, implemented it and it had shipped in
products.
the problem at hand was to show global LRU replacement significantly
better than local LRU.
much of the 70s, i was at the cambridge science center ... and in the
early 70s the grenoble science center had done a study using the same
operating system and the same hardware and the same type of workload
... but implementing a "working set dispatcher" and even gotten a
paper published in the cacm. they had come by cambridge and left me
with rough draft ... as well as a lot of the the detailed backup study
information. The primary difference between cambridge and grenoble was
that grenoble had a 1mbyte real storage 360/67 (154 4k "pageable
pages" after fixed memory requirements) and were running 35 concurrent
users while cambridge had a 768k real storage 360/67 (104 4k "pageable
pages" after fixed memory requirements) and 75-80 users. Cambridge
with approx. twice the load and significantly smaller real storage
using a lobal LRU replacment strategy was getting about the same
performance as the Grenoble "working set dispatcher" and local LRU
(with half the users and 50 percent more effective paging storage).
In any case, all the backup material showing local LRU significantly
outperforming global LRU on directly compareable hardware, software,
and workload help tip the balance in getting the Phd approved.
Not including in the comparison ... but about that time in the early
70s, I was also playing with some coding tricks with global LRU
implementation. Normally any sort of LRU-like implementation
effectively degrades to FIFO when there isn't sufficient information
to otherwise distinguish reference patterns between pages (except in
some rare cases, FIFO isn't a particularly good replacement strategy).
I had this slight of hand coding tricks for global LRU ... which
continued to look, smell and taste like standard global LRU
replacement ... but had the unusual characteristic of degradding to
random replacement (instead of FIFO ... in lots of detailed simulation
studies it was shown to out-perform true LRU across wide-variety of
conditions ... as compared to the LRU-approximation implementations
which strived just to be nearly as good as true LRU).
lots of past replacement algorithm postings
http://www.garlic.com/~lynn/subtopic.html#wsclock
some specific past references to the thesis (as well as grenoble paper):
http://www.garlic.com/~lynn/93.html#4 360/67, was Re: IBM's Project F/S ?
http://www.garlic.com/~lynn/94.html#1 Multitasking question
http://www.garlic.com/~lynn/99.html#18 Old Computers
http://www.garlic.com/~lynn/2001c.html#10 Memory management - Page
replacement
http://www.garlic.com/~lynn/2002c.html#49 Swapper was Re: History of Login
Names
for some topic drift mention of jim leaving sjr and going to tandem:
http://www.garlic.com/~lynn/2002k.html#39 Vnet : Unbelievable
http://www.garlic.com/~lynn/2002o.html#73 They Got Mail: Not-So-Fond
Farewells
http://www.garlic.com/~lynn/2002o.html#75 They Got Mail: Not-So-Fond
Farewells
http://www.garlic.com/~lynn/2004c.html#15 If there had been no MS-DOS
http://www.garlic.com/~lynn/2004l.html#31 Shipwrecks
some of the cambridge detailed trace and simulation work was
eventually released as a product called vs/repack in the mid-70s (i.e.
took detailed trace of application and attempted semi-reorg of large
application to minimize paging). it was also used by a number of
corporate products that were making the transition from real stroage
environment to virtual memory (compilers, database managers, etc) ..
as well as starting to study cache-sensitivity issues ...
random past vs/repack references:
http://www.garlic.com/~lynn/94.html#7 IBM 7090 (360s, 370s, apl, etc)
http://www.garlic.com/~lynn/99.html#68 The Melissa Virus or War on
Microsoft?
http://www.garlic.com/~lynn/2000g.html#30 Could CDR-coding be on the way
back?
http://www.garlic.com/~lynn/2001b.html#83 Z/90, S/390, 370/ESA (slightly
off topic)
http://www.garlic.com/~lynn/2001c.html#31 database (or b-tree) page sizes
http://www.garlic.com/~lynn/2001c.html#33 database (or b-tree) page sizes
http://www.garlic.com/~lynn/2001i.html#20 Very CISC Instuctions (Was: why
the machine word size ...)
http://www.garlic.com/~lynn/2002c.html#28 OS Workloads : Interactive etc
http://www.garlic.com/~lynn/2002c.html#45 cp/67 addenda (cross-post
warning)
http://www.garlic.com/~lynn/2002c.html#46 cp/67 addenda (cross-post
warning)
http://www.garlic.com/~lynn/2002c.html#49 Swapper was Re: History of Login
Names
http://www.garlic.com/~lynn/2002e.html#50 IBM going after Strobe?
http://www.garlic.com/~lynn/2002f.html#50 Blade architectures
http://www.garlic.com/~lynn/2003f.html#15 Alpha performance, why?
http://www.garlic.com/~lynn/2003f.html#21 "Super-Cheap" Supercomputing
http://www.garlic.com/~lynn/2003f.html#53 Alpha performance, why?
http://www.garlic.com/~lynn/2003g.html#15 Disk capacity and backup
solutions
http://www.garlic.com/~lynn/2003h.html#8 IBM says AMD dead in 5yrs ... --
Microsoft Monopoly vs. IBM
http://www.garlic.com/~lynn/2003j.html#32 Language semantics wrt exploits
http://www.garlic.com/~lynn/2004c.html#21 PSW Sampling
http://www.garlic.com/~lynn/2004.html#14 Holee shit! 30 years ago!
http://www.garlic.com/~lynn/2004m.html#22 Lock-free algorithms
http://www.garlic.com/~lynn/2004n.html#55 Integer types for 128-bit
addressing
http://www.garlic.com/~lynn/2004o.html#7 Integer types for 128-bit
addressing
--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/ |
|
|
| Back to top |
|
 |
Bernd Paysan
Guest
|
| Posted:
Thu Dec 30, 2004 2:10 am Post subject:
Re: Athlon cache question. |
|
|
Dan Koren wrote:
| Quote: | I don't mean to spoil your fun, but there is a lot
of empirical evidence that frequency of reference
based replacement strategies tend to outperform
pure LRU (whether local or global).
|
I remember a discussion on the Linux kernel mailing list several years
ago, where I proposed something like that for the page aging algorithm.
This started with fork bombs and other memory DOS programs, which tend
to page all other programs out. My point was that despite these new
pages all have been recently accessed, they don't have much value. The
resident and well-used pages of the other programs deserve to stay in
memory. LRU just makes sure that the fork bomb swaps out everything
else as fast as possible, making it impossible to intervene. I had a
strong feeling that the Linux developers back then didn't understand
what I said.
One problem with frequency of reference aging is that you need to keep
the statistics longer than the items (pages, lines). An item that is
thrown out and soon reloaded deserves to keep the priority. Access
frequency would be age/accesses; perhaps a digital filter on the
accesses would provide even more useful data.
--
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://www.jwdt.com/~paysan/ |
|
| Back to top |
|
|
Anne & Lynn Wheeler
Guest
|
| Posted:
Thu Dec 30, 2004 2:35 am Post subject:
Re: Athlon cache question. |
|
|
"Dan Koren" <dankoren@yahoo.com> writes:
| Quote: | Sorry to top post. I do not want to force people to
read your entire post before they get to the point.
I don't mean to spoil your fun, but there is a lot
of empirical evidence that frequency of reference
based replacement strategies tend to outperform
pure LRU (whether local or global).
|
there was acm paper from somebody at brown univsity in the early 70s
proposing a per page hardware reference counter ... but the studies of
70s (which was the statement somebody made in the original post that i
was responding to) was primarily focused on the practical hardware
that was available at the time. It wasn't about having fun ... it was
about responding to somebody's comments about cache studies in the 70s
at a certain company.
the very late 70s saw some looking more complex solutions as it looked
like more complex hardware might be available ... a lot of it being
done with some form of the vs/repack technology that we had done at
cambridge
http://www.garlic.com/~lynn/subtopic.html#545tech
multics had an acm paper in the mid-70s about having 1-4bits of page
reference information ... but it was orientated towards extending the
lemgth of history .... i.e. more like shift register with one bit per
cycle of the global LRU clock algorithm (as opposed to count of
references). i did mention in the early 70s coming up with variation
on standard global "wsclock" algorithm that would beat true/strick
global LRU (because even with small memories, short residencies, and
short term history, there was ways of beating global LRU).
the late 70s found some more detailed studies, many of them using the
product released from the science center, which in its product version
was primarily focused on re-organizing programs for better virtual
memory characteristics ... but the detailed i & d traces were also
used for other kinds of analysis (yes we could break out i & d
information as well as changed & non-changed references).
going into the late 70s ... real storages were starting to get so big
that virtual page lifetimes in real storage was far exceeding the
recent reference history information that the earlier replacement
algorithms had been oriented towards; in fact, the whole system
resource balance was starting to significantly change ... I was making
the comment that the relative disk system performance had declined by
a factor of ten times over a period of 10-15 years. this initially
upset the disk division so much that they assigned their performance
organization to refute the statement. after a couple months they
managed to come up with the fact that i had slightly understated the
problem. this eventually turned into a presentation made to the share
user group on recommendations to improve disk thruput.
the issue then for real storage management was using the excess
capacity to offset the bottlenecks with disks ... i.e. more use of
real storage for record caching, paging done in much larger blocks (in
the 3380 case, transfer rate had increased by a factor of ten while
arm access only increased by 3-4; ... so it was possible to do much
larger transfers; possibly wasting real storage and transfer capacity
.... if you could reduce the number of transfers).
as part of the resource manager in the mid-70s ... I had included done
some tweaks to reduce cache thrashing under heavy load as well as some
slight of hand in the smp support ... to improve cache affinity.
the "big" change in cache stuff made in that time-period was operating
system changes for 3081. went thru the operating system and very
carefully aligned kernel (and various other structures) on cache
boundaries. there was stuff to get buffers aligned on cache boundaries
and allocated in multiples of cache lines ... and attempts to make
sure that different data structures that might be concurrently
accessed by different processors didn't co-reside in the same cache
line.
i had helped with the vs/repack stuff when i was at the cambridge
science center ... and when i went out to SJR ... i did some work on a
disk record trace. it was here that we really started to see effects
of much more complicated longer term history ... than was useful in
the smaller real memory & shorter lifetimes of the 60s and early 70s.
it was initially used to establish that for a given amount real
storage ... that a global cache was nearly always better than local
caches aka ran detailed simulation with huge amounts of live trace
data where there was a fixed amount of electronic cache ... and it
could either be used for a single global cache ... or partitioned
amoung the channels, or finer partitioning amoung the disk controllers
.... or even much finer partitioning out to individual disks. This
somewhat validated the earlier GLOBAL vis-a-vis LOCAL observations
.... regardless of the exact pattern used for line management in the
cache.
with various enhancements to the disk record tracing in the early 80s
we started to see much longer lifetimes histories and started to
identify other types of longer term reference pattern information (and
program and applications "pages" were just another type of disk
record). the short period access patterns involving LRU from the 60s &
70s ... starting looking more & more like background noise compared to
the longer period and more complex real world use patterns that we
were starting to identify.
minor references to the resource manager interrupt window that I
played with to reduce cache thrashing (aka the system would go from
completely asyncronous, anarchy I/O interrupts which could cause a lot
of cache thrashing to very controlled mechanism for periodicly
handling i/o interrupts)
http://www.garlic.com/~lynn/2002l.html#25 Do any architectures use instruction count instead of timer
http://www.garlic.com/~lynn/2002n.html#29 why does wait state exist?
http://www.garlic.com/~lynn/2004f.html#21 Infiniband - practicalities for small clusters
http://www.garlic.com/~lynn/2004q.html#18 PR/SM Dynamic Time Slice calculation
random past references to disk cache work and studies:
http://www.garlic.com/~lynn/94.html#9 talk to your I/O cache
http://www.garlic.com/~lynn/94.html#13 talk to your I/O cache
http://www.garlic.com/~lynn/98.html#6 OS with no distinction between RAM and HD ?
http://www.garlic.com/~lynn/2000d.html#11 4341 was "Is a VAX a mainframe?"
http://www.garlic.com/~lynn/2000d.html#13 4341 was "Is a VAX a mainframe?"
http://www.garlic.com/~lynn/2001c.html#74 database (or b-tree) page sizes
http://www.garlic.com/~lynn/2001.html#18 Disk caching and file systems. Disk history...people forget
http://www.garlic.com/~lynn/2001.html#40 Disk drive behavior
http://www.garlic.com/~lynn/2001l.html#55 mainframe question
http://www.garlic.com/~lynn/2001n.html#79 a.f.c history checkup... (was What specifications will the standard year 2001 PC have?)
http://www.garlic.com/~lynn/2002b.html#16 hollow files in unix filesystems?
http://www.garlic.com/~lynn/2002b.html#20 index searching
http://www.garlic.com/~lynn/2002d.html#55 Storage Virtualization
http://www.garlic.com/~lynn/2002f.html#1 Blade architectures
http://www.garlic.com/~lynn/2002n.html#50 EXCP
http://www.garlic.com/~lynn/2003f.html#5 Alpha performance, why?
http://www.garlic.com/~lynn/2003f.html#9 Alpha performance, why?
http://www.garlic.com/~lynn/2003h.html#6 IBM says AMD dead in 5yrs ... -- Microsoft Monopoly vs. IBM
http://www.garlic.com/~lynn/2003j.html#7 A few Z990 Gee-Wiz stats
http://www.garlic.com/~lynn/2003j.html#65 Cost of Message Passing ?
http://www.garlic.com/~lynn/2004b.html#54 origin of the UNIX dd command
http://www.garlic.com/~lynn/2004e.html#25 Relational Model and Search Engines?
http://www.garlic.com/~lynn/2004g.html#17 Infiniband - practicalities for small clusters
http://www.garlic.com/~lynn/2004g.html#18 Infiniband - practicalities for small clusters
http://www.garlic.com/~lynn/2004g.html#21 Infiniband - practicalities for small clusters
http://www.garlic.com/~lynn/2004i.html#0 Hard disk architecture: are outer cylinders still faster than
http://www.garlic.com/~lynn/2004i.html#1 Hard disk architecture: are outer cylinders still faster than inner cylinders?
http://www.garlic.com/~lynn/2004k.html#41 Vintage computers are better than modern crap !
http://www.garlic.com/~lynn/2004l.html#36 FW: Looking for Disk Calc program/Exec
http://www.garlic.com/~lynn/2004o.html#8 Integer types for 128-bit addressing
http://www.garlic.com/~lynn/2004q.html#70 CAS and LL/SC
--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/ |
|
| Back to top |
|
|
Anne & Lynn Wheeler
Guest
|
| Posted:
Thu Dec 30, 2004 2:40 am Post subject:
Re: Athlon cache question. |
|
|
Bernd Paysan <bernd.paysan@gmx.de> writes:
| Quote: | One problem with frequency of reference aging is that you need to
keep the statistics longer than the items (pages, lines). An item
that is thrown out and soon reloaded deserves to keep the
priority. Access frequency would be age/accesses; perhaps a digital
filter on the accesses would provide even more useful data.
|
in some respects that is what my slight-of-hand trick did ... where
the standard LRU-approximation algorithms all tend to emulate LRU and
degenerate to FIFO under various conditions (preserving recent
reference history regardless of whether it was useful or not) ... the
slight-of-hand trick would effectively start ignoring strict recent
reference history when it wasn't helping.
i had done the original global LRU stuff in the 60s while i was an
undergraduate ... and it was incorporated and shipped in products. in
the early '70s while i was at the science center ... i came up with
this efficient slight-of-hand to effectively ignore recent reference
history information when it wasn't being much use.
--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/ |
|
| Back to top |
|
|
Dan Koren
Guest
|
| Posted:
Thu Dec 30, 2004 4:35 am Post subject:
Re: Athlon cache question. |
|
|
"Bernd Paysan" <bernd.paysan@gmx.de> wrote in message
news:23jba2-ivd.ln1@vimes.paysan.nom...
| Quote: | Dan Koren wrote:
I don't mean to spoil your fun, but there is a lot
of empirical evidence that frequency of reference
based replacement strategies tend to outperform
pure LRU (whether local or global).
I remember a discussion on the Linux kernel mailing list several years
ago, where I proposed something like that for the page aging algorithm.
This started with fork bombs and other memory DOS programs, which tend
to page all other programs out. My point was that despite these new
pages all have been recently accessed, they don't have much value.
|
Not to offend anyone, but I would not turn to OS developers
for deep knowledge about paging. The truth of the matter is
that most OS's do only a so-so job at best when it comes to
virtual memory. The database community has more experience
and is far more knowledgeable about caching and paging than
the OS community.
The
| Quote: | resident and well-used pages of the other programs deserve to stay in
memory. LRU just makes sure that the fork bomb swaps out everything
else as fast as possible, making it impossible to intervene. I had a
strong feeling that the Linux developers back then didn't understand
what I said.
|
The Linux community has done an absolutely outstanding job
of replacing the proverbial NIH (not invented here) attitude
by NIBOOU (not invented by one of us). If you're not buddies
with Linus, Alan Cox, David Miller or Stephen Tweedie, no one
will pay any attention to anything you might say. I found the
only reasonable person in the entire Linux cast of characters
is Ted T'so -- which is probably why we don't hear very often
from or about him.
| Quote: | One problem with frequency of reference aging is that you
need to keep the statistics longer than the items (pages,
lines).
|
Which is why FR techniques are more applicable to software
based paging (OS, databases, file system) than to hardware
caches. We should clearly distinguish the two subjects, and
split the discussion accordingly. On a theoretical and model
level the two may have a lot in common. In practice, they do
not, since the sets of constraints are quite different.
| Quote: | An item that is thrown out and soon reloaded deserves to
keep the priority. Access frequency would be age/accesses;
perhaps a digital filter on the accesses would provide even
more useful data.
|
Has been done. It is one of the standard techniques in web
caching.
dk |
|
| Back to top |
|
|
Anne & Lynn Wheeler
Guest
|
| Posted:
Thu Dec 30, 2004 5:00 am Post subject:
Re: Athlon cache question. |
|
|
somewhat random aside ... we had the disk record cache trace stuff
installed at sjr ... so we got to include a lot of system/r and later
rdbms research stuff ... random refs:
http://www.garlic.com/~lynn/subtopic.html#systemr
..... at longer period history than they were doing themselves (besides
just run of the mill data center use). we also got it installed at stl
.... which was another type of development house ... lots of ims
database development as well as use ... and then later db2 development
and use ... course stl was doing other types of development
.... languages like apl and pli and misc. other things. we also got
some of the machines that were doing the disk division commercial data
processing for the operation of the business ... so there was a
variety of data processing type things ... some with cyclic daily,
weekly and monthly operation (database, vsam, flat files, etc).
we were also able to get long term use traces out of some number of
other sites around the silicon valley area,
--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/ |
|
| Back to top |
|
|
Dan Koren
Guest
|
| Posted:
Thu Dec 30, 2004 5:11 am Post subject:
Re: Athlon cache question. |
|
|
"Dan Koren" <dankoren@yahoo.com> wrote in message
news:41d33f59$1@news.meer.net...
| Quote: | "Bernd Paysan" <bernd.paysan@gmx.de> wrote in message
news:23jba2-ivd.ln1@vimes.paysan.nom...
An item that is thrown out and soon reloaded deserves to
keep the priority. Access frequency would be age/accesses;
perhaps a digital filter on the accesses would provide even
more useful data.
Has been done. It is one of the standard techniques in web
caching.
|
On second thought, I'm wondering if I misunderstood what
you wrote.
What exactly did you have in mind when you wrote "digital
filtering"? Something like Bloom filters or some kind of
(partial) thumbprint?
Or did you have in mind DSP style smoothing filters? If
so, what would be the metric that is smoothed, and how
would the result be used in determining which pages to
page in or out? Also, how would this avoid the need to
store per page counters?
dk |
|
| Back to top |
|
|
Greg Lindahl
Guest
|
| Posted:
Thu Dec 30, 2004 2:12 pm Post subject:
Re: Athlon cache question. |
|
|
In article <23jba2-ivd.ln1@vimes.paysan.nom>,
Bernd Paysan <bernd.paysan@gmx.de> wrote:
| Quote: | LRU just makes sure that the fork bomb swaps out everything
else as fast as possible, making it impossible to intervene. I had a
strong feeling that the Linux developers back then didn't understand
what I said.
|
Maybe you forgot to explain why the entire paging design should be
screwed up to improve execution of a fork bomb? Thinking about corner
cases can be fun, but it isn't obvious that it's the most important
issue to think about.
| Quote: | One problem with frequency of reference aging is that you need to keep
the statistics longer than the items (pages, lines). An item that is
thrown out and soon reloaded deserves to keep the priority.
|
That's an assertion; what's the proof? And is this a *significant*
problem with pages, where keeping such information isn't hard, just a
bit tedious?
-- greg |
|
| Back to top |
|
|
glen herrmannsfeldt
Guest
|
| Posted:
Sun Jan 02, 2005 11:04 pm Post subject:
Re: Athlon cache question. |
|
|
Anne & Lynn Wheeler wrote:
(snip)
| Quote: | the problem was that the thesis was basically on global LRU
replacement strategy and stanford was getting a lot of pushback from
strong advocate of local LRU replacement.
in the late 60s there was some academic literature on local LRU
replacement and working sets. at that time i was an undergraduate and
doing lots of operating system changes ... including having come up
with this global LRU idea, implemented it and it had shipped in
products.
the problem at hand was to show global LRU replacement significantly
better than local LRU.
|
(snip)
I would think it would depend on the workload, and also the
expectation of the users. In a case where you have mixed
long and short term jobs, where the long term jobs are supposed
to have lower priority, I would have thought that global would
not work so well. That is, everyone is not equal, but global
assumes that they are.
In the case where all users are to be treated equally, my
guess would be that global would be better.
-- glen |
|
| Back to top |
|
|
Anne & Lynn Wheeler
Guest
|
| Posted:
Sun Jan 02, 2005 11:38 pm Post subject:
Re: Athlon cache question. |
|
|
glen herrmannsfeldt <gah@ugcs.caltech.edu> writes:
| Quote: | I would think it would depend on the workload, and also the
expectation of the users. In a case where you have mixed
long and short term jobs, where the long term jobs are supposed
to have lower priority, I would have thought that global would
not work so well. That is, everyone is not equal, but global
assumes that they are.
In the case where all users are to be treated equally, my
guess would be that global would be better.
|
in general, global ... regardless of cache type, processor, paging,
file, etc ... attempts to keep the most highly used information around
for the overall system thruput. local tends to perform less well since
it doesn't optimize for the overall system performance ... resulting
in keeping around lower-use pages for one process while kicking out
higher-user pages for another process.
The issue of LRU is whether or not recent acdess patterns accurately
predict future access patterns (but that is somewhat independent of
the global/local issue).
so another way of viewing it is the impact on overall system
performance (cpu stall) of not having a specific page compared to not
having another specific page.
so long ago, i did this policy based resource manager ... and a lot of
customers started calling it the fair share scheduler because the
default policty was fair share. processes using less then their target
(possibly target as defined as fair share) got better dispatching
priority ... so that they ran faster (at least until they caught up in
their resource utilization objective).
The issue on total system thruput isn't so much dispatching priority
.... but total resource thruput objectives. If you have a process that
is supposed to get 80 percent of the processor ... and it is having
frequent page faults ... then the impact on the overall system is much
worse than a process that is supposed to get 5 percent of the
processor (having the same page fault rate).
Some number of strategies were developed to directly address page
fault rate of processess that deemed critical (supposed to get some
significant resources) and behind schedule. One such stratigy was to
run the standard global replacement ... but sporadically
bypass/skip-over selected pages for process in such situation. There
effectively was a global ordering of pages based on projected overall
benefit to the system .... but (in effect) pages belonging to
behind-schedule critical processess could be given extra points).
Part of the issue wasn't strictly page-fault rate ... is was the
combination of relative page-fault rate and page-fault service time
.... resulting in total non-executable time for a specific process
.... and to what extent that total page-fault related non-executable
time was contributing to the process not meeting the established
resource utilization objective.
this is all predicated that some existing used virtual page in any way
relates to future use of that same virtual page. there are the edge
and pathelogical cases where there is no virtual page re-use and
caching provide no benefit what-so-ever.
--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/ |
|
| Back to top |
|
|
Anne & Lynn Wheeler
Guest
|
| Posted:
Mon Jan 03, 2005 2:13 am Post subject:
Re: Athlon cache question. |
|
|
the other slightly related access pattern "cache" management is more
associated with (longer term) file access ... although the basic
principle applies to all kinds of caches.
i mentioned having done the disk record access trace (highly
optimized, capable of being used in high-thruput production
environments) ... first for modeling lots of different file behavior
(system file caches, controller caches, disk device caches, etc)
.... and then later looking at using the information as part of
standard production system operation
http://www.garlic.com/~lynn/2004q.html#76 Athlon cache question
http://www.garlic.com/~lynn/2004q.html#79 Athlon cache question
i had also done the implementation for an internal backup/archive
system that was used in a lot of internal locations (especially
in the silicon valley area) ... which went thru a few internal
releases ... and then became available as workstation datasave ...
morphing into adsm and currently tsm
http://www.garlic.com/~lynn/subtopic.html#backup
there have been a couple of infrastructures that attempt to manage the
disk storage space as file cache ... HSM (hierarchical storage
manager), SMS (system managed storage), and ADSM/TSM.
In the vs/repack traces and program re-ordering ... it attempted to
pack things into the same page(s) that tended to be used together
.... this frequently would take large, relatively weak working sets and
turn them into much more compact, stronger working sets.
the file storage stuff has coming up with similar constructus ... i
think in TSM they are currently referred to as something like
containers ... collection of files that will tend to be migrated
together ... because the indications are that they tend to be used
together.
--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/ |
|
| Back to top |
|
|
Gene Wirchenko
Guest
|
| Posted:
Sat Jan 22, 2005 9:28 pm Post subject:
Re: Athlon cache question. |
|
|
"Dan Koren" <dankoren@yahoo.com> wrote:
| Quote: | Sorry to top post. I do not want to force people to
read your entire post before they get to the point.
|
Surely you have heard of trimming?
[snip]
Like that.
Sincerely,
Gene Wirchenko
Computerese Irregular Verb Conjugation:
I have preferences.
You have biases.
He/She has prejudices.
----== Posted via Newsfeeds.Com - Unlimited-Uncensored-Secure Usenet News==----
http://www.newsfeeds.com The #1 Newsgroup Service in the World! >100,000 Newsgroups
---= East/West-Coast Server Farms - Total Privacy via Encryption =--- |
|
| Back to top |
|
|
|
|
|
|
FAQ
Memberlist
Register
Profile
Log in to check your private messages
Log in