| Author |
Message |
ssubbarayan
Guest
|
 Posted:
Wed Sep 28, 2005 8:15 am Post subject:
Regarding calculation of free memory |
|
|
Gurus,
I was just wondering what could be the best possible way to calculate
free memory while our code is running in SDRAM.I have used vxworks and
it uses a approach to calculate the free memory as follows:
Fill up the entire stack with 0xeeeee and then when you start using the
stack,if at all a particular address in memory is used,the value of
0xeeeee in that location should be overwritten with real data.So if we
are able to track the first occurance of the pattern 0xeeeee we should
be able to subtract it from the entire size of memory and make a fair
assesment of the available free memory.
How ever I beleive such a algorithm has drawbacks.First drawback which
I can think is
1)What is the guarentee that the real data is not ur pattern chosen(In
vxworks case,its 0xeeeee,if 0xeeeee itself is real data,we cant find
out real size!)?
2)What should you look for to identify unique pattern?Or other way how
would you arrive at the pattern word to be used?
3)Are there any other good techniques for calculating free memory?
4)We are making software for a consumer electronics appliance.It uses a
premitive propreitory RTOS which does not give enough tools to find out
free memory.We are in a forced situation to reduce memory
consumption,so we have to get a way to calculate free available memory
first.To add to the complexity,our application uses
heap,partitions,partition with in partitions.At any point of time we
have to give the user memory utilised like most RTOS tools
provide.Since Heap is dynamic how do u arrive at a fair judgment?
If we wanted to get total free space,we need to calculate free space in
heap,free space in partition and then arrive at it.
Are any one aware of any good way to achieve it in the given case of
RTOS vendor not providing tools for memory calculations?
Looking farward for all your replys and Advanced thanks for the same.
Regards,
s.subbarayan |
|
| Back to top |
|
 |
Guest
|
| Posted:
Wed Sep 28, 2005 9:15 pm Post subject:
Re: Regarding calculation of free memory |
|
|
| Quote: | I was just wondering what could be the best possible way to calculate
free memory while our code is running in SDRAM.I have used vxworks and
it uses a approach to calculate the free memory as follows:
Fill up the entire stack with 0xeeeee and then when you start using the
stack,if at all a particular address in memory is used,the value of
0xeeeee in that location should be overwritten with real data.So if we
are able to track the first occurance of the pattern 0xeeeee we should
be able to subtract it from the entire size of memory and make a fair
assesment of the available free memory.
|
It sounds like your 0xeeeee method you outline is perfectly suited for
your purposes. Even if 0xeeeee is actually part of your real data that
happens to be in the stack, consider that it's highly unlikely that
0xeeeee will fill your stack to the extent that your stack usage figure
is way off. The point is, even if 0xeeeee is real stack data, the
information you get from hunting for unused stack will be close enough.
Get your usage total and add some margin for safety and you're done.
| Quote: | How ever I beleive such a algorithm has drawbacks.First drawback which
I can think is
1)What is the guarentee that the real data is not ur pattern chosen(In
vxworks case,its 0xeeeee,if 0xeeeee itself is real data,we cant find
out real size!)?
|
That's application specific. Look at your operating scenarios. Ask
yourself if the pattern you've selected is likely to be real data or
not.
| Quote: | 2)What should you look for to identify unique pattern?Or other way how
would you arrive at the pattern word to be used?
|
See #1 reply.
| Quote: | 3)Are there any other good techniques for calculating free memory?
|
Don't know but... This method is commonly used and usually works very
well. A good white-box test designed to stress the stack usage will
give you the real info you need to accurately size your RAM.
| Quote: | 4)We are making software for a consumer electronics appliance.It uses a
premitive propreitory RTOS which does not give enough tools to find out
free memory.We are in a forced situation to reduce memory
consumption,so we have to get a way to calculate free available memory
first.To add to the complexity,our application uses
heap,partitions,partition with in partitions.At any point of time we
have to give the user memory utilised like most RTOS tools
provide.Since Heap is dynamic how do u arrive at a fair judgment?
If we wanted to get total free space,we need to calculate free space in
heap,free space in partition and then arrive at it.
|
IMO, it's like gambling when you size how much stack margin you want to
add. If you believe your SW tests are "NASA grade" (nearly
exhaustive), you will not need to add much stack margin. If you don't
do SW testing, it's a roll of the dice. You probably fit in the middle
of the two extremes.
| Quote: | Are any one aware of any good way to achieve it in the given case of
RTOS vendor not providing tools for memory calculations?
|
Good way is the way you outlined for run-time stack checking. Maybe
some CPUs feature some sort of built-in high water mark stack pointer
register. That would be nice but I don't know if it exists.
JJS |
|
| Back to top |
|
|
John Devereux
Guest
|
| Posted:
Wed Sep 28, 2005 9:56 pm Post subject:
Re: Regarding calculation of free memory |
|
|
"ssubbarayan" <ssubba@gmail.com> writes:
| Quote: | Gurus,
I was just wondering what could be the best possible way to calculate
free memory while our code is running in SDRAM.I have used vxworks and
it uses a approach to calculate the free memory as follows:
Fill up the entire stack with 0xeeeee and then when you start using the
stack,if at all a particular address in memory is used,the value of
0xeeeee in that location should be overwritten with real data.So if we
are able to track the first occurance of the pattern 0xeeeee we should
be able to subtract it from the entire size of memory and make a fair
assesment of the available free memory.
How ever I beleive such a algorithm has drawbacks.First drawback which
I can think is
1)What is the guarentee that the real data is not ur pattern chosen(In
vxworks case,its 0xeeeee,if 0xeeeee itself is real data,we cant find
out real size!)?
2)What should you look for to identify unique pattern?Or other way how
would you arrive at the pattern word to be used?
|
<SNIP>
I believe the traditional value is "0xDEADBEEF" :)
--
John Devereux |
|
| Back to top |
|
|
Joe Pfeiffer
Guest
|
| Posted:
Wed Sep 28, 2005 10:05 pm Post subject:
Re: Regarding calculation of free memory |
|
|
John Devereux <jdREMOVE@THISdevereux.me.uk> writes:
| Quote: | "ssubbarayan" <ssubba@gmail.com> writes:
Gurus,
I was just wondering what could be the best possible way to calculate
free memory while our code is running in SDRAM.I have used vxworks and
it uses a approach to calculate the free memory as follows:
Fill up the entire stack with 0xeeeee and then when you start using the
stack,if at all a particular address in memory is used,the value of
0xeeeee in that location should be overwritten with real data.So if we
are able to track the first occurance of the pattern 0xeeeee we should
be able to subtract it from the entire size of memory and make a fair
assesment of the available free memory.
How ever I beleive such a algorithm has drawbacks.First drawback which
I can think is
1)What is the guarentee that the real data is not ur pattern chosen(In
vxworks case,its 0xeeeee,if 0xeeeee itself is real data,we cant find
out real size!)?
2)What should you look for to identify unique pattern?Or other way how
would you arrive at the pattern word to be used?
SNIP
I believe the traditional value is "0xDEADBEEF" :)
|
Which has the same problem. But really, if you pick a pattern that's
unlikely to turn up by coincidence, you'll come close enough to the
right answer for all practical purposes. Don't use, say, 00000000,
but beyond that it doesn't much matter.
--
Joseph J. Pfeiffer, Jr., Ph.D. Phone -- (505) 646-1605
Department of Computer Science FAX -- (505) 646-1002
New Mexico State University http://www.cs.nmsu.edu/~pfeiffer
skype: jjpfeifferjr |
|
| Back to top |
|
|
Peter Dickerson
Guest
|
| Posted:
Wed Sep 28, 2005 10:13 pm Post subject:
Re: Regarding calculation of free memory |
|
|
"John Devereux" <jdREMOVE@THISdevereux.me.uk> wrote in message
news:87d5mtb06d.fsf@cordelia.devereux.me.uk...
| Quote: | "ssubbarayan" <ssubba@gmail.com> writes:
Gurus,
I was just wondering what could be the best possible way to calculate
free memory while our code is running in SDRAM.I have used vxworks and
it uses a approach to calculate the free memory as follows:
Fill up the entire stack with 0xeeeee and then when you start using the
stack,if at all a particular address in memory is used,the value of
0xeeeee in that location should be overwritten with real data.So if we
are able to track the first occurance of the pattern 0xeeeee we should
be able to subtract it from the entire size of memory and make a fair
assesment of the available free memory.
How ever I beleive such a algorithm has drawbacks.First drawback which
I can think is
1)What is the guarentee that the real data is not ur pattern chosen(In
vxworks case,its 0xeeeee,if 0xeeeee itself is real data,we cant find
out real size!)?
2)What should you look for to identify unique pattern?Or other way how
would you arrive at the pattern word to be used?
SNIP
I believe the traditional value is "0xDEADBEEF" :)
|
There is also a slight advantage to using this value because it is odd. On
many processors using an odd address for larger than byte size data will
thow an exception. As a result uninitialised data on the stack used as
pointers can be caught more often.
The bigest problem I have though is that I am a vegetarian so I prefer
0xFACEFEED.
Peter |
|
| Back to top |
|
|
Hank Oredson
Guest
|
| Posted:
Thu Sep 29, 2005 12:15 am Post subject:
Re: Regarding calculation of free memory |
|
|
"John Devereux" <jdREMOVE@THISdevereux.me.uk> wrote in message
news:87d5mtb06d.fsf@cordelia.devereux.me.uk...
| Quote: | "ssubbarayan" <ssubba@gmail.com> writes:
Gurus,
I was just wondering what could be the best possible way to calculate
free memory while our code is running in SDRAM.I have used vxworks and
it uses a approach to calculate the free memory as follows:
Fill up the entire stack with 0xeeeee and then when you start using the
stack,if at all a particular address in memory is used,the value of
0xeeeee in that location should be overwritten with real data.So if we
are able to track the first occurance of the pattern 0xeeeee we should
be able to subtract it from the entire size of memory and make a fair
assesment of the available free memory.
How ever I beleive such a algorithm has drawbacks.First drawback which
I can think is
1)What is the guarentee that the real data is not ur pattern chosen(In
vxworks case,its 0xeeeee,if 0xeeeee itself is real data,we cant find
out real size!)?
2)What should you look for to identify unique pattern?Or other way how
would you arrive at the pattern word to be used?
SNIP
I believe the traditional value is "0xDEADBEEF" :)
|
Seen in various 16 bit codes:
#define STACKPAT 0x55aa // Stack fill value for high water mark
checking.
--
... Hank
http://home.earthlink.net/~horedson
http://home.earthlink.net/~w0rli |
|
| Back to top |
|
|
TOUATI Sid
Guest
|
| Posted:
Thu Sep 29, 2005 1:38 pm Post subject:
Re: Regarding calculation of free memory |
|
|
Isn't exploring the stack pointer register sufficient to determine
where is the top of the stack ?
For the heap, maybe the garbage collector of the OS can tell you how
much free memory you have.
S |
|
| Back to top |
|
|
Peter Dickerson
Guest
|
| Posted:
Thu Sep 29, 2005 2:56 pm Post subject:
Re: Regarding calculation of free memory |
|
|
"TOUATI Sid" <touati-nospamplease@nospam-prism.uvsq.fr> wrote in message
news:dhg96p$6ae$1@io.uvsq.fr...
| Quote: | Isn't exploring the stack pointer register sufficient to determine
where is the top of the stack ?
For the heap, maybe the garbage collector of the OS can tell you how
much free memory you have.
|
It tells you where it is but doesn't tell you where it has been. Usually you
want to know the max stack that is used by the app not how much stack is
curently being used.
Peter |
|
| Back to top |
|
|
TOUATI Sid
Guest
|
| Posted:
Thu Sep 29, 2005 4:15 pm Post subject:
Re: Regarding calculation of free memory |
|
|
| Quote: | It tells you where it is but doesn't tell you where it has been. Usually you
want to know the max stack that is used by the app not how much stack is
curently being used.
Peter
|
In this case, maybe I suggest to just instrument the code : insert some
instructions after each "pop" to memorize the maximal value of the stack
register.
For the heap, insert some instructions after each malloc to check the
memorize heap size (if there is any OS service for that, otherwise
create it by hacking linux sources )
This would be better than filling the stack with an arbitrary value in
order to determine any erased data ; the conception of such method does
not provide any guarantee to work.
S |
|
| Back to top |
|
|
ssubbarayan
Guest
|
| Posted:
Fri Sep 30, 2005 8:15 am Post subject:
Re: Regarding calculation of free memory |
|
|
TOUATI Sid wrote:
| Quote: | It tells you where it is but doesn't tell you where it has been. Usually you
want to know the max stack that is used by the app not how much stack is
curently being used.
Peter
In this case, maybe I suggest to just instrument the code : insert some
instructions after each "pop" to memorize the maximal value of the stack
register.
For the heap, insert some instructions after each malloc to check the
memorize heap size (if there is any OS service for that, otherwise
create it by hacking linux sources )
This would be better than filling the stack with an arbitrary value in
order to determine any erased data ; the conception of such method does
not provide any guarantee to work.
S
|
Touti,
Would it not really create latency if u try to put some instructions
after "POP" to memorize the maximal value of stack register?
Further I believe though really not well aware,adding your own
instruction would
take up more memory incase the calculations you are going to make is
huge.Now given that case,if you keep on doing that for every pop,then I
believe you need to do some subtractions for the calculation operations
each time.In turn that would need more time!
How can I handle this in proper way?
Regards,
s.subbarayan |
|
| Back to top |
|
|
Basile Starynkevitch [new
Guest
|
| Posted:
Sun Oct 02, 2005 10:25 am Post subject:
Re: Regarding calculation of free memory |
|
|
On 2005-10-02, Nick Maclaren <nmm1@cus.cam.ac.uk> wrote:
| Quote: |
extreme_pedant_mode=ON
[....]
The whole area of memory management schemes is sadly neglected, and
much of garbage collection work is based on unrealistic assumptions,
so much of what is currently believed isn't so.
extreme_pedant_mode=OFF
|
While I tend to agree with the above position, I'm not sure at all to
understand what you mean (you might be thinking the opposite of my
guess).
Do you mean that some GC or memory management is based on unrealistic
assumption, in the sense that other or better GC (I'm thinking of
generational copying ones vs referenc counting) techniques should be
more widely used, or do you mean that GC never really works and that
on the contrary programs should only use static (à la Fortran77) or
manual (à la C++) memory management?
I think that old Appel's paper "garbage collection can be faster than
stack allocation" does provide valuable insight, even if the actual
figures may not be true today (because of cache issues, etc..)
Regards
--
Basile STARYNKEVITCH http://starynkevitch.net/Basile/
email: basile(at)starynkevitch(dot)net
8, rue de la Faïencerie, 92340 Bourg La Reine, France |
|
| Back to top |
|
|
Steve at fivetrees
Guest
|
| Posted:
Sun Oct 02, 2005 1:59 pm Post subject:
Re: Regarding calculation of free memory |
|
|
"ssubbarayan" <ssubba@gmail.com> wrote in message
news:1127886917.777311.119870@z14g2000cwz.googlegroups.com...
| Quote: | Gurus,
I was just wondering what could be the best possible way to calculate
free memory while our code is running in SDRAM.
|
<snip>
| Quote: | 4)We are making software for a consumer electronics appliance.It uses a
premitive propreitory RTOS which does not give enough tools to find out
free memory.We are in a forced situation to reduce memory
consumption,so we have to get a way to calculate free available memory
first.To add to the complexity,our application uses
heap,partitions,partition with in partitions.At any point of time we
have to give the user memory utilised like most RTOS tools
provide.Since Heap is dynamic how do u arrive at a fair judgment?
|
<pedant_mode=ON>
If you want your appliance to be totally reliable, avoid using a heap at
all. Free memory calculation then becomes rather simple [1]. Logic: heaps
are (usually) non-deterministic and *will* cause eventual malloc failures
due to fragmentation [2].
[1] Excluding stack use, but that's not too hard either.
[2] Unless you allocate all memory at startup; a common trick. But then the
linker could do the same...
<pedant_mode=OFF>
Steve
http://www.fivetrees.com |
|
| Back to top |
|
|
Nick Maclaren
Guest
|
| Posted:
Sun Oct 02, 2005 3:08 pm Post subject:
Re: Regarding calculation of free memory |
|
|
In article <T7adnWGND7X4PKLeRVnyiw@pipex.net>,
Steve at fivetrees <steve@NOSPAMTAfivetrees.com> wrote:
| Quote: |
pedant_mode=ON
If you want your appliance to be totally reliable, avoid using a heap at
all. Free memory calculation then becomes rather simple [1]. Logic: heaps
are (usually) non-deterministic and *will* cause eventual malloc failures
due to fragmentation [2].
[1] Excluding stack use, but that's not too hard either.
[2] Unless you allocate all memory at startup; a common trick. But then the
linker could do the same...
|
< extreme_pedant_mode=ON >
Not so. In Fortran 77, it was possible to allocate all memory statically,
but it isn't generally possible. It can't be done in Fortran 90 or C.
Also, of the memory management schemes that are lumped under the term of
'heap' ones, there are several that have deterministic behaviour and do
not suffer from fragmentation. The simplest one is if all allocations
are of the same size, and chains are used instead of arrays.
The whole area of memory management schemes is sadly neglected, and
much of garbage collection work is based on unrealistic assumptions,
so much of what is currently believed isn't so.
< extreme_pedant_mode=OFF >
Regards,
Nick Maclaren. |
|
| Back to top |
|
|
Steve at fivetrees
Guest
|
| Posted:
Sun Oct 02, 2005 3:21 pm Post subject:
Re: Regarding calculation of free memory |
|
|
"Nick Maclaren" <nmm1@cus.cam.ac.uk> wrote in message
news:dhobjl$e2v$1@gemini.csx.cam.ac.uk...
| Quote: | extreme_pedant_mode=ON
Not so. In Fortran 77, it was possible to allocate all memory statically,
but it isn't generally possible. It can't be done in Fortran 90 or C.
|
<even_more_extreme_pedant_mode=ON>
It *can* be done in C - by avoiding malloc ;).
<even_more_extreme_pedant_mode=OFF>
| Quote: | Also, of the memory management schemes that are lumped under the term of
'heap' ones, there are several that have deterministic behaviour and do
not suffer from fragmentation. The simplest one is if all allocations
are of the same size, and chains are used instead of arrays.
|
Agreed. I've used techniques such as these (effectively an application-level
memory manager) in certain situations where extreme reliability was
required, but the platform insisted on the use of its memory manager...
<sigh>...
| Quote: | The whole area of memory management schemes is sadly neglected, and
much of garbage collection work is based on unrealistic assumptions,
so much of what is currently believed isn't so.
|
Completely agreed. I've often fantasized about a hardware memory manager
that remaps blocks in such a way that fragmentation cannot occur. OTOH, I've
also fantasized that one day, a majority will write good code that doesn't
leak memory and traps malloc errors properly...
Steve
http://www.fivetrees.com |
|
| Back to top |
|
|
David Hopwood
Guest
|
| Posted:
Sun Oct 02, 2005 4:15 pm Post subject:
Re: Regarding calculation of free memory |
|
|
Nick Maclaren wrote:
| Quote: | Also, of the memory management schemes that are lumped under the term of
'heap' ones, there are several that have deterministic behaviour and do
not suffer from fragmentation. The simplest one is if all allocations
are of the same size, and chains are used instead of arrays.
The whole area of memory management schemes is sadly neglected, and
much of garbage collection work is based on unrealistic assumptions,
so much of what is currently believed isn't so.
|
That's a worthlessly vague statement. As far as I can see, the problem isn't
lack of research; it's lack of application of that research in the most
commonly used systems.
--
David Hopwood <david.nospam.hopwood@blueyonder.co.uk> |
|
| Back to top |
|
|
|
|
|
|
FAQ
Memberlist
Register
Profile
Log in to check your private messages
Log in