CASTalk.com

Andy Glew <andy.glew@intel.com> wrote:
+---------------
| If I was really nice I would show you a sample memory map. But I'm
| lazy, and am afraid that I would embarass myself. But that's what you
| need to look for - the so-called memory map, the virtual address map,
| of a typical process on your OS.
|
| I expect somebody will post the current address maps on their favorite
| OS, and I'll learn something.
+---------------

On 32-bit FreeBSD:

$ cat /proc/curproc/map # "cat" itself
0x8048000 0x8057000 15 16 0xdf4cc4ac r-x 2 1 0x0 COW NC vnode
0x8057000 0x8058000 1 0 0xdf6c7730 rw- 1 0 0x2180 COW NNC vnode
0x8058000 0x805a000 2 0 0xdf6d0508 rw- 2 0 0x2180 NCOW NNC default
0x805a000 0x805c000 2 0 0xdf6d0508 rwx 2 0 0x2180 NCOW NNC default
0x28057000 0x28058000 1 0 0xdf4ca114 rwx 1 0 0x2180 NCOW NNC default
0xbfbe0000 0xbfc00000 2 0 0xdf6d32e0 rwx 1 0 0x2180 NCOW NNC default

$ cat /proc/232/map # a Common Lisp process (CMUCL)
0x8048000 0x805e000 22 0 0xdf4cc05c r-x 2 0 0x0 COW NC vnode
0x805e000 0x8069000 7 0 0xdf4ca33c rw- 2 0 0x2180 NCOW NNC default
0x8069000 0x81ed000 388 0 0xdf4ca33c rwx 2 0 0x2180 NCOW NNC default
0x10000000 0x1130f000 2546 0 0xdf4fdbdc rwx 1 0 0x2180 COW NNC vnode
0x1130f000 0x1ffff000 0 0 0xdf4ed114 rwx 1 0 0x2000 NCOW NNC default
0x2805e000 0x28071000 19 0 0xc033250c r-x 101 47 0x0 COW NC vnode
0x28071000 0x28072000 1 0 0xdf4ed398 rw- 1 0 0x2180 COW NNC vnode
0x28072000 0x28074000 2 0 0xdf4ed5c0 rw- 2 0 0x2180 NCOW NNC default
0x28074000 0x2807c000 7 0 0xdf4ed5c0 rwx 2 0 0x2180 NCOW NNC default
0x2807c000 0x28092000 22 0 0xc0334000 r-x 20 12 0x0 COW NC vnode
0x28092000 0x28093000 1 0 0xdf4d7c94 r-x 1 0 0x2180 COW NNC vnode
0x28093000 0x28097000 4 0 0xdf4ec78c rwx 1 0 0x2180 COW NNC vnode
0x28097000 0x28118000 84 0 0xc0332848 r-x 148 94 0x0 COW NC vnode
0x28118000 0x28119000 1 0 0xdf4ed4ac r-x 1 0 0x2180 COW NNC vnode
0x28119000 0x2811e000 5 0 0xdf4ccac8 rwx 1 0 0x2180 COW NNC vnode
0x2811e000 0x2813e000 15 0 0xdf4ed564 rwx 1 0 0x2180 NCOW NNC default
0x28f00000 0x291b3000 691 0 0xdf4ed05c rwx 1 0 0x2180 COW NNC vnode
0x291b3000 0x37fff000 0 0 0xdf4ed508 rwx 1 0 0x2180 NCOW NNC default
0x38000000 0x3ffff000 1 0 0xdf4fa958 rwx 1 0 0x2180 NCOW NNC default
0x40000000 0x40010000 0 0 0xdf4ecac8 r-x 2 0 0x2180 NCOW NNC default
0x40010000 0x47fe2000 11 0 0xdf4ecac8 rwx 2 0 0x2180 NCOW NNC default
0x48000000 0x48001000 1 0 0xdf4ccda8 rwx 1 0 0x2180 COW NNC vnode
0x48001000 0x68000000 3330 0 0xdf4ed0b8 rwx 1 0 0x2180 NCOW NNC default
0xb0000000 0xb0100000 1 0 0xdf4fae60 rwx 1 0 0x2180 NCOW NNC default
0xbfbe0000 0xbfc00000 3 0 0xdf4ece60 rwx 1 0 0x2180 NCOW NNC default
$

On a 64-bit Linux:

$ cat /proc/self/maps # "cat" itself
0000000000400000-0000000000404000 r-xp 0000000000000000 09:01 65643 /bin/cat
0000000000504000-0000000000505000 rw-p 0000000000004000 09:01 65643 /bin/cat
0000000000505000-0000000000526000 rwxp 0000000000000000 00:00 0
0000002a95556000-0000002a9566b000 r-xp 0000000000000000 09:01 524380 /lib64/ld-2.3.2.so
0000002a9566b000-0000002a9566c000 rw-p 0000000000015000 09:01 524380 /lib64/ld-2.3.2.so
0000002a9566c000-0000002a9566d000 rw-p 0000000000000000 00:00 0
0000002a95687000-0000002a957c2000 r-xp 0000000000000000 09:01 196701 /lib64/tls/libc-2.3.2.so
0000002a957c2000-0000002a95887000 ---p 000000000013b000 09:01 196701 /lib64/tls/libc-2.3.2.so
0000002a95887000-0000002a958c7000 rw-p 0000000000100000 09:01 196701 /lib64/tls/libc-2.3.2.so
0000002a958c7000-0000002a958cc000 rw-p 0000000000000000 00:00 0
0000002a958cc000-0000002a97775000 r--p 0000000000000000 09:01 491604 /usr/lib/locale/locale-archive
0000007fbfee9000-0000007fc0000000 rw-p fffffffffff23000 00:00 0
$


-Rob

-----
Rob Warnock <rpw3@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607

Andrew Reilly <andrew-newspost@areilly.bpc-users.org> writes:

Then there are the JVMs, which historically instantiate "activation
records" on the heap, and let the garbage collector clean up the mess
later. This ensures that references to locals returned to callers don't
wind up hanging.

I have my doubts that this would be the reason. I hadn't noticed any
facility in Java or the JVM for returning references to locals... And I
can't see any reason why a JVM would need to heap allocate activation
records (it's not like it's going to make implementation easier, AFAICT).

Apparently this is such a performance loser (stacks have
nice refrerence locality properties that work well with caches) that
recent JVMs do "escape analysis" to catch the hanging return issue and
allocate everything else on a real stack, as the scope syntax might lead
one to expect.

This sounds like a misunderstanding of an optimisation to stack-allocate
objects which provably don't escape the function that allocates them (or,
some "obvious" enclosing function). A typical example in Java is
Enumeration objects.

This is relevant in the context of the OP because there
exists at least one C compiler that targets the JVM, so therefore there is
at least one C compiler that uses heap allocated stacks. I expect that
there are others, though.

Can you name a JVM with heap-allocated activation records?

--
David Gay
dgay@acm.org

Greg Lindahl wrote:

In article <438B6648.F691DA73@sympaticoREMOVE.ca>,
Eric P. <eric_pattison@sympaticoREMOVE.ca> wrote:

However with multi-threading, I think the only solutions are the
between-a-rock-and-a-hard-place types, at least the ones that I
have heard of.

The common Unix solution is to treat the initial stack specially, so
non-threaded programs behave fairly sanely. And that's the case which
was at question here.

That does seem the obviously correct approach but for some bizzare
reason Windows choose not to do that. But there are also many other
problems with how Windows manages thread stacks.

There is no place to linearly grow dynamic stacks,
and if you reserve too much space for each thread stack, you can
run out of space and limit the number of threads.

Apparently you aren't a convert to 64-bit computing.
...

Bludgeon the problem with bits eh? Simple but effective.

Eric

Eric P. wrote:

So you pick a number and Windows picked 1 MB as their default.

Is it one megabyte? IIRC, Windows doesn't allow to grow the stack by some
arbitrary constant, but if you grow it step by step, you could grow it much
larger than the 1 MB limit above. The stack could grow only by a rather
small amount at a time (I ended up to grow in pages).

--
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://www.jwdt.com/~paysan/

On Tue, 29 Nov 2005 16:23:47 +0100 in comp.arch, Bernd Paysan
<bernd.paysan@gmx.de> wrote:

Eric P. wrote:
So you pick a number and Windows picked 1 MB as their default.

Is it one megabyte? IIRC, Windows doesn't allow to grow the stack by some
arbitrary constant, but if you grow it step by step, you could grow it much
larger than the 1 MB limit above. The stack could grow only by a rather
small amount at a time (I ended up to grow in pages).

AIUI, the 1 MB is the default address space allocation, but at thread
startup, only 1 page is committed. As the stack grows, additional
pages are committed one at a time, but the stack is not allowed to
grow beyond the allocated (e.g., 1 MB) limit.

The 1 MB limit can be overridden, either at link time or run time.
Similarly, the stack committment default can also be overridden, in
multiples of pages.

This was true of NT4 about 6 years ago. Subsequent versions of
Windoze may be different.

Regards,
-=Dave

--
Change is inevitable, progress is not.

Dave Hansen wrote:

On Tue, 29 Nov 2005 16:23:47 +0100 in comp.arch, Bernd Paysan
bernd.paysan@gmx.de> wrote:

Eric P. wrote:
So you pick a number and Windows picked 1 MB as their default.

Is it one megabyte? IIRC, Windows doesn't allow to grow the stack by some
arbitrary constant, but if you grow it step by step, you could grow it much
larger than the 1 MB limit above. The stack could grow only by a rather
small amount at a time (I ended up to grow in pages).

AIUI, the 1 MB is the default address space allocation, but at thread
startup, only 1 page is committed. As the stack grows, additional
pages are committed one at a time, but the stack is not allowed to
grow beyond the allocated (e.g., 1 MB) limit.

The 1 MB limit can be overridden, either at link time or run time.
Similarly, the stack committment default can also be overridden, in
multiples of pages.

This was true of NT4 about 6 years ago. Subsequent versions of
Windoze may be different.

The Reserve value is fixed and link time. Only the Commit value can
be overridden by specifying it to the CreateThread function.

Eric

Bernd Paysan wrote:

Eric P. wrote:
So you pick a number and Windows picked 1 MB as their default.

Is it one megabyte? IIRC, Windows doesn't allow to grow the stack by some
arbitrary constant, but if you grow it step by step, you could grow it much
larger than the 1 MB limit above. The stack could grow only by a rather
small amount at a time (I ended up to grow in pages).

It has been a while since I last looked.
This might have been updated since, though I doubt it
since it has remained unchanged for over 10 years.
In a nut shell...

In Win32, the stack has two values, the Reserved and Commit sizes.
The Reserved size is the number of address bytes allocated
to each thread. The Commit size is the number that are actually
mapped to memory/backing_store. Reserve can be larger than Commit
to allow for future expension without diddling those PTE's.

The stack Reserved value is specified at link time and stored in the
EXE file. The default linker value is 1 MB but can be changed with
the -stack linker option.

The same Reserved value is used for ALL threads created by the
application, even ones created by DLL's that the app knows nothing
about!!! That means if you change the value at link time, then
you have no idea what the consequences will be to library threads!!!

The Commit size is specified as an argument to the CreateThread
function. It specifes the number of page file backed demand zero pages.
This is really pretty useless since it is the Commit value that
automatically grows toward the Reserve amount.

When a thread is created, the stack is allocated and managed,
not by the kernel, but by the Win32 user mode code. The stack top
and low water mark values are recorded in fields of a user mode data
structure called the Thread Environment Block (TEB).

A stack is allocated covering a Reserved size address range, and a set
of one or more no access Guard pages is located at top and bottom of
stack. A Guard Page is a special page type indicated by a bit in the
PTE that triggers a GuardPageFault when touched, and resets the guard
bit (a one shot trigger).
The Commit pages are bound to page file pages (demand zero).
I think the reserved but non committed pages must also be guards.

If a thread touches the guard page, it gets a guard page exception
which the kernel(?) handles by extending the stack down and updating
the low water mark in the TEB. If the thread touches the Guard page
at the stack bottom, the documentation CLAIMS it will try to extend
the stack range, but I have never seen it do so. If it cannot
it gets a Stack Overflow exception.

BUT... since the TEB does not contain the lower stack bound user
mode cannot check it at allocate time. It must rely on touching
the bottom guard page. But where is it? Only the kernel knows.
So it must touch the pages one by one to check.

When the C compiler or its runtime library want to allocate a
stack block larger than 4 KB, it performs a polling loop touching
each page, one by one, between the current ESP and the potential
new ESP, thereby causing a flood of guard page faults.
Each fault extends the stack by just one page.
It could at least check the low water mark to avoid the
polling loop when possible, but doesn't. Go figure.

Eric

On 32 bit Linux

08048000-080d1000 r-xp 00000000 16:03 5097658 /bin/zsh
080d1000-080d5000 rw-p 00088000 16:03 5097658 /bin/zsh
080d5000-08190000 rw-p 080d5000 00:00 0 [heap]
b7cc0000-b7cd5000 rw-p b7cc0000 00:00 0
b7cd5000-b7cdb000 r-xp 00000000 16:03 5101446 /usr/lib/zsh/4.2.5/zsh/zutil.so
b7cdb000-b7cdc000 rw-p 00005000 16:03 5101446 /usr/lib/zsh/4.2.5/zsh/zutil.so
b7cdc000-b7cf8000 r-xp 00000000 16:03 5101459 /usr/lib/zsh/4.2.5/zsh/complete.so
b7cf8000-b7cf9000 rw-p 0001c000 16:03 5101459 /usr/lib/zsh/4.2.5/zsh/complete.so
b7cf9000-b7d23000 r-xp 00000000 16:03 5101465 /usr/lib/zsh/4.2.5/zsh/zle.so
b7d23000-b7d28000 rw-p 0002a000 16:03 5101465 /usr/lib/zsh/4.2.5/zsh/zle.so
b7d32000-b7d35000 rw-p b7d32000 00:00 0
b7d35000-b7d43000 r-xp 00000000 16:03 5101439 /usr/lib/zsh/4.2.5/zsh/computil.so
b7d43000-b7d44000 rw-p 0000e000 16:03 5101439 /usr/lib/zsh/4.2.5/zsh/computil.so
b7d45000-b7d48000 rw-p b7d45000 00:00 0
b7d48000-b7d50000 r-xp 00000000 16:03 8574043 /lib/libnss_files-2.3.5.so
b7d50000-b7d52000 rw-p 00007000 16:03 8574043 /lib/libnss_files-2.3.5.so
b7d52000-b7d5a000 r-xp 00000000 16:03 8574185 /lib/libnss_nis-2.3.5.so
b7d5a000-b7d5c000 rw-p 00007000 16:03 8574185 /lib/libnss_nis-2.3.5.so
b7d5c000-b7d63000 r-xp 00000000 16:03 8575562 /lib/libnss_compat-2.3.5.so
b7d63000-b7d65000 rw-p 00006000 16:03 8575562 /lib/libnss_compat-2.3.5.so
b7d65000-b7d67000 rw-p b7d65000 00:00 0
b7d67000-b7e75000 r-xp 00000000 16:03 8573962 /lib/tls/libc-2.3.5.so
b7e75000-b7e76000 r--p 0010e000 16:03 8573962 /lib/tls/libc-2.3.5.so
b7e76000-b7e79000 rw-p 0010f000 16:03 8573962 /lib/tls/libc-2.3.5.so
b7e79000-b7e7b000 rw-p b7e79000 00:00 0
b7e7b000-b7e9c000 r-xp 00000000 16:03 8571549 /lib/tls/libm-2.3.5.so
b7e9c000-b7e9e000 rw-p 00020000 16:03 8571549 /lib/tls/libm-2.3.5.so
b7e9e000-b7ed8000 r-xp 00000000 16:03 8628335 /lib/libncurses.so.5.5
b7ed8000-b7ee0000 rw-p 00039000 16:03 8628335 /lib/libncurses.so.5.5
b7ee0000-b7ee1000 rw-p b7ee0000 00:00 0
b7ee1000-b7ef2000 r-xp 00000000 16:03 8576480 /lib/libnsl-2.3.5.so
b7ef2000-b7ef4000 rw-p 00010000 16:03 8576480 /lib/libnsl-2.3.5.so
b7ef4000-b7ef6000 rw-p b7ef4000 00:00 0
b7ef6000-b7ef8000 r-xp 00000000 16:03 8575629 /lib/libdl-2.3.5.so
b7ef8000-b7efa000 rw-p 00001000 16:03 8575629 /lib/libdl-2.3.5.so
b7efa000-b7efb000 rw-p b7efa000 00:00 0
b7efe000-b7f04000 r-xp 00000000 16:03 5101456 /usr/lib/zsh/4.2.5/zsh/parameter.so
b7f04000-b7f05000 rw-p 00006000 16:03 5101456 /usr/lib/zsh/4.2.5/zsh/parameter.so
b7f05000-b7f12000 r-xp 00000000 16:03 5101448 /usr/lib/zsh/4.2.5/zsh/compctl.so
b7f12000-b7f13000 rw-p 0000d000 16:03 5101448 /usr/lib/zsh/4.2.5/zsh/compctl.so
b7f13000-b7f17000 rw-p b7f13000 00:00 0
b7f17000-b7f2c000 r-xp 00000000 16:03 8576515 /lib/ld-2.3.5.so
b7f2c000-b7f2d000 r--p 00014000 16:03 8576515 /lib/ld-2.3.5.so
b7f2d000-b7f2e000 rw-p 00015000 16:03 8576515 /lib/ld-2.3.5.so
bfe17000-bfe2c000 rw-p bfe17000 00:00 0 [stack]
ffffe000-fffff000 ---p 00000000 00:00 0 [vdso]

--

Seek simplicity and mistrust it.
Alfred Whitehead

A witty saying proves nothing.
Voltaire

On Tue, 29 Nov 2005 09:58:37 -0800, dgay wrote:

Can you name a JVM with heap-allocated activation records?

I thought that I'd read about one, but now can't find one. It's pretty
easy to find references to Smalltalk and lisp/scheme compilers that do
this, but not JVM ones.

Sorry for the noise. Chalk it up to crossed neurones.

Cheers,

--
Andrew

Andrew Reilly <andrew-newspost@areilly.bpc-users.org> writes:
Then there are the JVMs, which historically instantiate "activation
records" on the heap, and let the garbage collector clean up the mess
later. This ensures that references to locals returned to callers don't
wind up hanging.

On Tue, 29 Nov 2005 09:58:37 -0800, dgay wrote:
Can you name a JVM with heap-allocated activation records?

I thought that I'd read about one, but now can't find one. It's pretty
easy to find references to Smalltalk and lisp/scheme compilers that do
this, but not JVM ones.

Sorry for the noise. Chalk it up to crossed neurones.

Andrew is not completely wrong. In Java programming you do

public class C {
public void foo() {
MyClass var = new MyClass();
...
}
}

whereas in C++ (the assembly of OOP) you can do either:

class C {
public:
void foo() {
MyClass var;
...
}
}

or

class C {
public:
void foo() {
MyClass* var = new MyClass();
...
}
}


I.e. in C++ you can choose to allocate the object on the heap or on the stack.
In either case the reference or pointer to the object is on the stack.

Whereas in Java the reference s on the stack, but the object has to behave
as if it is on the heap.




It is a big deal and highly touted feature for a JVM to be able to
allocate some such objects on the stack rather than the heap.

Either it uses static analysis to do so.

Or it allocates on the stack, and detects when it should be moved to the heap.

Ask wrote:

Hi

On a pentium machine the following program crashes. Was guessing has it
something to do with the architecture ?

int main()
{
int ar[530000];
return 0;
}

You're overrunning the stack limit. Try it this way, with the array allocated
statically instead of on the stack:

int ar[530000];
int main()
{
return 0;
}

On 2005-11-30, Andy Glew <andy.glew@intel.com> wrote:

Whereas in Java the reference s on the stack, but the object has to behave
as if it is on the heap.

It is a big deal and highly touted feature for a JVM to be able to
allocate some such objects on the stack rather than the heap.

Either it uses static analysis to do so.

Or it allocates on the stack, and detects when it should be moved to the heap.

It would be great if the next gen x86 (and other arcitectures) cpus
could implement some hw support to help improve this sort of
optimizations. The hw support could likely be built to also improve
pauseless garbage collectors.
They key is to provide hw support for read and write barriers (as they
are named in GC terminalogy, not in the memory coherence meaning).
http://www.azulsystems.com apparently already have an implementation of
these concepts.

lupus

--
-----------------------------------------------------------------
lupus@debian.org debian/rules
lupus@ximian.com Monkeys do it better