CASTalk.com

Hi

On a pentium machine the following program crashes. Was guessing has it
something to do with the architecture ?

int main()
{
int ar[530000];
return 0;
}

Thanks

Ask wrote:

Hi

On a pentium machine the following program crashes. Was guessing has it
something to do with the architecture ?

int main()
{
int ar[530000];
return 0;
}

You're absolutely right!

Since the Pentium is a P5 class cpu, it cannot work with arrays larger
than 5 * 10^5 (5e5) elements. Trying to allocate 5.3e5 above is what
causes the crash.

Terje

--
- <Terje.Mathisen@hda.hydro.com>
"almost all programming can be viewed as an exercise in caching"

"Ask" <ask.q@indiatimes.com> wrote in message
news:1133159462.099573.78970@f14g2000cwb.googlegroups.com...

Hi

On a pentium machine the following program crashes. Was guessing has it
something to do with the architecture ?

int main()
{
int ar[530000];
return 0;
}

This looks like a compiler bug to me. There is no reason why such program
should crash - assuming we are talking about a C or C-like language. It
works for me unless I turn off optimization completely - I think there is a
bug in the compiler when optimization is turned off. Perhaps a bug report to
the compiler vendor is in order.

Peter

You're absolutely right!

Since the Pentium is a P5 class cpu, it cannot work with arrays larger
than 5 * 10^5 (5e5) elements. Trying to allocate 5.3e5 above is what
causes the crash.

It doesn't seem so. The same amount of memory created on heap doesn't
cause a crash. Is it related to the stack size given to the process ?

In general, what determines the maximum size of the array for a given
architecture ?

Thanks

Ask wrote:

Hi

On a pentium machine the following program crashes. Was guessing has it
something to do with the architecture ?

int main()
{
int ar[530000];
return 0;
}

Thanks

It could be. The array is about 2 megabytes. On many (most?) platforms,
there is a maximum stack size that is less than "all available memory".

On Win32, for example, only 1 megabyte of address space is reserved for
the stack of the first thread to grow into, although you can increase
that with a linker option. (Notice that it isn't *memory* that you are
running out of, it is address space.)

Now, to put this thread on-topic for comp.arch we need someone to note
how Linux differs from Windows (not much, I expect) and how the program
runs fine on (ooo, I don't know) an AS-400, and then move on to how it
would have run happily on twenty different systems before the rise of
UNIX bull-dozed all the competently designed systems out of the market).

This program does not crash on my pentium machine under linux.

S

In article <dmela1$qp9$1$830fa7a5@news.demon.co.uk>,
Ken Hagan <K.Hagan@thermoteknix.co.uk> wrote:

It could be. The array is about 2 megabytes. On many (most?) platforms,
there is a maximum stack size that is less than "all available memory".

On platforms where Fortran is commonly used, the stack limit isn't
that low. Low stack limits are e-vile.

-- greg

It doesn't seem so. The same amount of memory created on heap doesn't
cause a crash. Is it related to the stack size given to the process ?

In general, what determines the maximum size of the array for a given
architecture ?

It's not the architecture, it's the OS. And the compiler. And the
linker. And the loader. Usually all of these, by convention.

In theory, an OS could look at what you ask for, and arrange to give
it all to you, so long as it fits inside the 32b or 64b or whatever
limit.

In practice, all UNIX OSes tend to allocate the stack at a fixed
address, and the heap somewhere else. The traditional way is to have
the stack and heap growing from opposite sides of the address space,
so either can use all of the address space. However, that doesn't
work when there are other "segments" (not x86 segments) in the memory
image - typically things like memory mapped files.

If I was really nice I would show you a sample memory map. But I'm
lazy, and am afraid that I would embarass myself. But that's what you
need to look for - the so-called memory map, the virtual address map,
of a typical process on your OS.

I expect somebody will post the current address maps on their favorite
OS, and I'll learn something.

---

By the way, let me recommend a good book on a related topic:
Linkers and Loaders, by John Levine.

http://www.iecc.com/linker/

Andy Glew <andy.glew@intel.com> writes:

In practice, all UNIX OSes tend to allocate the stack at a fixed
address, and the heap somewhere else. The traditional way is to have
the stack and heap growing from opposite sides of the address space,
so either can use all of the address space. However, that doesn't
work when there are other "segments" (not x86 segments) in the memory
image - typically things like memory mapped files.

Modern 32bit Linux and I believe some other Unixes solve
the problem of the heap bumping into mmap by placing
the mmaps down from the bottom of the stack instead of up
from an arbitary boundary.

Disadvantage: ulimit -s has to be set properly if you
want more main program stack than the default (8MB normally)

64bit Linux doesn't bother because there is enough room.

-Andi

In article <438B6648.F691DA73@sympaticoREMOVE.ca>,
Eric P. <eric_pattison@sympaticoREMOVE.ca> wrote:

However with multi-threading, I think the only solutions are the
between-a-rock-and-a-hard-place types, at least the ones that I
have heard of.

The common Unix solution is to treat the initial stack specially, so
non-threaded programs behave fairly sanely. And that's the case which
was at question here.

There is no place to linearly grow dynamic stacks,
and if you reserve too much space for each thread stack, you can
run out of space and limit the number of threads.

Apparently you aren't a convert to 64-bit computing. One nice aspect
of it is that it papers over this problem. The OpenMP programming
model involves programs which tend to use lots of stack, and our
implementation Quietly Does The Right Thing for such programs.

-- greg
(employed by, not speaking for, PathScale.)

Ken Hagan <K.Hagan@thermoteknix.co.uk> writes:

and then move on to how it would have run happily on twenty
different systems before the rise of UNIX bull-dozed all the
competently designed systems out of the market).

I was about to do that...

Although my pet peeve is memory mapped files breaking up the address
space.

Greg Lindahl wrote:

In article <dmela1$qp9$1$830fa7a5@news.demon.co.uk>,
Ken Hagan <K.Hagan@thermoteknix.co.uk> wrote:

It could be. The array is about 2 megabytes. On many (most?) platforms,
there is a maximum stack size that is less than "all available memory".

On platforms where Fortran is commonly used, the stack limit isn't
that low. Low stack limits are e-vile.

On a single threaded OS like old VMS or Unix, the answer was to
assume that touching invalid memory was a stack access and
caused an automatic stack expansion. Not necessarily a correct
assumption but what most programs found useful.

However with multi-threading, I think the only solutions are the
between-a-rock-and-a-hard-place types, at least the ones that I
have heard of. There is no place to linearly grow dynamic stacks,
and if you reserve too much space for each thread stack, you can
run out of space and limit the number of threads.
So you pick a number and Windows picked 1 MB as their default.

Another option would be to dynamically allocate the stack in heap
chunks and chain them together. You could estimate the required
storage as the distance from the current stack point.
It would make it difficult to catch wild pointers though,
and you need special exception handlers to unwind the stack
chain and ensure space is properly recovered, so it gets messy.

Are there any other mechanisms (for traditional languages)?

Eric

On Mon, 28 Nov 2005 11:14:55 +0000, Ken Hagan wrote:

Ask wrote:
Hi

On a pentium machine the following program crashes. Was guessing has it
something to do with the architecture ?

int main()
{
int ar[530000];
return 0;
}
Now, to put this thread on-topic for comp.arch we need someone to note
how Linux differs from Windows (not much, I expect) and how the program
runs fine on (ooo, I don't know) an AS-400, and then move on to how it
would have run happily on twenty different systems before the rise of
UNIX bull-dozed all the competently designed systems out of the market).

Well, not to go as far as the AS-400, the early ARM personal computers,
under the native RISC-OS (which, like early Mac OS and Windows had all
processes live in one undifferentiated address space) had page-allocated
stacks. This resulted in a wildly hideous C calling convention, where all
sorts of run-time checks and exceptional conditions were required to chain
all of the (discontiguous) chunks together. I'm not sure that this
particular program would have run on such a system at all, though: I can't
remember whether auto (stack) allocations were limited to less than
page-size objects, or whether there was another mechanism to handle large
objects.

Then there are the JVMs, which historically instantiate "activation
records" on the heap, and let the garbage collector clean up the mess
later. This ensures that references to locals returned to callers don't
wind up hanging. Apparently this is such a performance loser (stacks have
nice refrerence locality properties that work well with caches) that
recent JVMs do "escape analysis" to catch the hanging return issue and
allocate everything else on a real stack, as the scope syntax might lead
one to expect. This is relevant in the context of the OP because there
exists at least one C compiler that targets the JVM, so therefore there is
at least one C compiler that uses heap allocated stacks. I expect that
there are others, though.

Cheers,

--
Andrew

"Ask" <ask.q@indiatimes.com> wrote ...

Hi

On a pentium machine the following program crashes. Was guessing has it
something to do with the architecture ?

Nope. Guess again, in another newsgroup.

int main()
{
int ar[530000];
return 0;
}

In article <p734q5whygi.fsf@verdi.suse.de>,
Andi Kleen <freitag@alancoxonachip.com> wrote:

Modern 32bit Linux and I believe some other Unixes solve
the problem of the heap bumping into mmap by placing
the mmaps down from the bottom of the stack instead of up
from an arbitary boundary.

This is a new definition of "solve" I must not be aware of ;-)

Disadvantage: ulimit -s has to be set properly if you
want more main program stack than the default (8MB normally)

Many machines running numeric programs have a default of unlimited
stacks, either because the admin or the user upped it. I guess you
mostly run C programs that don't use automatic arrays for temporary
variables.

Whence my comment about OSes which are e-vile.

-- greg