Mon Dec 23 2002

SUMMARY
-------

	SCO Compiler OU8.0.0bl7 produces badly incorrect code for alloca()
	under some circumstances.

BUG REPORTER
------------

	Steve Friedl
	Software Consultant
	Tustin, California USA
	steve at unixwiz dot net

ENVIRONMENT
-----------

	# cc -V
	UX:cc: INFO: Optimizing C Compilation System  (CCS) 4.0  06/09/01 (OU8.0.0bl7)

	# uname -a
	OpenUNIX foo 5 8.0.0 i386 x86at Caldera UNIX_SVR5

I'm reporting a bug in the C compiler where it generates incorrect code
in all modes (regular, debug, optimization). I've reproduced it and can
describe in detail why it's wrong.

This program compiles to incorrect code:

	/* testalloca.c */
	#include <string.h>
	#include <alloca.h>

	char *goodalloc(register char *str)
	{
	register char *p = alloca(16);

		return strcpy(p, str);
	}

	char *badalloc(register char *str)
	{
		return strcpy(alloca(16), str);
	}

The problem is when alloca() is used as a parameter to another function
(as in "badalloc), it mucks with the stack *while the arglist are being
assembled*, and the generated assembler code is clearly wrong.

This is the assembler produced from "cc -O testalloca.c"

	goodalloc:
		pushl	%ebp		// setup this function's frame
		movl	%esp,%ebp

		pushl	%esi		// save local registers
		pushl	%ebx

		movl	%esp,%ecx
		movl	8(%ebp),%esi
		subl	$16,%esp
		andl	$-4,%esp

		// save the base of new memory in %EAX
		// replicate saved variables
		movl	8(%ecx),%edx
		movl	%esp,%eax
		pushl	%edx
		movl	4(%ecx),%ebx
		pushl	%ebx
		movl	(%ecx),%edx
		pushl	%edx

		pushl	%esi			// second param ("str")
		pushl	%eax			// first param (new mem)
		call	strcpy
		popl	%edx

		popl	%ecx			// restore local registers
		popl	%ebx
		popl	%esi
		movl	%ebp,%esp
		popl	%ebp
		ret	

	badalloc:
		pushl	%ebp			// setup this function's frame
		movl	%esp,%ebp

		pushl	%edi			// save local registers
		pushl	%ebx

		movl	8(%ebp),%edi		// register-ize the "str" param

--->		pushl	%edi			// PUSH SECOND PARAM TO strcpy()

		movl	%esp,%ecx

		subl	$16,%esp		// allocate space for locals
		andl	$-4,%esp

		// save the base of new memory in %EAX
		// replicate the saved variables
		movl	%esp,%eax
		movl	8(%ecx),%edx		// replicate saved vars
		pushl	%edx
		movl	4(%ecx),%ebx
		pushl	%ebx
		movl	(%ecx),%edx
		pushl	%edx

		// second param was pushed earlier!
		pushl	%eax			// first param (new memory)
		call	strcpy
		popl	%edx

		popl	%ecx			// restore local registers
		popl	%ebx
		popl	%edi
		movl	%ebp,%esp
		popl	%ebp
		ret

In the "bad" version, the compiler pushes the "str" parameter (in %edi)
just before it allocates the local space, so strcpy() sees an old saved
register


The compiler pushes the "str" parameter for strcpy() very early in the
"bad" version, it not having any idea that alloca() is about to muck with
the stack.  Then -- ignoring the wrong strcpy() behavior for the moment --
when the saved registers get saved, they're all wrong because the stack
frame is misaligned.  This causes havoc in the caller that expects its
registers to be unmolested.

This shows the stack arrangment just before the call to strcpy() for
both the good and bad versions:


GOOD VERSION:

	| old EIP	|
	| old EBP	| <-- EBP
	| old EDI	|
	| old ESI	|
	| old EBI	|
	| .. alloca	|
	| .. alloca	|
	| .. alloca	|
	| .. alloca	|
	| old EDI	|
	| old ESI	|
	| old EBI	|
	| "str" param	|
	| new memory	| <-- ESP

	strcpy(newmem, "str")

BAD VERSION

	| old EIP	|
	| old EBP	| <-- EBP
	| old EDI	|
	| old ESI	|
	| old EBI	|
	| "str" param	|
	| .. alloca	|
	| .. alloca	|
	| .. alloca	|
	| .. alloca	|
	| old EDI	|
	| old ESI	|
	| old EBI	|
	| new memory	| <-- ESP

	strcpy(newmem, old %EBI)	BUG!

WORKAROUND
----------

	Perform the alloca() outside of other function calls by saving the
	result to a temporary.