[Solved] Why does it return a random value other than the value I give to the function?

by


First of all, C functions are call-by-value: the int x arg in the function is a copy. Modifying it doesn’t modify the caller’s copy of whatever they passed, so your swap makes zero sense.

Second, you’re using the return value of the function, but you don’t have a return statement. In C (unlike C++), it’s not undefined behaviour for execution to fall off the end of a non-void function (for historical reasons, before void existed, and function returns types defaulted to int). But it is still undefined behaviour for the caller to use a return value when the function didn’t return one.

In this case, returning 100 was the effect of the undefined behaviour (of using the return value of a function where execution falls off the end without a return statement). This is a coincidence of how GCC compiles in debug mode (-O0):

GCC -O0 likes to evaluate non-constant expressions in the return-value register, e.g. EAX/RAX on x86-64. (This is actually true for GCC across architectures, not just x86-64). This actually gets abused on codegolf.SE answers; apparently some people would rather golf in gcc -O0 as a language than ANSI C. See this “C golfing tips” answer and the comments on it, and this SO Q&A about why i=j inside a function putting a value in RAX. Note that it only works when GCC has to load a value into registers, not just do a memory-destination increment like add dword ptr [rbp-4], 1 for x++ or whatever.

In your case (with your code compiled by GCC10.2 on the Godbolt compiler explorer)

int y=100; stores 100 directly to stack memory (the way GCC compiles your code).

int a = swap(y); loads y into EAX (for no apparent reason), then copies to EDI to pass as an arg to swap. Since GCC’s asm for swap doesn’t touch EAX, after the call, EAX=y, so effectively the function returns y.

But if you call it with swap(100), GCC doesn’t end up putting 100 into EAX while setting up the args.

The way GCC compiles your swap, the asm doesn’t touch EAX, so whatever main left there is treated as the return value.

main:
...
        mov     DWORD PTR [rbp-4], 100          # y=100

        mov     eax, DWORD PTR [rbp-4]          # load y into EAX
        mov     edi, eax                        # copy it to EDI (first arg-passing reg)
        call    swap                            # swap(y)

        mov     DWORD PTR [rbp-8], eax          # a = EAX as the retval = y
...

But with your other main:

main:
...                                    # nothing that touches EAX
        mov     edi, 100
        call    swap
        mov     DWORD PTR [rbp-4], eax   # a = whatever garbage was there on entry to main
...

(The later ... reloads a as an arg for printf, matching the ISO C semantics because GCC -O0 compiles each C statement to a separate block of asm; thus the later ones aren’t affected by the earlier UB (unlike in the general case with optimization enabled), so do just print whatever’s in a‘s memory location.)

The swap function compiles like this (again, GCC10.2 -O0):

swap:
        push    rbp
        mov     rbp, rsp
        mov     DWORD PTR [rbp-4], edi
        mov     DWORD PTR [rbp-4], 20
        nop
        pop     rbp
        ret

Keep in mind none of this has anything to do with valid portable C. This (using garbage left in memory or registers) one of the kinds of things you see in practice from C that invokes undefined behaviour, but certainly not the only thing. See also What Every C Programmer Should Know About Undefined Behavior from the LLVM blog.

This answer is just answering the literal question of what exactly happened in asm. (I’m assuming un-optimized GCC because that easily explains the result, and x86-64 because that’s a common ISA, especially when people forget to mention any ISA.)

Other compilers are different, and GCC will be different if you enable optimization.

5

solved Why does it return a random value other than the value I give to the function?