Studying note of GCC-3.4.6 source (74)

5.3.1.2. Macro invocation

For our case, this next token is “FUN”. For this node, its type of course is NT_MACRO. As long as its not disable macro and not in the environment that prevent macro expansion (e.g., in handling assertion, pragma, #if series directives; parsing marco argument, function-like marco invocation), enter_macro_context will be invoked.

709 static int
710 enter_macro_context (cpp_reader *pfile, cpp_hashnode *node) in cppmacro.c
711 {
712 /* The presence of a macro invalidates a file's controlling macro. */
713 pfile->mi_valid = false;
714
715 pfile->state.angled_headers = false;
716
717 /* Handle standard macros. */
718 if (! (node->flags & NODE_BUILTIN))
719 {
720 cpp_macro *macro = node->value.macro;
721
722 if (macro->fun_like)
723 {
724 _cpp_buff *buff;
725
726 pfile->state.prevent_expansion++;
727 pfile->keep_tokens++;
728 pfile->state.parsing_args = 1;
729 buff = funlike_invocation_p (pfile, node);
730 pfile->state.parsing_args = 0;
731 pfile->keep_tokens--;
732 pfile->state.prevent_expansion--;
733
734 if (buff == NULL)
735 {
736 if (CPP_WTRADITIONAL (pfile) && ! node->value.macro->syshdr)
737 cpp_error (pfile, CPP_DL_WARNING,
738 "function-like macro /"%s/" must be used with arguments in traditional C",
739 NODE_NAME (node));
740
741 return 0;
742 }
743
744 if (macro->paramc > 0)
745 replace_args (pfile, node, macro, (macro_arg *) buff->base);
746 _cpp_release_buff (pfile, buff);
747 }
748
749 /* Disable the macro within its expansion. */
750 node->flags |= NODE_DISABLED;
751
752 macro->used = 1;
753
754 if (macro->paramc == 0)
755 push_token_context (pfile, node, macro->exp.tokens, macro->count);
756
757 return 1;
758 }
759
760 /* Handle built-in macros and the _Pragma operator. */
761 return builtin_macro (pfile, node);
762 }

Pay attention to the setting of pfile before and after the call of funlike_invocation_p at line 729.
5.3.1.2.1. Funciton-like macro expansion – arguments collection
For non-builtin macro, it needs setup context to take place the macro expansion. At line 720, remember that the whole definition is saved into macro field in the identifier node of the macro (see create_iso_definition). For non-function-like macro, put these tokens into a new context is OK, as line 755 above does. But for function-like macro, it needs macro argument replacement, in which the new context is created. Note that at line 728, slot parsing_args is set as 1 before invoking funlik_invocation_p, and reset to 0 after it.

669 static _cpp_buff *
670 funlike_invocation_p (cpp_reader *pfile, cpp_hashnode *node) in cppmacro.c
671 {
672 const cpp_token *token, *padding = NULL;
673
674 for (;;)
675 {
676 token = cpp_get_token (pfile);
677 if (token->type != CPP_PADDING)
678 break;
679 if (padding == NULL
680 || (!(padding->flags & PREV_WHITE) && token->val.source == NULL))
681 padding = token;
682 }
683
684 if (token->type == CPP_OPEN_PAREN)
685 {
686 pfile->state.parsing_args = 2;
687 return collect_args (pfile, node);
688 }
689
690 /* CPP_EOF can be the end of macro arguments, or the end of the
691 file. We mustn't back up over the latter. Ugh. */
692 if (token->type != CPP_EOF || token == &pfile->eof)
693 {
694 /* Back up. We may have skipped padding, in which case backing
695 up more than one token when expanding macros is in general
696 too difficult. We re-insert it in its own context. */
697 _cpp_backup_tokens (pfile, 1);
698 if (padding)
699 push_token_context (pfile, NULL, padding, 1);
700 }
701
702 return NULL;
703 }

Routine collect_args collects the real arguments for the macro invocation. These arguments are saved for later replacement. Note that before entering the function, field parsing_args is set as 2 – open parentheses is seen.

535 static _cpp_buff *
536 collect_args (cpp_reader *pfile, const cpp_hashnode *node) in cppmacro.c
537 {
538 _cpp_buff *buff, *base_buff;
539 cpp_macro *macro;
540 macro_arg *args, *arg;
541 const cpp_token *token;
542 unsigned int argc;
543
544 macro = node->value.macro;
545 if (macro->paramc)
546 argc = macro->paramc;
547 else
548 argc = 1;
549 buff = _cpp_get_buff (pfile, argc * (50 * sizeof (cpp_token *)
550 + sizeof (macro_arg)));
551 base_buff = buff;
552 args = (macro_arg *) buff->base;
553 memset (args, 0, argc * sizeof (macro_arg));
554 buff->cur = (unsigned char *) &args[argc];
555 arg = args, argc = 0;

Slot paramc in cpp_macro records the number of arguments of the macro. Type macro_arg is designed to hold necessary information for the argument replacement.

32 struct macro_arg in cppmacro.c
33 {
34 const cpp_token **first; /* First token in unexpanded argument. */
35 const cpp_token **expanded; /* Macro-expanded argument. */
36 const cpp_token *stringified; /* Stringified argument. */
37 unsigned int count; /* # of tokens in argument. */
38 unsigned int expanded_count; /* # of tokens in expanded argument. */
39 };

Below, see cpp_get_token is invoked at line 578 to fetch the tokens of argument. For our case, directive of #ifdef will be read in as the third token, and according to [12] the behavor of the directive is undefined by standard, but GCC decides to let those directives behave as normal – that is invoking _cpp_handle_directive. Note, during parsing arguments, argument being macro itself is prohibited from expansion as defined by standards (but it is expanded in argument replacement).

collect_args (continue)

557 /* Collect the tokens making up each argument. We don't yet know
558 how many arguments have been supplied, whether too many or too
559 few. Hence the slightly bizarre usage of "argc" and "arg". */
560 do
561 {
562 unsigned int paren_depth = 0;
563 unsigned int ntokens = 0;
564
565 argc++;
566 arg->first = (const cpp_token **) buff->cur;
567
568 for (;;)
569 {
570 /* Require space for 2 new tokens (including a CPP_EOF). */
571 if ((unsigned char *) &arg->first[ntokens + 2] > buff->limit)
572 {
573 buff = _cpp_append_extend_buff (pfile, buff,
574 1000 * sizeof (cpp_token *));
575 arg->first = (const cpp_token **) buff->cur;
576 }
577
578 token = cpp_get_token (pfile);
579
580 if (token->type == CPP_PADDING)
581 {
582 /* Drop leading padding. */
583 if (ntokens == 0)
584 continue;
585 }
586 else if (token->type == CPP_OPEN_PAREN)
587 paren_depth++;
588 else if (token->type == CPP_CLOSE_PAREN)
589 {
590 if (paren_depth-- == 0)
591 break;
592 }
593 else if (token->type == CPP_COMMA)
594 {
595 /* A comma does not terminate an argument within
596 parentheses or as part of a variable argument. */
597 if (paren_depth == 0
598 && ! (macro->variadic && argc == macro->paramc))
599 break;
600 }
601 else if (token->type == CPP_EOF
602 || (token->type == CPP_HASH && token->flags & BOL))
603 break;
604
605 arg->first[ntokens++] = token;
606 }
607
608 /* Drop trailing padding. */
609 while (ntokens > 0 && arg->first[ntokens - 1]->type == CPP_PADDING)
610 ntokens--;
611
612 arg->count = ntokens;
613 arg->first[ntokens] = &pfile->eof;
614
615 /* Terminate the argument. Excess arguments loop back and
616 overwrite the final legitimate argument, before failing. */
617 if (argc <= macro->paramc)
618 {
619 buff->cur = (unsigned char *) &arg->first[ntokens + 1];
620 if (argc != macro->paramc)
621 arg++;
622 }
623 }
624 while (token->type != CPP_CLOSE_PAREN && token->type != CPP_EOF);

Notice that even arguments are separated by comma, but ISO standard allows arbitrary number of whitespaces inserted around the comma, so it uses nested loops to handle the case. The inner FOR loop retrieves the may having tailing paddings argument (see that the paddings are also tokens), then while loop at line 609 strips these padding tokens, leaves the argument.

collect_args (continue)

626 if (token->type == CPP_EOF)
627 {
628 /* We still need the CPP_EOF to end directives, and to end
629 pre-expansion of a macro argument. Step back is not
630 unconditional, since we don't want to return a CPP_EOF to our
631 callers at the end of an -include-d file. */
632 if (pfile->context->prev || pfile->state.in_directive)
633 _cpp_backup_tokens (pfile, 1);
634 cpp_error (pfile, CPP_DL_ERROR,
635 "unterminated argument list invoking macro /"%s/"",
636 NODE_NAME (node));
637 }
638 else
639 {
640 /* A single empty argument is counted as no argument. */
641 if (argc == 1 && macro->paramc == 0 && args[0].count == 0)
642 argc = 0;
643 if (_cpp_arguments_ok (pfile, macro, node, argc))
644 {
645 /* GCC has special semantics for , ## b where b is a varargs
646 parameter: we remove the comma if b was omitted entirely.
647 If b was merely an empty argument, the comma is retained.
648 If the macro takes just one (varargs) parameter, then we
649 retain the comma only if we are standards conforming.
650
651 If FIRST is NULL replace_args () swallows the comma. */
652 if (macro->variadic && (argc < macro->paramc
653 || (argc == 1 && args[0].count == 0
654 && !CPP_OPTION (pfile, std))))
655 args[macro->paramc - 1].first = NULL;
656 return base_buff;
657 }
658 }
659
660 /* An error occurred. */
661 _cpp_release_buff (pfile, base_buff);
662 return NULL;
663 }

As abnormal case, end of file is encountered before closing parenthesis is found. Otherwise, after closing parenthesis finishes the retrieving, the arguments themselves need be validated.

494 bool
495 _cpp_arguments_ok(cpp_reader *pfile, cpp_macro *macro, const cpp_hashnode *node, unsigned int argc)
496 {
497 if (argc == macro->paramc)
498 return true;
499
500 if (argc < macro->paramc)
501 {
502 /* As an extension, a rest argument is allowed to not appear in
503 the invocation at all.
504 e.g. #define debug(format, args...) something
505 debug("string");
506
507 This is exactly the same as if there had been an empty rest
508 argument - debug("string", ). */
509
510 if (argc + 1 == macro->paramc && macro->variadic)
511 {
512 if (CPP_PEDANTIC (pfile) && ! macro->syshdr)
513 cpp_error (pfile, CPP_DL_PEDWARN,
514 "ISO C99 requires rest arguments to be used");
515 return true;
516 }
517
518 cpp_error (pfile, CPP_DL_ERROR,
519 "macro /"%s/" requires %u arguments, but only %u given",
520 NODE_NAME (node), macro->paramc, argc);
521 }
522 else
523 cpp_error (pfile, CPP_DL_ERROR,
524 "macro /"%s/" passed %u arguments, but takes just %u",
525 NODE_NAME (node), argc, macro->paramc);
526
527 return false;
528 }

The validation only concerns the matching of number of arguments.
5.3.1.2.2. Funciton-like macro expansion – arguments replacement
After collecting the parameters during the invocation, at line 745 in enter_macro_context, these parameters will replace the declared arguments in definition body by replace_args.

768 static void
769 replace_args(cpp_reader *pfile, cpp_hashnode *node, cpp_macro *macro, macro_arg *args)
770 {
771 unsigned int i, total;
772 const cpp_token *src, *limit;
773 const cpp_token **dest, **first;
774 macro_arg *arg;
775 _cpp_buff *buff;
776
777 /* First, fully macro-expand arguments, calculating the number of
778 tokens in the final expansion as we go. The ordering of the if
779 statements below is subtle; we must handle stringification before
780 pasting. */
781 total = macro->count;
782 limit = macro->exp.tokens + macro->count;
783
784 for (src = macro->exp.tokens; src < limit; src++)
785 if (src->type == CPP_MACRO_ARG)
786 {
787 /* Leading and trailing padding tokens. */
788 total += 2;
789
790 /* We have an argument. If it is not being stringified or
791 pasted it is macro-replaced before insertion. */
792 arg = &args[src->val.arg_no - 1];
793
794 if (src->flags & STRINGIFY_ARG)
795 {
796 if (!arg->stringified)
797 arg->stringified = stringify_arg (pfile, arg);
798 }
799 else if ((src->flags & PASTE_LEFT)
800 || (src > macro->exp.tokens && (src[-1].flags & PASTE_LEFT)))
801 total += arg->count - 1;
802 else
803 {
804 if (!arg->expanded)
805 expand_arg (pfile, arg);
806 total += arg->expanded_count - 1;
807 }
808 }

At line 781 slot count records the number of tokens in the definition body, but with replacement, the number may not correct. FOR loop beginning at line 784 updates this number. For those tokens of argument in definition body, they are of type CPP_MACRO_ARG.
According to ISO C/C++, token following ‘#’ (the stringify operator) and tokens around ‘##’ (the paste operator) will not be expanded even they are macro themselves. For stringify case, one token is allowed after ‘#’, so no token number change. For paste case, in fact, we can pass an expression for the argument, for example:

#define fA(A, B) c = A##B;
fA (a+b, B) // output a+bB;

So the number of tokens added is the argument’s token number minus 1. While for the left case, argument being macro itself should be expanded normally.

979 static void
980 expand_arg (cpp_reader *pfile, macro_arg *arg) in cppmacro.c
981 {
982 unsigned int capacity;
983 bool saved_warn_trad;
984
985 if (arg->count == 0)
986 return;
987
988 /* Don't warn about funlike macros when pre-expanding. */
989 saved_warn_trad = CPP_WTRADITIONAL (pfile);
990 CPP_WTRADITIONAL (pfile) = 0;
991
992 /* Loop, reading in the arguments. */
993 capacity = 256;
994 arg->expanded = xmalloc (capacity * sizeof (cpp_token *));
995
996 push_ptoken_context (pfile, NULL, NULL, arg->first, arg->count + 1);
997 for (;;)
998 {
999 const cpp_token *token;
1000
1001 if (arg->expanded_count + 1 >= capacity)
1002 {
1003 capacity *= 2;
1004 arg->expanded = xrealloc (arg->expanded,
1005 capacity * sizeof (cpp_token *));
1006 }
1007
1008 token = cpp_get_token (pfile);
1009
1010 if (token->type == CPP_EOF)
1011 break;
1012
1013 arg->expanded[arg->expanded_count++] = token;
1014 }
1015
1016 _cpp_pop_context (pfile);
1017
1018 CPP_WTRADITIONAL (pfile) = saved_warn_trad;
1019 }

To parsing arbitrary nested macro correctly, it is necessary to setup parsing context as push_ptoken_context does.

931 static void
932 push_ptoken_context (cpp_reader *pfile, cpp_hashnode *macro, in cppmacro.c
933 _cpp_buff *buff, const cpp_token **first, unsigned int count)
934 {
935 cpp_context *context = next_context (pfile);
936
937 context->direct_p = false;
938 context->macro = macro;
939 context->buff = buff;
940 FIRST (context).ptoken = first;
941 LAST (context).ptoken = first + count;
942 }

After setting up the context, in expand_arg at line 1008, cpp_get_token next fetches token from the context. If the argument is macro too, enter_macro_context at line 1099 will be invoked again. Telling from the invocation sequence, the handling of nested macros are depth first, so when exit FOR loop at line 1011, all macros nested within must be handled. Even note that for this expanded argument, the tokens generated are saved within expanded field of corresponding object of macro_arg.

replace_args (continue)

810 /* Now allocate space for the expansion, copy the tokens and replace
811 the arguments. */
812 buff = _cpp_get_buff (pfile, total * sizeof (cpp_token *));
813 first = (const cpp_token **) buff->base;
814 dest = first;
815
816 for (src = macro->exp.tokens; src < limit; src++)
817 {
818 unsigned int count;
819 const cpp_token **from, **paste_flag;
820
821 if (src->type != CPP_MACRO_ARG)
822 {
823 *dest++ = src;
824 continue;
825 }
826
827 paste_flag = 0;
828 arg = &args[src->val.arg_no - 1];
829 if (src->flags & STRINGIFY_ARG)
830 count = 1, from = &arg->stringified;
831 else if (src->flags & PASTE_LEFT)
832 count = arg->count, from = arg->first;
833 else if (src != macro->exp.tokens && (src[-1].flags & PASTE_LEFT))
834 {
835 count = arg->count, from = arg->first;
836 if (dest != first)
837 {
838 if (dest[-1]->type == CPP_COMMA
839 && macro->variadic
840 && src->val.arg_no == macro->paramc)
841 {
842 /* Swallow a pasted comma if from == NULL, otherwise
843 drop the paste flag. */
844 if (from == NULL)
845 dest--;
846 else
847 paste_flag = dest - 1;
848 }
849 /* Remove the paste flag if the RHS is a placemarker. */
850 else if (count == 0)
851 paste_flag = dest - 1;
852 }
853 }
854 else
855 count = arg->expanded_count, from = arg->expanded;
856
857 /* Padding on the left of an argument (unless RHS of ##). */
858 if ((!pfile->state.in_directive || pfile->state.directive_wants_padding)
859 && src != macro->exp.tokens && !(src[-1].flags & PASTE_LEFT))
860 *dest++ = padding_token (pfile, src);
861
862 if (count)
863 {
864 memcpy (dest, from, count * sizeof (cpp_token *));
865 dest += count;
866
867 /* With a non-empty argument on the LHS of ##, the last
868 token should be flagged PASTE_LEFT. */
869 if (src->flags & PASTE_LEFT)
870 paste_flag = dest - 1;
871 }
872
873 /* Avoid paste on RHS (even case count == 0). */
874 if (!pfile->state.in_directive && !(src->flags & PASTE_LEFT))
875 *dest++ = &pfile->avoid_paste;
876
877 /* Add a new paste flag, or remove an unwanted one. */
878 if (paste_flag)
879 {
880 cpp_token *token = _cpp_temp_token (pfile);
881 token->type = (*paste_flag)->type;
882 token->val.str = (*paste_flag)->val.str;
883 if (src->flags & PASTE_LEFT)
884 token->flags = (*paste_flag)->flags | PASTE_LEFT;
885 else
886 token->flags = (*paste_flag)->flags & ~PASTE_LEFT;
887 *paste_flag = token;
888 }
889 }
890
891 /* Free the expanded arguments. */
892 for (i = 0; i < macro->paramc; i++)
893 if (args[i].expanded)
894 free (args[i].expanded);
895
896 push_ptoken_context (pfile, node, buff, first, dest - first);
897 }

As the space needed for expanding the macro has been calculated, a buffer of this size is created and holds the tokens of the expanded macro. And as it is possible to pass a token sequence for an argument, for token paste, it needs mark the first (on right hand side of ##) or the last (on the left hand side of ##) token in the sequence as PASTE_LEFT. A new parsing context is created again at line 896, solely to make them the next fetched tokens.
For paste case, in cpp_get_token, paste_all_tokens will be invoked at line 1071.

448 static void
449 paste_all_tokens (cpp_reader *pfile, const cpp_token *lhs) in cppmacro.c
450 {
451 const cpp_token *rhs;
452 cpp_context *context = pfile->context;
453
454 do
455 {
456 /* Take the token directly from the current context. We can do
457 this, because we are in the replacement list of either an
458 object-like macro, or a function-like macro with arguments
459 inserted. In either case, the constraints to #define
460 guarantee we have at least one more token. */
461 if (context->direct_p)
462 rhs = FIRST (context).token++;
463 else
464 rhs = *FIRST (context).ptoken++;
465
466 if (rhs->type == CPP_PADDING)
467 abort ();
468
469 if (!paste_tokens (pfile, &lhs, rhs))
470 {
471 _cpp_backup_tokens (pfile, 1);
472
473 /* Mandatory error for all apart from assembler. */
474 if (CPP_OPTION (pfile, lang) != CLK_ASM)
475 cpp_error (pfile, CPP_DL_ERROR,
476 "pasting /"%s/" and /"%s/" does not give a valid preprocessing token",
477 cpp_token_as_text (pfile, lhs),
478 cpp_token_as_text (pfile, rhs));
479 break;
480 }
481 }
482 while (rhs->flags & PASTE_LEFT);
483
484 /* Put the resulting token in its own context. */
485 push_token_context (pfile, NULL, lhs, 1);
486 }

Note that at line 424, only ‘/’ and ‘=’ can be pasted to generate ‘/=’ operator, other ‘/’ and not ‘=’ pair will have white space inserted between. Then at line 433 below, _cpp_temp_token creates a new token, which has its line NO., column NO., copied fromcur_token of file (e.g., the pasted token). It can ensure that the diagnositc system can give out correct information.

407 static bool
408 paste_tokens (cpp_reader *pfile, const cpp_token **plhs, const cpp_token *rhs)
409 {
410 unsigned char *buf, *end;
411 const cpp_token *lhs;
412 unsigned int len;
413 bool valid;
414
415 lhs = *plhs;
416 len = cpp_token_len (lhs) + cpp_token_len (rhs) + 1;
417 buf = alloca (len);
418 end = cpp_spell_token (pfile, lhs, buf);
419
420 /* Avoid comment headers, since they are still processed in stage 3.
421 It is simpler to insert a space here, rather than modifying the
422 lexer to ignore comments in some circumstances. Simply returning
423 false doesn't work, since we want to clear the PASTE_LEFT flag. */
424 if (lhs->type == CPP_DIV && rhs->type != CPP_EQ)
425 *end++ = ' ';
426 end = cpp_spell_token (pfile, rhs, end);
427 *end = '/n';
428
429 cpp_push_buffer (pfile, buf, end - buf, /* from_stage3 */ true);
430 _cpp_clean_line (pfile);
431
432 /* Set pfile->cur_token as required by _cpp_lex_direct. */
433 pfile->cur_token = _cpp_temp_token (pfile);
434 *plhs = _cpp_lex_direct (pfile);
435 valid = pfile->buffer->cur == pfile->buffer->rlimit;
436 _cpp_pop_buffer (pfile);
437
438 return valid;
439 }

Above at line 429, the concatenated literal name is pushed into the buffer slot of file, then following _cpp_lex_direct will fetch this concatenated name as token and assign it to plhs. Also see that when return back to paste_all_tokens, at line 485, this token is placed into a new context too, which will be fetched when next invoke cpp_get_tokenl.
Before leaving this section, we have a look at where token in lookahead slot comes from in _cpp_lex_token. It dues to we step back one or more tokens in base context.

1153 void
1154 _cpp_backup_tokens (cpp_reader *pfile, unsigned int count) in cppmacro.c
1155 {
1156 if (pfile->context->prev == NULL)
1157 {
1158 pfile->lookaheads += count;
1159 while (count--)
1160 {
1161 pfile->cur_token--;
1162 if (pfile->cur_token == pfile->cur_run->base
1163 /* Possible with -fpreprocessed and no leading #line. */
1164 && pfile->cur_run->prev != NULL)
1165 {
1166 pfile->cur_run = pfile->cur_run->prev;
1167 pfile->cur_token = pfile->cur_run->limit;
1168 }
1169 }
1170 }
1171 else
1172 {
1173 if (count != 1)
1174 abort ();
1175 if (pfile->context->direct_p)
1176 FIRST (pfile->context).token--;
1177 else
1178 FIRST (pfile->context).ptoken--;
1179 }
1180 }

At the end of enter_macro_context, notice that at line 750, now the identifier node of the macro is flagged as expansion disable. That means if the macro is found in the expanded body in later read-in, it will be treated as normal identifier. It prevents macro to invoke itself then cause indefinitely recursion and breakdown the compiler.
And when the context of the macro expansion is handled and popped out, the macro is enable again.

1024 void
1025 _cpp_pop_context (cpp_reader *pfile) in cppmacro.c
1026 {
1027 cpp_context *context = pfile->context;
1028
1029 if (context->macro)
1030 context->macro->flags &= ~NODE_DISABLED;
1031
1032 if (context->buff)
1033 _cpp_release_buff (pfile, context->buff);
1034
1035 pfile->context = context->prev;
1036 }