Studying note of GCC-3.4.6 source (1)

About 4 years ago, I joined GDNT – the coventure of Nortel in China and worked in project of Radio access network of 3G UMTS; where I used GCC first time. At that time Nortel widely used GCC as the official compiler for its giant project of UMTS, CDMA, self developed tools etc (later GCC is also used in the 4G projects of Wimax and LTE). Every week I needed build several project loads to verify my code of bug fix; in every loadbuild the files got compiled amounted to thousands, and the final object file was larger than 100 megabytes. GCC worked with surprising stability. In these years I only encounter two collapses of GCC, one in which dued to I try to make separate compilation of template which GCC still doesn’t support yet (as far as I can tell, till now only the front-end of EDG can do that. And before the C++ standard being stable【ISO-IEC - 14882-1998】, the EDG front-end is the pratical standard. At http://www.edg.com/ can find something about EDG), and fed GCC with weird codes. And cause generating wrong immediate tree code which triggers assertion within GCC (GCC can’t find out the semantics error). Though GCC exitted immediately, it gave detailed diagnostical dump – an elegant give up.
This fantastic tool attracts me deeply! Though I had learnt principle of compilation before, but facing it I just found I knew little about it. Thanks to its open source, I can peep into its mystical body (at least to me). By these years digging into its source code, I have known something about this compiler but its veil is still not fully opened. I am just glad to share all of you with my studying note of GCC source (the GCC concerned is v3.4.6, C++ front-end, host: x86/Linux, target: x86/Linux); the note is far from complete and still growing.
Reference
[1] Programming language pragmatics, 2nd edition
[2] gccint, version 3.4.6
[3] ISO-IEC-14882-2003
[4] The C Preprocessor April 2001, for GCC V3
[5] cppinternals
[6] Using the GNU Compiler Collection
[7] Inside The C++ Object Model, Stanley B.Lippman
[8] GCC Complete Reference
[9] The design and evolution of C++, by Bjarne Stroustrup
[10] Linkers & Loaders, by John R. Levine
[11] Efficient Instruction Scheduling Using Finite State Automata, by Vasanth Bala, Norman Rubin
[12] Compilers: Principles, Techniques, and Tools, 2nd edition
[2] and [5] can be found at YOUR-GCC-SOURCE-DIR/gcc/doc. [1], [7], [9], [10] and [12] give some useful background knowledge.
Preparation before going ahead
Some non-trivial GCC’s codes are generated by GCC’s tools. Before really dipping into the source code; we need first compile the GCC project to make them generated. I just skip the steps to download the source(http://gcc.gnu.org/mirrors.html is the official site for download), configure and compile it. There are plent of links in the net tell about it (Note: if already installed GCC, using g++ -### can check the configure command).
Software architecture of GCC
The GCC as a whole can be separated into two parts: the front-end, and the back-end. Preprocessor (if there is), lexer (for syntax analysis), parser (for semantic analysis) are implemented in front-end, the purpose of front-end is to transform the source language into intermediate form which is source language independent. In theory, to introduce new language, what needs to do is to realize preprocessor, lexer and parser, but in fact, we still need to wrtie codes to setup the needed environment.
In GCC of version 3.4.6, the common intermediate language is RTL (register transfer language). RTL is a simple langauge, which can be easily degraded into assemble language; so it is inappropriate to reduce the source code into it. In fact, the front-end will reduce the code first into an intermediate tree and do lot of manipulation before turning it into RTL form which will be fed to the back-end.
The aim of back-end is to generate assemble code. As a widely used compiler, GCC can be built to support varies host platforms. To achieve this goal, GCC uses files called machine description which describes not only the instruction set, but also the pipeline’s characteristices, and chance for optimization due to the architecture. For specified target, these files will be handled by several tools to generate related files which will be used in compiling the GCC. To introduce new machine, the major work is to offer the new correct machine description file (usually, it also needs define some helper functions to implement the necessary processing logic).
And At here http://www.ibm.com/developerworks/linux/library/l-gcc4/index.html?S_TACT=105AGX52&S_CMP=content, find information about the newest version of GCC.
The front end
When we call “gcc –o xxx xxx.c”, we just call a shell, this shell will parse the command line, do some preparation upon the host platform (pay attention to the difference between host machine and target machine) following to the options it recognizes, and then calls the appropriate compiler according to the postfix of the file and passes those unrecognized options to the compiler. We don’t see this shell here, what we focus is the real compiler.

1. Overview

The front-end will read in and parse the program written in specified language and transform it into a language independent form. In theory, every front-end can generate distinctive form; however, in GCC to reuse code as possible, all intermediate trees built by different front-ends just use the same set of tree node. Of course, the nodes in tree must be diverse enough to cover the languages. This form of tree is so important in handling C/C++ program; it is worth focus and effort first.

1.1. Tree representation in front-ends

To accommodate the already existing and to be added front-ends, there are tens kinds of nodes (not all can be leaf). All nodes begin with a common part as below. It occupies the beginning of all the structures.

129 struct tree_common GTY(()) in tree.h
130 {
131 tree chain;
132 tree type;
133
134 ENUM_BITFIELD(tree_code) code : 8;
135
136 unsigned side_effects_flag : 1;
137 unsigned constant_flag : 1;
138 unsigned addressable_flag : 1;
139 unsigned volatile_flag : 1;
140 unsigned readonly_flag : 1;
141 unsigned unsigned_flag : 1;
142 unsigned asm_written_flag: 1;
143 unsigned unused_0 : 1;
144
145 unsigned used_flag : 1;
146 unsigned nothrow_flag : 1;
147 unsigned static_flag : 1;
148 unsigned public_flag : 1;
149 unsigned private_flag : 1;
150 unsigned protected_flag : 1;
151 unsigned deprecated_flag : 1;
152 unsigned unused_1 : 1;
153
154 unsigned lang_flag_0 : 1;
155 unsigned lang_flag_1 : 1;
156 unsigned lang_flag_2 : 1;
157 unsigned lang_flag_3 : 1;
158 unsigned lang_flag_4 : 1;
159 unsigned lang_flag_5 : 1;
160 unsigned lang_flag_6 : 1;
161 unsigned unused_2 : 1;
162 };

Above at line 134, ENUM_BITFIELD for Version 3.46, will be expanded into __extension__ enum, and chain at line 131 will link the node into the tree if needed.
The meaning of some flags defined in the structure, and macros defined to access them are shown in below (words in red).
Ø TREE_TYPE ((NODE)->common.type)
In all nodes that are expressions, this is the data type of the expression.
² In POINTER_TYPE nodes, this is the type that the pointer points to.
² In ARRAY_TYPE nodes, this is the type of the elements.
² In VECTOR_TYPE nodes, this is the type of the elements.
Ø TREE_ADDRESSABLE((NODE)->common.addressable_flag)
² In VAR_DECL nodes, nonzero means address of this is needed. So it cannot be in a register.
² In a FUNCTION_DECL, nonzero means its address is needed. So it must be compiled even if it is an inline function.
² In a FIELD_DECL node, it means that the programmer is permitted to construct the address of this field. This is used for aliasing purposes: see record_component_aliases
² In CONSTRUCTOR nodes, it means object constructed must be in memory.
² In LABEL_DECL nodes, it means a goto for this label has been seen from a place outside all binding contours that restore stack levels.
² In *_TYPE nodes, it means that objects of this type must be fully addressable. This means that pieces of this object cannot go into register parameters, for example.
² In IDENTIFIER_NODEs, this means that some extern decl for this name had its address taken. That matters for inline functions.
Ø TREE_STATIC ((NODE)->common.static_flag)
² In a VAR_DECL, nonzero means allocate static storage.
² In a FUNCTION_DECL, nonzero if function has been defined.
² In a CONSTRUCTOR, nonzero means allocate static storage.
Ø TREE_VIA_VIRTUAL ((NODE)->common.static_flag)
² Nonzero for a TREE_LIST or TREE_VEC node means that the derivation chain is via a `virtual' declaration.
Ø TREE_CONSTANT_OVERFLOW ((NODE)->common.static_flag)
² In an INTEGER_CST, REAL_CST, COMPLEX_CST, or VECTOR_CST this means there was an overflow in folding. This is distinct from TREE_OVERFLOW because ANSI C requires a diagnostic when overflows occur in constant expressions.
Ø TREE_SYMBOL_REFERENCED
(IDENTIFIER_NODE_CHECK (NODE)->common.static_flag)
² In an IDENTIFIER_NODE, this means that assemble_name was called with this string as an argument.
Ø CLEANUP_EH_ONLY ((NODE)->common.static_flag)
² In a TARGET_EXPR, WITH_CLEANUP_EXPR, CLEANUP_STMT, or element of a block's cleanup list, means that the pertinent cleanup should only be executed if an exception is thrown, not on normal exit of its scope.
Ø TREE_OVERFLOW ((NODE)->common.public_flag)
² In an INTEGER_CST, REAL_CST, COMPLEX_CST, or VECTOR_CST, this means there was an overflow in folding, and no warning has been issued for this subexpression. TREE_OVERFLOW implies TREE_CONSTANT_OVERFLOW, but not vice versa.
Ø TREE_PUBLIC((NODE)->common.public_flag)
² In a VAR_DECL or FUNCTION_DECL, nonzero means name is to be accessible from outside this module. In an IDENTIFIER_NODE, nonzero means an external declaration accessible from outside this module was previously seen for this name in an inner scope.
Ø TREE_PRIVATE ((NODE)->common.private_flag)
² Used in classes in C++.
Ø CALL_EXPR_HAS_RETURN_SLOT_ADDR ((NODE)->common.private_flag)
² In a CALL_EXPR, means that the address of the return slot is part of the argument list.
Ø TREE_PROTECTED ((NODE)->common.protected_flag)
² Used in classes in C++. In a BLOCK node, this is BLOCK_HANDLER_BLOCK.
Ø CALL_FROM_THUNK_P ((NODE)->common.protected_flag)
² In a CALL_EXPR, means that the call is the jump from a thunk to the thunked-to function.
Ø TREE_SIDE_EFFECTS ((NODE)->common.side_effects_flag)
² In any expression, nonzero means it has side effects or reevaluation of the whole expression could produce a different value. This is set if any subexpression is a function call, a side effect or a reference to a volatile variable.
² In a *_DECL, this is set only if the declaration said `volatile'.
Ø TREE_THIS_VOLATILE ((NODE)->common.volatile_flag)
² Nonzero means this expression is volatile in the C sense: its address should be of type `volatile WHATEVER *'. In other words, the declared item is volatile qualified. This is used in *_DECL nodes and *_REF nodes.
² In a *_TYPE node, means this type is volatile-qualified. But use TYPE_VOLATILE instead of this macro when the node is a type, because eventually we may make that a different bit. If this bit is set in an expression, so is TREE_SIDE_EFFECTS.
Ø TYPE_VOLATILE (TYPE_CHECK (NODE)->common.volatile_flag)
² Nonzero in a type considered volatile as a whole.
Ø TREE_READONLY ((NODE)->common.readonly_flag)
² In a VAR_DECL, PARM_DECL or FIELD_DECL, or any kind of *_REF node, nonzero means it may not be the lhs of an assignment.
² In a *_TYPE node, means this type is const-qualified (but the macro TYPE_READONLY should be used instead of this macro when the node is a type).
Ø TYPE_READONLY (TYPE_CHECK (NODE)->common.readonly_flag)
² Means this type is const-qualified.
Ø TREE_CONSTANT ((NODE)->common.constant_flag)
² Value of expression is constant. Always appears in all *_CST nodes. May also appear in an arithmetic expression, an ADDR_EXPR or a CONSTRUCTOR if the value is constant.
Ø TREE_UNSIGNED ((NODE)->common.unsigned_flag)
² In INTEGER_TYPE or ENUMERAL_TYPE nodes, means an unsigned type. In FIELD_DECL nodes, means an unsigned bit field.
Ø TREE_ASM_WRITTEN ((NODE)->common.asm_written_flag)
² Nonzero in a VAR_DECL means assembler code has been written.
² Nonzero in a FUNCTION_DECL means that the function has been compiled. This is interesting in an inline function, since it might not need to be compiled separately.
² Nonzero in a RECORD_TYPE, UNION_TYPE, QUAL_UNION_TYPE or ENUMERAL_TYPE if the sdb debugging info for the type has been written.
² In a BLOCK node, nonzero if reorder_blocks has already seen this block.
Ø TREE_USED ((NODE)->common.used_flag)
² Nonzero in a *_DECL if the name is used in its scope.
² Nonzero in an expr node means inhibit warning if value is unused.
² In IDENTIFIER_NODEs, this means that some extern decl for this name was used.
Ø TREE_NOTHROW ((NODE)->common.nothrow_flag)
² In a FUNCTION_DECL, nonzero means a call to the function cannot throw an exception. In a CALL_EXPR, nonzero means the call cannot throw.
Ø TYPE_ALIGN_OK (TYPE_CHECK (NODE)->common.nothrow_flag)
² In a type, nonzero means that all objects of the type are guaranteed by the language or front-end to be properly aligned, so we can indicate that a MEM of this type is aligned at least to the alignment of the type, even if it doesn't appear that it is. We see this, for example, in object-oriented languages where a tag field may show this is an object of a more-aligned variant of the more generic type.
Ø TREE_DEPRECATED ((NODE)->common.deprecated_flag)
² Nonzero in an IDENTIFIER_NODE if the use of the name is defined as a deprecated feature by __attribute__((deprecated)).
List 1 flags in tree_common

1.1.1. Tree node definition

The definition of tree node is given below.

45 typedef union tree_node *tree; in coretypes.h

1772 union tree_node GTY ((ptr_alias (union lang_tree_node), in tree.h
1773 desc ("tree_node_structure (&%h)")))
1774 {
1775 struct tree_common GTY ((tag ("TS_COMMON"))) common;
1776 struct tree_int_cst GTY ((tag ("TS_INT_CST"))) int_cst;
1777 struct tree_real_cst GTY ((tag ("TS_REAL_CST"))) real_cst;
1778 struct tree_vector GTY ((tag ("TS_VECTOR"))) vector;
1779 struct tree_string GTY ((tag ("TS_STRING"))) string;
1780 struct tree_complex GTY ((tag ("TS_COMPLEX"))) complex;
1781 struct tree_identifier GTY ((tag ("TS_IDENTIFIER"))) identifier;
1782 struct tree_decl GTY ((tag ("TS_DECL"))) decl;
1783 struct tree_type GTY ((tag ("TS_TYPE"))) type;
1784 struct tree_list GTY ((tag ("TS_LIST"))) list;
1785 struct tree_vec GTY ((tag ("TS_VEC"))) vec;
1786 struct tree_exp GTY ((tag ("TS_EXP"))) exp;
1787 struct tree_block GTY ((tag ("TS_BLOCK"))) block;
1788 };

No doubt, the node is defined as union. Notice ptr_alias at line 1772, it tells GTY (garbage collection service in GCC, we ignore it here for the moment) that pointer to tree_node in fact points to lang_tree_node which is front-end specified and builds up from tree_node with extra fields for the language characteristics (so it can be pointed by pointer of tree_node). In C++ front-end, lang_tree_node has following definition.

472 union lang_tree_node GTY((desc ("cp_tree_node_structure (&%h)"), in cp-tree.h
473 chain_next ("(union lang_tree_node *)TREE_CHAIN (&%h.generic)")))
474 {
475 union tree_node GTY ((tag ("TS_CP_GENERIC"),
476 desc ("tree_node_structure (&%h)"))) generic;
477 struct template_parm_index_s GTY ((tag ("TS_CP_TPI"))) tpi;
478 struct ptrmem_cst GTY ((tag ("TS_CP_PTRMEM"))) ptrmem;
479 struct tree_overload GTY ((tag ("TS_CP_OVERLOAD"))) overload;
480 struct tree_baselink GTY ((tag ("TS_CP_BASELINK"))) baselink;
481 struct tree_wrapper GTY ((tag ("TS_CP_WRAPPER"))) wrapper;
482 struct tree_default_arg GTY ((tag ("TS_CP_DEFAULT_ARG"))) default_arg;
483 struct lang_identifier GTY ((tag ("TS_CP_IDENTIFIER"))) identifier;
484 };