Character set: To study a language first we have to familiarize the character set. For example to study English language first we have to study the alphabets. Similarly here the character set includes letters(A to Z & a to z), digits(0 to 9), special characters+,-,?,*,/, ) white spaces(non printable) etc..
Token: It is the smallest individual units similar to a word in English or Malayalam language. C++ has 5 tokens
1) Keywords
Eg: float is used to declare variable to store numbers with decimal point. We can’t use this for any other purpose.
2) identifier: These are user defined words.
Eg: variable name, function name, class name, object name etc…
3) Literals (Constants): Its value does not change during execution
Eg: In maths π = 3.14157 and boiling point of water is 100.
4) Punctuators: In English or Malayalam language punctuation mark are used to increase the read-ability but here it is used to separate the tokens.
Eg:{,},(,)……….
5) Operators: These are symbols used to perform an operation(Arithmetic, relational, logical, etc…)
These are reserved words for the compiler. We can’t use for any other purposes.
