Tokens

• Topaz source code tokens
• Comments
• Constants
• Integer constants
• Float constants
• Character constants
• Simple string constants
• Enhanced string constants
• Boolean constants
• Reserved words
• Keywords
• Hint directives
• Identifiers

Topaz source code tokens

The fundamental lexical building blocks of source code are called tokens. They are the words of the language, formed by combining characters in accordance with the programming language's rules. Tokens fall into five categories:

• Constants
  Actual values in the source code are indicated by numerical or character constants, such as 1 (a tiny integer constant), 1.5 (a float constant), or 'text' (an unicode string constant).
• Keywords
  These are words that are reserved and have a set meaning in the language. They can't be altered or reinterpreted.
  
• Identifiers
  These are the names of symbols defined by the programmer. They are adaptable and reusable. They are governed by the language's scope rules.
  
• Operators
  Typically, they are the symbols +, -, *, div and so forth, which represent mathematical or other operations.
  
• Separators
  Commonly, white space and a semicolon are used here.
  
  

Comments

Constants

Integer constants

Float constants

Character constants

Simple string constants

Enhanced string constants

Boolean constants

Reserved words

Keywords

Hint directives

Identifiers