DocToken¶
struct DocToken
Token.
A token is only valid for the lifetime of the Tokenizer that produced it, as it contains a reference to a substring of the source file.
Variables¶
| Name | Description |
|---|---|
| type | Token type. |
| value | Iterator to first character. |
Functions¶
| Name | Description |
|---|---|
| DocToken | Constructor. |
| str | Get token as string. |
| substr | Get substring of the token as a string. |
Variable Details¶
type¶
DocTokenType type
Token type.
value¶
std::string_view value
Iterator to first character.
Function Details¶
DocToken¶
DocToken(const DocTokenType type = NONE, std::string_view value = std::string_view())
Constructor.
-
type - Token type.
-
value - Token value.
str¶
std::string_view str() const
Get token as string.
substr¶
std::string_view substr(size_t pos = 0) const
Get substring of the token as a string.
-
pos - Position of the first character.