Lexical elements

DreamShader » DreamShaderLang » Lexical elements

The character-level rules of DreamShaderLang: whitespace, comments, identifiers, case sensitivity, string and numeric literals, and how a section body is split into statements.

(since 2.0.0) One lexer reads every source, .dss and 1.x alike, into one token stream; the 1.x grammar is read from those tokens by the legacy front end. Through 1.9.x the declaration grammar and the Graph grammar each had their own character scanner, with rules of their own. Three options apply to a .dsm / .dsf: a /// line is an ordinary comment, a '…' pair on one line is a string (the Content Browser's Texture2D'…' shell), and a # line at the start of a line is a directive token.

Synopsis

<token> := <identifier> | <keyword> | <string-literal> | <number> | <punctuation>

<identifier>     := { <letter> | _ } { <letter> | <digit> | _ }…
<qualified-name> := <identifier> :: <identifier>
<string-literal> := " { <character> | \<escape> }… "
<punctuation>    := { | } | ( | ) | [ | ] | ; | = | , | . | : | # | the operators of a Graph expression

Whitespace and comments may appear between any two tokens and are otherwise insignificant.

Whitespace

Space, tab, vertical tab, form feed, carriage return and line feed separate tokens (since 2.0.0: ASCII only; other Unicode whitespace, such as a no-break space, is DSH2101). Newlines carry no syntactic weight; they matter only to preprocessor directives, #Region directives inside a Graph body (Layout), and diagnostic line numbers.

(since 2.0.0) Any whitespace separates any two words, in every section: a tab between a type and a name is as good as a space.

Comments

• Block comments do not nest. /* a /* b */ c */ ends at the first */; the trailing c */ is code again.
• An unterminated block comment is an error (since 2.0.0): DSH2102, reported at the opening /*. 1.x accepted it and dropped the rest of the file.
• Comments are recognized identically everywhere, including inside Graph bodies and while braces, parentheses and brackets are counted, so a brace or quote inside a comment never unbalances a block.
• A comment hides an import (since 2.0.0): import is a token, so one inside /* … */ is commented out. See import. It does not hide a preprocessor directive: the #if family is found by a line scan that runs before any comment is understood, so a #if inside /* … */ is still a directive. A line whose first non-whitespace characters are // never is. See Preprocessor.

Identifiers

• There is no length limit. (since 2.0.0) Only ASCII letters count; a letter outside ASCII is DSH2101.
• Block words, section names and type names are not reserved. A property, variable, parameter or function may be named Shader, Graph or float; whether it then resolves is a matter for the context it appears in. The lower-case reserved words — in, out, if, return, const, … — are reserved (since 2.0.0).
• A declaration name is one identifier token in every section (since 2.0.0); 1.x checked only that a Properties name was non-empty, so Properties { float 1Bad = 0; } parsed. Now 1Bad is a malformed number (DSH2105) and the declaration is DSH3250.
• Namespace-qualified names use ::, as in Common::ApplyTint. See Namespace.

When a name reaches generated HLSL it is sanitized: every character outside A–Z a–z 0–9 _ becomes _, a leading digit gains a _ prefix, runs of consecutive __ collapse to one, and a result that is empty or entirely underscores becomes DreamShaderSymbol. This is why Common::ApplyTint and Common_ApplyTint collide in generated code.

Case sensitivity

Top-level block keywords and the lower-case reserved words are case-sensitive. Everything else keyword-like in the 1.x grammar is matched ignoring case, with the exceptions marked below. Names the engine defines — material attributes, node classes, pins — are matched exactly and, in a 1.x file, also ignoring case with the warning DSH5276; names the file declares, with DSH5275 (since 2.0.0, legacy rule L19). The preprocessor, which runs before the grammar and is not part of it, is case-sensitive throughout.

So shader(Name="X") and SHADER(Name="X") are both syntax errors (DSH2240), while properties = { … }, settings { domain = "ui"; } and Shader(name="X") are all accepted.

A keyword is a whole word: ShaderFunction is its own identifier, never Shader followed by something, and ShaderLayerBlend is never ShaderLayer.

String literals

A string is opened by ", ends at the next unescaped " on the same line, and is unescaped by the lexer.

• (since 2.0.0) A string never spans lines: a line break before the closing " ends it, and an unclosed string is DSH2103, in a section and in a Graph body alike. 1.x let a string run across lines and closed one in a Graph body silently at the end of the input.
• In a 1.x file and a .dsh, a pair of ' on one line is a string too, with no escapes — the shell of a Content Browser reference, Texture2D'/Game/T.T'. A lone ' is DSH2101.

Anywhere a value may be quoted — settings values, metadata values, attribute values, argument values — quoting is optional: Domain = UI; and Domain = "UI"; are equivalent. A quoted value is the string's text; an unquoted one is the text of its tokens.

[!NOTE] A quoted Settings or Options value is not trimmed: Domain = " UI "; stores UI with its spaces.

Numeric literals

Declaration grammar

(since 2.0.0) A number is a token everywhere, lexed by the rules below; a section then reads the token's text the way its consumer asks:

What is not accepted is DSH3254 for a default, DSH3258 for a SortPriority and DSH3275 for a Layout coordinate.

The vector-literal form is deliberately loose, as in 1.x. The first ( and the last ) delimit the components, and the text before ( is ignored entirely — float3(1,0,0), vec3(1,0,0), (1,0,0) and Nonsense(1,0,0) all read identically. The interior is split on , without tracking nesting, so a nested call in a component breaks the split. Each component may be a number (with an exponent and a trailing f) or true / false.

The unfilled default is (0, 0, 0, 1).

Graph expression grammar

The suffix is part of the token text, so 0.55f is the number 0.55.

(since 2.0.0) Every operator of the 2.0 grammar is a token: %, <, !, &&, ?, [, ] no longer end a Graph expression and drop the rest of it. What a 1.x Graph did not have is refused by name instead — DSH2200 to DSH2222. See Unsupported constructs.

Statement separation

A section body is a list of ;-separated statements, read token by token (since 2.0.0).

• A ; separates statements only outside (), [] and {}. A string is one token, so a ; inside one is just text.
• Empty statements are skipped, so stray ;; is harmless.
• The ; of the last statement before the section's closing } is optional.
• The ; after a section's closing } is optional, and so is the = between a section name and its block (since 1.5.0).
• A value — a default, a setting, a binding source, an argument — runs to the ;, , or ) that ends it outside brackets, so an unquoted attribute value may hold parentheses and commas: Asset = Path(Game, "MaterialFunctions/F_X") is one value.

(since 2.0.0) 1.x split statements with three different splitters, two of which looked for a literal space, so float3\tColor; failed in Inputs, Outputs, Results and a Shader's Outputs, and opt\tfloat X did not mark an input optional. Both work now.

Diagnostics

Every diagnostic carries the file, line and column of the construct it is about, and is printed <file>(<line>,<column>): DSHnnnn: <message> (since 2.0.0); only a few that concern a whole file, such as an include that cannot be resolved, report line 1. 1.x located only the messages that ended in near index {Index}. See import.

Example

// Line comment before the top-level declaration.
Shader(Name="DreamShaderTests/Corpus/M_Comments")
{
    /* Block comment
       spanning multiple lines. */
    Settings = {
        Domain = "UI";        // trailing line comment
        ShadingModel = Unlit; // quotes are optional
    }

    Properties {
        // Any whitespace may separate a type from its name.
        ScalarParameter Rough = 0.5 [Group="Surface"; Slider(0, 1)];
        vec3            Tint  = vec3(1.0, 0.4, 0.1);
        float           Fudge = 1.0f;   // reads as 1.0
    }

    Outputs {
        vec3	Color;                  // a tab is whitespace too
        Base.EmissiveColor = Color
    }                                   // the last ';' in a block is optional

    Graph {
        vec2 UV = UE.TexCoord(Index = 0);
        Color = vec3(Rough, Rough, UV.x) * Tint;
    }
}

See also

• Keyword index — every reserved word, with its case rule
• Source files — the file kinds these tokens live in
• import — a token of the language since 2.0.0, and where positions come from
• Preprocessor — the line-oriented directives, and the one case-sensitive keyword set
• Types — the full type-token catalogue and the GLSL aliases
• Properties — the section with Group("…") { … } scopes
• Inputs / Outputs / Results — the typed-parameter sections
• Layout — #Region / #EndRegion, the only directive-like syntax
• Literals — literal forms inside a Graph block
• Unsupported constructs — what a 1.x Graph expression refuses
• Diagnostics index — every code