Add: "include" keyword, allowing to include files without external tools

[glx22]

Many GRF workflows use tools to combine multiple files into a single NML (C preprocessor, python, ...).

So I though it would be nice to be able to do that natively in NML. It's done during parsing by switching lexer data.

I also added optional <ID> = <expression> as include parameters, allowing to replace some parts inside the included file during parsing (very basic equivalent of #define from C preprocessor).

Include syntax is include(<filename>[, <ID> = <expression>[, ...]]);.

Added identifier concatenation support using <identifier>##<identifier>.

--nml output for 042_include.nml:

param[0] = 0;

param[0] = (param[0] + 1);

const _NAMExy = 1;
param[0] = (param[0] + _NAMExy);


param[0] = (param[0] + 2);

const _testxy = 2;
param[0] = (param[0] + _testxy);


param[0] = (param[0] + 3);

const _test2xy = 3;
param[0] = (param[0] + _test2xy);

[andythenorth]

Tested simplest case with FIRS, works as expected.

andythenorth/firs@0beca16

[andythenorth]

Tested locally as working with params.

include("src/polar_fox/cargo_classes/cargo_class_constants.nml", cabbage=2, ham=1);

[zephyris]

Is there anything us NewGRF authors can do to help test this? It would be nice to get it out of draft status!

[frosch123]

The character to concat identifiers is .
Classic preprocessors would use ##

Do we block pontential future "member access" by using "." for concatenation?

[glx22]

Well it's still very hacky, and needs more work.

[Pixel-Tony]

I wholeheartedly approve of the idea, because more code reusability is always welcomed, but I stand against token concatenation and identifier mutation capabilities being included.

One of my main arguments against it is, if user chooses to use this feature to create templated names a. e. purchase spriteset names with common prefix, then he will still need to write (and remember) full prefix in the place of its usage.

I am currently developing a language server for NML and ability to join identifiers makes a lot of things ambiguous and error-prone, just like complex defines in C do. IMHO, we should fix boilerplate-y sections instead of providing generic macros support.

[Pixel-Tony]

Also, we should consider incomplete code inside included file: is the intended behavior to allow users to use includes inside other AST nodes or parse file separately and include received statements to the resulting unit at the top level?

[andythenorth]

I am still strongly in favour of this, but I'm unsure how to help move it forward.

I am not convinced that we need parameterised includes. Simplicity is the goal for an include feature IMO.

That said, I wonder if we could target, not a file, but a script with params, returning nml as a string?
E.g. include("src/foo.py cabbage=2 ham=1")

Then the author could template the output of foo.py using whatever means they choose, whilst keeping it simple for other cases to use static nml via include("src/bar.nml")

[audigex]

I feel like step 1 here is just to allow the user to write NML in multiple files and then compile them without needing CMAKE or Python as a pre-build step

Being able to parameterise includes or pull python scripts etc certainly seem interesting as concepts but it's solving an extra problem beyond what most newGRF authors are actually doing and IMO should be left to a later step

Simply being able to include other NML files inline would solve a huge chunk of the problem

Eg the user's newGRF file could define the grf{} block, params, rail types, and then include() steam, diesel, electric as sub-files which then includes the required units. That alone would solve the vast majority of organisational/pre-processing woes here

Then we can add clever bells and whistles later

[reldred]

I concur with Audigex here, keep it simple, get it merged, flesh it out later. I rarely found myself going fully parametrized in any of the .grf's I've made, the simple ability to break a giant NFO or NML codebase into bite sized chunks was all I was after when patchman originally taught me how to (ab)use the GCC preprocessor. An in-house solution shipped would help a lot of NML developers who aren't terribly technical, and then we can start to guide them towards more modern development practices.