Support for pre-declared "external" globals for sourced scripts to avoid SC2154 warnings.

[deryni]

We have a series of scripts that operate on a set of shared variables that are created by the top-level script.

The secondary scripts use these variables without declaring them as they are known to exist when run.

shellcheck, obviously, doesn't know this. It would be nice if we could tell it about acceptable external global variables to disable SC2154 warnings about them.

I realize we could use # shellcheck disable=SC2154 for them but that's a larger hammer than I would like as it will also disable other warnings from typos and the like if they happen to appear on that line.

The context=sourced suggestion from #281 is somewhat related to this request.

[koalaman]

Global variables are conventionally in all uppercase (PATH, DISPLAY, SSH_CLIENT, etc). For this reason, ShellCheck generally doesn't warn when using undefined uppercase variables.

The best solution would probably be for ShellCheck to process sourced files though.

[deryni]

That's true for system variables but I'm not sure that's as true for script-internal global variables (but maybe it is though that feels like playing in the system's/shell's playground a little bit).

Processing sourced files would be interesting though, assuming you mean when you pass shellcheck the toplevel file it would "recurse" into the sourced files and consider them in that context.

That wouldn't solve all of my cases (we have some scripts that get sourced and others that get run that all expect the external globals) but it would solve some of them.

[moeriki]

When writing scripts for preset systems this uppercase convention can't always be adhered to.

Case in point. I'm writing scripts for Atlassian Bamboo. There is a global variable bamboo you can refer to for certain configurational values.

[deryni]

Bamboo uses many variables with the bamboo_ prefix but not a single variable. (Though using a single associative array variable would certainly be nice.) But yes, same idea.

[koalaman]

I ran into the same issue with a library set FLAGS_verbosity style variables.

How about a flag for setting a pattern/regex to ignore? This would let you do something like

export SHELLCHECK_OPTS="--ignore-variable-pattern=FLAGS_|bamboo"

to not get these warnings anymore. Alternatively as a directive, starting the file with

# shellcheck ignore-variable-pattern=FLAGS_|bamboo

[deryni]

Seems reasonable and would work for me. Though I'd probably suggest making it require explicit pattern markings (or having both pattern and non-pattern versions so that literal variables could be included in the list safely as well (i.e.

--ignore-unassigned-variables='FLAGS_*|bamboo_*|explicit_global'

or

--ignore-unassigned-variable-patterns='FLAGS_|bamboo_' --ignore-unassigned-variables=explicit_global

That said it also might make sense to handle this more generically (the way we discussed in #380, that is with names/patterns attached to the normal directives).

As the link in my comment on #380 is apparently now dead here's an updated link to the luacheck documentation for how it handles this idea.

The gist is that in addition to a directive like # shellcheck disable=SC2154 you could use # shellcheck disable=SC2154/FLAGS_* or # shellcheck disable=SC2154/bamboo_* or # shellcheck disable=SC2154/explicit_global to scope the disabling to that specific variable/pattern. (I realize that is likely a fair bit more complex of an addition though. Though if my haskell was anything but non-existent I'd be interested in taking a crack at it.)

[moeriki]

Bamboo uses many variables with the bamboo_ prefix but not a single variable.

@deryni That depends. When running inline scripts it's a single variable. ${bamboo.myvar}
When running linked scripts it's all separate. ${bamboo_myvar}

Bamboo variables are exported as bash shell variables. All full stops (periods) are converted to underscores. For example, the variable bamboo.my.variable is $bamboo_my_variable in bash. This is related to File Script tasks (not Inline Script tasks).

• https://confluence.atlassian.com/bamboo/bamboo-variables-289277087.html

[deryni]

@moeriki ${bamboo.myvar} isn't a shell variable. It isn't a legal variable name. (And even if it was that would still be many variables and not a single variable as that isn't how you index into a shell array variable that would be ${bamboo["myvar"]}.)

What's happening there is that bamboo is pre-processing the script for you and replacing that (and I didn't know that worked in inline script bodies, I've never used it, but did know it worked in other places in bamboo itself).

The fact that it works in inline script bodies is sort of scary actually (likely safe-ish and possibly even entirely safe but still not something I want). Just one more reason for me to work on removing them from my builds entirely. =)

And I'm not sure what that "This is related to File Script tasks (not Inline Script tasks)." bit is supposed to mean but the shell variables work in inline scripts just fine as well (I suppose it would have been possible for Atlassian not to export the variables to the environment for inline scripts but that would be odd and they didn't.)

[moeriki]

@deryni Ah I see. Thanks for the insight!

@koalaman your proposition would work fine for me.

[dflock]

I have the same issue. Maybe it would be possible to use a new directive to point to the parent script, where the variables are defined?

[tomprince]

For example, for this file, I get (slightly reformated):

   7:18  warning  srcdir is referenced but not assigned.                    2154 (shellcheck)
  13:19  warning  _objdir is referenced but not assigned.                   2154 (shellcheck)
  18:5   warning  moztopsrcdir appears unused. Verify it or export it.      2034 (shellcheck)
  19:5   warning  commtopsrcdir appears unused. Verify it or export it.     2034 (shellcheck)
  21:5   warning  mozreltopsrcdir appears unused. Verify it or export it.   2034 (shellcheck)
  22:5   warning  commreltopsrcdir appears unused. Verify it or export it.  2034 (shellcheck)
  24:5   warning  commtopobjdir appears unused. Verify it or export it.     2034 (shellcheck)

I would like to specify something like

# shellcheck external_vars="srcdir _objdir"
# shellcheck exported_vars="moztopsrcdir commtopsrcdir mozreltopsrcdir commreltopsrcdir commtopobjdir"

to be explicit about what variables are defined by the sourcing script, and which are used by the sourcing script.

[mat813]

In the FreeBSD ports tree, we use environment variables extensively to pass values from the make(1) framework to the shell scripts doing the more complex stuffs, and I am in the process of cleaning things up it would be nice to have a directive where one could add an exhaustive list of already existing variables, to be able to turn SC2154 on.

[plambert]

This'd be very useful for me, as well; while I'm not a strong enough developer to make a patch or pull request, if there's anything I can do to help accomplish this, let me know.

[matthewpersico]

Am I missing something? Seems to me that if you allow

# shellcheck external_vars="srcdir _objdir"

in one file, you're gonna need it in others. And then you'll get the bright idea to put the common declarations in one file, which you'll need to...

wait for it...

source in! :-)

So instead of a directive, how about just dropping a bunch of declare statements in a file foo.bashinc and sourcing that in with a hard coded path?

[mat813]

So, your idea is nice, for people who have variables that are the same in all their scripts.

It is absolutely not the case for the FreeBSD ports framework usage, a directive is really the only way to go for us to be able to use SC2154.