Improvement of identifiers

[olivhoenen]

Non-backward compatible changes in identifiers are rare but not without impact on users/codes (cf. comments from @DavidPCoster in #133 (comment) and older mentions from @fcasson in https://jira.iter.org/browse/IMAS-3434).

While simple renames shall be avoided, we also don't want to keep misleading ones for sake of clarity in the Data Dictionary or not follow best practice for standardization purpose (e.g. https://github.com/iterorganization/IMAS-Data-Dictionary/pull/133/files#diff-8a9927da42874e8e11e733a5e46329b23df7a4396d43da70b291173bad3ecc1f).

A first improvement proposal could simply build a list of compatible names (or alias) for the same identifier (or use the same kind of mechanism already in place for field names using the NBC tags). Different alias names would share the same index value and the same description.

A second improvement proposal would be to stop adding the full description as a field in the IDS, this shall be available in the Data Dictionary and as meta-data only. I understand that the rational for keeping this is to allow describing the non-official identifiers, but unless there are many success stories of groups using locally defined identifiers before requesting the addition as an official identifier, this seems to be a case of adding many fields for a very small use-case. Also note that one can still describe the non-standard identifier in the free-form ids_properties/comment for instance.

Please share comments and ideas. @SimonPinches @imbeauf @maarten-ic @hrdoktorx

[imbeauf]

I think the "first improvement" proposed is a good idea. Such metadata is easy to add, then to make it work it requires the development of some library/tool that can handle/recognize the multiple aliases (otherwise the flexibility will give confusion to the user's codes).

About the "second improvement" proposed, I always thought that the description could be more free-form than the two other fields, and could be used to refine the description for a specific use case or bring more context (while keeping consistent with the "default" version of the DD identifier list). I don't know whether this unofficial way of using the description has ever been used. Otherwise, if that field is not customizable, I agree with Olivier that the description node could be removed from the IDS structure and be used only for documentation purposes.

[SimonPinches]

Allowing alternative names (aliases) for the same identifier (e.g. poloidal_flux_differential and diamagnetic_differential) is a good way to evolve the DD without breaking existing codes but it must be clear they all mean the same thing and thus map to one standard name, (e.g. differential).

I agree that the full description should only appear in the documentation (as meta-data).

[hrdoktorx]

I agree with the first proposal of aliases, which is reasonable.
The second proposal however, I do not see as practical for codes such as SOLPS that make an extensive use of private identifiers for GGD grid subsets, neutral particle types, sources, radiation processes, etc... Shoving all the clarifications the long descriptions provide, which come for several IDSs, to one single comment in a different IDS almost guarantees the information get drowned out and lost and the average user will never find it. It seems to me infinitely simpler to just keep the long descriptions as an optional field that the data provider may or may not fill in, and, if it is not filled, then a "default" long description can be provided for identifiers matching existing values listed as part of the standard Data Dictionary.

[olivhoenen]

Would it make sense to try gathering the list of private identifiers (at least the least recent ones) being used and attempt to add them in the DD as official ones? I fear that private identifiers kept for too long will never make it to a public list, if it is used in production in the code developers may not like to have to change the index once it becomes official, unless we care only about the name.

[hrdoktorx]

This would only make sense if the private identifiers are foreseen to eventually become "official". This is not the case for most of them. One still needs to have some flexibility to add useful code-specific information that does not strictly adhere to the Data Dictionary but is valuable for other users of the same code that want to analyze a run in more detail that the DD offers, and would make no sense to add to the DD because the use case is unique to that code.

[SimonPinches]

I'm not sure that identifiers are the right place to store this information, so maybe what's needed are additional capabilities to store more descriptive free-form text for human (rather than machine) consumption? (Assuming ids_properties/comment is insufficient).

[maarten-ic]

Some thoughts (ideal world):

• index should be leading. Once a certain index is standardized its meaning shouldn't change (although this has happened in the past). Codes should be able to assume that checking the index number is sufficient when reading data from an IDS. identifiers could be simplified to just their index, as they are for some "fake" identifiers like coil/geometry/geometry_type in pf_active.
• name is a short string description. Support libraries in the HLI can use this to allow code developers to use these names instead of a magic integer number. An index could have multiple aliases defined (proposal 1), which just map to the same index value. The name is also metadata, and would not have to be stored in the IDS.
• description is metadata, describing precisely what is meant with the identifier. Descriptions can be clarified in future versions of the Data Dictionary. Like the documentation of an IDS node, this is metadata that does not need to be stored as data in the IDS.

Unfortunately, the world isn't ideal:

• Some codes do string matching for identifier names, for example 🔒CYRANO.
• This only works for standardized identifiers, and (as @hrdoktorx mentions) has a risk of loosing the meaning of a private identifier.

Given the current situation of identifiers and their usage, I think that changing the name of identifiers, and/or adding aliases has the risk to break existing codes. The only backwards compatible changes that we can make are updates to the description to clarify potential misunderstandings...

[olivhoenen]

Given the current situation of identifiers and their usage, I think that changing the name of identifiers, and/or adding aliases has the risk to break existing codes. The only backwards compatible changes that we can make are updates to the description to clarify potential misunderstandings...

The main idea behind aliases is to avoid breaking existing codes, they would serve no other purpose, so if this is not the case they should not be introduced.

[maarten-ic]

I'm not certain if adding aliases will or will not break existing codes, but there's a risk that it will.
If someone has written a code like the following, then the changes in PR#133 or adding aliases will break the code:

! look for D_T_to_He4_n process in distribution sources:
do i1=1,size(distribution_sources%source)
    do i2=1,size(distribution_sources%source(i1)%process)
        if(distribution_sources%source(i1)%process(i2)%type%name(1)=='D_T_to_He4_n')then
             ! do something with T+D -> 4He+neutron nuclear reaction source:
             ! ...
        endif
    enddo
enddo

Also note that the description of source(i1)/process(i2)/type doesn't match with the values in the identifier. I've added a comment to PR#133 for this.

[olivhoenen]

DD 4.1 will introduce the possibility to keep an alias name in the identifier, as well as units. We do not foresee at this stage to remove the description from the structure.