new xtype="strings"

[mbtaylor]

Similar but not identical to the wontfix issues #12 and #13, an xtype that can robustly represent a 1-d array of variable-length strings, which is not otherwise possible in VOTable. This results from VOTable discussions with @msdemlei who wasn't keen on those earlier suggestions because of possible trouble with delimiters, but supports this one.

The suggestion here is to nominate a delimiter - say pipe '|' - with an escaping mechanism, say a backslash '\'. So:

• strings are delimited with '|'
• a literal '|' is represented by '\|'
• a literal '\' is represented by '\\'

This would address the long-standing annoyance in VOTable that arrays of variable-length strings are not possible. It does not permit multidimensional string arrays (N-d char/unicodeChar arrays for N>2), but such things are very rarely seen.

It would be nice to have this in DALI 1.2, but if it's too late for that, in absence of major objections, I plan to prototype it soon in STIL anyway. I can write a PR if/when that's appropriate.

[pdowler]

So support for this in VOTable itself is a non-starter? That was the goal even when I created the referenced wontfix issues (just to provoke disussion, not a completely thought out solution)

I am OK with this in principle as there are several use cases that could take advantage of it, but I have concerns:

1. By using xtype="strings" we still do not have a way to do mutliple values that use anotgher xtype already. I am thinking about mutliple URIs in particular. Maybe that's OK and consistent with our acceptance of multipolygon and multiinterval... if we decide we need it later on we will need to add xtype="multiuri" and maybe there are only a handful of such use cases.
2. There is the fact that ObsCore.pol_states uses / as the separator, so this xtype cannot describe that serialization...
3. The ObsCore.pol_states also requires leading and trailing separators to make ADQL queries (LIKE) easier to write...

[mbtaylor]

It's not impossible to imagine a solution in VOTable itself, but it would be quite disruptive. I consider an xtype-based solution much better since it can accommodate existing parsers without major backward compatibility issues. If done in xtype, then existing VOTable parsers will read the content as a (somewhat human-readable) long string, while xtype="strings"-aware parsers can make sense of the value as an array.

If it's done as some kind of VOTable primitive then for BINARY/BINARY2 we'd need some kind of rules about per-element character/code-unit counts either embedded in the binary stream or stored in some kind of heap at the end of the fixed-width part of the binary stream. The former would certainly completely break any attempt by existing BINARY/2 parsers to make sense of the stream; the latter could maybe(?) be fixed not to, but would prevent a streaming parse. I think this solution would also require some (also non-backward-compatible) reworking of the syntax of the arraysize attribute. Unless we just reinvent a new xtype-like attribute to fulfill this purpose...

To me the xtype solution seems much cleaner. But I take your point (1) about the possible requirement of xtyping string array elements. The delimiter doesn't have to be a "|" though "/" doesn't look like a particularly obvious choice ... but if adopting that as a solution for ObsCore.pol_states looks like a major win then your points (2) and (3) could be swallowed.

[pdowler]

When looking for a delimiter for CAOM keywords (which tend to be phrases rather than words so space was out) the | was by far the best choice and was adopted there. That has proven to be a good decision with zero issues/complaints for 8 years or so.

The / is more commonly used in text (ratios like Fe/H, for example). So I would be OK with this xtype using | delimiter.

Did you think about encoding literal values of delimiter rather than escaping? I guess everyone has to encode/decode or escape/unescape the same amount of times so it doesn't seem much different. I personally don't like either and would prefer just saying "reserved character" in the place the content is defined (eg a data model).

[msdemlei]

On Fri, Oct 10, 2025 at 11:00:42AM -0700, Patrick Dowler wrote: pdowler left a comment (ivoa-std/DALI#66) Did you think about encoding literal values of delimiter rather than escaping? I guess everyone has to encode/decode or escape/unescape the same amount of times so it doesn't seem much different. I personally don't like either and would prefer just saying "reserved character" in the place the content is defined (eg a data model).

I am totally opposed to saying "well, don't have | in your values, then". That is a certain way to make people very unhappy, and half the encoders won't bother to ascertain that the values indeed don't have a pipe. You could say "urlencoding is ok, and you must urlencode | in values" (say); whether that's simpler in practice than backslash-escaping I won't decide. But it's true that urlencoding is available already in most languages, so you'd have my blessing for that, too.