Add data origin in VOTable output

[gilleslandais]

I propose to add metadata

• to trace the resource queried: author, date of publication, etc
• to keep query parameter: URL, parameters, protocol, version
  see https://github.com/gilleslandais/ivoa-dcp-data-origin

the metadata could be added then in SCS and other protocol

[pdowler]

I am not sure this needs or should be included in DALI. The note gives advice on how services that output VOTable can enhance the metadata, but since none of this is intended to be "machine-readable" and acted upon by client software I don't think DALI is the place for it.

In CADC TAP servers we include the ADQL query in the VOTable as it is valuable later to understand exactly what the result table means (on if someone saves it and looks at it later)... so I can envision some very specific recommendations about reflecting the input/request in the output VOTable (parameters, maybe the ivo resource ID of the service).

For now, the scope of what we can/should say in DALI is still unclear.

[gilleslandais]

Thank to take into account IVOID, and other parameters concerning the query execution - we talked about them in Tucson.

The other information concern Provenance and are may be not usefull for protocols, but they improve the fairness of the output (when VOTable is used). It is an information which can be used in other acces such as sia, ssa, tap, etc.
That's why I think about DALI.

The information can already be easily reported in software such as Topcat or Aladin. Furthermore, astropy includes in its API DataOrigin (https://docs.astropy.org/en/latest/api/astropy.io.votable.dataorigin.DataOrigin.html).

So, the information is like that, at least a little bit, machine-readable. But may be you expect (for querying or provenance info) something more precise/robust? I would be very pleased to discuss with you to improve or complete the information.

[msdemlei]

On Fri, Sep 19, 2025 at 12:43:22PM -0700, Patrick Dowler wrote: I am not sure this needs or should be included in DALI. The note gives advice on how services that output VOTable can enhance the metadata, but since none of this is intended to be "machine-readable" and acted upon by client software I don't think DALI is the place for it.

I'm not sure that machine-readability should be the criterion for inclusion in DALI; for one, there already is citation, which is on the turf of data origin, and then I think the criterion should be "is it something that ought to be part of many/all DAL standards?". For that latter criterion, I think data origin passes: as a VizieR user, I have been grateful for data origin information multiple times already. Giving as much of what is in data origin as possible *does* make a difference when you have a VOTable you wrote yesterday or a month ago. In that sense, I *wish* DAL protocols had at least a strong "should" for it. But then I, too, wonder whether DO shouldn't become a REC or EN of itself. This is partly because it is quite a bit of material; but that's a weak point, because when DALI references it, people have to read it whether or not it is part of the DALI document. No, I rather think it would be preferable if DO could mature a bit more outside of DALI, and I suspect that it could also evolve a bit faster that DALI. So, to me it's more a question of weighing the advantages of having a bit of flexibility against the benefit of having one standard less.

In CADC TAP servers we include the ADQL query in the VOTable as it is valuable later to understand exactly what the result table means

Oh, all of DO is valuable, and I'd totally try to talk you into adopting it for opencadc. If that should still be necessary: Let me know :-)

[molinaro-m]

I will be facing this issue of adding DataOrigin to a specification also for ConeSearch (where a specific issue has bee set up).
In my opinion it would be better if dealt at a general level, DALI or not.
I wonder if it is strictly DALI, or, as Markus proposes, it would be better to reference it within DALI (Sec. 4).

I would vote for the latter solution and provide some time to figure out how many <INFO> elements we are using there and make them more easily manageable by clients.

[pdowler]

It is true that DO can be added to any VOTable (following the note) whether or not it is in DALI, so providers can and do add this now. Putting it into DALI would raise awareness, but I'm not sure 1 larger vs 2 smaller documents is a win.

What would people expect of a provider that aggregates data from many different sources (eg CADC has a TAP service with metadata from 30+ telescopes/missions)? What could such a provider sanely include? I see the main complication that the DO is nominally dependent on which rows are returned in the query result...

And also if it was in DALI and was a SHOULD, what would validators do? They can check for the presence/absence of DO but they can't feasibly check the value/specificity of it? So warnings? Does that encourage people to put less-than-useful placeholders in to score or not lose points?

[pdowler]

So concensus is for DO to mature outside DALI, as a note... maybe EN at some point.

Do we want to add a reference to it now (WD-DALI-1.2)?

[gilleslandais]

We are going to think about an endorsed note.

It could be an opportunity to review the information - in particular the information about the query parameters (which are, for my point of view , with the ivoid of the resources the most interesting part for DALI)

The basic Provenance is clearly an option and it could be a reference to DO (in WD-DALI-1.2? or later ?)

About the cadc use-case where the Origin granularity is at the record level.
Simple INFO can't make the job and if you want that, we need a model (but, I don't think that include model in output serialisation is desirable for DALI)

[molinaro-m]

So concensus is for DO to mature outside DALI, as a note... maybe EN at some point.

Do we want to add a reference to it now (WD-DALI-1.2)?

I wonder if an extra sentence in current Sec. 5.4.3 (Additional Information) is worth.
The balancing is the usual: to put or not an optional suggestion referencing an external document that is under development.

Purely personal view in this case: an extra sentence there won't hurt.

[gmantele]

As @molinaro-m said, it is always a bit annoying to point toward an in-progress external document. However, I agree that adding a simple sentence mentioning the DataOrigin note does not hurt, especially in Sec. 5.4.3 (Additional Information). On the contrary, it can be an appropriate example highlighting the mention of standardID and citation. Otherwise, a footnote could also be enough in my sense ; something like "note that the DataOrigin note proposes the addition of other valid INFO elements to ....".

[molinaro-m]

With DALI-1.2 in RFC but also DataOrigin-1.1 moved to ProposedEN, I would suggest to figure out whether DataOrigin gets Endorsed.
If it happens so, a proper reference could be the right way to proceed, without affecting the RFC process for DALI-1.2 (that I suspect will take longer then the decision on endorsing or not the DataOrigin-1.1).