Index pattern with nested time field

[IdanWo]

Issue:
When creating an index pattern, a time field must be chosen. When choosing a nested time field - no matching results can be discovered or aggregated.

Workaround:
A hack is possible by adding a non-nested LastModified field in the mapping (a regular date datatype field), and each time nested document is added, an update operation is made for this field to the current timestamp (now). This way I was managed to filter all my dashboard.

[ppadovani]

I ran into this years ago when I first started down this custom Kibana path. There really only is one main issue:

There is no way to specify that an index has multiple time fields. One workaround I proposed to the Kibana team that they rejected was to allow the user to create multiple index patterns for the same index by defining an ID... So you could have 'HomesByPurchaseDate' and 'HomesByCarPurchaseDate' that both point to the 'homes' index. This works around some built in assumptions that were coded into Kibana around the time chooser. For nested time fields, the time chooser code does not properly inject the nested path, and hence you cannot properly search on it.

In thinking through this, there are two options I'm not sure which would be the best until I dig into the code and see which is the most feasible to implement.

1. I implement the above feature, which minimizes user impact to just defining the index pattern correctly.

2. I change the time chooser code that is shown in the search bar to allow the user to select the time field (if other than the default time field) to use.

Honestly, I think option 1 is the correct approach because when dealing with dashboards, choosing a time field on a dashboard doesn't make sense when multiple indices are represented. But if each index represented is specific to an index pattern that is configured for a specific time field, then the integration is seamless.

[IdanWo]

Didn't know there were any limiations uppon index patterns so far - so thanks for letting me know.

In my specific use case, I don't need to create more than 1 index pattern per index. The reason for that is that, unfortuently, I already duplicate the same data to different indices. As of peformance considerations, I maintain periodic indices per different time fields. This allows me to make each search request directly to the relevant indices right away instead of all indices (in price of higher write throughput and storage, of course). So option 1 is great for me, I only need this for dashboards uppon the peridoic indices correlated to my UserActionTime - which is a nested field.

I think that in general, the best approch for searching through different time fields would be to index them accordingly to multiple indices. In case of big data sets, that's a must for any search made on ElasticSearch - and not only for kibana dashboards.

Remarks:

• If, in the future, ElasticSearch will support index sort on nested fields - no duplicates will be necessary and then this feature request will need to get higher priority. That's what I think.
• The concept of mandatory main time field for each dashboard / creating a dashboard on a specific index pattern is problomatic of its own. It means that making graphs on different indices patterns and different time fields in general is not well supported in Kibana. That's bad, but the problem needs to be solved in Kibana and not in this plugin. I don't like the idea that this time field must be part of the query of all the graphs in the dashboard.
• Unfourtently, this feature is needed in my current Kibana 5.6.10. If you could port it in this release and not only 6.x that would be great.

[IdanWo]

After a few months trying the workaround, I gotta say that maintaining it by updating a non nested 'last update time' is quite uncomfortable.

How difficult it is to add this improvement? The logic is already implemented, so I guess the current challenge is how to plug it and wire it into the time filter, so the query it produces would be nested.

In case the dashboard needs to be periodically update by a nested time filter this feature is a MUST.