Issue when using a date (in a nested field) for X-Axis in a Date Histogram

[mackermann2]

Plugin version: 6.4.2
Kibana version: 6.4.2

Description of the problem including expected versus actual behavior:
I would like to use a field which contains a timestamp in a nested field (userActions.startTime) as below :

There is no error but unfortunately the graph is empty:

That seems to be a bug ? :)

[IdanWo]

In most cases, it means that the generated query has no results or wrong.

Can you add the generated ElasticSearch query here? It is accessable from the graph by Request tab. Try also to identify errors shown in the Response tab.

[mackermann2]

Hi IdanWo, I don't find the way to display these buttons below the chart (as in your screenshot).
Maybe I have to activate this feature in Advanced Settings ?

[mackermann2]

If that can help, a sample of data (not all the json file, just an extract) :

  "_index": "dynatrace",
  "_type": "session",
  "_id": "nqdxxxxx-LSKOLGNNWNBNLVLXSTREEGEEZIGANHYJ-1544514266703",
  "_version": 1,
  "_score": null,
  "_source": {
    "userSessionId": "LSKOLGNNWNBNLVLXSTREEGEEZIGANHYJ",
    "startTime": 1544514266703,
    "endTime": 1544517951093,
    "duration": 3684390,
    "internalUserId": "1543305377680LRQOGC7OGHSQLADMG4BRNFP0O3HCJ14E",
    "applicationType": "WEB_APPLICATION",
    "userActionCount": 56,
    "totalErrorCount": 0,
    "userActions": [
      {
        "name": "PROD_ViewOrder",
        "startTime": 1544514266703,
        "endTime": 1544514270193,
        "duration": 3490,
        "internalApplicationId": "APPLICATION-4AD43E503369C9FC",
        "networkTime": 343,
        "serverTime": 199,
        "frontendTime": 2948
      },
      {
        "name": "PROD_QT_QuoteToOrder",
        "startTime": 1544514309545,
        "endTime": 1544514315263,
        "duration": 5718,
        "networkTime": 8,
        "serverTime": 201,
        "frontendTime": 5509
      },
      {
        "name": "PROD_ViewOrder",
        "startTime": 1544514315355,
        "endTime": 1544514318049,
        "duration": 2694,
        "networkTime": 49,
        "serverTime": 198,
        "frontendTime": 2447
      },
      {
        "name": "PROD_ProductScreen",
        "startTime": 1544514337432,
        "endTime": 1544514338453,
        "duration": 1021,
        "networkTime": 18,
        "serverTime": 415,
        "frontendTime": 588
        }
    ],
  },
  "fields": {
    "startTime": [
      "2018-12-11T07:44:26.703Z"
    ],
    "endTime": [
      "2018-12-11T08:45:51.093Z"
    ]
  },
  "sort": [
    1544514266703
  ]
}

with the associated mapping :

{
  "mappings": {
    "session": {
      "dynamic_templates": [
        {
          "num_fields": {
            "path_match": "numTags.*",
            "mapping": {
              "norms": "false",
              "type": "double"
            }
          }
        },
        {
          "date_fields": {
            "path_match": "dateTags.*",
            "mapping": {
              "norms": "false",
              "type": "date"
            }
          }
        },
        {
          "string_fields": {
            "match": "*",
            "match_mapping_type": "string",
            "mapping": {
              "norms": "false",
              "type": "keyword"
            }
          }
        }
      ],
      "properties": {
        "applicationType": {
          "type": "keyword"
        },
        "dateTags": {
          "type": "object"
        },
        "duration": {
          "type": "integer"
        },
        "endTime": {
          "type": "date"
        },
        "internalUserId": {
          "type": "keyword"
        },
		"startTime": {
          "type": "date"
        },
        "totalErrorCount": {
          "type": "integer"
        },
        "userActionCount": {
          "type": "integer"
        },
        "userActions": {
          "type": "nested",
          "properties": {
            "duration": {
              "type": "integer"
            },
            "endTime": {
              "type": "date"
            },
            "frontendTime": {
              "type": "integer"
            },
            "name": {
              "type": "keyword"
            },
            "networkTime": {
              "type": "integer"
            },
            "serverTime": {
              "type": "integer"
            },
            "startTime": {
              "type": "date"
            }
          }
        },
        "userSessionId": {
          "type": "keyword"
        }
      }
    }
  }
}

[mackermann2]

Hello, I move forward on my problem.
I recreated my index by letting elasticsearch create it with the first entry (so new mapping file is used).
It is now possible to select userActions.startTime in sub aggregation.
.
My problem is now the following : "userActions.duration" value is always the same for each "userActions.name". Instead of displaying each time the specified duration in the correct order on the X-Axis, all actions appear to be merged over the same timeframe :

Any idea about that ? ^^

[mackermann2]

Considering the following json :

"userActions": [
{
"name": "Action1",
"startTime": 1544537662007,
"duration": 4231
}
{
"name": "Action2",
"startTime": 1544537674072,
"duration": 558
}
{
"name": "Action3",
"startTime": 1544537761154,
"duration": 3146
}
]

If I want to apply a filter on a specific userActions.name (Action1 for example), all the other actions in the array (Action2 and Action3 in the case above) are selected in my chart, not only Action1.
I'm not a elasticsearch or kibana expert but maybe "inner_hits" option could resolved this issue ?
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-inner-hits.html#top-level-inner-hits

The screenshot of the chart with the filter :

I have finally found how to get the request in my version of Kibana :

{
  "aggs": {
    "2": {
      "date_histogram": {
        "field": "userActions.startTime",
        "interval": "1h",
        "time_zone": "Europe/Berlin",
        "min_doc_count": 1
      },
      "aggs": {
        "3": {
          "terms": {
            "field": "userActions.name.keyword",
            "size": 3,
            "order": {
              "1": "desc"
            },
            "include": "PROD.*"
          },
          "aggs": {
            "1": {
              "avg": {
                "field": "userActions.duration"
              }
            }
          }
        }
      }
    }
  },
  "size": 0,
  "_source": {
    "excludes": []
  },
  "stored_fields": [
    "*"
  ],
  "script_fields": {},
  "docvalue_fields": [
    {
      "field": "endTime",
      "format": "date_time"
    },
    {
      "field": "events.startTime",
      "format": "date_time"
    },
    {
      "field": "query.constant_score.filter.bool.filter.range.startTime.gte",
      "format": "date_time"
    },
    {
      "field": "query.constant_score.filter.bool.filter.range.startTime.lte",
      "format": "date_time"
    },
    {
      "field": "startTime",
      "format": "date_time"
    },
    {
      "field": "userActions.endTime",
      "format": "date_time"
    },
    {
      "field": "userActions.startTime",
      "format": "date_time"
    }
  ],
  "query": {
    "bool": {
      "must": [
        {
          "range": {
            "startTime": {
              "gte": 1544526899838,
              "lte": 1544613299838,
              "format": "epoch_millis"
            }
          }
        },
        {
          "match_phrase": {
            "userActions.name.keyword": {
              "query": "PROD_PriceScreen"
            }
          }
        }
      ],
      "filter": [],
      "should": [],
      "must_not": []
    }
  }
}

Thanks for your assistance :)

[ppadovani]

See the discussion in #102.

Duplicate of #90