Convert a Dataset row containing EmbeddedList to Array<String>

[barrettc]

I realize it's probably a long-shot that anyone is monitoring these issues but just in case...

I'm using the connector to retrieve data from OrientDB in which my column out is an EmbeddedList:

    val dfOrient = sqlContext.read
      .format("org.apache.spark.orientdb.graphs")
      .option("dburl", dbUrl)
      .option("user", user)
      .option("password", password)
      .option("vertextype", "character")
      .option("query", "select name, out('appears_in').title from character where outE('appears_in').size() > 0")
      .schema(struct)
      .load()

I know the data in out is an array of strings like:

["Iron Man"]
["Captain America: The First Avenger","Captain America: The Winter Soldier","Captain America: Civil War","Avengers: Infinity War"]
["Captain America: The Winter Soldier"]

I'm not sure how to convert the out column to an Array<String> such that I can run something like:

    val vertices = df
      .select(explode(concat(array('name), 'out)) as "x")
      .distinct.rdd.map(_.getAs[String](0))
      .zipWithIndex.map(_.swap)

Any hints would be appreciated - new to both Spark/OrientDB

[barrettc]

I was able to get this to work by casting the EmbeddedList to a String and then splitting on the comma. I have to believe there's a better way to do this but it works for now.

    val vertices = df
      .withColumn("out", col("out").cast("String"))
      .withColumn("out", split(col("out"), ", "))
      .select(explode(concat(array('name), 'out)) as "x")
      .distinct.rdd.map(_.getAs[String](0))
      .zipWithIndex.map(_.swap)

[astroecom]

I could see a good solution for using spark connector for orientDB especially casting the EmbeddedList to a String.