Skip to content

Json parsing speed too slow #10

@johniez

Description

@johniez

JsonCpp library used to parse scroll results is extremely slow. Scrolling through larger indices show the delay much more. In numbers: what the current implementation iterates in 85 seconds, scroll implemented using rapidjson did in 25 seconds.

Unfortunately JsonCpp is used in Scroll header interface, so the change would break API/ABI. But the speed is worth it.

Possible implementation with rapidjson (as it is header only library):

  • hide it completely inside elasticlient, without publishing it in public interface
  • make the rapidjson parsing public (header only), so each client could compile it again

First option means that parsing has to be done twice. Internally and then for data parsing itself. Otherwise there is a risk of using different versions (binary incompatible) of rapidjson.
The second option allows shared parsing (internall and client's data) without possibility of using different versions of rapidjson (one compiled into elasticlient, another version in client code).

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions