WIP : DNM : Kinesis Integration

[boneill42]

Just putting this up as a place holder for now. I may have time to hack on this over the next couple days, and will try to fold in the work we've done.

[Gauravshah]

working on similar code :) should have opened a pull request

[boneill42]

@Gauravshah I'm happy to close this one and contribute to yours?
Or if you want to merge into this one, have at it.

[Gauravshah]

Thanks. I’ll add to your pull request. Mine works but requires lot of cleanup.

…

On Sun, 15 Oct 2017 at 10:14 PM, boneill42 ***@***.***> wrote: @Gauravshah <https://github.com/gauravshah> I'm happy to close this one and contribute to yours? Or if you want to merge into this one, have at it. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#254 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAl4D-_ECz2RB4tJMzfB9q2Nu3UBN6dRks5ssjaAgaJpZM4P0znl> .

[Gauravshah]

@boneill42 please have a look at https://github.com/druid-io/tranquility/compare/master...GoshPosh:kinesis?expand=1 , should give some directions if you need something.

[gianm]

@boneill42 is this still WIP or are you ready for review?

[boneill42]

Still WIP... i have the rest of the code, just didn't push it up yet. Give me until the end of the weekend (10/29).

[gianm]

No rush, was just curious

[guptaraul89]

I am looking for kinesis to druid connectors. Is there any way possible / any documents to ingest data into druid from Kinesis data streams?

[boneill42]

Ugh. We have one... I've been trying to find time to open source it for over a year. Ping me directly. I can maybe hand off to you what I have. (bone@alumni.brown.edu <mailto:bone@alumni.brown.edu>)

…

-brian -- Brian O'Neill CTO @ Monetate m: 215.588.6024 boneill@monetate.com <mailto:boneill@monetate.com>

On Jan 18, 2018, at 12:51 PM, Rahul Gupta ***@***.***> wrote: I am looking for kinesis to druid connectors. Is there any way possible / any documents to ingest data into druid from Kinesis data streams? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#254 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA2wzTOBejo7uZWWtdjj72Fw5OTVO4_qks5tL4SagaJpZM4P0znl>.

[guptaraul89]

Awesome! thank you. Sent you an email.

[Gauravshah]

@gianm should we consider building something similar to http://druid.io/docs/0.9.1.1/development/extensions-core/kafka-ingestion.html for kinesis instead of doing it on tranquility ?

[boneill42]

@Gauravshah I believe that's the plan (pull vs. push model), which is honestly why I haven't spent a ton of time getting this in a stage to merge. The pull model is the way to go.