FLIP-532 Donate connector to Flink

[davidradl]

Initial implementation of Flip-532 on a server side branch

tasks

• find the documented process for this - legal requirements
• Add Apache 2 licenses to every file, java pom etc
• change connector name Flink connector http
• change java package names to follow the standard
• split up the poms to have a sql one
• assess shading is in line with SQL non-sql way that are expected for Flink connectors
• change the config to use Flink config parameter names and gid fallback keys
• move content of README to documentation folder
• review pom to ensure it have the same standard checking, licenses, format etc
• keep coverage checks if possible.
• publish to Maven snapshot
• Check github actions are inline with the other Flink connectors

Follow on tasks

• new component in Jira / Github
• knit in the http connector into the Flink docs
• first release

Initial assumptions:

• changes go to main
• release at some stage later,
• v120 only currently

[davidradl]

Grzegorz Kołakowski (@grzegorz8) Krzysztof Zarzycki (@kzarzycki)

I have spent a few days on this. I have:

• a clean compile with licenses in all files, with the connector check style and the new package names.
• the unit tests complete successfully in the IDE.
• I have issues running the unit tests in the maven build
  • I see errors that the ports in the unit tests are already being used
  • I have seen this in the past , so I thought I would try to make it better by getting rid of the duplicate port numbers referred to by the wire mock server, and to allocate them from a helper class. I have 2 counters a secure and non secure base port - each time I need a port I give the caller in incrementing the number, so the ports should not clash.
  • unfortunately the ports still clashed , so I now check to see if the socket is reachable and only allocate the port if there is no listening server socket on on that port. I still get the address is already in use
  • I then noticed that the junit suite is being run more than once at the same time on different processes in the build
  • I removed the coverage test run, but I still see these errors.

I notice that in the Flink connectors they do not use a test phase and do not use jacoco. It appears that test jars are built . I am looking into how the junit tests are driven in Flink and other Flink connectors, so we can do the same.

any thoughts ?

[davidradl]

This fix for this was to force forkcount=1 for surefire - this stops the parallelism and prevents all the tests from being driven 4 times .

[davidradl]

I needed to remove the references to guava - the checker complained of import com.google.common.collect.ImmutableList;
I changed this to a List.
Mockito is also banned in Flink - I will raise a separate issue to fix that after the initial commit

[davidradl]

Removed references to jetbrains annotations
Removed references to shaded dependancies

[davidradl]

Adding in the dependancy checker meant that I needed to exclude the following dependancy from the checker. These dependancies are required for a successful unit test run, but are viewed as unused without these ignores.

        <ignoredDependency>org.apache.flink:flink-table-test-utils:jar:1.20.1</ignoredDependency>
        <ignoredDependency>org.mockito:mockito-inline:jar:4.6.1</ignoredDependency>

[davidradl]

associated PR

[davidradl]

I have populated the Flink branch in the Flink repo with this PR content.

[davidradl]

kristoffSC Grzegorz Kołakowski (@grzegorz8) - it is merged! Leaving this open for now to check whether there are any other tasks to do