Task/argis refactor stdr by AlvaroVega · Pull Request #1979 · telefonicaid/fiware-cygnus

AlvaroVega · 2020-11-17T15:34:42Z

Comes from #1860

This PR will be used to align original branch to current master

New Arcgis sink to avoid licence problems with SDK file.

[cygnus-ngsi][feature][NGSIArcgisSink] create NGSIArcgisSink #1672
[cygnus-ngsi][NameMapping] Allow deep Json attribute mapping. #1856

…nto ArcgisRest

Co-authored-by: Fermín Galán Márquez <[email protected]>

AlvaroVega

LGTM

fgalan · 2020-11-18T11:49:25Z

cygnus-common/pom.xml

@@ -16,6 +16,12 @@
    </properties>

    <dependencies>


Weird indent in the added XML fragments...

fgalan · 2020-11-18T11:54:23Z

doc/cygnus-ngsi/flume_extensions_catalogue/ngsi_arcgis_featuretable_sink.md

@@ -1,4 +1,4 @@
-# <a name="top"></a>NGSIArcGisSink


This file has been renamed. Review references to it in other parts of the documenation (with grep it should be easy), so links can be fixed.

At least I think that .mkdocs.yml should be adjusted.

CHANGES_NEXT_RELEASE

Co-authored-by: Fermín Galán Márquez <[email protected]>

IvanHdzC · 2020-11-18T12:24:02Z

cygnus-ngsi/src/main/java/com/telefonica/iot/cygnus/sinks/NGSISink.java

-import static com.telefonica.iot.cygnus.sinks.Enums.DataModel.DMBYATTRIBUTE;
-import static com.telefonica.iot.cygnus.sinks.Enums.DataModel.DMBYENTITY;
-import static com.telefonica.iot.cygnus.sinks.Enums.DataModel.DMBYSERVICE;
-import static com.telefonica.iot.cygnus.sinks.Enums.DataModel.DMBYSERVICEPATH;
-import java.util.Map;
 import com.telefonica.iot.cygnus.utils.CommonConstants;
 import com.telefonica.iot.cygnus.utils.NGSIConstants;
-import java.util.ArrayList;
-import java.util.Arrays;
-import java.util.Date;
-import org.apache.flume.Channel;
-import org.apache.flume.Context;
-import org.apache.flume.Event;
-import org.apache.flume.EventDeliveryException;
-import org.apache.flume.Sink.Status;
-import org.apache.flume.Transaction;
-import org.apache.flume.ChannelException;
-import org.apache.flume.conf.Configurable;
-import org.apache.log4j.MDC;


I don't quite understand why this lines were removed. I thing the code will not compile without this lines.

They should be aggregated.

My mistake. I didn't look closely, the lines were moved from its place.

NTC

fgalan · 2020-11-18T13:54:29Z

cygnus-common/pom.xml

+                <!-- SKIP TEST -->
+	        <plugin>
+	          <groupId>org.apache.maven.plugins</groupId>
+	          <artifactId>maven-surefire-plugin</artifactId>
+	          <version>2.12.4</version>
+	          <configuration>
+		    <skipTests>true</skipTests>
+	          </configuration>
+	        </plugin>


Really needed? Which new code is using it?

Used by com.telefonica.iot.cygnus.backends.arcgis.CredentialRestApiTest

As you can see in https://travis-ci.org/github/telefonicaid/fiware-cygnus/jobs/744408999 without this plugin we obtain:

Results : Failed tests: getObjectIdTest(com.telefonica.iot.cygnus.backends.arcgis.FeatureTest): Encuentra id cuando no debería haberlo. checkResponseTest(com.telefonica.iot.cygnus.backends.arcgis.RestApiTest): Response should be ok. Tests in error: testUserCredential(com.telefonica.iot.cygnus.backends.arcgis.CredentialRestApiTest): Can´t parse token generation url, MalformedURLException. getSetObjectId(com.telefonica.iot.cygnus.backends.arcgis.FeatureTest): java.lang.Integer cannot be cast to java.lang.Long Tests run: 220, Failures: 2, Errors: 2, Skipped: 4 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test (default-test) on project cygnus-common: There are test failures. [ERROR] [ERROR] Please refer to /home/travis/build/telefonicaid/fiware-cygnus/cygnus-common/target/surefire-reports for the individual test results. [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException <<<<<<<<<<<< Run cygnus-ngsi tests >>>>>>>>>>>>

Ok. NTC

The intent stills looking a bit ugly. I have tried to fixed, without success. It seems to be some kind of tabs vs space issue, but it doesn't worth the time to keep trying.

fgalan · 2020-11-18T13:59:52Z

...s-common/src/main/java/com/telefonica/iot/cygnus/backends/arcgis/NGSIArcgisFeatureTable.java

+     * @param username
+     * @param password
+     * @param getTokenUrl
+     * @param b


Suggested change

* @param b

* @param timeoutSecs

Weird... maybe there are another cases of "not fully replaced JavaDoc templates" like this?

Fixed in 6cc4a5a

…/arcgis/NGSIArcgisFeatureTable.java Co-authored-by: Fermín Galán Márquez <[email protected]>

IvanHdzC

LGTM

doc/cygnus-ngsi/flume_extensions_catalogue/ngsi_arcgis_featuretable_sink.md

…table_sink.md Co-authored-by: Ivan Hernandez <[email protected]>

…are-cygnus into task/argis_reactor_stdr

fgalan

LGTM

AlvaroVega · 2020-11-19T13:56:23Z

LGTM

IvanHdzC · 2020-11-19T17:13:54Z

LGTM

dmartinezgomez added 30 commits March 25, 2020 17:05

Limpieza y añadida nueva dependencia

cd3d787

Primera versión del Sink sin pasar test aún

1c516f7

pequeños parches

f7eacd0

Cache concurrente

a5fa5e8

Logging y control de errores

7238cce

Fix error parseando geo:json

78099b3

borrar código inútil

da4898f

Fix problema de limpieza de url

d58f30d

Tratamiento de timeouts a nivel de batch

06f39d7

Comprobación de timeouts en método process

947b20c

Eliminación de duplicado de logs

c017414

Comprueba si las features existen antes de añadirlas

a64aa91

maxBatchSize parametrizable desde el agente

74444e2

Argis Backend project

6dc647a

Argis Backend project

6448610

Fix unit tests

b134a13

Fix unit tests

b1b775a

Telefónica Checkstyle

c68dc9c

Telefónica Checkstyle

376095d

Telefónica Checkstyle

4053921

Eliminando

e88c691

Eliminando

ce78864

Merge branch 'ArcgisRest' of https://github.com/Stdri/fiware-cygnus i…

33342c4

…nto ArcgisRest

Merge branch 'ArcgisRest' of https://github.com/Stdri/fiware-cygnus i…

7b90f75

…nto ArcgisRest

Delete ArcgisBaseTest.java

27abb46

Delete ArcgisBaseTest.java

15e9b43

Merge branch 'ArcgisRest' of https://github.com/Stdri/fiware-cygnus i…

7e9502e

…nto ArcgisRest

Create ArcgisBaseTest.java

5ca1bfd

Refactor

d61290e

Refactorización

6f594dc

Update CHANGES_NEXT_RELEASE

1ccb2c1

Co-authored-by: Fermín Galán Márquez <[email protected]>

AlvaroVega commented Nov 18, 2020

View reviewed changes

fgalan reviewed Nov 18, 2020

View reviewed changes

CHANGES_NEXT_RELEASE Outdated Show resolved Hide resolved

AlvaroVega and others added 5 commits November 18, 2020 12:56

Update pom.xml

b65f15d

Update CHANGES_NEXT_RELEASE

004f1bd

Co-authored-by: Fermín Galán Márquez <[email protected]>

Update pom.xml

2e1226d

Update NGSIArcgisFeatureTableSink.java

0868d9b

Update mkdocs.yml

6505526

IvanHdzC reviewed Nov 18, 2020

View reviewed changes

AlvaroVega and others added 3 commits November 18, 2020 13:34

revert ngsisink.java

22d8822

add maven-surefire-plugin with skip test

a25903d

FIX .gitignore to remove too broad filter

704e06e

fgalan reviewed Nov 18, 2020

View reviewed changes

AlvaroVega and others added 2 commits November 18, 2020 15:26

Update cygnus-common/src/main/java/com/telefonica/iot/cygnus/backends…

6cc4a5a

…/arcgis/NGSIArcgisFeatureTable.java Co-authored-by: Fermín Galán Márquez <[email protected]>

FIX some tabs vs space issue in pom.xml indentation

59bd69f

IvanHdzC approved these changes Nov 19, 2020

View reviewed changes

FIX link to md

de75f59

IvanHdzC reviewed Nov 19, 2020

View reviewed changes

doc/cygnus-ngsi/flume_extensions_catalogue/ngsi_arcgis_featuretable_sink.md Outdated Show resolved Hide resolved

AlvaroVega and others added 3 commits November 19, 2020 12:14

Update doc/cygnus-ngsi/flume_extensions_catalogue/ngsi_arcgis_feature…

4b3e926

…table_sink.md Co-authored-by: Ivan Hernandez <[email protected]>

RENAME NGSIArcgisFeatureTableSink -> NGSIArcGisSink in docs

262d543

Merge branch 'task/argis_reactor_stdr' of github.com:telefonicaid/fiw…

3526a69

…are-cygnus into task/argis_reactor_stdr

fgalan approved these changes Nov 19, 2020

View reviewed changes

fgalan merged commit 3a46935 into master Nov 19, 2020

fgalan deleted the task/argis_reactor_stdr branch November 19, 2020 17:21

fgalan mentioned this pull request Nov 19, 2020

[cygnus-ngsi][feature][NGSIArcgisSink] create NGSIArcgisSink #1672

Closed

Conversation

AlvaroVega commented Nov 17, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AlvaroVega left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

IvanHdzC left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

fgalan left a comment

Choose a reason for hiding this comment

Uh oh!

AlvaroVega commented Nov 19, 2020

Uh oh!

IvanHdzC commented Nov 19, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

AlvaroVega commented Nov 17, 2020 •

edited

Loading