the tutorial compilation issue #118
Comments
|
Is there a specific reason you're trying to use such an old version? We are currently on 3.13.0. |
|
Hi, Mr. Pat,
I picked the 1.2.2.0 because it is the version used in this tutorial I am
trying to follow. I am not sure the difference of building new version and
the old version. and I am not sure the steps/code in the tutorial work with
new version of streamsets datacollector. If you have equivalent tutorial on
how to create custom java destination, that will be awesome. Can you share
with me if you have?
if i want to try the new version, which version number do you recommend?
and all three modules are with the same version number?
Thanks,
Wei
…On Wed, Mar 4, 2020 at 2:14 PM Pat Patterson ***@***.***> wrote:
Is there a specific reason you're trying to use such an old version? We
are currently on 3.13.0.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#118?email_source=notifications&email_token=AOXQL4H2RPMD4GL3IRDS2TLRF2SC7A5CNFSM4LBQXRV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENZWNGA#issuecomment-594765464>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AOXQL4AG6KNE34ZJDRDI25LRF2SC7ANCNFSM4LBQXRVQ>
.
|
|
Check the version of StreamSets Data Collector that you are using. It needs to be the same as that. The same instructions should still work. |
|
Hi, Mr. Pat,
I got all three project from master and I am trying to build following the
order. I am still having issue with building datacollector. I managed to
solve as many as I can and I am stuck at this:
(very similar warning about the same line 201, column 19 for many
dependencies)
[WARNING]
'dependencyManagement.dependencies.dependency.(groupId:artifactId:type:classifier)'
must be unique: com.streamsets:streamsets-datacollector-kafka_multiso
urce-protolib:test-jar -> duplicate declaration of version
${project.version} @
com.streamsets:streamsets-datacollector-root-lib:3.15.0-SNAPSHOT,
C:\Users\g706872\
git\datacollector\root-lib\pom.xml, line 201, column 19
[WARNING]
[WARNING] Some problems were encountered while building the effective model
for
com.streamsets:streamsets-datacollector-mongodb_3-lib:jar:3.15.0-SNAPSHOT
[WARNING]
'dependencyManagement.dependencies.dependency.(groupId:artifactId:type:classifier)'
must be unique: com.streamsets:streamsets-datacollector-kafka_multiso
urce-protolib:test-jar -> duplicate declaration of version
${project.version} @
com.streamsets:streamsets-datacollector-root-lib:3.15.0-SNAPSHOT,
C:\Users\g706872\
git\datacollector\root-lib\pom.xml, line 201, column 19
[WARNING]
[WARNING] Some problems were encountered while building the effective model
for
com.streamsets:streamsets-datacollector-databricks-ml_2-lib:jar:3.15.0-SNAPSHOT
[WARNING]
'dependencyManagement.dependencies.dependency.(groupId:artifactId:type:classifier)'
must be unique: com.streamsets:streamsets-datacollector-kafka_multiso
urce-protolib:test-jar -> duplicate declaration of version
${project.version} @
com.streamsets:streamsets-datacollector-root-lib:3.15.0-SNAPSHOT,
C:\Users\g706872\
git\datacollector\root-lib\pom.xml, line 201, column 19
[WARNING]
[WARNING] It is highly recommended to fix these problems because they
threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support
building such malformed projects.
[WARNING]
[INFO]
------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] StreamSets Data Collector Jython 2.7.0 Lib
[INFO] StreamSets Data Collector MapR 6.1.0 Lib
[INFO] StreamSets Data Collector MapR 6.1.0 MEP 6 Lib
[INFO] Streamsets Data Collector MapR Spark 2.1 MEP 3.0 Lib
[INFO] StreamSets Data Collector MongoDB 3.0.2 Lib
[INFO] StreamSets Data Collector Databricks ML 2 Lib
[INFO]
[INFO]
------------------------------------------------------------------------
[INFO] Building StreamSets Data Collector Jython 2.7.0 Lib 3.15.0-SNAPSHOT
[INFO]
------------------------------------------------------------------------
[INFO]
------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] StreamSets Data Collector Jython 2.7.0 Lib ......... FAILURE [
1.247 s]
[INFO] StreamSets Data Collector MapR 6.1.0 Lib ........... SKIPPED
[INFO] StreamSets Data Collector MapR 6.1.0 MEP 6 Lib ..... SKIPPED
[INFO] Streamsets Data Collector MapR Spark 2.1 MEP 3.0 Lib SKIPPED
[INFO] StreamSets Data Collector MongoDB 3.0.2 Lib ........ SKIPPED
[INFO] StreamSets Data Collector Databricks ML 2 Lib ...... SKIPPED
[INFO]
------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 12.922 s
[INFO] Finished at: 2020-03-06T14:06:40-05:00
[INFO] Final Memory: 175M/917M
[INFO]
------------------------------------------------------------------------
[ERROR] Failed to execute goal on project
streamsets-datacollector-jython_2_7-lib: Could not resolve dependencies for
project com.streamsets:streamsets-datacollect
or-jython_2_7-lib:jar:3.15.0-SNAPSHOT: Failure to find
com.streamsets:streamsets-datacollector-jython-protolib:jar:tests:3.15.0-SNAPSHOT
in http://packages.conflue
nt.io/maven/ was cached in the local repository, resolution will not be
reattempted until the update interval of confluent has elapsed or updates
are forced -> [He
lp 1]
While I was trying to build
streamsets-datacollector-jython-protolib:jar:tests:3.15.0-SNAPSHOT
separately, I got:
[ERROR]
C:\users\g706872\git\datacollector\jython-protolib\src\test\java\com\streamsets\pipeline\stage\processor\scripting\TestJythonAbstractScriptingProcessor.jav
a:[19,43] package com.streamsets.datacollector.runner does not exist
[ERROR]
C:\users\g706872\git\datacollector\jython-protolib\src\test\java\com\streamsets\pipeline\stage\origin\scripting\TestJythonAbstractScriptingSource.java:[19,
43] package com.streamsets.datacollector.runner does not exist
Then, I tried to rebuild the container, since I know the
com.streamsets.datacollector.runner is in the container project. the
container was build successfully, but still the jython-protolib test jar
build failed for the same reason.
Can you help me on this compilation issue? or is it enough to create my own
java custom destination if my custom destination has nothing to do with
jython, MarR, MongoDB, databricks?
Thanks,
Wei
…On Wed, Mar 4, 2020 at 2:53 PM Pat Patterson ***@***.***> wrote:
Check the version of StreamSets Data Collector that you are using. It
needs to be the same as that. The same instructions should still work.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#118?email_source=notifications&email_token=AOXQL4CGBJPYHKLM66SKAELRF2WSPA5CNFSM4LBQXRV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENZ5XUA#issuecomment-594795472>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AOXQL4EBOXZ7OZPHPUBKCUDRF2WSPANCNFSM4LBQXRVQ>
.
|
|
Hi Wei - I'll investigate the build failure. Yes - try creating your custom destination now. I don't think those specific issues will be a problem. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi, I git cloned datacollector 1.2.2.0, datacollector-api 1.2.2.0, can you tell me which version of datacollector-plugin-api I should get? it has no 1.2.2.0.
While compiling the datacollector: mvn clean install -DskipTests, the container failed. the log:
[INFO] Building StreamSets Data Collector Container 1.2.2.0
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ streamsets-datacollector-container ---
[INFO] Deleting C:\Users\g706872\git\datacollector\container\target
[INFO]
[INFO] --- hadoop-maven-plugins:2.5.0:version-info (version-info) @ streamsets-datacollector-container ---
[WARNING] [svn, info] failed with error code 1
[INFO] SCM: GIT
[INFO] Computed MD5: 62ac5d1ee141cfb7921c48f46b28db
[INFO]
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ streamsets-datacollector-container ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 9 resources
[INFO] Copying 1 resource
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ streamsets-datacollector-container ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 364 source files to C:\Users\g706872\git\datacollector\container\target\classes
[INFO] -------------------------------------------------------------
[WARNING] COMPILATION WARNING :
[INFO] -------------------------------------------------------------
[WARNING] /C:/Users/g706872/git/datacollector/container/src/main/java/com/streamsets/datacollector/memory/MultipleMonitorLocker.java:[24,16] sun.misc.Unsafe is internal proprietary API and may be removed in a future release
[WARNING] /C:/Users/g706872/git/datacollector/container/src/main/java/com/streamsets/datacollector/memory/MultipleMonitorLocker.java:[24,16] sun.misc.Unsafe is internal proprietary API and may be removed in a future release
[WARNING] /C:/Users/g706872/git/datacollector/container/src/main/java/com/streamsets/datacollector/memory/MultipleMonitorLocker.java:[24,16] sun.misc.Unsafe is internal proprietary API and may be removed in a future release
[WARNING] /C:/Users/g706872/git/datacollector/container/src/main/java/com/streamsets/datacollector/memory/MultipleMonitorLocker.java:[24,16] sun.misc.Unsafe is internal proprietary API and may be removed in a future release
[WARNING] /C:/Users/g706872/git/datacollector/container/src/main/java/com/streamsets/datacollector/memory/MultipleMonitorLocker.java:[33,24] sun.misc.Unsafe is internal proprietary API and may be removed in a future release
[WARNING] /C:/Users/g706872/git/datacollector/container/src/main/java/com/streamsets/datacollector/memory/MultipleMonitorLocker.java:[35,5] sun.misc.Unsafe is internal proprietary API and may be removed in a future release
[WARNING] /C:/Users/g706872/git/datacollector/container/src/main/java/com/streamsets/datacollector/memory/MultipleMonitorLocker.java:[37,27] sun.misc.Unsafe is internal proprietary API and may be removed in a future release
[WARNING] /C:/Users/g706872/git/datacollector/container/src/main/java/com/streamsets/datacollector/memory/MultipleMonitorLocker.java:[39,17] sun.misc.Unsafe is internal proprietary API and may be removed in a future release
[WARNING] /C:/Users/g706872/git/datacollector/container/src/main/java/com/streamsets/datacollector/execution/runner/cluster/ClusterRunner.java: Some input files use or override a deprecated API.
[WARNING] /C:/Users/g706872/git/datacollector/container/src/main/java/com/streamsets/datacollector/execution/runner/cluster/ClusterRunner.java: Recompile with -Xlint:deprecation for details.
[WARNING] /C:/Users/g706872/git/datacollector/container/src/main/java/com/streamsets/datacollector/config/StageDefinition.java: Some input files use unchecked or unsafe operations.
[WARNING] /C:/Users/g706872/git/datacollector/container/src/main/java/com/streamsets/datacollector/config/StageDefinition.java: Recompile with -Xlint:unchecked for details.
[INFO] 12 warnings
[INFO] -------------------------------------------------------------
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /C:/Users/g706872/git/datacollector/container/src/main/java/com/streamsets/datacollector/definition/ConfigDefinitionExtractor.java:[391,12] an enum switch case label must be the unqualified name of an enumeration constant
[INFO] 1 error
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
The text was updated successfully, but these errors were encountered: