Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incompatibility of the plugin with kafka-connect #737

Open
zTranquilizer opened this issue Nov 20, 2024 · 0 comments
Open

Incompatibility of the plugin with kafka-connect #737

zTranquilizer opened this issue Nov 20, 2024 · 0 comments

Comments

@zTranquilizer
Copy link

Hey, everybody!

I have this situation:

I was using image confluentinc/cp-kafka-connect:7.7.0 in conjunction with clickhouse-kafka-connect v.1.2.0 and everything worked fine.

After a certain period of time I updated image confluentinc/cp-kafka-connect to version 7.7.1. And everything stopped working, an error appeared:


java.lang.VerifyError: Bad return type
Exception Details:
  Location:
    io/confluent/protobuf/MetaProto$Meta.internalGetMapFieldReflection(I)Lcom/google/protobuf/MapFieldReflectionAccessor; @24: areturn
  Reason:
    Type 'com/google/protobuf/MapField' (current frame, stack[0]) is not assignable to 'com/google/protobuf/MapFieldReflectionAccessor' (from method signature)
  Current Frame:
    bci: @24
    flags: { }
    locals: { 'io/confluent/protobuf/MetaProto$Meta', integer }
    stack: { 'com/google/protobuf/MapField' }
  Bytecode:
    0000000: 1bab 0010 0001 0018 0100 0001 0000 0002
    0000010: 0300 2013 2ab7 0002 b1bb 000f 59bb 1110
    0000020: 59b7 0011 1212 b601 131b b660 14b6 0015
    0000030: b702 11bf                              
  Stackmap Table:
    same_frame(@20)
    same_frame(@25)

at io.confluent.protobuf.MetaProto.<clinit>(MetaProto.java:1112)
at io.confluent.kafka.schemaregistry.protobuf.ProtobufSchema.<clinit>(ProtobufSchema.java:246)
at io.confluent.kafka.schemaregistry.protobuf.ProtobufSchemaProvider.parseSchemaOrElseThrow(ProtobufSchemaProvider.java:38)
at io.confluent.kafka.schemaregistry.SchemaProvider.parseSchema(SchemaProvider.java:75)
at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.parseSchema(CachedSchemaRegistryClient.java:301)
at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.getSchemaByIdFromRegistry(CachedSchemaRegistryClient.java:347)
at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.getSchemaBySubjectAndId(CachedSchemaRegistryClient.java:472)
at io.confluent.kafka.serializers.protobuf.AbstractKafkaProtobufDeserializer.deserialize(AbstractKafkaProtobufDeserializer.java:138)
at io.confluent.kafka.serializers.protobuf.AbstractKafkaProtobufDeserializer.deserializeWithSchemaAndVersion(AbstractKafkaProtobufDeserializer.java:294)
at io.confluent.connect.protobuf.ProtobufConverter$Deserializer.deserialize(ProtobufConverter.java:200)
at io.confluent.connect.protobuf.ProtobufConverter.toConnectData(ProtobufConverter.java:132)

I have this setting in docker-copmpose:

kafka-connect:
   image: 'confluentinc/cp-kafka-connect:7.7.1'
    container_name: connect
    hostname: connect
    ports:
      - '8083:8083'
    environment:
      CONNECT_BOOTSTRAP_SERVERS: 'kafka:9092'
      CONNECT_REST_ADVERTISED_HOST_NAME: connect
      CONNECT_GROUP_ID: compose-connect-group
      CONNECT_CONFIG_STORAGE_TOPIC: docker-connect-configs
      CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR: 1
      CONNECT_OFFSET_FLUSH_INTERVAL_MS: 10000
      CONNECT_OFFSET_STORAGE_TOPIC: docker-connect-offsets
      CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR: 1
      CONNECT_STATUS_STORAGE_TOPIC: docker-connect-status
      CONNECT_STATUS_STORAGE_REPLICATION_FACTOR: 1
      CONNECT_KEY_CONVERTER: org.apache.kafka.connect.storage.StringConverter
      CONNECT_VALUE_CONVERTER: org.apache.kafka.connect.storage.StringConverter
      CONNECT_VALUE_CONVERTER_SCHEMA_REGISTRY_URL: http://schema-registry:8081
      CLASSPATH: /usr/share/java/monitoring-interceptors/monitoring-interceptors-7.6.0.jar
      CONNECT_PRODUCER_INTERCEPTOR_CLASSES: "io.confluent.monitoring.clients.interceptor.MonitoringProducerInterceptor"
      CONNECT_CONSUMER_INTERCEPTOR_CLASSES: "io.confluent.monitoring.clients.interceptor.MonitoringConsumerInterceptor"
      CONNECT_PLUGIN_PATH: "/usr/share/java,/usr/share/confluent-hub-components,/usr/share/dockershare"
      CONNECT_LOG4J_LOGGERS: org.apache.zookeeper=ERROR,org.I0Itec.zkclient=ERROR,org.reflections=ERROR,com.clickhouse=DEBUG

I put the clickhouse-kafka-connect plugin in the /usr/share/confluent-hub-components folder. I don't add anything else. All other packages like com.google.java-protobuf etc. - come by default in the image (at least I see jar files with those names). I can also run the clickhouse-kafka-connect connector itself and it runs on version 7.7.0 and 7.7.1.

But when I post a message to a topic and kafka-connect starts pulling the message and apparently deserializing it, an error occurs.

By the way, I have the same settings as for confluentinc/cp-kafka-connect:7.7.0. But as soon as I change the image version from 7.7.0 to 7.7.1 - the error occurs.

A little searching for a solution - there was a suggestion that it is connected with some incompatibility of package versions, but I can't say for sure.
Can you tell me if someone has encountered this problem and knows how to solve it?
Or maybe someone has some ideas what can be tried to solve the problem.
I will be very grateful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant