問題描述
我們已經(jīng)成功地使用了 mySQL - 使用 jdbc 獨立連接器的 kafka 數(shù)據(jù)攝取，但現(xiàn)在面臨在分布式模式下使用相同的問題(作為 kafka 連接服務(wù)).
We have successfully used mySQL - kafka data ingestion using jdbc standalone connector but now facing issue in using the same in distributed mode (as kafka connect service ).
connect-distributed.properties 文件-
connect-distributed.properties file-
bootstrap.servers=IP1:9092,IP2:9092
group.id=connect-cluster
key.converter.schemas.enable=true
value.converter.schemas.enable=true
offset.storage.topic=connect-offsets
offset.storage.replication.factor=2
config.storage.topic=connect-configs
config.storage.replication.factor=2
status.storage.topic=connect-status
status.storage.replication.factor=2
offset.flush.interval.ms=10000
plugin.path=/usr/share/java,/usr/share/java/kafka-connect-jdbc
我有我的連接器罐子-
/usr/share/java/kafka-connect-jdbc
-rw-r--r-- 1 root root 906708 Jul 29 01:18 zookeeper-3.4.13.jar
-rw-r--r-- 1 root root 74798 Jul 29 01:18 zkclient-0.10.jar
-rw-r--r-- 1 root root 5575351 Jul 29 01:18 sqlite-jdbc-3.8.11.2.jar
-rw-r--r-- 1 root root 41203 Jul 29 01:18 slf4j-api-1.7.25.jar
-rw-r--r-- 1 root root 658466 Jul 29 01:18 postgresql-9.4-1206-jdbc41.jar
-rw-r--r-- 1 root root 1292696 Jul 29 01:18 netty-3.10.6.Final.jar
-rw-r--r-- 1 root root 489884 Jul 29 01:18 log4j-1.2.17.jar
-rw-r--r-- 1 root root 211219 Jul 29 01:18 kafka-connect-jdbc-5.0.0.jar
-rw-r--r-- 1 root root 317816 Jul 29 01:18 jtds-1.3.1.jar
-rw-r--r-- 1 root root 87325 Jul 29 01:18 jline-0.9.94.jar
-rw-r--r-- 1 root root 20844 Jul 29 01:18 common-utils-5.0.0.jar
-rw-r--r-- 1 root root 20437 Jul 29 01:18 audience-annotations-0.5.0.jar
-rw-r----- 1 root root 2132635 Nov 11 16:31 mysql-connector-java-8.0.13.jar
我可以通過這種方式運行腳本來運行獨立模式-
I am able to run the standalone mode by running the script in this way-
/usr/bin/connect-standalone /etc/kafka/connect-standalone.properties /etc/kafka-connect-jdbc/source-quickstart-mysql.properties
但是當(dāng)我嘗試調(diào)用 REST API 來運行分布式模式連接器時，出現(xiàn)錯誤:
But when I try to invoke the REST API to run a distributed mode connector , I get the error:
curl -X POST -H "Accept:application/json" -H "Content-Type:application/json" X.X.X.X:8083/connectors/ -d '{"name": "linuxemp-connector", "config": { "connector.class": "io.confluent.connect.jdbc.JdbcSourceConnector", "tasks.max": "1", "connection.url": "jdbc:mysql://Y.Y.Y.Y:3306/linux_db?user=groot&password=pwd","table.whitelist": "emp","mode": "timestamp","incrementing.column.name":"empid","topic.prefix": "mysqlconnector-" } }'
錯誤-
{"error_code":400,"message":"Connector configuration is invalid and contains the following 2 error(s):\nInvalid value java.sql.SQLException: No suitable driver found for jdbc:mysql://Y.Y.Y.Y:3306/linux_db?user=groot&password=pwd for configuration Couldn't open connection to jdbc:mysql://Y.Y.Y.Y:3306/linux_db?user=groot&password=pwd\nInvalid value java.sql.SQLException: No suitable driver found for jdbc:mysql://Y.Y.Y.Y:3306/linux_db?user=groot&password=pwd for configuration Couldn't open connection to jdbc:mysql://Y.Y.Y.Y:3306/linux_db?user=groot&password=pwd\nYou can also find the above list of errors at the endpoint `/{connectorType}/config/validate`"}
注意-連接器 jars 放置在所有連接節(jié)點上，plugin.path 在所有連接節(jié)點上都相同，并且 kafka-connect 服務(wù)已啟動并運行.
note- connector jars are placed on all connect nodes , plugin.path is same on all connect nodes and kafka-connect service is up and running .
我錯過了什么?為什么我無法提交 REST 調(diào)用來啟動此 mysql 管道的分布式連接工作器/任務(wù)?這在獨立模式下非常有效.但是在分布式模式下拋出錯誤.
What am I missing ? Why am I not able to submit REST call to start a distributed connect worker/task for this mysql pipeline ? This works absolutely fine with standalone mode . But throws error with distributed mode .
請幫忙！
謝謝！
推薦答案
通過修復(fù)以下問題，問題得到解決 -
By fixing below things, the issue got resolved -
1.修改/usr/share/java/kafka-connect-jdbc/mysql-connector-java-8.0.13.jar的權(quán)限為7552.在插件路徑中只保留/usr/share/java.
1.Changed permissions of /usr/share/java/kafka-connect-jdbc/mysql-connector-java-8.0.13.jar to 755 2.Keep only /usr/share/java in plugin path.
3.更改我的sql表結(jié)構(gòu)，使其具有一個主鍵和一個具有增量性質(zhì)或時間戳的列屬性.
3.Change my sql table structure to have one primary key and one column property with incremental nature or timestamp.
這篇關(guān)于kafka-connect-jdbc:SQLException:僅在使用分布式模式時沒有合適的驅(qū)動程序的文章就介紹到這了，希望我們推薦的答案對大家有所幫助，也希望大家多多支持html5模板網(wǎng)！