Unable to set the cluster property due to following error : Error setting cluster property
Error: Unable to set the cluster property in ZK
ERROR: Failed to configure urlScheme property for Solr cluster in Zookeeper
そして更に調査したところ、そもそもSpark MllibのSVDの実装が古いらしく、性能が出ない情報がネット上で結構上がっています。Workaroundとしては、SVDの代わりにAlternating Least Squares（ALS）などを使うか、あるいはネイティブな数学演算ライブラリを使って高速化するか などがあります。
pyspark --jars /usr/hdp/current/hive_warehouse_connector/hive-warehouse-connector-assembly-1.0.0.3.1.0.0-78.jar --py-files /usr/hdp/current/hive_warehouse_connector/pyspark_hwc-1.0.0.3.1.0.0-78.zip
Error
py4j.protocol.Py4JJavaError: An error occurred while calling o72.executeQuery.
: java.lang.RuntimeException: java.io.IOException: shadehive.org.apache.hive.service.cli.HiveSQLException: java.io.IOException: org.apache.hadoop.hive.ql.metadata.HiveException: Failed to compile query: org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAccessControlException: Permission denied: user [anonymous] does not have [USE] privilege on [default]
at com.hortonworks.spark.sql.hive.llap.HiveWarehouseDataSourceReader.readSchema(HiveWarehouseDataSourceReader.java:130)
at org.apache.spark.sql.execution.datasources.v2.DataSourceV2Relation$.apply(DataSourceV2Relation.scala:56)
2019-07-19T14:35:13,842 WARN [main] io.druid.metadata.SQLMetadataConnector - Exception creating table
org.skife.jdbi.v2.exceptions.CallbackFailedException: io.druid.java.util.common.ISE: Database default character set is not UTF-8.
Druid requires its MySQL database to be created using UTF-8 as default character set.
at org.skife.jdbi.v2.DBI.withHandle(DBI.java:284) ~[jdbi-2.63.1.jar:2.63.1]
at io.druid.metadata.SQLMetadataConnector$2.call(SQLMetadataConnector.java:135) ~[druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
at io.druid.java.util.common.RetryUtils.retry(RetryUtils.java:63) ~[java-util-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
at io.druid.java.util.common.RetryUtils.retry(RetryUtils.java:81) ~[java-util-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
at io.druid.metadata.SQLMetadataConnector.retryWithHandle(SQLMetadataConnector.java:139) ~[druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
at io.druid.metadata.SQLMetadataConnector.retryWithHandle(SQLMetadataConnector.java:148) ~[druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
at io.druid.metadata.SQLMetadataConnector.createTable(SQLMetadataConnector.java:189) [druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
at io.druid.metadata.SQLMetadataConnector.createRulesTable(SQLMetadataConnector.java:287) [druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
at io.druid.metadata.SQLMetadataConnector.createRulesTable(SQLMetadataConnector.java:553) [druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
at io.druid.metadata.SQLMetadataRuleManagerProvider$1.start(SQLMetadataRuleManagerProvider.java:71) [druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
at io.druid.java.util.common.lifecycle.Lifecycle.start(Lifecycle.java:311) [java-util-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
at io.druid.guice.LifecycleModule$2.start(LifecycleModule.java:134) [druid-api-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
at io.druid.cli.GuiceRunnable.initLifecycle(GuiceRunnable.java:101) [druid-services-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
at io.druid.cli.ServerRunnable.run(ServerRunnable.java:50) [druid-services-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
at io.druid.cli.Main.main(Main.java:116) [druid-services-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
Caused by: io.druid.java.util.common.ISE: Database default character set is not UTF-8.
Druid requires its MySQL database to be created using UTF-8 as default character set.
at io.druid.metadata.storage.mysql.MySQLConnector.tableExists(MySQLConnector.java:174) ~[?:?]
at io.druid.metadata.SQLMetadataConnector$4.withHandle(SQLMetadataConnector.java:195) ~[druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
at io.druid.metadata.SQLMetadataConnector$4.withHandle(SQLMetadataConnector.java:191) ~[druid-server-0.12.1.3.1.0.0-78.jar:0.12.1.3.1.0.0-78]
at org.skife.jdbi.v2.DBI.withHandle(DBI.java:281) ~[jdbi-2.63.1.jar:2.63.1]
... 14 more
2019-07-19T14:35:13,846 ERROR [main] io.druid.cli.CliCoordinator - Error when starting up. Failing.
org.skife.jdbi.v2.exceptions.CallbackFailedException: org.skife.jdbi.v2.exceptions.UnableToExecuteStatementException: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'druid.druid_rules' doesn
't exist [statement:"SELECT id from druid_rules where datasource=:dataSource", located:"SELECT id from druid_rules where datasource=:dataSource", rewritten:"SELECT id from druid_rules where datasource=?", arg
uments:{ positional:{}, named:{dataSource:'_default'}, finder:[]}]
MySqlの文字コードがUTF-8ではないため、テーブルが見つからない というようなエラー
早速MySqlでdatabase（druid)の文字コードを変更してみます。
alter database druid character set utf8 collate utf8_general_ci;
use druid;
show variables like "character_set_database";
+------------------------+-------+
| Variable_name | Value |
+------------------------+-------+
| character_set_database | utf8 |
+------------------------+-------+
1 row in set (0.00 sec)
