Pentaho Open Source BA Server 6.0.0.0-353
CDH 5.4.8
I followed https://help.pentaho.com/Documentati...H0/070/030/010 to set up Pentaho to connect to CDH5.4 Cluster. It was OK for an unkerberized cluster, so I should be following all necessary steps.
I try to connect to a Kerberized Cluster, and added the following lines in config.properties
When I try to configure a Cloudera Impala data source, I get the following error stack in catalina.out
The error stack mentioned "cdh-kerberos", so it is reading my config.properties. (Tried both hive 2 and Cloudera jdbc driver, same error)
Traced the source code and searched some issues (http://jira.pentaho.com/browse/PDI-13283), I added the 3 commented lines above, copied kettle-engine-6.0.0.0-353.jar from pdi to tomcat/lib (for KerberosAuthenticationProvider ). Same result.
As suggested by the document, I also tried the setting in spoon. It failed in the "Active Shim Load" step, "Learn more" link displayed nothing. In the console, spoon displayed :
CDH 5.4.8
I followed https://help.pentaho.com/Documentati...H0/070/030/010 to set up Pentaho to connect to CDH5.4 Cluster. It was OK for an unkerberized cluster, so I should be following all necessary steps.
I try to connect to a Kerberized Cluster, and added the following lines in config.properties
Code:
authentication.superuser.provider=cdh-kerberos
authentication.kerberos.principal=cdh@HADOOP.COM
authentication.kerberos.password=real_password
pentaho.oozie.proxy.user=oozie
#authentication.kerberos.id=cdh-kerberos
#authentication.kerberos.class=org.pentaho.di.core.auth.KerberosAuthenticationProvider
#authentication.provider.list=authentication.kerberosCode:
10:20:05,510 ERROR [ConnectionServiceImpl] ::: ConnectFactory.ERROR_0001 - Database connection could not be established to: org.apache.hive.jdbc.ImpalaSimbaDriver jdbc:impala://bigdata2:21050/bigdatabase;AuthMech=1;KrbServiceName=impala;KrbHostFQDN=bigdata2.mydomain;KrbRealm=HADOOP.COM
java.sql.SQLException: Unable to load Impala Simba JDBC driver for the currently active Hadoop configuration
at org.apache.hive.jdbc.HiveDriver.getActiveDriver(HiveDriver.java:112)
at org.apache.hive.jdbc.HiveSimbaDriver.checkBeforeCallActiveDriver(HiveSimbaDriver.java:67)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:147)
at org.pentaho.platform.plugin.services.connections.sql.SQLConnection.init(SQLConnection.java:162)
at org.pentaho.platform.plugin.services.connections.sql.SQLConnection.connect(SQLConnection.java:586)
at org.pentaho.platform.plugin.services.connections.sql.SQLConnection.setProperties(SQLConnection.java:106)
at org.pentaho.platform.engine.services.connection.PentahoConnectionFactory.getConnection(PentahoConnectionFactory.java:124)
at org.pentaho.platform.engine.services.connection.PentahoConnectionFactory.getConnection(PentahoConnectionFactory.java:98)
at org.pentaho.platform.dataaccess.datasource.wizard.service.impl.ConnectionServiceImpl.testConnection(ConnectionServiceImpl.java:293)
at org.pentaho.platform.dataaccess.datasource.wizard.service.impl.ConnectionService.testConnection(ConnectionService.java:266)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1511)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1442)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1391)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1381)
at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716)
at org.pentaho.platform.web.servlet.JAXRSPluginServlet.service(JAXRSPluginServlet.java:112)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
at org.pentaho.platform.web.servlet.JAXRSPluginServlet.service(JAXRSPluginServlet.java:117)
at org.pentaho.platform.web.servlet.PluginDispatchServlet.service(PluginDispatchServlet.java:89)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.pentaho.platform.web.http.filters.PentahoWebContextFilter.doFilter(PentahoWebContextFilter.java:185)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.pentaho.platform.web.http.filters.PentahoRequestContextFilter.doFilter(PentahoRequestContextFilter.java:87)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:399)
at org.springframework.security.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:109)
at org.springframework.security.intercept.web.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:83)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:411)
at org.springframework.security.ui.ExceptionTranslationFilter.doFilterHttp(ExceptionTranslationFilter.java:101)
at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:411)
at org.springframework.security.providers.anonymous.AnonymousProcessingFilter.doFilterHttp(AnonymousProcessingFilter.java:105)
at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:411)
at org.pentaho.platform.web.http.security.RequestParameterAuthenticationFilter.doFilter(RequestParameterAuthenticationFilter.java:191)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:411)
at org.springframework.security.ui.basicauth.BasicProcessingFilter.doFilterHttp(BasicProcessingFilter.java:174)
at org.pentaho.platform.web.http.security.PentahoBasicProcessingFilter.doFilterHttp(PentahoBasicProcessingFilter.java:115)
at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:411)
at org.springframework.security.context.HttpSessionContextIntegrationFilter.doFilterHttp(HttpSessionContextIntegrationFilter.java:235)
at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:411)
at org.pentaho.platform.web.http.filters.HttpSessionPentahoSessionIntegrationFilter.doFilter(HttpSessionPentahoSessionIntegrationFilter.java:263)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:411)
at org.springframework.security.wrapper.SecurityContextHolderAwareRequestFilter.doFilterHttp(SecurityContextHolderAwareRequestFilter.java:91)
at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:411)
at org.springframework.security.util.FilterChainProxy.doFilter(FilterChainProxy.java:188)
at org.springframework.security.util.FilterToBeanProxy.doFilter(FilterToBeanProxy.java:99)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.pentaho.platform.web.http.filters.SystemStatusFilter.doFilter(SystemStatusFilter.java:55)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.pentaho.platform.web.http.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:114)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.pentaho.platform.web.http.filters.WebappRootForwardingFilter.doFilter(WebappRootForwardingFilter.java:70)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.pentaho.platform.web.http.filters.PentahoPathDecodingFilter.doFilter(PentahoPathDecodingFilter.java:34)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:617)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:518)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1091)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:668)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1527)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1484)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.Exception: Unable to locate Hadoop Configuration Registry
at org.pentaho.hadoop.hive.jdbc.HadoopConfigurationUtil.findHadoopConfigurationProvider(HadoopConfigurationUtil.java:101)
at org.pentaho.hadoop.hive.jdbc.HadoopConfigurationUtil.getProvider(HadoopConfigurationUtil.java:107)
at org.pentaho.hadoop.hive.jdbc.HadoopConfigurationUtil.getActiveConfiguration(HadoopConfigurationUtil.java:120)
at org.pentaho.hadoop.hive.jdbc.HadoopConfigurationUtil.getActiveHadoopShim(HadoopConfigurationUtil.java:131)
at org.apache.hive.jdbc.HiveDriver.getActiveDriver(HiveDriver.java:108)
... 99 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.pentaho.hadoop.hive.jdbc.HadoopConfigurationUtil.findHadoopConfigurationProvider(HadoopConfigurationUtil.java:94)
... 103 more
Caused by: org.pentaho.hadoop.shim.ConfigurationException: Unable to load Hadoop configuration
at org.pentaho.hadoop.shim.HadoopConfigurationLocator.loadHadoopConfiguration(HadoopConfigurationLocator.java:405)
at org.pentaho.hadoop.shim.HadoopConfigurationLocator.findHadoopConfigurations(HadoopConfigurationLocator.java:184)
at org.pentaho.hadoop.shim.HadoopConfigurationLocator.init(HadoopConfigurationLocator.java:150)
at org.pentaho.di.core.hadoop.HadoopConfigurationBootstrap.initializeHadoopConfigurationProvider(HadoopConfigurationBootstrap.java:217)
at org.pentaho.di.core.hadoop.HadoopConfigurationBootstrap.initProvider(HadoopConfigurationBootstrap.java:156)
at org.pentaho.di.core.hadoop.HadoopConfigurationBootstrap.getProvider(HadoopConfigurationBootstrap.java:129)
at org.pentaho.di.core.hadoop.HadoopConfigurationBootstrap.getHadoopConfigurationProvider(HadoopConfigurationBootstrap.java:98)
... 108 more
Caused by: java.lang.RuntimeException: Unable to find relevant provider for chosen authentication method (id of cdh-kerberos
at org.pentaho.hadoop.shim.cdh54.authorization.AuthenticatingHadoopShim.onLoad(AuthenticatingHadoopShim.java:70)
at org.pentaho.hadoop.shim.HadoopConfigurationLocator.loadHadoopConfiguration(HadoopConfigurationLocator.java:402)
... 114 moreTraced the source code and searched some issues (http://jira.pentaho.com/browse/PDI-13283), I added the 3 commented lines above, copied kettle-engine-6.0.0.0-353.jar from pdi to tomcat/lib (for KerberosAuthenticationProvider ). Same result.
As suggested by the document, I also tried the setting in spoon. It failed in the "Active Shim Load" step, "Learn more" link displayed nothing. In the console, spoon displayed :
Code:
2015/12/14 11:21:16 - Named cluster: bigdata - Cluster Test: Hadoop Configuration
2015/12/14 11:21:16 - Named cluster: bigdata - Active Shim Load: ERROR Unable to load the {0} Shim.
2015/12/14 11:21:16 - Named cluster: bigdata - Unable to load Hadoop configuration
2015/12/14 11:21:16 - Named cluster: bigdata - Shim Configuration Verification: SKIPPED This test was skipped because Active Shim Load was not successful.
2015/12/14 11:21:16 - Named cluster: bigdata - The Shim Configuration Verification test was skipped because test [hadoopConfigurationTestShimLoad] was not successful.
2015/12/14 11:21:16 - Named cluster: bigdata - Cluster Test: Hadoop File System
2015/12/14 11:21:16 - Named cluster: bigdata - Hadoop File System Connection: SKIPPED This test was skipped because Active Shim Load was not successful.
2015/12/14 11:21:16 - Named cluster: bigdata - The Hadoop File System Connection test was skipped because test [hadoopConfigurationTestShimLoad] was not successful.
2015/12/14 11:21:16 - Named cluster: bigdata - User Home Directory Access: SKIPPED This test was skipped because Active Shim Load was not successful.
2015/12/14 11:21:16 - Named cluster: bigdata - The User Home Directory Access test was skipped because test [hadoopConfigurationTestShimLoad] was not successful.
2015/12/14 11:21:16 - Named cluster: bigdata - Root Directory Access: SKIPPED This test was skipped because Active Shim Load was not successful.
2015/12/14 11:21:16 - Named cluster: bigdata - The Root Directory Access test was skipped because test [hadoopConfigurationTestShimLoad] was not successful.
2015/12/14 11:21:16 - Named cluster: bigdata - Verify User Home Permissions: SKIPPED This test was skipped because Shim Configuration Verification was not successful.
2015/12/14 11:21:16 - Named cluster: bigdata - The Verify User Home Permissions test was skipped because test [hadoopConfigurationTestShimXConfig, hadoopConfigurationTestShimLoad, hadoopFileSystemListHomeDirectoryTest] was not successful.
2015/12/14 11:21:16 - Named cluster: bigdata - Cluster Test: Map Reduce
2015/12/14 11:21:16 - Named cluster: bigdata - Ping Job Tracker / Resource Manager: SKIPPED This test was skipped because Active Shim Load was not successful.
2015/12/14 11:21:16 - Named cluster: bigdata - The Ping Job Tracker / Resource Manager test was skipped because test [hadoopConfigurationTestShimLoad] was not successful.
2015/12/14 11:21:16 - Named cluster: bigdata - Cluster Test: Oozie
2015/12/14 11:21:16 - Named cluster: bigdata - Oozie Host Connection: SKIPPED This test was skipped because Active Shim Load was not successful.
2015/12/14 11:21:16 - Named cluster: bigdata - The Oozie Host Connection test was skipped because test [hadoopConfigurationTestShimLoad] was not successful.
2015/12/14 11:21:16 - Named cluster: bigdata - Cluster Test: Zookeeper
2015/12/14 11:21:16 - Named cluster: bigdata - Zookeeper Ensemble Connection: SKIPPED This test was skipped because Active Shim Load was not successful.
2015/12/14 11:21:16 - Named cluster: bigdata - The Zookeeper Ensemble Connection test was skipped because test [hadoopConfigurationTestShimLoad] was not successful.