The TIBCO Platform is a real-time, composable data platform that will bring together an evolving set of your TIBCO solutions - and it's available now!
A chart showing the TIBCO Platform vision
Jump to content
Forums
Ask questions and gain insight from discussions

Unable to join the TDV Server to cluster - Remote Server Error

Nikesh V

By Nikesh V
in TDV

 Share

Recommended Posts

Nikesh V Newbie

Nikesh V

Posted
Posted

Hi,

I am working on creating a cluster, I have 3 Linux VM with TDV Server installed on it. I have made on VM as the Active Cluster, I am trying to join the other 2 VM to the main VM.

I checked the port on all the 3 VM, the ports are opened, I am able to ping from Server 1 to Server 2 and its vice versa. When I try to connect I get the following error

 

 

 

Can anyone please let me know what is the reason for with this error.

Nikesh V Newbie

Nikesh V

Posted
  • Author
Posted

@motsuaboshi

 

I have added the logs for your reference.

 

INFO 2019-12-20 21:47:56.706 +0000 ConnectionService - Accepted view: [10.17.89.105:9407|0] [10.17.89.105:9407]
INFO 2019-12-20 21:47:56.706 +0000 ConnectionService - Connection service started on channel: 1576779980050-270296162 using TCP unicasts
INFO 2019-12-20 21:47:56.706 +0000 HealthMonitorImpl - Changed Health Monitor Intervals,  PingInterval = 600, PongInterval = 30, Tolerance = 600, aggressivePingInterval = 300
INFO 2019-12-20 21:47:56.706 +0000 HealthMonitorImpl - Changed Health Monitor Ping Table Path: 
INFO 2019-12-20 21:47:56.706 +0000 HealthMonitorImpl - Started HealthMonitor
INFO 2019-12-20 21:47:56.707 +0000 NewClusterNode - Cluster state now set to INITIALIZING_REMOTE
INFO 2019-12-20 21:47:56.707 +0000 NewClusterNode - Starting as a member of cluster QA_Server_1 using a heartbeat of 5 seconds.
INFO 2019-12-20 21:47:56.707 +0000 NewClusterNode - Other cluster members are: 'ue1ahlxbls001.qa.ocean.com-9400--1260488998' (ue1ahlxbls001.qa.ocean.com:9400)
ERROR 2019-12-20 21:47:56.709 +0000 NewClusterNode - Failed to connect to ue1ahlxbls001.qa.ocean.com:9403. Cause:
java.net.UnknownHostException: ue1ahlxbls001.qa.ocean.com
at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:797)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1505)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1364)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1298)
at java.base/java.net.InetAddress.getByName(InetAddress.java:1248)
at com.compositesw.server.dbchannel.RequestSelectThread.openChannel(RequestSelectThread.java:525)
at com.compositesw.server.dbchannel.DefaultChannelManager.openChannel(DefaultChannelManager.java:58)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.compositesw.common.services.ServiceResolver$1.invoke(ServiceResolver.java:18)
at com.sun.proxy.$Proxy8.openChannel(Unknown Source)
at com.compositesw.cluster.NewClusterNode.openClusterChannel(NewClusterNode.java:8342)
at com.compositesw.cluster.NewClusterNode.join(NewClusterNode.java:8255)
at com.compositesw.cluster.NewStandaloneNode.joinCluster(NewStandaloneNode.java:251)
at com.compositesw.server.api.core.ServerApi.joinCluster(ServerApi.java:433)
at com.compositesw.server.api.ws.server.JoinClusterOperation.invokeInternal(JoinClusterOperation.java:116)
at com.compositesw.server.api.ApiOperation.invoke(ApiOperation.java:114)
at com.compositesw.server.api.BuiltInProcedureRequest.invoke(BuiltInProcedureRequest.java:53)
at com.compositesw.server.request.HookProcedureRequest.invoke(HookProcedureRequest.java:79)
at com.compositesw.server.wsapi.handlers.SoapInvocationHandler.execute(SoapInvocationHandler.java:350)
at com.compositesw.server.wsapi.WsapiServlet.doGetOrPost(WsapiServlet.java:318)
at com.compositesw.server.wsapi.WsapiServlet.doPost(WsapiServlet.java:151)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669)
at com.compositesw.server.services.AuthenticationFilter.doFilter(AuthenticationFilter.java:104)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)
at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:301)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at com.compositesw.server.services.SecurityHeadersFilter.doFilter(SecurityHeadersFilter.java:97)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:258)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.base/java.lang.Thread.run(Thread.java:834)

INFO 2019-12-20 21:47:56.715 +0000 HealthMonitorImpl - Stopped HealthMonitor
INFO 2019-12-20 21:47:57.726 +0000 ConnectionService - Connection service stopped.
ERROR 2019-12-20 21:47:57.728 +0000 NewStandaloneNode - Error joining the cluster: 
Failed to connect to the remote host ue1ahlxbls001.qa.ocean.com at port 9,403. Cause:
   ue1ahlxbls001.qa.ocean.com  [cluster-2900003]
at com.compositesw.cluster.NewClusterNode.join(NewClusterNode.java:8262)
at com.compositesw.cluster.NewStandaloneNode.joinCluster(NewStandaloneNode.java:251)
at com.compositesw.server.api.core.ServerApi.joinCluster(ServerApi.java:433)
at com.compositesw.server.api.ws.server.JoinClusterOperation.invokeInternal(JoinClusterOperation.java:116)
at com.compositesw.server.api.ApiOperation.invoke(ApiOperation.java:114)
at com.compositesw.server.api.BuiltInProcedureRequest.invoke(BuiltInProcedureRequest.java:53)
at com.compositesw.server.request.HookProcedureRequest.invoke(HookProcedureRequest.java:79)
at com.compositesw.server.wsapi.handlers.SoapInvocationHandler.execute(SoapInvocationHandler.java:350)
at com.compositesw.server.wsapi.WsapiServlet.doGetOrPost(WsapiServlet.java:318)
at com.compositesw.server.wsapi.WsapiServlet.doPost(WsapiServlet.java:151)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669)
at com.compositesw.server.services.AuthenticationFilter.doFilter(AuthenticationFilter.java:104)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)
at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:301)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at com.compositesw.server.services.SecurityHeadersFilter.doFilter(SecurityHeadersFilter.java:97)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:258)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.base/java.lang.Thread.run(Thread.java:834)

 

 

Motsu Aboshi Newbie

Motsu Aboshi

Posted
Posted

Strange. This doesn't look like a TDV issue, since it's not getting far enough for that: it's a hostname resolution failure.

 

If this is a new-ish VM, is it possible the DNS cache used by the source VM didn't reflect the new hostname Granted, in that case, it shouldn't resolve via ping, either.

 

As a troubleshooting step, you could try adding the custer node names to /etc/hosts, if they aren't there already.

 

 

Nikesh V Newbie

Nikesh V

Posted
  • Author
Posted

@motsuaboshi Currently, I see this in the host file, on the server on which I have created the cluster 127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4

::1         localhost localhost.localdomain localhost6 localhost6.localdomain610.17.89.189 ue1ahlxbls001.qa.ocean.com ue1ahlxbls001

 ISo do you want me to add the server details of another node to this host file and do the same on the other server

Nikesh V Newbie

Nikesh V

Posted
  • Author
Posted

@motsuaboshi

 

I added the server details in the host file and when I try to join the nodes, I keep seeing the message "Joining the cluster on host" and it stays and never joins to the server to cluster.

 

cluster_join.png

Motsu Aboshi Newbie

Motsu Aboshi

Posted
Posted

@nikeshv - I would call that progress, so now we're into further troubleshooting.

 

How long did you wait for the join to finish Since it's a new cluster, shouldn't take long, but still isn't instant. Also, load up the manager page in another browser tab, make sure it's not actually trying to join (CONNECTED status).

 

Presuming it's not trying at all the first thing to check would be: make sure all of the ports are reachable, for every cluster node. Telnet to each of these ports for each cluster node, on each cluster node:

 

https://docs.tibco.com/pub/tdv/8.2.0/doc/pdf/TIB_tdv_8.2.0_InstallationGuide.pdf ("Port Requirements")

 

the important one for clustering is 9407.

 

Additionally, check the cluster logs (logs/cluster/cs_cluster.log) for any obvious errors.

Nikesh V Newbie

Nikesh V

Posted
  • Author
Posted

@motsuaboshi I waited for around 5 minutes, today I gave one more try and I waited for around 15 minutes and closed the loading. Checked the cluster and I see the following. The 1st node is the cluster and the 2nd node is the server that is added to the cluster

 

clusters.png

 

 

Nikesh V Newbie

Nikesh V

Posted
  • Author
Posted
The default value was 60 min, I changed it to 120 Min and restarted the server. I tried with the new configuration and I still see the same behavior.
  • 3 months later...

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing
© 2024 Cloud Software Group, Inc. All rights reserved.
×
×
  • Create New...