Forum


Notice to all users

We are migrating towards a new forum system located at community.teamspeak.com, as such this forum will become read-only on January 29, 2020

Results 1 to 12 of 12
  1. #1
    Join Date
    December 2009
    Location
    Portugal
    Posts
    492

    Server stops accepting connections

    Hi

    I've been having this weird problem for a couple of months now. Every 2 weeks or so, the server stops accepting connections and then drops everyone. I originally posted on another thread, but this might not be the same problem, just shares one of the symptoms: http://forum.teamspeak.com/showthrea...-limit-reached

    What i got so far:
    - It starts with some clients not being able to connect, while others are connected and talking
    - After some minutes (about 20/30m), clients that were in the server start dropping, until everyone is out (including lan clients)
    - The server process keeps running, the ports listening and replying to nmap
    - telnet to 10011 gives the information "error id=515 msg=max\sclients\sprotocol\slimit\sreached" and closes
    - If i restart the server process, everything is fine again
    - Logs don't show anything about the "crash", but startup and last lines of the files are as follow:

    ts3server_0.log
    ================================================== ================================================== ==============================

    2014-04-27 11:47:24.996481|INFO |ServerLibPriv | | TeamSpeak 3 Server 3.0.10.3 (2014-01-01 16:28:39)
    2014-04-27 11:47:24.996849|INFO |ServerLibPriv | | SystemInformation: Linux 3.14.0-031400-lowlatency #201403310035 SMP PREEMPT Mon Mar 31 04:44:05 UTC 2014 x86_64 Binary: 64bit
    2014-04-27 11:47:25.599022|INFO |DatabaseQuery | | dbPlugin name: MySQL plugin, (c)TeamSpeak Systems GmbH
    2014-04-27 11:47:25.599155|INFO |DatabaseQuery | | dbPlugin version: 1
    2014-04-27 11:47:27.124092|INFO |Accounting | | Licensing Information
    2014-04-27 11:47:27.124237|INFO |Accounting | | type : Non-profit
    2014-04-27 11:47:27.124326|INFO |Accounting | | starting date : Mon Dec 16 00:00:00 2013
    2014-04-27 11:47:27.124375|INFO |Accounting | | ending date : Sat Jun 21 00:00:00 2014
    2014-04-27 11:47:27.124416|INFO |Accounting | | max virtualservers: 10
    2014-04-27 11:47:27.124456|INFO |Accounting | | max slots : 512
    2014-04-27 11:47:31.270963|INFO | | | Puzzle precompute time: 3348
    2014-04-27 11:47:31.271469|INFO |FileManager | | listening on 0.0.0.0:30033
    2014-04-27 11:47:32.267706|INFO |CIDRManager | | updated query_ip_whitelist ips: 127.0.0.1, 192.168.1.0/24, 85.25.120.233,
    2014-04-27 11:47:32.270576|INFO |Query | | listening on 0.0.0.0:10011
    2014-04-27 12:24:33.049281|ERROR | | | resolving 'xxx' to binary ip failed, using 0.0.0.0 instead

    ...

    2014-05-14 13:40:39.350931|ERROR | | | resolving 'xxx' to binary ip failed, using 0.0.0.0 instead
    2014-05-14 14:26:39.353351|ERROR | | | resolving 'xxx' to binary ip failed, using 0.0.0.0 instead
    2014-05-14 15:24:11.474561|ERROR | | | resolving 'xxx' to binary ip failed, using 0.0.0.0 instead
    2014-05-14 16:36:18.071362|ERROR | | | resolving 'xxx' to binary ip failed, using 0.0.0.0 instead
    2014-05-14 16:40:26.832235|ERROR | | | resolving 'xxx' to binary ip failed, using 0.0.0.0 instead
    2014-05-14 16:54:28.545330|ERROR | | | resolving 'xxx' to binary ip failed, using 0.0.0.0 instead
    2014-05-14 17:05:52.959505|ERROR | | | resolving 'xxx' to binary ip failed, using 0.0.0.0 instead
    2014-05-14 17:34:29.297148|ERROR | | | resolving 'xxx' to binary ip failed, using 0.0.0.0 instead
    2014-05-14 17:43:25.267566|ERROR | | | resolving 'xxx' to binary ip failed, using 0.0.0.0 instead
    2014-05-14 17:49:10.327547|ERROR | | | resolving 'xxx' to binary ip failed, using 0.0.0.0 instead
    Just as a small note before you ask, the error(s) you see above are caused by tsdns, even though everything works as it should. It is not the root of this problem, as i already ran the server without tsdns server running and it happened again after a couple of weeks.


    ts3server_1.log
    ================================================== ================================================== ==============================

    2014-04-27 11:47:31.561647|WARNING |PermGroupMgr | 1| cldbid: 2, assigned to unknown gid: 2, ignoring!
    2014-04-27 11:47:32.034959|INFO |VirtualServer | 1| listening on 0.0.0.0:9987

    ...

    2014-05-14 17:10:14.976755|INFO |VirtualServer | 1| query client connected 'Unknown from 192.168.1.1:51717'(id:906)
    2014-05-14 17:10:14.981864|INFO |VirtualServerBase| 1| query client disconnected 'Unknown from 192.168.1.1:51717'(id:906) reason 'reasonmsg=connection lost'
    2014-05-14 17:12:36.786354|INFO |VirtualServer | 1| query client connected 'Unknown from 85.25.120.233:50246'(id:906)
    2014-05-14 17:12:37.950694|INFO |VirtualServerBase| 1| query client disconnected 'TSViewer.com DBscanREG 929266'(id:906) reason 'reasonmsg=disconnecting'
    2014-05-14 17:15:13.565363|INFO |VirtualServer | 1| query client connected 'Unknown from 192.168.1.1:51976'(id:906)
    2014-05-14 17:15:13.571066|INFO |VirtualServerBase| 1| query client disconnected 'Unknown from 192.168.1.1:51976'(id:906) reason 'reasonmsg=connection lost'
    2014-05-14 17:20:14.321872|INFO |VirtualServer | 1| query client connected 'Unknown from 192.168.1.1:52241'(id:906)
    2014-05-14 17:20:14.326638|INFO |VirtualServerBase| 1| query client disconnected 'Unknown from 192.168.1.1:52241'(id:906) reason 'reasonmsg=connection lost'
    2014-05-14 17:21:24.167997|INFO |VirtualServer | 1| query client connected 'Unknown from 85.25.120.233:55720'(id:906)
    2014-05-14 17:21:25.349725|INFO |VirtualServerBase| 1| query client disconnected 'TSViewer.com DBscanREG 929266'(id:906) reason 'reasonmsg=disconnecting'
    What i see in the above logs is that the last successful query client connection was at 17:21, and the last time the server threw one error was at 17:49. The clients got disconnected at 17:50. That seems to be the 20/30m interval where clients are unable to connect while others are connected.

    I cant really remember the last thing i changed in the server related to TS, but it was probably the update to 3.0.10.3, but then again, i cant say that its been happening since that update or not. The kernel was updated already after the problem surfaced, and the system packages were updated from Ubuntu 13.10 to 14.04.

    Next thing im thinking is to disable both the local TS3 Web Interface viewer and the TSViewer, but that would be reaching out too much already, as im sure others would have complained already if there was a problem with either...

    Any suggestions are very much appreciated
    Last edited by Jonybat; July 15th, 2017 at 12:16 PM.

  2. #2
    Join Date
    June 2011
    Location
    Germany
    Posts
    4,368
    Why do you get kicked instantly but the TS3Viewer bot doesn't? That's really strange.

  3. #3
    Join Date
    December 2009
    Location
    Portugal
    Posts
    492
    It wasn't that way...Clients stopped being able to connect at the same time that the last TSViewer bot connected. When that happened, clients were still conected to the server...

  4. #4
    Join Date
    September 2012
    Posts
    6,079
    You not being able to connect to telnet with error 515 is because there are more than the allowed maximum of concurrent query clients connected to your instance for some reason. You should look into who is using your query and maybe block outside access to the query by setting query_ip=127.0.0.1 even if only for testing purposes.
    When sending PMs please make sure to include a reference link to the thread in question in the body of your message.

  5. #5
    Join Date
    December 2009
    Location
    Portugal
    Posts
    492
    You mean simultaneous connections? From what i see in the logs, the query clients are disconnecting shortly after connecting.

    Could something like that cause the problems that i mention above?

    As i said, i only have a local web viewer from the Web Interface beta3.4.2, that is being accessed in one external website, and the TSViewer.com. I will try to be on top of it next time it happens, and then disable those viewers and set the "query_ip=127.0.0.1" like you said...and then wait for it to happen again

    EDIT: I just remembered that i also have a local munin plugin that reads the number of users every 5 minutes. I have been using all these 3 systems that use SQ almost since the beginning of TS3.
    Last edited by Jonybat; May 23rd, 2014 at 08:49 PM.

  6. #6
    Join Date
    September 2012
    Posts
    6,079
    Well if there are more than the allowed number of connections to the query interface at the same time, the server refuses to accept any additional connections. So you should check the open connections to the query and reduce them to below the threshold.
    When sending PMs please make sure to include a reference link to the thread in question in the body of your message.

  7. #7
    Join Date
    December 2004
    Location
    RF
    Posts
    3,008
    Would it be acceptable suggestion to reserve one slot for connections from localhost, or always accept local connections?
    This way, a system administrator could connect, even if all slots are used up, and analyze the situation in close before making calls?
    Would also help local scripts, such as monitoring systems, to watch over situation without being hindered by the limits imposed.

  8. #8
    Join Date
    September 2012
    Posts
    6,079
    Quote Originally Posted by ANR Daemon View Post
    Would it be acceptable suggestion to reserve one slot for connections from localhost, or always accept local connections?
    This way, a system administrator could connect, even if all slots are used up, and analyze the situation in close before making calls?
    Would also help local scripts, such as monitoring systems, to watch over situation without being hindered by the limits imposed.
    In general that might not be that bad of an idea, but you know yourself what people do to their servers... (not just TeamSpeak but in general), they just install and run all the random pieces of stuff they can find. With that in mind it's not that good an option anymore because the limits are not just there for the sake of limits. It may cause problems and then we get all sorts of threads of people complaining about their TeamSpeak servers not working. That is different from the current situation because it would likely be way more threads and issues then there are caused through these limits.
    When sending PMs please make sure to include a reference link to the thread in question in the body of your message.

  9. #9
    Join Date
    December 2004
    Location
    RF
    Posts
    3,008
    Well, let's not forget, that you can use anything in any way... generally speaking.
    What people do, and how the thing is expected to work are not necessarily match, though may be related. (Pick a hammed for example? And number of objects you can smash with it. That is not limited to nails.)
    Many high-load production servers press hard to make sure an administrator have a way to call up on the service to analyze and correct the situation, unless it's WAY out of control. I.e. MySQL reserve a slot for super-user login.
    The fear of users being stupid is not what you should be concerned with. Users ARE stupid, you can only endure this fact.
    Your concern should be to ensure server operation under most thinkable circumstances. And then some unthinkable, but likely possible.

  10. #10
    Join Date
    December 2009
    Location
    Portugal
    Posts
    492
    Quote Originally Posted by Chris View Post
    Well if there are more than the allowed number of connections to the query interface at the same time, the server refuses to accept any additional connections. So you should check the open connections to the query and reduce them to below the threshold.
    What im saying is that i never thought that the normal clients would be affected by any problem caused by server query connections, and if they really are (which seems to be the case), that would be something that should be looked into imho.

    Anyway, i think i may have just found the culprit. I looked into the logs yet again, and although the last 500 lines or so dont show anything different from what i posted, but a few more lines back i found this:

    2014-05-14 16:44:57.159780|INFO |VirtualServer | 1| query client connected 'Unknown from 192.168.1.1:50409'(id:908)
    2014-05-14 16:44:57.313659|INFO |VirtualServer | 1| query client connected 'Unknown from 192.168.1.1:50410'(id:908)
    2014-05-14 16:44:57.414205|INFO |VirtualServerBase| 1| query client disconnected 'Unknown from 192.168.1.1:50409'(id:908) reason 'reasonmsg=deselected virtualserver'
    2014-05-14 16:44:57.564911|INFO |VirtualServerBase| 1| query client disconnected 'Unknown from 192.168.1.1:50410'(id:908) reason 'reasonmsg=deselected virtualserver'
    2014-05-14 16:45:03.541275|INFO |VirtualServer | 1| query client connected 'Unknown from 192.168.1.1:50420'(id:908)
    2014-05-14 16:45:03.786484|INFO |VirtualServerBase| 1| query client disconnected 'Unknown from 192.168.1.1:50420'(id:908) reason 'reasonmsg=deselected virtualserver'
    2014-05-14 16:45:12.719733|INFO |VirtualServer | 1| query client connected 'Unknown from 192.168.1.1:50436'(id:908)
    2014-05-14 16:45:12.725037|INFO |VirtualServerBase| 1| query client disconnected 'Unknown from 192.168.1.1:50436'(id:908) reason 'reasonmsg=connection lost'
    2014-05-14 16:45:26.284276|INFO |VirtualServer | 1| query client connected 'Unknown from 192.168.1.1:50447'(id:908)
    2014-05-14 16:45:26.538289|INFO |VirtualServerBase| 1| query client disconnected 'Unknown from 192.168.1.1:50447'(id:908) reason 'reasonmsg=deselected virtualserver'
    2014-05-14 16:45:29.352282|INFO |VirtualServer | 1| query client connected 'Unknown from 192.168.1.1:50448'(id:908)
    2014-05-14 16:45:29.376866|INFO |VirtualServer | 1| query client connected 'Unknown from 192.168.1.1:50449'(id:908)
    2014-05-14 16:45:29.592748|INFO |VirtualServerBase| 1| query client disconnected 'Unknown from 192.168.1.1:50448'(id:908) reason 'reasonmsg=deselected virtualserver'
    2014-05-14 16:45:29.619535|INFO |VirtualServerBase| 1| query client disconnected 'Unknown from 192.168.1.1:50449'(id:908) reason 'reasonmsg=deselected virtualserver'
    2014-05-14 16:45:35.212881|INFO |VirtualServer | 1| query client connected 'Unknown from 192.168.1.1:50453'(id:908)
    2014-05-14 16:45:35.474995|INFO |VirtualServerBase| 1| query client disconnected 'Unknown from 192.168.1.1:50453'(id:908) reason 'reasonmsg=deselected virtualserver'
    2014-05-14 16:45:59.368387|INFO |VirtualServer | 1| query client connected 'Unknown from 192.168.1.1:50478'(id:908)
    2014-05-14 16:45:59.621058|INFO |VirtualServerBase| 1| query client disconnected 'Unknown from 192.168.1.1:50478'(id:908) reason 'reasonmsg=deselected virtualserver'
    ...and a few more like that along the rest of the log file. So, what i take from that is that somehow, 2 SQ clients are connecting in a row and only then disconnecting. Although they close the connection, i understand how the server could interpret this in a wrong way.

    Those connections that issue "logout" in the SQ are from the tsviewpub.php, from Psychokiller's Web Interface beta3.4.2, that is hosted locally and then sourced as an iframe to an external website, as i said previously. That php does a new SQ connection every time the page is reloaded, so the more the users in the website, the more likely something like this can happen. So, i just wrote a script that fetches the content of tsviewpub.php every minute and serves it as source to the iframe.

    Quote Originally Posted by Chris View Post
    That is different from the current situation because it would likely be way more threads and issues then there are caused through these limits.
    That is indeed my question...and my fear

    I will just wait and see if that was indeed the root of the problem...

  11. #11
    Join Date
    December 2009
    Location
    no
    Posts
    5
    Quote Originally Posted by Jonybat View Post
    I will just wait and see if that was indeed the root of the problem...
    Did you manage to resolve this issue?

    I currently have 1 instance running 1 virtual, and it is suffering from the same symptoms. It hits a point where it stops letting clients connect and then a little while after it drops all the clients that were already connected, with no errors in the log and only a number of query clients logged after it drops the normal clients.
    It is a high traffic server with high traffic query, now if it is somehow down to query connections/usage then it needs to throttle and work those connections better without affecting normal client connections.

  12. #12
    Join Date
    December 2009
    Location
    Portugal
    Posts
    492
    Its been running fine ever since, so i assume that the problem was indeed caused by the way that the viewers were implemented, as i have explained before.

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Server not accepting connections
    By Stylw in forum Linux / FreeBSD
    Replies: 5
    Last Post: January 6th, 2013, 05:05 PM
  2. TS 3 Not Accepting Server Password
    By smudger1767 in forum Windows
    Replies: 1
    Last Post: January 4th, 2013, 02:29 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •