Forum

Results 1 to 14 of 14
  1. #1
    Join Date
    November 2012
    Posts
    21

    ts3 3.0.6.1 amd64 hang/crash

    teamspeak3-server_linux-amd64-3.0.6.1 crash on 2.6.18-194.17.1.el5

    server dropped all users, I went in and stopped and started via ssh

    core environment file http://bit.ly/QFiTan 3.3 MB created by make_core_environment.sh as described here http://forum.teamspeak.com/showthrea...linux-properly

    core dump file http://bit.ly/Qk5NOm 39.4 MB

    Any idea what I can do to not have this happen again? Also, is there some (easy) way to monitor the TS server and automagically restart it should it hang? Thanks in advance for any assistance you can provide.

  2. #2
    Join Date
    November 2012
    Posts
    21
    The same thing happened again.

    The server hangs, all clients are kicked, no clients can connect, I am able to log in and stop start with via SSH

    ./ts3server_startscript.sh stop
    Stopping the TeamSpeak 3 serverdone

  3. #3
    Join Date
    September 2012
    Posts
    6,080
    Please post the complete server log of both the virtual server that had this problem and the instance log (_0.log)
    Some users wrote alternative scripts that check whether the process is still running and starts it again through a cronjob, just search the forums and you should find some.

  4. #4
    Join Date
    November 2012
    Posts
    21
    Quote Originally Posted by Chris View Post
    Please post the complete server log of both the virtual server that had this problem and the instance log (_0.log)
    Thanks for looking into this Chris, here are the logs http://bit.ly/10Q536E

  5. #5
    Join Date
    September 2012
    Posts
    6,080
    The server log doesn't show anything that would explain this problem.
    Make sure to enable all logging on the virtual server in the logging tab and that you start the server with logquerycommands=1 parameter

  6. #6
    Join Date
    November 2012
    Posts
    21
    Quote Originally Posted by Chris View Post
    The server log doesn't show anything that would explain this problem.
    Make sure to enable all logging on the virtual server in the logging tab and that you start the server with logquerycommands=1 parameter
    Chris, thank you very much for looking into this.
    I will enable all logging options and refresh this thread with new information should the server crash again. It has been stable for for the past couple of days without that I made any changes.

  7. #7
    Join Date
    November 2012
    Posts
    21
    Sadly there has been no noticeable improvement. I have confirmed with the VPS host that this wasn't some sort of DDOS issue either.

    New copy of the core_envrionment: http://bit.ly/RvNF5E

    Core dumps and logs: http://bit.ly/UjYQuK

    Sometimes it stays up for 1 hour before it crashes, sometimes for 14, but it's never been a lot more than that.

    It's also not really crashing, the process is still running and can be stopped with ./ts3server_startscript.sh stop just fine, the server just kicks everyone and doesn't accept connections till it's stopped and restarted.

    The VPS host can't find anything wrong with the VPS, so we are back to thinking that this is a problem with the TS3 server.

  8. #8
    Join Date
    September 2012
    Posts
    6,080
    It looks like network issue then, maybe iptables (or similar services) blocks traffic after a certain amount of packets in a certain amount of time or the hoster itself limits UDP traffic.

  9. #9
    Join Date
    November 2012
    Posts
    21
    Quote Originally Posted by Chris View Post
    It looks like network issue then, maybe iptables (or similar services) blocks traffic after a certain amount of packets in a certain amount of time or the hoster itself limits UDP traffic.
    Just to confirm, a core.* file is created every time the server is stopped or only if there is something wrong with the server? The reason I am asking is that a core.* file was created in each case where the server kicked everyone and didn't accept new connections.

    If there is a firewall issue, let's say it's limiting packets/second, then TS3 wouldn't really have any knowledge of that and wouldn't create a core file?

  10. #10
    Join Date
    November 2012
    Posts
    21
    Quote Originally Posted by Chris View Post
    It looks like network issue then, maybe iptables (or similar services) blocks traffic after a certain amount of packets in a certain amount of time or the hoster itself limits UDP traffic.
    The firewall is now completely disabled and the issue persists.
    Any chance to get a copy of the debugging symbols? I realize that there is no support for NPL licenses but if I could get the debugging symbols I could have someone start looking at the core.* file and try to figure this out.

    The problem has to be somehow related to the environment or the TS code itself. If you are interested to have someone look at it then please let me know, I can provide full root access to the server this is running on.
    Last edited by TheDude; December 6th, 2012 at 06:57 PM.

  11. #11
    Join Date
    July 2002
    Location
    Germany
    Posts
    2,192
    Hi,

    I have had a look at the core dumps, and the problem that occures in each of these is that the operator "new", which is used to request memory to be used by the programme, causes an exception. This can happen when "new" is not able to allocate the requested memory. So it seems to problem you are seeing is you are running out of (heap) memory. Please check the limitations to memory that are imposed on you (either by limited physical memory or by some configuration of the virtual machine or of the ulimits...My guess is if you provide sufficient memory you will also no longer run into that other problem you describe, where the server stops responding.

  12. #12
    Join Date
    November 2012
    Posts
    21
    Quote Originally Posted by Peter View Post
    This can happen when "new" is not able to allocate the requested memory. So it seems to problem you are seeing is you are running out of (heap) memory. Please check the limitations to memory that are imposed on you (either by limited physical memory or by some configuration of the virtual machine or of the ulimits...My guess is if you provide sufficient memory you will also no longer run into that other problem you describe, where the server stops responding.
    Peter, thank you so very much for looking into this, it's truly appreciated.

    This install runs a single virtual server. The ulimit for physical/virtual were set for 200MB on that VPS. It appears that the issue occurred at roughly 40-60 clients online at a time. Does the memory usage scale linearly, meaning if 40 users need 200MB, do 400 need 2000MB? Any other guidance on how much memory is required per new client connection?

    I have used this post you made a few years ago to check the memory usage of the TS3 process.

    There are currently 16 clients connected and here are the outputs:

    ps_mem.py
    10.6 MiB + 161.0 KiB = 10.7 MiB ts3server_linux_amd64

    meminfo-1.0.2
    Memory usage per user: ts3srv 1 9m11s 12m46s 6.30 MiB
    Memory usage by processes with same names: ts3server_linux 1 9m11s 12m46s 6.30 MiB

    Process memory usage sorted by unique resident set size (in KiB):
    PID UID URES SHR VIRT data exe hwres lckd lib pte stack virt-p MINFLT MAJFLT Started S CMD (n threads)
    3079 ts3srv 6456 5492 125536 38328 4580 12032 0 3936 204 88 125792 3139 108 2012-12-06 S ts3server_linux (25 T)

  13. #13
    Join Date
    November 2012
    Posts
    21
    Another update to this.

    I have completely removed any ulimits and everything shows as unlimited. During a "stress test" I had close to 60 clients connect and the server crashed again. I don't have the exact memory consumption at the time of the crash, but top reported around 124MB and the ps_mem and meminfo around 18MB at about the time of the crash. There was sufficient additional memory available.

    Here's the link to that particular core: http://bit.ly/XK7t79

    There has to be something that is specific to this environment that makes it so that the memory is not seen as available even though it is?

  14. #14
    Join Date
    September 2012
    Posts
    6,080
    I've send it to Peter again, who will have a look at this again. Please have patience, it might be a couple days before he is able to respond.

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. System Hang
    By Oaker in forum General Questions
    Replies: 10
    Last Post: February 12th, 2011, 03:38 PM
  2. beta21 crash on Debian/squeeze amd64
    By dibbe in forum Linux
    Replies: 4
    Last Post: June 9th, 2010, 07:24 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •