Server crash and possible DB corruption

[jknorr]

Hello all,

I have been running a TS3 rc1 server for a couple of months now without any problems. A couple of days ago, the server went down and the cron job could not start the server back up. When I try to start it manually, I get this:

theknorr@theknorrs.com [~/ts]# ./ts3server_minimal_runscript.sh inifile=ts3server.ini
TeamSpeak Server 3.0.0-rc1 [Build: 14468]
(c)TeamSpeak Systems GmbH

Logging started
2011-10-17 17:14:39.720749|INFO    |ServerLibPriv |   | Server Version: 3.0.0-rc1 [Build: 14468], Linux
2011-10-17 17:14:39.721087|INFO    |DatabaseQuery |   | dbPlugin name:    SQLite3 plugin, Version 2, (c)TeamSpeak Systems GmbH
2011-10-17 17:14:39.721120|INFO    |DatabaseQuery |   | dbPlugin version: 3.7.3
2011-10-17 17:14:39.721398|INFO    |DatabaseQuery |   | checking database integrity (may take a while)
2011-10-17 17:14:40.543046|CRITICAL|DatabaseQuery |   | setSQLfromFile( file:sql/update_1.sql) failed
theknorr@theknorrs.com [~/ts]#

I pulled the log from the error:

2011-10-14 10:50:02.045534|INFO    |ServerLibPriv |   | Server Version: 3.0.0-rc1 [Build: 14468], Linux
2011-10-14 10:50:02.056302|INFO    |DatabaseQuery |   | dbPlugin name:    SQLite3 plugin, Version 2, (c)TeamSpeak Systems GmbH
2011-10-14 10:50:02.056380|INFO    |DatabaseQuery |   | dbPlugin version: 3.7.3
2011-10-14 10:50:02.057731|INFO    |DatabaseQuery |   | checking database integrity (may take a while)
2011-10-14 10:50:03.001935|INFO    |SQL           |   | pruning old database log entries where timestamp is older than 90 days
2011-10-14 10:50:03.850471|INFO    |Accounting    |   | Licensing Information
2011-10-14 10:50:03.850537|INFO    |Accounting    |   | type              : Non-profit
2011-10-14 10:50:03.850585|INFO    |Accounting    |   | starting date     : Wed Dec 22 00:00:00 2010
2011-10-14 10:50:03.850611|INFO    |Accounting    |   | ending date       : Fri Dec 23 00:00:00 2011
2011-10-14 10:50:03.850637|INFO    |Accounting    |   | max virtualservers: 10
2011-10-14 10:50:03.850659|INFO    |Accounting    |   | max slots         : 512
2011-10-14 10:50:04.270085|INFO    |FileManager   |   | listening on 69.89.22.54:30033
2011-10-14 10:50:04.697676|INFO    |VirtualServer |  1| listening on 69.89.22.54:9987
2011-10-14 10:50:04.698477|INFO    |CIDRManager   |   | updated query_ip_whitelist ips: 127.0.0.1, 69.89.22.54, 
2011-10-14 10:50:04.698948|INFO    |Query         |   | listening on 69.89.22.54:10011
2011-10-14 10:55:04.360201|ERROR   |              |   | TS3ANetwork::Send failed error: 1
2011-10-14 10:55:04.369584|ERROR   |              |   | TS3ANetwork::Send failed error: 1

2011-10-15 10:45:13.096026|ERROR   |              |   | TS3ANetwork::Send failed error: 1
2011-10-15 10:45:15.097250|ERROR   |              |   | TS3ANetwork::Send failed error: 1
2011-10-15 10:45:15.097942|ERROR   |              |   | TS3ANetwork::Send failed error: 1
2011-10-15 10:50:03.013694|DEBUG   |VirtualSvrMgr |   | pruning old log entries
2011-10-15 10:55:11.720273|ERROR   |              |   | TS3ANetwork::Send failed error: 1
2011-10-15 10:55:11.738953|ERROR   |              |   | TS3ANetwork::Send failed error: 1
2011-10-15 10:55:13.040274|ERROR   |              |   | TS3ANetwork::Send failed error: 1
2011-10-15 10:55:13.041115|ERROR   |              |   | TS3ANetwork::Send failed error: 1
2011-10-15 10:55:15.044260|ERROR   |              |   | TS3ANetwork::Send failed error: 1
2011-10-15 10:55:15.044932|ERROR   |              |   | TS3ANetwork::Send failed error: 1
2011-10-15 11:05:11.752187|ERROR   |              |   | TS3ANetwork::Send failed error: 1
2011-10-15 11:05:11.768516|ERROR   |              |   | TS3ANetwork::Send failed error: 1
2011-10-15 11:05:13.069457|ERROR   |              |   | TS3ANetwork::Send failed error: 1
2011-10-15 11:05:13.070206|ERROR   |              |   | TS3ANetwork::Send failed error: 1
2011-10-15 11:05:15.071100|ERROR   |              |   | TS3ANetwork::Send failed error: 1
2011-10-15 11:05:15.071893|ERROR   |              |   | TS3ANetwork::Send failed error: 1
2011-10-15 11:11:24.068022|INFO    |ServerMain    |   | Received signal SIGTERM, shutting down.
2011-10-15 11:11:24.201798|INFO    |VirtualServer |  1| stopped
2011-10-15 11:11:24.286929|ERROR   |DatabaseQuery |   | db_exec failed disk I/O error
2011-10-15 11:11:24.287030|ERROR   |DatabaseQuery |   | db_exec() insert into instance_properties ( server_id, id, ident, value, string_ error: disk I/O error

And the next log started showing the error I get now.

2011-10-15 11:15:03.801933|INFO    |ServerLibPriv |   | Server Version: 3.0.0-rc1 [Build: 14468], Linux
2011-10-15 11:15:03.809326|INFO    |DatabaseQuery |   | dbPlugin name:    SQLite3 plugin, Version 2, (c)TeamSpeak Systems GmbH
2011-10-15 11:15:03.809389|INFO    |DatabaseQuery |   | dbPlugin version: 3.7.3
2011-10-15 11:15:03.814605|INFO    |DatabaseQuery |   | checking database integrity (may take a while)
2011-10-15 11:15:05.600492|CRITICAL|DatabaseQuery |   | setSQLfromFile( file:sql/update_1.sql) failed

Is there any way I can fix this without destroying everything and starting over? Please excuse the "Send failed error:1 entries. They always seem to be there in the logs. I removed about 1000 lines of it just to fit it in the post.

[jknorr]

Oh well, I just reinstalled the whole thing. I will try to find a script to backup the database on a regular basis.

[dante696]

This could mean, that your harddrive was not available or full at this moment, while the server tried to write his entrys int othe database.

2011-10-15 11:11:24.286929|ERROR   |DatabaseQuery |   | db_exec failed disk I/O error
2011-10-15 11:11:24.287030|ERROR   |DatabaseQuery |   | db_exec() insert into instance_properties ( server_id, id, ident, value, string_ error: disk I/O error

There are several SqLite tools to export and import such a databse.
I use the plugin for Firefox SQLite Manager