Get voice input that ts3 "hears" with a plugin

[Jhhhj_original]

Is there a way in the pluginsdk to get the voice data of a user? I don't just want to constantly listen to the mic, but instead only get what teamspeak "hears".
So only get the voice data when the client is actually talking.
Basically the recording has to match the voice activation or ptt boundaries from the client.

Should I use functions that tell me when the client has started / stopped talking?
Is there a function to get the sound from the users microphone using the pluginsdk or do I have to use other libraries?
Or is there a better (intended) way?

It has to be a live transmission, so doing a recording doesnt help, unless I can record into a network stream or socket file

[Thomas]

There are indeed sound processing functions.

The sdk documentation is not perfect, but these functions are contained, so just take a look at it.

[Jhhhj_original]

Is the sample the raw data or encoded voice data?
Does the sample need to be freed by me or the client (I assume the client?)
And can I get access to TS's built in opus encoder somehow?

[Thomas]

The sample is freed by the client only. If I remember correctly, you can even manipulate the data, so there is no reason to free the memory.

The TeamSpeak Client Lib allows users to access the raw playback and capture voice data and even modify it, for example to
add effects to the voice. These callbacks are also used by the TeamSpeak client for the voice recording feature.

So it seems to be the raw data.

Looking into the code of CrossTalk, which processes (and manipulate) the sound data might help as well.

I never worked with the sound data, so I'm no expert on that and I'm out for any further question

[Jhhhj_original]

Alright, that already helped a lot, thanks.
But a couple more things I need:

When does the function get called?
Is it when a user starts speaking?
Or maybe every x milliseconds?
And what is the sample count in those cases?
Is it a number that makes for a valid 48 kHz opus encoder frame size?
And is there ANY decent documentation about the ts3plugin_onEditPlaybackVoiceDataEvent function?

[Chris]

Alright, that already helped a lot, thanks.
But a couple more things I need:

When does the function get called?
Is it when a user starts speaking?
Or maybe every x milliseconds?
And what is the sample count in those cases?
Is it a number that makes for a valid 48 kHz opus encoder frame size?
And is there ANY decent documentation about the ts3plugin_onEditPlaybackVoiceDataEvent function?

Easy to figure out yourself, through just taking a look at those events when using the client.

onEditPlaybackVoiceDataEvent / onEditPostProcessVoiceDataEvent: For every client in your channel (other than self) while they are talking, every 10ms.
onEditCapturedVoiceDataEvent: As long as your capture device is open, every 20ms.
onEditMixedPlaybackVoiceDataEvent: As long as your playback device is open, every 10ms.