{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Realtime ASR without VAD\n",
"\n",
"Let say you want to transcribe realtime recording / input using PyAudio without VAD, malaya-speech able to do that."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"\n",
"This tutorial is available as an IPython notebook at [malaya-speech/example/realtime-asr-without-vad](https://github.com/huseinzol05/malaya-speech/tree/master/example/realtime-asr-without-vad).\n",
" \n",
"
\n",
"\n",
"This module is not language independent, so it not save to use on different languages. Pretrained models trained on hyperlocal languages.\n",
" \n",
"
\n",
"\n",
"This is an application of malaya-speech Pipeline, read more about malaya-speech Pipeline at [malaya-speech/example/pipeline](https://github.com/huseinzol05/malaya-speech/tree/master/example/pipeline).\n",
" \n",
"
"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Cannot import beam_search_ops from Tensorflow Addons, ['malaya.jawi_rumi.deep_model', 'malaya.phoneme.deep_model', 'malaya.rumi_jawi.deep_model', 'malaya.stem.deep_model'] will not available to use, make sure Tensorflow Addons version >= 0.12.0\n",
"check compatible Tensorflow version with Tensorflow Addons at https://github.com/tensorflow/addons/releases\n",
"/Users/huseinzolkepli/.pyenv/versions/3.9.4/lib/python3.9/site-packages/tqdm/auto.py:22: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
" from .autonotebook import tqdm as notebook_tqdm\n",
"torchaudio.io.StreamReader exception: FFmpeg libraries are not found. Please install FFmpeg.\n",
"`torchaudio.io.StreamReader` is not available, `malaya_speech.streaming.torchaudio.stream` is not able to use.\n",
"`openai-whisper` is not available, native whisper processor is not available, will use huggingface processor instead.\n",
"`torchaudio.io.StreamReader` is not available, `malaya_speech.streaming.torchaudio` is not able to use.\n"
]
}
],
"source": [
"import malaya_speech\n",
"from malaya_speech import Pipeline"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Streaming interface\n",
"\n",
"```python\n",
"def stream(\n",
" vad_model=None,\n",
" asr_model=None,\n",
" classification_model=None,\n",
" sample_rate: int = 16000,\n",
" segment_length: int = 2560,\n",
" num_padding_frames: int = 20,\n",
" ratio: float = 0.75,\n",
" min_length: float = 0.1,\n",
" max_length: float = 10.0,\n",
" realtime_print: bool = True,\n",
" **kwargs,\n",
"):\n",
" \"\"\"\n",
" Stream an audio using pyaudio library.\n",
"\n",
" Parameters\n",
" ----------\n",
" vad_model: object, optional (default=None)\n",
" vad model / pipeline.\n",
" asr_model: object, optional (default=None)\n",
" ASR model / pipeline, will transcribe each subsamples realtime.\n",
" classification_model: object, optional (default=None)\n",
" classification pipeline, will classify each subsamples realtime.\n",
" device: None, optional (default=None)\n",
" `device` parameter for pyaudio, check available devices from `sounddevice.query_devices()`.\n",
" sample_rate: int, optional (default = 16000)\n",
" output sample rate.\n",
" segment_length: int, optional (default=2560)\n",
" usually derived from asr_model.segment_length * asr_model.hop_length,\n",
" size of audio chunks, actual size in term of second is `segment_length` / `sample_rate`.\n",
" ratio: float, optional (default = 0.75)\n",
" if 75% of the queue is positive, assumed it is a voice activity.\n",
" min_length: float, optional (default=0.1)\n",
" minimum length (second) to accept a subsample.\n",
" max_length: float, optional (default=10.0)\n",
" maximum length (second) to accept a subsample.\n",
" realtime_print: bool, optional (default=True)\n",
" Will print results for ASR.\n",
" **kwargs: vector argument\n",
" vector argument pass to malaya_speech.streaming.pyaudio.Audio interface.\n",
"\n",
" Returns\n",
" -------\n",
" result : List[dict]\n",
" \"\"\"\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Check available devices"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"> 0 MacBook Air Microphone, Core Audio (1 in, 0 out)\n",
"< 1 MacBook Air Speakers, Core Audio (0 in, 2 out)"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import sounddevice\n",
"\n",
"sounddevice.query_devices()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"By default it will use `0` index."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Load ASR model"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"scrolled": true
},
"outputs": [
{
"data": {
"text/plain": [
"{'mesolitica/conformer-tiny': {'Size (MB)': 38.5,\n",
" 'malay-malaya': {'WER': 0.17341180814, 'CER': 0.05957485024},\n",
" 'malay-fleur102': {'WER': 0.19524478979, 'CER': 0.0830808938},\n",
" 'Language': ['malay']},\n",
" 'mesolitica/conformer-base': {'Size (MB)': 121,\n",
" 'malay-malaya': {'WER': 0.122076123261, 'CER': 0.03879606324},\n",
" 'malay-fleur102': {'WER': 0.1326737206665, 'CER': 0.05032914857},\n",
" 'Language': ['malay']},\n",
" 'mesolitica/conformer-medium': {'Size (MB)': 243,\n",
" 'malay-malaya': {'WER': 0.12777757303, 'CER': 0.0393998776},\n",
" 'malay-fleur102': {'WER': 0.1379928549, 'CER': 0.05876827088},\n",
" 'Language': ['malay']},\n",
" 'mesolitica/emformer-base': {'Size (MB)': 162,\n",
" 'malay-malaya': {'WER': 0.175762423786, 'CER': 0.06233919000537},\n",
" 'malay-fleur102': {'WER': 0.18303839134, 'CER': 0.0773853362},\n",
" 'Language': ['malay']},\n",
" 'mesolitica/conformer-singlish': {'Size (MB)': 121,\n",
" 'singlish': {'WER': 0.08535878149,\n",
" 'CER': 0.0452357273822,\n",
" 'WER-LM': 0.08516293835,\n",
" 'CER-LM': 0.044870668858},\n",
" 'Language': ['singlish']},\n",
" 'mesolitica/conformer-medium-mixed': {'Size (MB)': 243,\n",
" 'malay-malaya': {'WER': 0.122076123261, 'CER': 0.03879606324},\n",
" 'malay-fleur102': {'WER': 0.1326737206665, 'CER': 0.05032914857},\n",
" 'singlish': {'WER': 0.08535878149,\n",
" 'CER': 0.0452357273822,\n",
" 'WER-LM': 0.08516293835,\n",
" 'CER-LM': 0.044870668858},\n",
" 'Language': ['malay', 'singlish']}}"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"malaya_speech.stt.transducer.available_pt_transformer()"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"model = malaya_speech.stt.transducer.pt_transformer(model = 'mesolitica/conformer-base')"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"_ = model.eval()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### ASR Pipeline\n",
"\n",
"Because pyaudio will returned int16 bytes, so we need to change to numpy array then normalize to float, feel free to add speech enhancement or any function, but in this example, I just keep it simple."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAToAAACbCAYAAADydVejAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nO3deViT154H8C+EBNmDIMiOrApEhQLWCmJFrQWlWvciWJ0O1d5pvbfjWOdWW+/U9t6pz9zrtdrejl1cq45atVpFBVvcHgtFkE1lUwkk7CTsSSC/+cPJO0RWLSQQz+d58iR53+Sc35s3+eW82zlGRERgGIYxXApjfUfAMAwz1FiiYxjG4LFExzCMwTPRdwCM4SAiyGQyNDQ0QCaTQaFQoKWlhZsvl8uhVqsBADweD9bW1tw8S0tLmJqawsbGBra2thAKhTAyMtL5MjCGiSU6pl8dHR14+PAhiouLIZFIUF5eDqlUivLyckgkEtTU1EAmk6GxsXFQ67WxsYFQKMSYMWPg7OwMV1dXODk5wdXVFS4uLvD29oaHhwd4PN6g1ssYHiN21JXRUKlUKCgoQFZWFgoKClBUVIS7d++itLQUSqUSAGBubg43NzeMHTsWbm5ucHJygoODA4RCoVZrzMbGBgKBoFurjc/nc3U1Nzdz8+RyOVQqFWQyGeRyOdcqlMvlqKqqglQqhVgsRmVlJcrLy9Ha2goAMDU1hbe3N/z9/eHr64uAgACEhIRgwoQJMDFh/+MMAEDBEt0z7O7du7h69SoyMjKQlZWF3NxcKBQKmJmZYcKECfDz84Ofnx/8/f3h5+cHHx8fCIVCfYcNAKivr0dxcTEKCwtx7949FBYWoqioCHfu3EF7eztGjRqFiRMnIjg4GGFhYZg+fTp8fX31HTajHyzRPUsKCwuRkpKCK1euIC0tDZWVlbC0tERISAhCQkIQHByMkJAQjB8/fsS2hjo6OlBQUIBbt24hKysLt27dwq1bt9Da2gpnZ2dERUUhKioK0dHR8PHx0Xe4jG6wRGfIOjs7kZ2djTNnzuDYsWMoKCiAhYUFpk6dimnTpiEiIgKRkZEwNTXVd6hDqqOjA7dv30ZKSgquXbuGa9euQSaTwcvLC/PmzcP8+fMRFRXFbVYzBoclOkN048YN7Nu3D8ePH0d9fT38/f0RFxeHefPm4YUXXhixrbXB0tHRgatXr+Ls2bP44YcfUFxcDHt7eyxduhSJiYmYMmWKvkNkBhdLdIaioqIC33zzDQ4cOICioiKIRCKsXLkSCxcuZPum+nHnzh2cOnUKBw8eREFBAcaPH4+EhASsXr0aTk5O+g6P+e1YohvpsrOz8cUXX2D//v0wMzPDkiVLkJCQgIiICH2HNiLl5+fjwIED+PbbbyGTyfDKK69gw4YNCA8P13dozNNTgJgRKTU1laKioggATZ48mfbt20cKhULfYRmM9vZ2+vrrr0kkEhEAio6OpitXrug7LObptLNLwEaYnJwcxMTEIDo6GmZmZkhNTUVWVhYSExMhEAj0HZ7BMDU1xZo1a3D79m1cvHgRADB9+nS88sorKCgo0HN0zJNiiW6EkMlk+Od//mcEBwejpqYGly9fxvnz5zFz5kx9h2bQjIyMMHv2bKSkpODChQsoKyvDxIkT8dZbb6GpqUnf4TEDxPbRjQDnzp1DUlISOjs78be//Q3Lli1j14HqiVqtxsGDB7FhwwaYm5vjq6++wqxZs/QdFtM31h/dcKZUKvHmm28iNjYWM2bMQH5+PpYvX86SnB4ZGxsjMTEReXl5CAsLw5w5c7B+/Xp0dHToOzSmD6xFN0zV1dVh8eLFyMzMxL59+7Bw4UJ9h8T04PDhw3jjjTcQERGBo0ePDptL5Bgt7PSS4aisrAyzZs2CUqnEmTNnIBKJ9B0S04dbt24hLi4O1tbWSElJgbOzs75DYrSxRDfc1NXVITIyEnw+HxcvXoSjo6O+Q2IGQCKRYNasWeDz+UhLS2Mtu+GF7aMbTtra2jBv3jy0t7cjOTl5yJPctWvXcPDgQe526NAhXLp0CcXFxVCpVFqvbWhoQEBAAPbt2zdo9Xd0dODEiROYNWsW/v73vw9pXUPN2dkZFy5cQH19PRYsWMB1a8UMDyzRDSP/8R//gTt37iA5OVknlx5NmzYNdnZ2SEhIwB/+8AdIpVLcuHEDcXFxcHd3584fAwATExPY2dnB0tJy0OrXdNyZmpqqlViHoi5dcHNzQ3JyMjIzM/GXv/xF3+EwXenvZGWmq5ycHOLz+fTFF1/ovG5ra2sKDAzkntfV1ZGPjw8ZGRlRVlbWkNZdW1tLAGj79u1DWo8u/fWvfyVTU1MqKCjQdyjMI+zKiOFi06ZNCAsLQ1JSks7rFggEWqesjB49GsuWLQMR4ciRI0NatyH2pPLOO+8gKCgImzdv1ncozP9hiW4YkEgkuHDhAv71X/8VxsbDY5VoukAvLy8HALS3t+PAgQNam7MlJSXYsWMHgEf7+95//33s27ePGwBHo7GxEXv27MG7776LXbt2aXWh3tM5gT3VVVxcjM2bN0OtVqOoqAgff/wx9uzZ021fYl916QqPx8Mf/vAHnD17FjU1NTqvn+mBvtuUDNGnn35K9vb2erso397enoKCgrSmTZ48mQDQN998Q3fu3KEFCxYQAPrP//xPIiL67LPPyNLSkpycnOjQoUMkEonIzMyMANCiRYu4cgoLC2n+/Pl04cIFys7OpqCgIPL29qaGhgYiIpLL5Vqbrj3VtXfvXnJ0dCQA9MMPP9Crr75KsbGxBIC2bNky4Lp0qbW1lWxsbGjnzp06r5vppp0lumEgLi6Oli9frrf67e3tycPDgzIyMujatWu0fPlyAkCvv/46qdVqIiKqqKjQSj5ERMuWLSMLCws6ePAgERFJJBKaOnUqAaALFy4QEdHs2bPp5MmT3HvOnz+vlaAeT3S91bVx40YCQKdPn+amvfjii+Tn58c9768uXXvllVdo2bJleqmb0dJueDtIRqAHDx4gLi5OrzHweDw8fPgQYrEYc+bMwaZNmzBp0iRufk9HQC0sLGBtbY34+HgAgJOTE/785z9jxowZuHTpEkQiES5duoTJkyfjl19+AQA0NzcjNDSUG8WrJ73VBQAxMTHctKCgIK5cqVT6VHUNJR8fH1y9elUvdTPaWKIbBtrb22FmZqbXGMzNzbFo0aJe5/e27/DxfWxhYWEAALFYjKKiIgDAxo0bYW9vP+BYeqqrp2kWFhbcNaZPW9dQMjc3R1tbm77DYMAORgwLNjY2qK+v13cYg0IgEMDU1BTu7u5c/3i3bt3q9rrB7uJIl3UNVF1dHWxtbfVSN6ONJbphYMKECbh9+7be6ici0FNeCdje3q71/MaNG1AoFAgPD4e/vz94PB4+/PBDrSsFampqcOjQod8U8+N0WddAZWdnY8KECXqpm9HGEt0wEBMTg59//hmVlZU6r1upVKKhoQGNjY19vk5zmkZLS4vWdLlcjrKyMu55cnIyQkNDsWjRItja2mLt2rW4efMmoqKi8N1332Hv3r2Ij4/HihUrei23p2maFm/XTcGOjg6oVCooFIoB1aVL5eXl+OWXX7T2KTJ6pO/DIQxRW1sbjR49WudXB6SlpdGrr75KAAgArVu3jtLT07u9rqysjNatW0cAKCAggM6fP09ERGvWrCELCwuKi4uj3bt3U1JSEkVERND9+/e597a0tFBiYiJXh7W1NXdkVCKRaJV76tSpHus6deoUeXp6EgBav349lZaW0pEjR2jcuHEEgP7t3/6Nqqqq+qxL1z766CNydHQkpVKpl/oZLe2s95JhYsOGDdi/fz8KCgqGzc70/vzTP/0TkpOTcf/+fRQUFMDGxgbjxo3r8bW1tbUoKyvDhAkThvzAiy7r6kllZSUCAgLw1ltvYdu2bTqvn+mGddM0XDQ1NSEgIAAzZ84cMb12aBJdRUWFvkMZVpYuXYr09HTk5+dzp8UwesW6aRourKyssGvXLhw4cABff/21vsMZkNbW1m777J51n3/+OY4fP47//u//ZkluGOFt3bp1q76DYB4ZP3481Go1Nm7cCJFINGyP2KlUKnz55Zc4cOAA6uvr0dHRAT8/P+762GfV6dOnsXr1anzyySdYtWqVvsNh/l8n23QdhpKSknDo0CEcPHiQjRUxQhw5cgSvv/46kpKSsHPnTn2Hw2hjm67D0RdffIFVq1Zh8eLFrAPHYY6IsHXrVrz22mtYt24d15sLM8zo8ZAv04+dO3cSj8ejJUuWUE1Njb7DYR4jlUopLi6O+Hw+ffnll/oOh+kd63hzOHv77beRnJyMmzdvIigoCN9//72+Q2L+z+HDhxEUFITc3FykpqbqpcNUZuBYohvmZs2ahdzcXMybNw+LFy/GggULcPfuXX2H9czKy8tDbGws4uPjsWzZMuTk5CAyMlLfYTH9YIluBLCxscFXX32Fixcv4v79+xCJRHjzzTchkUj0HdozQywWY82aNZg8eTIqKyvx008/Yffu3SNuAJ9nFUt0I8isWbOQlZWF7777DikpKfDy8kJiYiLy8/P1HZrBKiwsxPr16+Hv74/U1FR8/vnnyMjIQFRUlL5DY54AS3QjjLGxMZYsWYL8/Hzs2LED6enpEIlEiI2NxYULF9DZ2anvEEe8zs5OnDt3Di+99BLGjx+PlJQU7Ny5E0VFRUhKSho243owA8fW2Ag1atQorF27FgUFBTh9+jTa2towd+5ceHh44L333mOtvKeQk5ODDRs2wM3NDfPmzYNarcaPP/6IvLw8vPHGG1yfd8zIw04YNiCFhYU4cOAADhw4gIcPHyI4OBgLFizA/PnzERwcrO/whh0iQmZmJs6cOYNTp04hJycHXl5eSEhIQEJCAry9vfUdIjM42EX9hkitVuPKlSs4cuQIzp49i4qKCq6VMnfuXERGRj6zPd/W1dXh6tWrSE5O1vps5s+fj+XLlyMiIqLHIRiZEY0lOkPXtdVy5swZZGdnw8jICCKRCFFRUYiKisKUKVPg4uKi71CHhFgsxs2bN3HlyhX8/PPPyM/Ph5GREUJCQjB//nzW2n02sET3rNG0aNLS0pCWloacnBx0dnbC0dERISEh3E0kEmHcuHEwMRkZ4yepVCrcv38fOTk5uHXrFnerqakBj8dDcHAwpk+fjqioqGe6RfuMYonuWSeXy7USQ1ZWFu7duwe1Wg0+n49x48bBz88P/v7+8Pb2hqurK1xcXODs7AxHR0edbeap1WpUVVVBIpFAIpFALBajuLgY9+7dQ1FREe7fv4+Ojg7weDz4+/trJe2QkBBYWVnpJE5mWGKJjumuubmZSyCFhYXc45KSEq3Ryvh8PhwdHeHo6AhbW1sIhULY2NhAKBRCKBSCz+fD0tISfD4fwKMjxZoef9va2riBdVQqFZqbm6FUKiGTySCXy7n7hoYGVFVVobKykhvaEADs7Ozg7e0NPz8/7ubr6wt/f3/WDxzzOJbomCfT1taG8vJySKVSiMViVFZWorq6GjKZDA0NDVqJqqOjA3K5HGq1GsCjwW40I3SZmprC3NwcwKPBs62trcHn87USpVAohK2tLRwcHODk5ARXV1c4OzvDxcUFo0aN0ttnwIw4LNExurVo0SIIBAIcPnxY36Ewzw7WHx3DMIaPJTqGYQweS3QMwxg8lugYhjF4LNExDGPwWKJjGMbgsUTHMIzBY4mOYRiDxxIdwzAGjyU6hmEMHkt0DMMYPJboGIYxeCzRMQxj8FiiYxjG4LFExzCMwWOJjmEYg8cSHcMwBo8lOoZhDB5LdAzDGDyW6BiGMXgs0TEMY/BYomMYxuCxRMcwjMFjiY5hGIPHEh3DMAaPJTqGYQweS3QMwxg8lugYhjF4LNExDGPwWKJjGMbgGRER6TsIxjDt3bsXO3bsQGdnJzetvLwcRkZGcHFx4abxeDxs3LgRr732mj7CZAyfgiU6Zsjcu3cP48eP7/d1RkZGKCkpwbhx43QQFfMMUrBNV2bI+Pv7QyQSwcjIqNfXGBkZITQ0lCU5ZkixRMcMqcTERPB4vF7n83g8JCYm6jAi5lnENl2ZISWRSODq6orevmbGxsaQSCRwdHTUcWTMM4RtujJDy9nZGS+88AKMjbt/1Xg8HmbMmMGSHDPkWKJjhlxCQkKv++kSEhJ0HA3zLGKbrsyQq6+vh6OjIzo6OrSm8/l81NTUwMbGRk+RMc8ItunKDL3Ro0dj9uzZMDEx4aaZmJggNjaWJTlGJ1iiY3Ri5cqVWicOd3Z2YuXKlXqMiHmWsE1XRidaWlpgb2+P9vZ2AIC5uTlqa2thZmam58iYZwDbdGV0w8LCAnFxceDz+eDz+Vi8eDFLcozOsETH6Ex8fDxUKhVUKhW7rpXRKbbpyuiMUqnEmDFjwOPxUF1drXVwgmGGkIJ90xidEQgEWLp0KUxNTVmSY3SKteiYblQqFZqbmwEAbW1t3AGE5uZmqFQq7nUtLS1QKpV9ltXU1KR1/lx+fj5MTEzg7+/PTTMxMYGVlVWf5ZiamsLc3Jx7zufzYWlpCQAwMzPDqFGjAABWVlYsiTKPYy26kaSzsxONjY1oaGiAXC5HY2MjWltb0dzcjJaWFigUCshkMigUCrS2tqKpqQlKpRJyuRzt7e1oa2tDY2MjFAoFmpqaAACNjY3o7OwEEUEmk+l5CQeXra0tAO1Eam1tDYFAAGtray5BCoVCCAQCWFpawsLCAgKBALa2thAIBLCwsIClpSXMzc1hY2MDa2tr2NrawsbGpsfL2pjhibXodKy5uRl1dXWoqalBXV0dd5PJZGhsbIRcLodMJoNcLueSmeZe08rqiaWlJQQCAYRCIdf66WmalZUVBAIBd6Ku5ocNgPvxGhsbc/M1P3bg0SkhpqamXJ1dW1W9ebwl1pPW1lYoFIo+X/N4y1CTzAHtlqVcLodarYZarYZcLgfwaN9gS0sLAEAmk0GpVKK5uZmrt+s0TVkNDQ1a7+uJpaUll/y63tva2nKPbWxsYGdnBzs7O4wZM4Z73N/nxgwq1vHmbyWXyyGRSFBVVYWKigpUV1ejrq4OtbW1qK2t1UpmdXV13X7QfD4fdnZ2EAqF3I9DKBRqPX/8XigUcj8ic3NzLhExQ0OTFDV/Og0NDVp/QI//KXWdL5PJUFdX1+3yN1NTUy7p2dnZwd7eHvb29txjBwcHuLi4wNHREc7OzrC2ttbT0hsEluh6U19fD7FYjPLycq0kVlFRgaqqKkilUlRWVqKtrY17D5/Ph4ODA/eF7foP/vhN88VmX+Bng1wu79aK73rr+sdYW1uL6upqreRoZmYGJycnODk5YezYsXB2duaSoYODA9zc3ODu7g6hUKjHpRy2nt1E19DQgNLSUu4mkUgglUpRWlqKkpISrf1Vo0aNgq2tLZydneHk5NTrvaOjY5+dTDLMk2hoaIBEIkFDQwOkUin3He16L5FIun1XnZ2d4eXlBS8vL+77qXnu7u7+LB6sMdxE19bWhuLiYhQVFWndFxcXQyqVctddCgQCuLm5cf+IHh4ecHd35567u7uzTUNmWGtubkZZWRnKysogFotRVlaGhw8fctMqKiq4fZg8Hg8uLi7w9vaGr68vfHx84OPjwz3WHL02MCM70anVapSWliIvL69bQisvLwcRwcjICG5ublor1cPDA25ubvDw8MDYsWP7HNOAYUY6tVqNyspKreQ3kN+Kr68vfH19ERgYiHHjxo3ko8wjJ9E1NDQgPz8fmZmZKCgoQH5+PrKzs7mjYra2tvDy8kJAQAACAwO5prq/vz87wsUwfVAqlSgvL0dpaSny8/NRUFDA7dJ58OAB1Go1BAIBfHx88NxzzyEwMFDrdzYCDL9ER0S4d+8e0tPTkZGRgZycHOTl5aG+vh4AMHbsWAQFBSEoKAiBgYGYOHEiAgICWDJjmCHQ1NSE/Px85ObmIi8vD/n5+cjJyUFNTQ0AwM7ODhMnToRIJEJYWBjCw8Ph6+s73LaS9J/opFIp0tPTuVtGRgbkcjlMTU0xefJkTJo0CSKRCIGBgRCJRLC3t9dnuAzDAKiurkZubi7y8/ORl5eH7OxsZGdnQ6VSwdbWFuHh4QgPD+eSn57HBdF9oisuLsbly5dx+fJl3LhxA2KxGMbGxvDz8+M+nPDwcEyaNIk7kZVhmOFPoVAgKysLGRkZXMOlqKgIRAQPDw9MmzYNM2fOxMyZM3U9ju/QJzqJRILU1FQuuZWVlcHCwgKRkZGIjIzElClTEBoayrrUZhgDJJPJuKR35coVXL9+Ha2trRg3bhxmzpyJ6OhovPjiixg7duxQhjH4iY6I8Ouvv+L48eP44YcfcPfuXQgEAjz//PPcgk2ZMgV8Pn8wq2UYZgRQKBS4efMmLl++jNTUVKSnp0OlUiEoKAhxcXFYtGgRQkJCBr3aQUl0RISbN2/i+PHjOHHiBB4+fAgvLy+8+uqrmD17NiIiIvq93pFhmGdPc3Mzrly5gkuXLuHkyZNc7li8eDEWLVqEsLCwwTiw8dsS3d27d/HVV1/h6NGjKC8vh5+fHxfgEGRlhmEMWNetwRMnTqCkpAQeHh5Yvnw53njjDfj4+Dxt0U+e6NRqNU6dOoWdO3fiypUrcHd3R2JiIhYvXoyJEyc+bSAMwzBasrKycPz4cezfvx8VFRWYOXMm3nnnHcyfP/9JW3kK0AB1dHTQt99+S/7+/mRsbExxcXF07tw56uzsHGgRTA/EYjF98MEH5ObmRm1tbfoOh+mDSqWi48ePU3R0NO3YsWPI68vPz6d33nmHQkNDn+h9Dx48oD/+8Y/k4eExNIHpWEdHB505c4ZiYmLIyMiIAgMD6eDBg0+Se9oHdE1HSkoKQkJCkJSUhGnTpiE/Px+nT5/Gyy+/PJIvCxkWSkpKcO3aNYjFYtBT7EXorx83fZWli3J1rby8nDuLoGtPy0Pl/v37SE5ORm1t7RO9r7S0FD///DPKy8uHKDLd4vF4mDdvHn788Ufk5OQgJCQEq1atQnh4OK5cuTKgMvrMUs3NzVi7di3mzJkDT09P5Obm4uuvv8b48eMHZQEYICoqCtOnT3/q97///vtQq9WDEstglqWLcnXN09NTp6OXxcbGPtW+7hdffBHTpk0bgoj0LygoCPv370d2djbGjBmDGTNmYP369VrdpfWk10QnlUoxffp0HD16FPv378fp06e1+vlnBs/TnmqTm5uLf/zjH4MSw2CWpYty9UXXXRw97XfD0E/fCgoKwvnz53H06FEcPHgQ06dPR1VVVa+v73Gt1dbWYtq0aTAzM0N2djY8PDyGLGBduXbtGs6fPw83NzcYGxsjKSkJwKNNxzNnzuD3v/899xo/Pz8kJCRobZY3Njbi6NGjuHPnDry8vPD6669rXV/b3/z29nYcO3YM+fn5mDp1KubMmdNtAGcjIyPcuXMHR48ehaenJ+Lj43v9wl6/fh2vvfYaWlpacOTIEfD5fCxZsgTAo03FtLQ0pKWlwdnZGXPnzoW3t3evn01fZfW2XKmpqaiurubKiImJQWFhIYqLiwEAc+bMwd27d3st93EPHjzAt99+i02bNqGqqgp79+6Fo6MjVqxYAaFQiJKSEhw7dgwCgQCrV6/mxoPQKCoqwrlz5yCTyRAeHo6XX36ZmzeQdVxfX48dO3Zg9erVfZ6139tO8L7q/y3LZmRkxNWZnJyMtLQ0BAcHY+nSpVr1q1QqnDx5EllZWZgxY0aPLei+YhyplixZguDgYMTExCAqKgo3b97sufPRnvbcvfTSS+Tl5UW1tbWDtkNRnzZu3EiHDh2ilpYWOnz4MFlaWhIR0WeffUaWlpbk5OREhw4dIpFIRGZmZgSAFi1axL2/sLCQ5s+fTxcuXKDs7GwKCgoib29vamhoGND8Bw8eUGRkJO3Zs4fEYjFFR0eTl5cXd/Dh448/JgB08uRJSkxMpMTERAJAH3/8ca/LdPXqVYqPjycAdPbsWbpw4QIREbW1tdGMGTPoyJEj1NDQQJ999hlZWVnRiRMnnrisvparvr6eVq9eTQBo5cqVRPTowIqZmRmdPXuW1Gp1r+U+7vDhw+Tq6koA6NixY5SQkEDx8fHE4/Ho1VdfpbS0NFq+fDnFx8eTiYkJxcTEaL3/7bffpsjISKqtraWLFy+SkZER/eUvf3midfw///M/BIA++eSTXj8nIiK5XE4AaPv27QOq/7cuW2JiInl6etIf//hHev7558nFxUXrMycikslkFB0dTVu3bqW6ujrat28fCQQC4vF4A4rREEilUnJ1daUFCxb0NLu9W6K7cuUKAaCrV68OfXQ6oFQqyc7Oju7du8dNW79+Pfd42bJlZGFhQQcPHiQiIolEQlOnTiUA3A9z9uzZdPLkSe4958+fJwC0ZcuWAc9PSkri5p89e5aMjIzo+++/J6L/T3Rdk1FcXBx5enr2uWx/+tOfCACp1Wpu2muvvUarV6/Wet3ixYvJzMyMxGLxE5XV33IplUqKiIggKysrKisro/Xr13PL1Fe5fdV/+vRpbtpbb71FAGj//v3ctM2bNxMAksvl3DQbGxvatm0b9zwgIICef/557vlA1rFSqaSzZ89SU1NTn3H2lOj6q/+3LFtiYiJZWVlReno6ERG1t7fTrFmzCABdvHiRK+vxH/i8efO0El1/MRqCixcvEgDus+qivdum6/nz5yESiRARETEYLUu94/P5sLKywqxZs/Dll1/i5Zdfxvvvv8/Nt7CwgLW1NeLj4wEATk5O+POf/4wZM2bg0qVLEIlEuHTpEiZPnoxffvkFwKODNKGhoWhtbYVUKu1zfnFxMS5duoRz585xdcbGxqKyshIODg5ascbExHCPvb29kZqa+kTL2traimPHjuG//uu/tKavW7cOx48fx7fffostW7YMqKz+lgt49Nnu378fkyZNwvz58zF37lwsXLjwiWLW0Gzmdz0wM2nSJADQ+i5qDoRVVFRw4238+OOPmDBhAgAgPT0dRKS1c7q/dTxnzhzw+XzExsY+Vez91f9blg0A7O3tERYWBuDRoDpJSUlISUnBxYsXMWnSJOzZswd///vftWKaOHEizp8/P+AYDcHs2bPh6+uL8+fPc5+XRrdEV11dDScnJ3Xr9G8AAA0tSURBVJ0Fpwu7du1CQkICYmJiMHXqVOzduxdjxozh5j++30XzIYnFYhQVFQEANm7c2GMXUZrD273NP3PmDAB06y/v8ST3OD6f/8SnMNy4cQMqlarbDnNfX18AQGFh4YDL6m+5NcaNG4dPPvkEb7/9Nt59990nircrzb6yruui69CKGpoebbp+NtOmTcPJkyfx/fff46WXXoKnpycqKiq03tfXOv6t+qv/tyxbT+bMmQMTExNIJBLcvn0bKpWq20Xxjy/vQD4jQ+Ds7NzjQYluR139/Pxw+/btfkdgH0liY2NRXFyM3//+98jMzERoaCju3LnT6+sFAgFMTU3h7u7Offlu3brV7XVNTU39ztcccLhw4UK3+ZrOCweLZhyMGzduaE3XJCo/P78Bl9Xfcmmo1Wpcu3YN0dHReOeddwYlcTypjRs34ptvvsGePXuwcuXKHpPI47quY33U/1vY2NjAzMwMvr6+3LqQSqXDKkZ9aGtrQ25ubo+nv3VLdPHx8ZDJZNi1a5dOghtqLS0t2LNnD0aPHo2//e1v+Pnnn9Hc3IzDhw9zr2lvb9d6z40bN6BQKBAeHg5/f3/weDx8+OGHWsm/pqYGhw4d6nd+QEAAjI2NcebMGS4RAY+OBP7666+DsoyacoODg2Fqaorr169rzdck1MjIyAGX1d9yaWzbtg2rVq3CoUOHwOfzsWrVqh5PfO667IMpMzMT27dvx+9+9zutgV0ej6GvdayL+gdTZWUlmpqaMH36dO5H3XUzVUNz5FUfMerDX//6VygUCixfvrzbvG6JzsXFBR988AE2bdqE5ORknQQ4lNRqNT788EPuiz516lT4+vpqbbrK5XKUlZVxz5OTkxEaGopFixbB1tYWa9euxc2bNxEVFYXvvvsOe/fuRXx8PFasWNHvfGdnZyQmJiInJwdLlizB5cuXsXv3bmzZsgVz584FAG6fl+YeeJQYVCpVny1rzTJkZmbi6tWrsLa2xttvv4379+/jp59+4l536tQpLFmyBFFRUQMuy8zMrM/l0nxOdXV1ePnll+Ho6IhPP/0UP/30Ez799NNey3084Wg0NzcDADcGCPD/m3Bd9yVpxjrVvE7TK86pU6fQ0dGBlJQU3L59Gw0NDSgqKsL9+/cB9L2OASA1NRUxMTFIS0vr9TPqKc6B1P+0y6bR2tqq9d3Yvn07Xn/9dcycORMBAQGYO3cuzp49i7179wJ4NAZEdnY2iAhisZhrnff3GY1kp06dwocffoht27Zp/bY5PR29UKvVtHr1auLz+bRnz56hO0yiA42NjWRmZkYikYh27txJW7dupdWrV5NSqSQiojVr1pCFhQXFxcXR7t27KSkpiSIiIuj+/ftcGS0tLdwpHwDI2tpa62hkf/PlcjktXLiQm+/p6UkZGRlERPTDDz/Q+PHjCQD97ne/o+LiYjpy5Ah5eXkRANq4cWOvp/mUlpaSo6Mj2dra0ldffUVERJ2dnfTuu+/SmDFj6L333qNVq1bR0qVL+72Otqey+lqu48ePk5WVFa1Zs4Y6OjqIiOjzzz8nAMTj8WjTpk3U2traY7mPu3jxIgUFBREAWrt2Ld27d4/OnTtHYWFh3KkUOTk5dPnyZYqIiCAAtHjxYiooKCAiooSEBDI2NiZHR0f6xz/+Qdu2bSNjY2PasGHDgNfx2rVrCQBNnjy5189IIpHQunXrCAAFBATQqVOn+q3/ty7b9evX6fnnnyd/f3/605/+RKtXr6Z3332XWltbubgqKyspMjKSAJCfnx/FxcXRypUrydLSkv7lX/6FysvL+/2MRrJdu3aRiYkJvfXWW729pPvpJRpqtZo2b95MxsbGtHDhQpJKpUMT5RBTq9XU0tJCjY2NlJmZ2e30gTVr1pCzszMpFArKysqi0tLSXsuqqamhzMxMrS/Zk8yvqKig7OxsLskOBqVS2WN9ra2tdOvWrSfqKKC3svpbrqctdzBVV1drfa719fXc44Gs47a2NsrPz6dXXnll0OsfDFKplDIyMqi5ubnX1xQXF9O9e/dIrVZTaWmp1mkquohR18RiMcXGxhKPx9M6daYHvSc6jcuXL5OnpydZWlrSRx991OcHPRJpfgSM4RroOs7MzDSok2gNVWNjI23ZsoXMzc3Jx8eHrl271t9b+u+95MUXX0RBQQH+/d//Hdu3b4enpye2bt36xD0qDFetra3d9okwhmUg6zg5ORkPHjzAe++9p6OomCdVXV2NzZs3w8PDA5999hm2bt2KvLy8gXVg8CSZtKamhj744AOys7OjUaNGUWJiIl2/fv3pU7MeKZVK2r17Nzk5OZGxsTFt3ry5zysHmJHnSdYx61dx+EpLS6P4+HgyNTUlBwcH+uijj550s7v9qbpSb2lpwcGDB/Hll18iKyuLdaHOMMygof/rUv3EiRM4fvw4SkpKEB4ejjfffBMrVqzo1hnGAPz2wXEyMjJw5MgRrUFxFi1ahMWLFw/WwBYMwxg4tVqNmzdv4sSJE91yyYoVKxAcHPxbih+84Q6ph4EtXF1dMWvWLG6YQ2dn58GoimEYAyAWi7lhD1NSUiCVSodq63DoBrDOysrCmTNnkJqaips3b0KpVGLChAncSN0zZszA6NGjh6JqhmGGodraWvz0009ccisqKsKoUaMwdepUREdHIy4uDiKRaCiqHrpE11VrayuuXbuG1NRUXL58GVlZWSAiBAUFISwsDFOmTEF4eDgCAwN13oMrwzCDT6VSITc3F+np6dztzp07MDIyQmhoKKKjozFz5ky88MILT7PP7UnpJtE9rqGhAWlpabhx4wbS09ORmZmJ5uZmmJubIyQkBGFhYQgPD0d4eDi8vLx0HR7DME+AiFBSUsIltIyMDGRlZaGtrQ1WVlYIDQ1FeHg4pk2bhunTp8PGxkbXIeon0T2us7MTd+/eRWZmJnf79ddfoVAoYG1tDV9fXwQEBCAwMBABAQEICwvr1i0NwzBDr6GhAfn5+SgoKEB+fj4yMzORk5ODpqYmmJiYwM/PD8899xyee+45REREIDg4eDiMFDg8El1PWltbkZ2djZycHOTm5iIvLw+5ubloaGgA8KjzxKCgIIhEIgQGBsLPzw++vr5wdHTUc+QMM/JVVlaiqKgIhYWFyMvL435/mr7e7OzsIBKJEBQUhKCgIEyaNAmTJ0/W6h1lGBm+ia43FRUV3IeuWQEFBQVcLxDW1tbw8fGBr68vd6+59dirAcM8o6qqqlBUVISioiIUFxejuLiYe6zp587c3BwBAQFcg2LixIkICgoaaZ3zjrxE1xP6v+5ouq4ozeOSkhKuayAbGxt4e3vDw8MD7u7u3L2bmxvc3d3Z5jBjMIgIlZWVKCsrQ1lZGcRiMR4+fIiHDx+irKwMJSUlaGxsBACMGjWKaxh0bST4+PjA1dXVEM6FNYxE1xe1Wo3y8nIuAZaUlHAr/sGDB6isrOQ6KNT0OKtJfh4eHnB1dcXYsWMxduxYODs7w8HBgR0ZZvRKpVKhuroaEokElZWVqKysREVFBR48eMB9t8ViMRQKBYBHXbk7OTnB09OT+257e3tzCc1AkllfDD/R9UepVKK8vLzbP55YLObuNR0nAo/64ndwcICDgwNcXFzg6OgIJycnrWRoZ2fH3VhSZAZCpVKhrq6Ou1VUVKCqqgpSqZRLZhKJBNXV1Vrj6QKAlZUVt1XS9U9as8Xi4uJi8ANa94MluoFobW3V+veUSqWoqqpCRUUF989aVVWF6urqbl2GC4VCjBkzRiv52dnZwd7enrvZ2dlBKBTC2toaQqEQQqHQ0P9hDZZarYZcLodMJoNcLodcLkddXR1qampQW1vLJbKuj2tqaiCXy7XK4fF4cHBw0NqScHFxgYODA5ydnbk/VicnJ66XY6ZXLNENps7Ozh6/0JrnXadrvuAymazHsqysrGBtbQ0bGxtYW1tzj21tbbWejxo1Cra2thAIBLCwsIClpSUEAgGEQiFMTU1hbm4OKysr1rLsRUdHB5qamtDa2gqFQgGZTAalUonm5ma0tLRAqVSioaEBbW1taGxshFwuR2Njo1Yia2xs5OZ1bf13NXr0aO5P7fHbmDFjus1zcHAYDqdlGAqW6PSto6MDdXV1/f6ANPea+Zp5bW1tkMlk/Q50YmxszCVGMzMzWFtbg8fjwcTEBFZWVgDAzQMeDc+o2dzRtDB5PJ7WeKNdy+2Lpq6edHZ2cjvFe9PT8snlcqjVaq4FBTzaDaHpd06TuACgsbERnZ2dXF1tbW1ob2/nyuiLkZERhEIhzMzMtP50NC3vrn86Pf0h2djYwM7OrtflZ3SCJTpDofmRNzc3Q6lUQiaTob29nWuJKBSKbi0XIoJCodAanOfx5EBEXKtTpVJ1a7Fo6ujNQBKZjY1Nn60Xc3PzbsPzdW2l9pSINa1Z4NEA1gKBgEtaXZO9qakprKysuDo0LWFN6/gZ37dlKFiiYxjG4CnYTgCGYQweS3QMwxg8lugYhjF4JgCO6TsIhmGYIaT6X3ocDi5fl5IbAAAAAElFTkSuQmCC\n",
"text/plain": [
""
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"p_asr = Pipeline()\n",
"pipeline_asr = (\n",
" p_asr.map(lambda x: model.beam_decoder([x])[0], name = 'speech-to-text')\n",
")\n",
"p_asr.visualize()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**You need to make sure the last output should named as `speech-to-text` or else the streaming interface will throw an error**."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Start Recording"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Again, once you start to run the code below, it will straight away recording your voice**. \n",
"\n",
"If you run in jupyter notebook, press button stop up there to stop recording, if in terminal, press `CTRL + c`."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**If you are not provide VAD model, make sure `max_length` set to proper value to chunk it**."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"scrolled": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"helo nama saya sin bin zulkifli hussein zulkifli saya suka bercakap tentang nasi sebab nasib ayam sedap status bukan nasib nasi status betullah tu terima kasih "
]
}
],
"source": [
"samples = malaya_speech.streaming.pyaudio.stream(asr_model = p_asr, max_length = 3.0)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Actually it is pretty nice. As you can see, it able to transcribe realtime, you can try it by yourself."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"11"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"len(samples)"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"scrolled": true
},
"outputs": [
{
"data": {
"text/plain": [
"{'wav_data': array([0.02052168, 0.01281819, 0.01248555, ..., 0.00667683, 0.00603234,\n",
" 0.00554827], dtype=float32),\n",
" 'timestamp': datetime.datetime(2023, 2, 16, 16, 31, 17, 749795),\n",
" 'asr_model': 'sin bin zulkifli'}"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"samples[1]"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [],
"source": [
"import IPython.display as ipd\n",
"import numpy as np"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
" \n",
" "
],
"text/plain": [
""
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"ipd.Audio(samples[1]['wav_data'], rate = 16000)"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [],
"source": [
"ipd.Audio(np.concatenate([s['wav_data'] for s in samples]), rate = 16000)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.4"
},
"varInspector": {
"cols": {
"lenName": 16,
"lenType": 16,
"lenVar": 40
},
"kernels_config": {
"python": {
"delete_cmd_postfix": "",
"delete_cmd_prefix": "del ",
"library": "var_list.py",
"varRefreshCmd": "print(var_dic_list())"
},
"r": {
"delete_cmd_postfix": ") ",
"delete_cmd_prefix": "rm(",
"library": "var_list.r",
"varRefreshCmd": "cat(var_dic_list()) "
}
},
"types_to_exclude": [
"module",
"function",
"builtin_function_or_method",
"instance",
"_Feature"
],
"window_display": false
}
},
"nbformat": 4,
"nbformat_minor": 4
}