{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Text-to-Speech GlowTTS"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"GlowTTS, Text to Melspectrogram."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"\n",
"This tutorial is available as an IPython notebook at [malaya-speech/example/tts-glowtts](https://github.com/huseinzol05/malaya-speech/tree/master/example/tts-glowtts).\n",
" \n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"\n",
"This module is not language independent, so it not save to use on different languages. Pretrained models trained on hyperlocal languages.\n",
" \n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"\n",
"This is an application of malaya-speech Pipeline, read more about malaya-speech Pipeline at [malaya-speech/example/pipeline](https://github.com/huseinzol05/malaya-speech/tree/master/example/pipeline).\n",
" \n",
"
"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import malaya_speech\n",
"import numpy as np\n",
"from malaya_speech import Pipeline\n",
"import matplotlib.pyplot as plt\n",
"import IPython.display as ipd"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### GlowTTS description\n",
"\n",
"1. Malaya-speech GlowTTS will generate melspectrogram with feature size 80.\n",
"2. Use Malaya-speech vocoder to convert melspectrogram to waveform."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### List available GlowTTS"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" Size (MB) \n",
" Quantized Size (MB) \n",
" Combined loss \n",
" Understand punctuation \n",
" Is lowercase \n",
" \n",
" \n",
" \n",
" \n",
" male \n",
" 119 \n",
" 27.6 \n",
" -1.429 \n",
" True \n",
" True \n",
" \n",
" \n",
" female \n",
" 119 \n",
" 27.6 \n",
" -1.464 \n",
" True \n",
" True \n",
" \n",
" \n",
" haqkiem \n",
" 119 \n",
" 27.6 \n",
" -1.649 \n",
" True \n",
" True \n",
" \n",
" \n",
" female-singlish \n",
" 119 \n",
" 27.6 \n",
" -1.728 \n",
" True \n",
" True \n",
" \n",
" \n",
" yasmin \n",
" 119 \n",
" 27.6 \n",
" -1.908 \n",
" True \n",
" False \n",
" \n",
" \n",
" osman \n",
" 119 \n",
" 27.6 \n",
" -1.908 \n",
" True \n",
" False \n",
" \n",
" \n",
" multispeaker \n",
" 404 \n",
" 79.9 \n",
" -1.882 \n",
" True \n",
" True \n",
" \n",
" \n",
"
\n",
"
"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"fig = plt.figure(figsize=(10, 8))\n",
"ax1 = fig.add_subplot(311)\n",
"ax1.set_title('Predicted Mel-Spectrogram')\n",
"im = ax1.imshow(np.rot90(r_male['mel-output']), aspect='auto', interpolation='none')\n",
"fig.colorbar(mappable=im, shrink=0.65, orientation='horizontal', ax=ax1)\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAhQAAAGoCAYAAAAemnx2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xm4ZWV55/3vjxLBCRGhkQAKGrSb2IqGoEaNOCQirw3andjghMYO2i1G36gRTV4H0nlDNOIQp64IMsSARETRYNA2TtGgFArIoG2JGgoRBGRQIkLV3X+sdXRzOMM+tce19/dzXes6az1r2M9ZaJ1738+UqkKSJGkQ20y6ApIkqfsMKCRJ0sAMKCRJ0sAMKCRJ0sAMKCRJ0sAMKCRJ0sAMKKQhS3Jikv/Z7j8+ybcmXSdJGjUDCmkrJflckh8n2W65a6rqi1X1kHHWaxiSfC/JU8Z9r6TuMqCQtkKSvYDHAwUcMtHKSNIUMKCQts7zgXOBE4EjlrsoyYFJNvUcPzLJ15PcnOTvk3yop3nkwCSbkrwyyTVJrkrywp57T0zyniSfTPKTJF9Kcr8kb28zJd9M8oie638lyRlJfpTku0n+sOfcG5OcnuTkti6XJNm/PXcKcH/g4+3n/PESv9fOST6R5IYk1yf5YpJtlrs3yaOTfLm9/sIkB/Y863NJ/iLJV5PclORjSXZqz22f5G+TXNfee16SXdf430rSGBhQSFvn+cAH2+2p/fyRS3JX4EyaIGQn4FTgmYsuux9wb2B34EXAu5Pcp+f8s4A/BXYGbgX+Bfhae/xh4Lj2s7YBPg5c2D7rycArkjy151mHAKcBOwJnAe8CqKrnAf8K/KequmdVvXmJX+eVwCZgF2BX4HXNrXe+N8nuwD8A/7P9vV8FnJFkl57nPR/4fWA34HbgnW35Ee372BO4L/AS4N+WqI+kCTOgkNYoyeOABwCnV9X5wHeAZ/dx66OBuwDvrKrbquojwFcXXXMbcEx7/mzgJ0BvH4wzq+r8qvoZTXDys6o6uao2Ax8CFjIUvwHsUlXHVNXPq+py4G+Aw3qe9c9VdXZ77ynAw/t/C9xG88f/AW1dv1jLLwz0XODs9rO2VNWngQ3AwT3XnFJVF1fVT4H/D3hWknXt59wX+NWq2tz+7jetoZ6SxsSAQlq7I4BPVdW17fHfsUKzR49fAa5c9If3ikXXXFdVt/cc3wLcs+f46p79f1vieOHaBwC/0jYT3JDkBposQm8m5YeLPmf7JHfp4/cAeAuwEfhUksuTHL3CtQ8Afm9RXR5HE5As6H0P3we2pcm6nAKcA5yW5AdJ3pxk2z7rKGmM+v3HQxKQ5G40zQ7rkiz8Qd4O2DHJw6vqwhVuvwrYPUl6goo9aTIcw3YF8N2q2mcr719xGeKqupmm2eOVSR4K/FOS86rqM0vcewVNBuIPVnjknj3796fJTFzbZk/eBLyp7Qh7NvAt4Pg1/C6SxsAMhbQ2zwA2A/sC+7XbfwC+SNMPYCX/0t57VJK7JDkUOGBE9fwqcHOS1yS5W5J1SR6a5Df6vP9q4IHLnUzy9CS/miTAjTS/15Zl7v1b4D8leWpbj+3bDqh79Fzz3CT7Jrk7cAzw4aranOSJSf5j2/xxE02gsQVJU8eAQlqbI4APVNW/VtUPFzaaDo3PWanJoKp+Dvxnms6WN9D0LfgETefKoWq/2T+dJuD5LnAt8H6aDo79+AvgT9smilctcX4f4H/T9PH4F+A9VfXZpe6tqiuAQ2maXH5Ek7F4NXf89+cUms6qPwS2BxZGpNyPprPpTcBlwOfbayVNmSzfj0rSqCX5CvC+qvrApOsyKUk+B/xtVb1/0nWRtPXMUEhjlOQJ7dwRd0lyBPAw4B8nXS9JGpQBhTReD6GZG+IGmk6Nv1tVV022SpLmTZIT2gn0Ll7mfJK8M8nGJBcleeSqz7TJQ5Kk+ZLkt2j6QJ1cVQ9d4vzBwMto5ot5FPCOqnrUSs80QyFJ0pypqi8A169wyaE0wUZV1bk0Q+N3W+H6bs9DcddsV9tzj0lXQ5I0wx78sFs4/6Jbr62qXVa/eus99Yn3qOuu3zyUZ51/0a2XAD/rKVpfVevX8IjdueOEc5vasmWbaDsdUGzPPXhUnjzpakiSZtg551zAut02fn/Un3Pd9Zv56jn3H8qz1u327Z9V1f5DeVifOh1QSJI0Kuf84AIAnvor+9HMND9aBWyZnnnbruSOM9ju0ZYty4BCkqRFzvnBBW0gMbfOopnV9zSaTpk3rjYizYBCkqSpUGyu8WQokpwKHAjsnGQT8AaaRfmoqvfRrJtzME1q5hbghas904BCkiR+2cQBTCQ70TR5jGcqh6o6fJXzBbx0Lc902KgkSRqYGQpJ0ly7Y+fLyZqiTplrZkAhSdIUKIrNHZ692oBCkjS3pm00x7j6UIyCfSgkSXNp2oKJrjNDIUnSFChgc4czFAYUkqS5MemhoauxyUOSJM01MxSSpLkw7X0mChzlIUnSNJv2YGJBd2ehsMlDkiQNwcgCiiR7JvlskkuTXJLk5W35Tkk+neTb7c/7tOVJ8s4kG5NclOSRo6qbJGl+dCU7URSbh7RNwigzFLcDr6yqfYFHAy9Nsi9wNPCZqtoH+Ex7DPA0YJ92OxJ47wjrJknSdCnYPKRtEkbWh6JdN/2qdv/mJJcBuwOH0iyZCnAS8DngNW35ye0KZ+cm2THJbqutvy5J0lKmaY2OfjSrjXbXWPpQJNkLeATwFWDXniDhh8Cu7f7uwBU9t21qyxY/68gkG5JsuI1bR1ZnSZLUv5GP8khyT+AM4BVVdVOSX5yrqkqypuRMVa0H1gPskJ26O75GkjQSXctM/FLYTFa/bEqNNKBIsi1NMPHBqvpIW3z1QlNGkt2Aa9ryK4E9e27foy2TJKkvXemAuZQCtnT4a/IoR3kEOB64rKqO6zl1FnBEu38E8LGe8ue3oz0eDdxo/wlJkrphlBmKxwLPA76RZGHy9NcBxwKnJ3kR8H3gWe25s4GDgY3ALcALR1g3SdKM6XJ2YoFNHkuoqn+GZd/Mk5e4voCXjqo+kiRNs2a1UQMKSZImZhayEwBbyoBCkqSx6V2GHLo4omP2GFBIkjQFbPKQJGkMerMSs5iRKMLmDq/Z2d2aS5KkqWGGQpLUCbOYlVjMTpmSJGkgXe9DYZOHJGnqLR7VoeljhkKSNNVmZY6J1YXN1d3v+QYUkiRNgQK2dLjhwIBCkjSVursM+dazD4UkSZprZigkSVNj1ievWkmVfSgkSRrY/HS+XN4WmzwkSdI8M0MhSZo4sxMLE1t193u+AYUkSVPBPhSSJK3ZPHfAXErX56Hobs0lSdLUMEMhSRqreZywql+bXW1UkqSVGUisrEinO2V2t+aSJGlqmKGQJI2UmYn+bXGUhyRJGoTzUEiStAQzE/PFgEKSpClQxFEekiT1cirtrdPlia0MKCRJQ2cwsXZVdHrq7e7WXJIkTQ0zFJKkoXBtjkGFLdiHQpIkDaDodpOHAYUkaSAODxUYUEiStpKBxPA5sZUkSRpIEbY4D8WdJTkBeDpwTVU9tC37EPCQ9pIdgRuqar8kewGXAd9qz51bVS8ZVd0kSYMzM6Feo8xQnAi8Czh5oaCq/uvCfpK3Ajf2XP+dqvJ/nZKkuWWTxxKq6gtt5uFOkgR4FvCkUX2+JGk07DsxGkW3VxudVM0fD1xdVd/uKds7ydeTfD7J45e7McmRSTYk2XAbt46+ppIkjUXYPKRtEibVKfNw4NSe46uA+1fVdUl+Hfhokl+rqpsW31hV64H1ADtkpxpLbSVpzjlplVYz9oAiyV2A/wz8+kJZVd0KTbqhqs5P8h3gwcCGcddPknRHLvQ1Hl1v8phEhuIpwDeratNCQZJdgOuranOSBwL7AJdPoG6SJE3MpJorhmGUw0ZPBQ4Edk6yCXhDVR0PHMYdmzsAfgs4JsltwBbgJVV1/ajqJklanZ0vtRajHOVx+DLlL1ii7AzgjFHVRZKkaVcVmzwkSbPFfhOT4eJgkqTOcyTHZBWMbfnyJAcB7wDWAe+vqmMXnb8/cBLNrNbrgKOr6uyVntndUEiSJK1ZknXAu4GnAfsChyfZd9FlfwqcXlWPoOn7+J7VnmuGQpLmnJ0vp0XG1eRxALCxqi4HSHIacChwac81BezQ7t8b+MFqDzWgkCRpCjTzUAytyWPnJL1zOa1vJ4YE2B24oufcJuBRi+5/I/CpJC8D7kEz5cOKDCgkaY7Z+XJmXVtV+w9w/+HAiVX11iSPAU5J8tCq2rLcDQYUkiRNiTGtNnolsGfP8R5tWa8XAQcBVNW/JNke2Bm4ZrmH2ilTkuaU2YnpUoQtNZxtFecB+yTZO8ldaTpdnrXomn8FngyQ5D8A2wM/WumhBhSSNIcMJuZXVd0OHAWcA1xGM5rjkiTHJDmkveyVwB8kuZBmdusXVNWKC3La5CFJ0pTYMqbv+e2cEmcvKnt9z/6lwGPX8kwDCkmaE05cNd2qYPPwRnmMnQGFJElTYojDRsfOgEKSZpwTV2kcDCgkaUYZSHRLM8qju2MlDCgkSZoSm8e0ONgoGFBI0gyx46UmxYBCkqQpMOS1PMbOgEKSZoSTVXWdfSgkSdIQbOlwH4ruhkKSpF8wO6FJM0MhSR3m0NDZ4UyZkiRpKOxDIUkaKzMTmjYGFJIkTYFmpkybPCRJY2BmYrZ1eZSHAYUkdYQjOTTNDCgkSZoCzpQpSRo5sxPzwVEekiRpMGWnTEnSiNgJU11hQCFJ0hQoHOUhSRoyMxPzySYPSdLQ2AFTXWRAIUnSFOj6sNGRjU9JckKSa5Jc3FP2xiRXJrmg3Q7uOffaJBuTfCvJU0dVL0maZmYn5tuWdqTHoNskjHLA64nAQUuUv62q9mu3swGS7AscBvxae897kqwbYd0kSdIQjSygqKovANf3efmhwGlVdWtVfRfYCBwwqrpJ0jQyOzHfFhYHM0PRv6OSXNQ2idynLdsduKLnmk1t2Z0kOTLJhiQbbuPWUddVksbCYELQDBsdxjYJ4w4o3gs8CNgPuAp461ofUFXrq2r/qtp/W7Ybdv0kSZqM6nYfirGO8qiqqxf2k/wN8In28Epgz55L92jLJGkmLcwzscDshLpurBmKJLv1HD4TWBgBchZwWJLtkuwN7AN8dZx1kyRpkhaGjZqhWCTJqcCBwM5JNgFvAA5Msh/Ne/se8GKAqrokyenApcDtwEuravOo6iZJk2R/CS2ny/NQjCygqKrDlyg+foXr/xz481HVR5IkjY4zZUrSGJmd0HIWho12lQGFJI2JwYRWUwYUkiRpUC5fLklaUu/wULMTmmUGFJIkTYEqR3lIkpZgnwmtlX0oJEm/sNDMYTCheWJAIUnSVHDYqCTNPTtfahi63OQxieXLJUnSjDFDIUlbyayEhmlhcbCuMqCQJGkaVDN0tKsMKCRpDcxKaJScKVOSZphBhLQ6AwpJkqZA0e1RHgYUkrQCZ7vU+HR7HgqHjUqSpIGZoZCkJTh9tibBUR6SNCMMJDRJ9qGQJEkDqTKgkKTOMzMhDcaAQpKkKdHlUR4GFJLmmpkJTZMud8p02KgkSRqYGQpJc8eptDWt7JQpSR1g84amWZFOBxQ2eUiSpIGZoZA0N8xMaNp1uE9mfwFFkvsBB9D8rudV1Q9HWitJkubNrE9sleS/Aa8H/gkI8NdJjqmqE0ZdOUkalB0w1SkdTlH0k6F4NfCIqroOIMl9gS8DBhSSJAnoL6C4Dri55/jmtkySpto5P7jArIQ6ZVxNHkkOAt4BrAPeX1XHLnHNs4A30uRNLqyqZ6/0zH4Cio3AV5J8rH3oocBFSf4IoKqOW8svIUnjYDChLhrHTJlJ1gHvBn4b2AScl+Ssqrq055p9gNcCj62qHyf5d6s9t59ho98BPsovW3Y+BnwXuFe7LVfhE5Jck+TinrK3JPlmkouSnJlkx7Z8ryT/luSCdntfH/WSJElrdwCwsaour6qfA6fRJAt6/QHw7qr6MUBVXbPaQ1fNUFTVmwCS3L2qbllDhU8E3gWc3FP2aeC1VXV7kr+kiX5e0577TlX5dULSwMxOqIuKoTZ57JxkQ8/x+qpa3+7vDlzRc24T8KhF9z8YIMmXaJpF3lhV/7jSB/YzyuMxwPHAPYH7J3k48OKq+h8r3VdVX0iy16KyT/Ucngv87mqfL0nSXChgeAHFtVW1/wD33wXYBzgQ2AP4QpL/WFU3LHdDP00ebweeStsRs6ouBH5rgEou+H3gkz3Heyf5epLPJ3n8EJ4vaQ6ZnZBWdSWwZ8/xHm1Zr03AWVV1W1V9F/g/NAHGsvqa2KqqrkjuEDVt7ue+5ST5E+B24INt0VXA/avquiS/Dnw0ya9V1U1L3HskcCTA9tx9kGpImiGu06FZMKbly88D9kmyN00gcRiweATHR4HDgQ8k2ZmmCeTylR7aT4biiiS/CVSSbZO8CrhsrbVfkOQFwNOB51Q1r66qbl2Y56KqzqfpCPrgpe6vqvVVtX9V7b8t221tNSRJmj41pG2lj6i6HTgKOIfm7/npVXVJkmOSHNJedg5wXZJLgc8Cr174O72cfjIUL6EZq7o7TSTzKWDF/hPLace9/jHwhN4Onkl2Aa6vqs1JHkiTVlkxEpIkZ8HUbBnfaqNVdTZw9qKy1/fsF/BH7daXfgKKh1TVc3oLkjwW+NJKNyU5laYzx85JNgFvoBnVsR3w6bYJ5dyqeglNn4xjktwGbAFeUlXX9/tLSJKkyeonoPhr4JF9lN1BVR2+RPHxy1x7BnBGH3WRNOfMSmimzeJaHu1w0d8EdlmYFbO1A82YVEmSNCwzvNroXWnmnrgLd5wR8yacP0LSmDmKQ5puywYUVfV54PNJTqyq7wMk2Qa451LDOSVpFAwkNFc63OTRz7DRv0iyQ5J7ABcDlyZ59YjrJUnSHMqQtvHrp1PmvlV1U5Ln0MxseTRwPvCWkdZM0lwzM6G5NOMZim2TbAs8g3YaTjr9K0uSpGHrJ0Pxv4DvARfSLA7yAJqOmZI0dGYmNNc6/HW9n+XL3wm8s6fo+0meOLoqSZpHBhKae8NdbXTsVm3ySLJrkuOTfLI93hc4YuQ1kyRJndFPk8eJwAeAP2mP/w/wIZaZ9VKS+uWsl9IdjWm10ZHop1PmzlV1Os0aGwurlA20fLkkSVrCGFYbHZV+MhQ/TXJf2iomeTRw40hrJWnmnfODC8xKSDOkn4Dij4CzgAcl+RKwC069LUnS8HW4U2Y/ozy+luQJwENopt/6VjsXhSStiX0mpJWlw30oVg0okqwDDgb2aq//nSRU1XEjrpukGWAQIfVpgv0fhqGfJo+PAz8DvkHbMVOSJKlXPwHFHlX1sJHXRNJMcaIqaa3S6T4U/Qwb/WSS3xl5TSRJmnczPmz0XODMJNsAt9F0zKyq2mGkNZPUSWYmpPnUT0BxHPAY4BtVXZ7DS9IoGUhIQ9Dhv7L9BBRXABcbTEiSNGId/kvbT0BxOfC5dnGwWxcKHTYqCcxMSGr0E1B8t93u2m6SJGnYOr58eT8zZb5pHBWR1A29E1WBmQlpmGZypswkb6+qVyT5OEu06lTVISOtmSRJ82YWAwrglPbnX42jIpKmkxkJSf1YNqCoqvPb3f2q6h2955K8HPj8KCsmabLsbClpLfqZKfOIJcpeMOR6SJI091LD2SZhpT4UhwPPBvZOclbPqXsB14+6YpImw8yEpK2xUh+KLwNXATsDb+0pvxm4aJSVkiRpLs3isNGq+j7wfZpptyXNgXN+cIGZCWlSJriw1zD0M7GVpBnWO4rDYEKasA4HFP10ypQkSVrRihmKJOuAk6vqOWOqj6QRc14JaXp1eabMFTMUVbUZeECSrVrDI8kJSa5JcnFP2U5JPp3k2+3P+7TlSfLOJBuTXJTkkVvzmZIkdVYNaZuAflcb/VI7dPSnC4V9rjZ6IvAu4OSesqOBz1TVsUmObo9fAzwN2KfdHgW8t/0paUjsdClpVPrpQ/Ed4BPttffq2VZVVV/gznNWHAqc1O6fBDyjp/zkapwL7Jhkt34+R5KkmTDLGYqF1UaT3L2qbhnCZ+5aVVe1+z8Edm33dweu6LluU1t2FZIGZnZCmm6TnOVyGFbNUCR5TJJLgW+2xw9P8p5hfHhVrTmWSnJkkg1JNtzGrcOohjTTzvnBBQYTkkaunyaPtwNPBa4DqKoLgd8a4DOvXmjKaH9e05ZfCezZc90ebdkdVNX6qtq/qvbflu0GqIYkSVOmMpxtAvqa2KqqrkjuUMHNA3zmWTQLjh3b/vxYT/lRSU6j6Yx5Y0/TiKQ1ck0OqYM63OTRT0BxRZLfBCrJtsDLgcv6eXiSU4EDgZ2TbALeQBNInJ7kRTRTez+rvfxs4GBgI3AL8MI1/B6SJHVel/tQ9BNQvAR4B00HySuBTwEv7efhVXX4MqeevMS11e9zJa3MPhOSxq2fUR7XAs6UKXWAzRxSx81ihiLJX7PCr1ZVfziSGkmSNI86Pmx0pQzFhvbnY4F9gQ+1x78HXDrKSknqn6uFSpoGywYUVXUSQJL/Djyuqm5vj98HfHE81ZMkaY7MaIZiwX2AHfjlFNr3bMskTZD9JaQZNOMBxbHA15N8FgjNpFZvHGWlJEmaR7PahwKAqvpAkk/STDZVwGuq6ocjr5mkO7G/hKRp1ddMmcABwOPb/QI+PprqSFqKzRuSpt2qAUWSY4HfAD7YFv1hksdU1etGWjNJkubNLDd50EyHvV9VbQFIchLwdcCAQhoxMxOSuqLfJo8d+eUoj3uPqC6SJM2vGZ7YasFfcOdRHkePtFbSnOrtdAlmJqS5M8sBRVWdmuRzNP0owFEe0ki4oJekLttmtQuSPBO4parOqqqzgJ8lecboqyZJ0pypIW0T0E+Txxuq6syFg6q6IckbgI+OrlrS/LDjpSRo+hTMeh+KpbIY/XbmlCRJ/ZrxgGJDkuOAd7fHLwXOH12VpPlhvwlJs2LVPhTAy4Cf0yxf/iHgVpqgQpIkDUs7bHQY2yT0M8rjpzhMVBo6sxOS7mRMwUCSg4B3AOuA91fVsctc91+ADwO/UVUbVnpmP1NvPxh4FbBX7/VV9aS+ay7pDgwmJE1KknU03Rh+G9gEnJfkrKq6dNF19wJeDnyln+f204fi74H3Ae8HNq+l0pIkaQ3Gk6E4ANhYVZcDJDkNOBS4dNF1fwb8JfDqfh7aT0Bxe1W9dw0VlbQElx6XtJoh9n/YOUlvE8X6qlrf7u8OXNFzbhPwqDvUI3kksGdV/UOSoQUUH0/yP4AzaTpkAlBV1y9/iyRJmqBrq2r/rbkxyTbAccAL1nJfPwHFEe3P3gilgAeu5YOkeWafCUl9GU+Tx5XAnj3He7RlC+4FPBT4XBKA+wFnJTlkpY6Z/Yzy2HurqivNOZs4JK3J+KbNPg/YJ8neNIHEYcCzf1GNqhuBnReO2/W8XrXaKI9l56FI8sc9+7+36Nz/v8bKS5KkVYxjHoqquh04CjgHuAw4vaouSXJMkkO2tu4rZSgOA97c7r+WZrTHgoOA123th0qzziYOSdOsqs4Gzl5U9vplrj2wn2euFFBkmf2ljiVJ0qBmdC2PWmZ/qWNJLbMTkrbWrK42+vAkN9FkI+7W7tMebz/ymkmSpM5YNqCoqnXjrIjUdQujOsxOSNpqM5qhkNQHAwlJQzG+YaMjYUAhSdIUCN0e8WBAIW0FJ62SpDsae0CR5CHAh3qKHgi8HtgR+APgR23569pxspIkzQebPPpXVd8C9oNfrMl+Jc3CYy8E3lZVfzXuOkn9sr+EpFHq8rDRZafeHpMnA9+pqu9PuB6SJGkAk+5DcRhwas/xUUmeD2wAXllVP158Q5IjgSMBtufuY6mkZGZC0lh0OEMxsYAiyV2BQ2jWCQF4L/BnNK/zz4C3Ar+/+L6qWg+sB9ghO3X41Wua9Xa6BAMJSWPS4b9qk2zyeBrwtaq6GqCqrq6qzVW1Bfgb4IAJ1k2SJK3BJJs8DqenuSPJblV1VXv4TODiidRKc8+1OCRNRB9Lj0+ziQQUSe4B/Dbw4p7iNyfZjybh871F5yRJmn0GFGtTVT8F7ruo7HmTqIvUy+yEpEkyQyF1nKM4JGkwBhSSJE0LMxRSN5mZkDRNutzkMemZMiVJ0gwwQ6G5ZGZC0tQpbPKQJElDYEAhdYdDQyVNo2AfCqkzDCYkaTTMUEiSNC06nKEwoNDM61051OyEpGmW6m5EYZOHJEkamBkKzSyHhkrqFIeNStPHzpeSushRHpIkaa6ZodBM6O14CTZzSOqoDmcoDCgkSZoSXW7yMKBQp9nxUtJM6XBAYR8KSZI0MDMU6hwnqpI0k8omD2lsHA4qaaZ1OKCwyUOSJA3MDIWmnk0ckuZB15cvN6CQJGladHhxMAMKTT2zEpI0/exDoam2eAZMSZplqeFsk2CGQpKkaeBqo9Jw2QlT0rzKlknXYOvZ5CFJkgZmhkJTw3U5JM09mzwkSdKgnIdCGpBTaktStxlQaKJs5pCkVuHEVpIkaXA2eUh9WjxRlZkJSephQLF2Sb4H3AxsBm6vqv2T7AR8CNgL+B7wrKr68aTqKEmS+jPpDMUTq+ranuOjgc9U1bFJjm6PXzOZqmmY7CshSStztdHhOhQ4sN0/CfgcBhSd5YyXkrQGVZ3ulDnJmTIL+FSS85Mc2ZbtWlVXtfs/BHZdfFOSI5NsSLLhNm4dV10lSdIKJpmheFxVXZnk3wGfTvLN3pNVVcmdkz9VtR5YD7BDdupuKDejzEpI0tbrcpPHxDIUVXVl+/Ma4EzgAODqJLsBtD+vmVT9JEkauxrSNgETyVAkuQewTVXd3O7/DnAMcBZwBHBs+/Njk6if+ucwUEkSTK7JY1fgzCQLdfi7qvrHJOcBpyd5EfB94FkTqp8kSWPX5SaPiQQUVXU58PAlyq8Dnjz+GmmtHAYqSUNWwJbuRhTTNmxUHeBCXpI0It2NJyY6bFSSJM0IMxRaE7MTkjQ69qGQJEmD6/BMmQYU6oudMCVJK7EPhVa10MxhMCFJo5Uazrbq5yQHJflWko3tYpyLz/9RkkuTXJTkM0kesNozDSgkSZoGw5olc5WAIsk64N3A04B9gcOT7Lvosq9d4Hk2AAAMZklEQVQD+1fVw4APA29erfo2eWhZNnNI0vg0y5ePpQ/FAcDGdk4okpxGs9r3pQsXVNVne64/F3juag81QyFJ0uzZeWFl7nY7sufc7sAVPceb2rLlvAj45GofaIZCd+BqoZI0QVuG9qRrq2r/QR+S5LnA/sATVrvWgEKSpCkxpiaPK4E9e473aMvuWJfkKcCfAE+oqltXe6gBhcxKSNJ8OQ/YJ8neNIHEYcCzey9I8gjgfwEHVdU1/TzUgGJOGURI0pTpY4TGUD6m6vYkRwHnAOuAE6rqkiTHABuq6izgLcA9gb9vVwb/16o6ZKXnGlBIkjQVamwzZVbV2cDZi8pe37P/lLU+04BiDrkehyRp2AwoJEmaEi4Opk5woipJmnIuDqZpZedLSeqIggxvHoqxc6ZMSZI0MDMUM8rmDUnqIJs8JEnSwLobTxhQzBozE5KkSTCgkCRpSoxpLY+RMKCYAY7kkKQZYUChceoNIMAgQpI0eQYUkiRNgwI6PA+FAUWH2OFSkmZXKPtQSJKkITCg0CiZmZAkTTsDiilmICFJc8YMhSRJGoidMjUK5/zgAjMTkqTOMKCQJGlKOMpDQ+GMl5I05zocUGwz7g9MsmeSzya5NMklSV7elr8xyZVJLmi3g8ddN0mSJqeagGIY2wRMIkNxO/DKqvpaknsB5yf5dHvubVX1VxOo08TZZ0KS1GVjDyiq6irgqnb/5iSXAbuPux7TwqGhkiSgGeVhk8fWSbIX8AjgK23RUUkuSnJCkvssc8+RSTYk2XAbt46pppIkjcGWIW0TMLGAIsk9gTOAV1TVTcB7gQcB+9FkMN661H1Vtb6q9q+q/bdlu7HVdxQWmjnMTkiSum4iozySbEsTTHywqj4CUFVX95z/G+ATk6ibJEmT4rDRNUgS4Hjgsqo6rqd8t7Z/BcAzgYvHXbdxsd+EJGlJBhRr8ljgecA3kixMvPA64PAk+9F0S/ke8OIJ1E2SJG2FSYzy+GcgS5w6e9x1mQSHh0qSllTAFjMUWoXNHJKklU1uUqphMKCQJGlaGFBoOWYmJEnzwIBCkqRpYYZCi5mZkCStiZ0ytcDlxyVJ88qAQpKkqVBQE1qIYwgMKAbQm5EAsxKSpAHZh0KSJA3EPhTzyRkvJUn6JQMKSZKmhU0e88PhoJKkkTGgmH0GEpIkLc+AQpKkqeDiYDPNzIQkaSwK2NLdeSi2mXQFJElS95mhWIFDQyVJY2WTx2yxmUOSNBEGFJIkaTDlTJmzwsyEJElbx4BCkqRpUFCuNtpdvSuGmpmQJE1Uh5s8HDYqSZIGNtcZCoeFSpKmiqM8usMmDknSVKpypkxJkjTfZj5D0ZuRALMSkqQpZpOHJEkaVHW4yWNmAwonqZIkdYvLl08dR29IkjReMxlQSJLUOUWnJ7aaqYDCZg5JUqd1eOpth41KkqSBzUyGwn4TkqQuK6A63OQxdRmKJAcl+VaSjUmOnnR9JEkai6qmyWMY2wRMVUCRZB3wbuBpwL7A4Un2Xe0+sxOSpFlQW2oo22pW+/KeZLskH2rPfyXJXqs9c6oCCuAAYGNVXV5VPwdOAw5d7uIHP+wWgwlJktagzy/vLwJ+XFW/CrwN+MvVnjttAcXuwBU9x5vasl9IcmSSDUk2/Oi6zWOtnCRJIzWeJo9+vrwfCpzU7n8YeHKSrPTQznXKrKr1wHqAJD9at9vGn8LGaydcrVm1M+C7HQ3f7Wj5fkdnXt/tA0b9ATfz43P+d3145yE9bvskG3qO17d/P2HpL++PWnT/L66pqtuT3AjclxX+209bQHElsGfP8R5t2ZKqapckG6pq/5HXbA75bkfHdztavt/R8d2OTlUdNOk6DGLamjzOA/ZJsneSuwKHAWdNuE6SJM2Sfr68/+KaJHcB7g1ct9JDpyqgqKrbgaOAc4DLgNOr6pLJ1kqSpJnSz5f3s4Aj2v3fBf6pauWVy6atyYOqOhs4ew23rF/9Em0l3+3o+G5Hy/c7Or7bjmv7RCx8eV8HnFBVlyQ5BthQVWcBxwOnJNkIXE8TdKwoqwQckiRJq5qqJg9JktRNBhSSJGlgnQ0oXPNjcElOSHJNkot7ynZK8ukk325/3qctT5J3tu/7oiSPnFzNp1+SPZN8NsmlSS5J8vK23Pc7oCTbJ/lqkgvbd/umtnzvdorgje2UwXdty9c8hfC8S7IuydeTfKI99t1qVZ0MKLZ2zQ/dyYnA4nHPRwOfqap9gM+0x9C8633a7UjgvWOqY1fdDryyqvYFHg28tP3fqO93cLcCT6qqhwP7AQcleTTN1MBva6cK/jHN1MGwFVMIi5fTjLRb4LvVqjoZULDGNT+0tKr6Ak3v3V69062eBDyjp/zkapwL7Jhkt/HUtHuq6qqq+lq7fzPNP8674/sdWPuOftIebttuBTyJZopguPO7XdMUwvMsyR7A/wO8vz0Ovlv1oasBxaprfmir7VpVV7X7PwR2bfd951upTQM/AvgKvt+haFPyFwDXAJ8GvgPc0M5lA3d8f3eYQhhYmEJYS3s78MfAwoIQ98V3qz50NaDQGLSTmDiueABJ7gmcAbyiqm7qPef73XpVtbmq9qOZ4e8A4N9PuEozIcnTgWuq6vxJ10Xd09WAYk1rfmhNrl5Itbc/r2nLfedrlGRbmmDig1X1kbbY9ztEVXUD8FngMTTNRAuT9fW+vzVPITzHHgsckuR7NE3JTwLege9WfehqQOGaH6PTO93qEcDHesqf345GeDRwY0/qXou07cjHA5dV1XE9p3y/A0qyS5Id2/27Ab9N00flszRTBMOd3+2aphCeV1X12qrao6r2ovl39Z+q6jn4btWHzs6UmeRgmra+hWlD/3zCVeqcJKcCB9IsR3w18Abgo8DpwP2B7wPPqqrr2z+Q76IZFXIL8MKq2rDUcwVJHgd8EfgGv2yLfh1NPwrf7wCSPIymI+A6mi9Fp1fVMUkeSPOteifg68Bzq+rWJNsDp9D0Y7keOKyqLp9M7bsjyYHAq6rq6b5b9aOzAYUkSZoeXW3ykCRJU8SAQpIkDcyAQpIkDcyAQpIkDcyAQpIkDcyAQhpQks1JLmhXvrwwySuTjOz/W0n26l0hdiuf8bpFx18erFYrftZeSZ49qudLmg4GFNLg/q2q9quqX6OZZOlpNHN6TIWeGQ573SGgqKrfHGEV9gIMKKQZZ0AhDVFVXUOz/PhR7ayX65K8Jcl5SS5K8uKFa5O8Jsk32qzGsW3ZfknOba89M8l92vJfb6+7EHhpzzOWfH6SA5N8MclZwKW9dWw/625tVuWDbdlPeu77fJKPJbk8ybFJnpPkq21dH9Ret0uSM9rPPS/JY9vyJ7TPvSDJ15PcCzgWeHxb9v+uUucvJPmHJN9K8r5RZnokDVlVubm5DbABP1mi7AaalUSPBP60LdsO2ADsTZPF+DJw9/bcTu3Pi4AntPvHAG/vKf+tdv8twMXt/nLPPxD4KbB3P3VeOG7vuwHYrX3elcCb2nMv76nP3wGPa/fvTzPFOMDHgce2+/cE7tI+8xM9n7VSnX8GPJBmFsxPA7876f++bm5u/W1LpUIlDc/vAA9LsrAOwr2BfYCnAB+oqlsAqpl++97AjlX1+fbak4C/b9et2LGqvtCWn0ITkKz0/J8DX62q725Fnc+rdh2RJN8BPtWWfwN4Yrv/FGDfZsZwAHZoV1b9EnBcm/n4SFVt6rlmtXeyUOfL288+FXgc8OGt+B0kjZkBhTRk7boHm2lWEg3wsqo6Z9E1Tx3Wxy3z/ANpMhRb49ae/S09x1v45b8Z2wCPrqqfLbr32CT/ABwMfGmZ33OlOi9eC8C1AaSOsH1SGqIkuwDvA95VVQWcA/z3NEuZk+TBSe5Bk85/YZK7t+U7VdWNwI+TPL593POAz1ezRPcN7YJjAM/p+cjlnr+a2xbu2UqfAl62cJBkv/bng6rqG1X1lzSrAv974GbgXn3W+YA0qwhvA/xX4J8HqKOkMTJDIQ3ubkkuALYFbqdpklhYsvz9NKMcvtauKPoj4BlV9Y/tH+ENSX4OnE0z8uII4H1toHE58ML2OS8ETkhS/LIJYtnn91Hn9cBFSb5WzfLUa/WHwLuTXETz78gXgJcAr0jyRJpsxiXAJ9v9zW2H0hOBd6xQ5/NoVl39VZols8/cirpJmgBXG5U0FdKzXPak6yJp7WzykCRJAzNDIUmSBmaGQpIkDcyAQpIkDcyAQpIkDcyAQpIkDcyAQpIkDez/AhZKgZ9P2GyzAAAAAElFTkSuQmCC\n",
"text/plain": [
""
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"fig = plt.figure(figsize=(8, 6))\n",
"ax = fig.add_subplot(111)\n",
"ax.set_title('Alignment steps')\n",
"im = ax.imshow(\n",
" r_male['alignment'],\n",
" aspect='auto',\n",
" origin='lower',\n",
" interpolation='none')\n",
"fig.colorbar(im, ax=ax)\n",
"xlabel = 'Decoder timestep'\n",
"plt.xlabel(xlabel)\n",
"plt.ylabel('Encoder timestep')\n",
"plt.tight_layout()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Load Vocoder model\n",
"\n",
"There are 2 ways to synthesize melspectrogram output from TTS models,\n",
"\n",
"1. If you are going to use individual speaker vocoder, make sure the speakers are the same If use female tacotron2, need to use female MelGAN also. Use `mel-output` from TTS model. Read more at https://malaya-speech.readthedocs.io/en/latest/load-vocoder.html\n",
"2. If you are going to use universal MelGAN, use `universal-output` from TTS model. Read more at https://malaya-speech.readthedocs.io/en/latest/load-universal-melgan.html\n",
"\n",
"We prefer individual speaker vocoder, size only 17MB and faster than universal vocoder."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"universal_melgan = malaya_speech.vocoder.melgan(model = 'universal-1024')"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
" \n",
" \n",
" "
],
"text/plain": [
""
]
},
"execution_count": 20,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"y_ = universal_melgan(r_male['universal-output'])\n",
"ipd.Audio(y_, rate = 22050)"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
" \n",
" \n",
" "
],
"text/plain": [
""
]
},
"execution_count": 21,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"r_female = female.predict(string1)\n",
"y_ = universal_melgan(r_female['universal-output'])\n",
"ipd.Audio(y_, rate = 22050)"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
" \n",
" \n",
" "
],
"text/plain": [
""
]
},
"execution_count": 22,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"r_haqkiem = haqkiem.predict(string1)\n",
"y_ = universal_melgan(r_haqkiem['universal-output'])\n",
"ipd.Audio(y_, rate = 22050)"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
" \n",
" \n",
" "
],
"text/plain": [
""
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"r_yasmin = yasmin.predict(string1)\n",
"y_ = universal_melgan(r_yasmin['universal-output'])\n",
"ipd.Audio(y_, rate = 22050)"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
" \n",
" \n",
" "
],
"text/plain": [
""
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"r_osman = osman.predict(string1)\n",
"y_ = universal_melgan(r_osman['universal-output'])\n",
"ipd.Audio(y_, rate = 22050)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Playing around with temperature and length ratio"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
" \n",
" \n",
" "
],
"text/plain": [
""
]
},
"execution_count": 23,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"r_male = male.predict(string1, length_ratio = 1.2)\n",
"y_ = universal_melgan(r_male['universal-output'])\n",
"ipd.Audio(y_, rate = 22050)"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
" \n",
" \n",
" "
],
"text/plain": [
""
]
},
"execution_count": 24,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"r_male = male.predict(string1, length_ratio = 1.2, temperature = 1.0)\n",
"y_ = universal_melgan(r_male['universal-output'])\n",
"ipd.Audio(y_, rate = 22050)"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
" \n",
" \n",
" "
],
"text/plain": [
""
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"r_yasmin = yasmin.predict(string1, length_ratio = 1.5, temperature = 0.6666)\n",
"y_ = universal_melgan(r_yasmin['universal-output'])\n",
"ipd.Audio(y_, rate = 22050)"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
" \n",
" \n",
" "
],
"text/plain": [
""
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"r_osman = osman.predict(string1, length_ratio = 1.5, temperature = 0.6666)\n",
"y_ = universal_melgan(r_osman['universal-output'])\n",
"ipd.Audio(y_, rate = 22050)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.7"
},
"varInspector": {
"cols": {
"lenName": 16,
"lenType": 16,
"lenVar": 40
},
"kernels_config": {
"python": {
"delete_cmd_postfix": "",
"delete_cmd_prefix": "del ",
"library": "var_list.py",
"varRefreshCmd": "print(var_dic_list())"
},
"r": {
"delete_cmd_postfix": ") ",
"delete_cmd_prefix": "rm(",
"library": "var_list.r",
"varRefreshCmd": "cat(var_dic_list()) "
}
},
"types_to_exclude": [
"module",
"function",
"builtin_function_or_method",
"instance",
"_Feature"
],
"window_display": false
}
},
"nbformat": 4,
"nbformat_minor": 4
}