{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# GPU Environment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">\n",
    "\n",
    "This tutorial is available as an IPython notebook at [malaya-speech/example/gpu-environment](https://github.com/huseinzol05/malaya-speech/tree/master/example/gpu-environment).\n",
    "    \n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 4.7 s, sys: 1.5 s, total: 6.19 s\n",
      "Wall time: 14.1 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "import malaya_speech\n",
    "import logging\n",
    "logging.basicConfig(level = logging.INFO)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### List available GPU"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "malaya_speech.utils.available_gpu()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Limit GPU memory"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By default `Malaya-Speech` will not set max cap for GPU memory, to put a cap, override `gpu_limit` parameter in any load model API. `gpu_limit` should 0 < `gpu_limit` < 1. If `gpu_limit` = 0.3, it means the model will not use more than 30% of GPU memory.\n",
    "\n",
    "`malaya_speech.vad.deep_model(gpu_limit = 0.3)`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Not all operations supported by GPU\n",
    "\n",
    "Yes, some models might faster in CPU due to head cost transitioning from CPU to GPU too frequently."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### N Models to N gpus"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To allocate a model to another GPU, set `device` to different GPU, eg, `GPU:1`, default is `GPU:0`.\n",
    "\n",
    "```python\n",
    "malaya_speech.emotion.deep_model(gpu_limit = 0.5, device = 'GPU:0')\n",
    "malaya_speech.language_detection.deep_model(gpu_limit = 0.5, device = 'GPU:1')\n",
    "malaya_speech.noise_reduction.deep_model(gpu_limit = 0.5, device = 'GPU:2')\n",
    "malaya_speech.vad.deep_model(gpu_limit = 0.5, device = 'GPU:3')\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### GPU Rules"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. `Malaya-Speech` will not consumed all available GPU memory, but slowly grow based on batch size. This growth only towards positive (use more GPU memory) dynamically, but will not reduce GPU memory if feed small batch size.\n",
    "2. Use `malaya_speech.utils.clear_session` to clear session from unused models but this will not free GPU memory."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}