Malaya-Speech Cache

This tutorial is available as an IPython notebook at malaya-speech/example/caching.

This module only useful if use default Malaya-Speech repository, for huggingface, read more at https://huggingface.co/docs/datasets/v1.12.0/cache.html

Default Cache location

You actually can know where is your Malaya default caching folder. Caching folder will save any models, vocabs, and rules downloaded for specific modules.

import malaya_speech

malaya_speech.__home__

'/Users/huseinzolkepli/Malaya-Speech'

Change default Cache location

To change default cache location, you need to set MALAYA_SPEECH_CACHE OS environment before import Malaya-Speech,

export MALAYA_SPEECH_CACHE=/Users/huseinzolkepli/Documents/malaya-speech-cache

Or you can set in bashenv to make it permanent if you want.

import os

os.environ['MALAYA_CACHE'] = '/Users/huseinzolkepli/Documents/malaya-speech-cache'

Cache subdirectories

Malaya-Speech put models in subdirectories, you can print it by simply,

malaya_speech.utils.print_cache()

Malaya-Speech/
├── gender/
│   ├── deep-speaker/
│   │   ├── model.pb
│   │   └── version
│   ├── vggvox-v2/
│   │   ├── model.pb
│   │   └── version
│   └── vggvox-v2-quantized/
│       ├── model.pb
│       └── version
├── language-model/
│   ├── bahasa/
│   │   ├── ctc-bahasa.json
│   │   ├── model.trie.klm
│   │   └── version
│   └── bahasa-combined/
│       ├── ctc-bahasa.json
│       ├── model.trie.klm
│       └── version
├── multispeaker-separation-wav/
│   ├── fastsep-4/
│   │   ├── model.pb
│   │   └── version
│   └── fastsep-4-quantized/
│       ├── model.pb
│       └── version
├── speaker-vector/
│   └── vggvox-v2/
│       ├── model.pb
│       └── version
├── speech-to-text/
│   ├── conformer-mixed/
│   │   ├── model.pb
│   │   ├── transducer-mixed.subword.subwords
│   │   └── version
│   ├── conformer-mixed-quantized/
│   │   ├── model.pb
│   │   ├── transducer-mixed.subword.subwords
│   │   └── version
│   ├── small-conformer-mixed/
│   │   ├── model.pb
│   │   ├── transducer-mixed.subword.subwords
│   │   └── version
│   └── small-conformer-mixed-quantized/
│       ├── model.pb
│       ├── transducer-mixed.subword.subwords
│       └── version
├── speech-to-text-ctc/
│   ├── wav2vec2-conformer/
│   │   ├── ctc-bahasa.json
│   │   ├── model.pb
│   │   └── version
│   └── wav2vec2-conformer-quantized/
│       ├── ctc-bahasa.json
│       ├── model.pb
│       └── version
├── super-resolution/
│   ├── srgan-128/
│   │   ├── model.pb
│   │   └── version
│   └── srgan-256/
│       ├── model.pb
│       └── version
├── tts/
│   ├── fastspeech2-female-singlish/
│   │   ├── model.pb
│   │   ├── quantized/
│   │   │   ├── model.pb
│   │   │   └── version
│   │   └── version
│   ├── stats/
│   │   └── female-singlish.npy
│   └── tacotron2-female-singlish/
│       ├── model.pb
│       ├── quantized/
│       │   ├── model.pb
│       │   └── version
│       └── version
├── vad/
│   └── vggvox-v2/
│       ├── model.pb
│       └── version
├── version
└── vocoder-melgan/
    ├── female-singlish/
    │   ├── model.pb
    │   └── version
    ├── universal-1024/
    │   ├── model.pb
    │   └── version
    └── universal-1024-quantized/
        ├── model.pb
        └── version

Deleting specific model

you can specifically choose which model you want to delete.

malaya_speech.utils.delete_cache('language-model/bahasa-combined')

True

What happen if a directory does not exist?

malaya_speech.utils.delete_cache('language-model/bahasa-combined2')

---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
<ipython-input-9-328c5a06876b> in <module>
----> 1 malaya_speech.utils.delete_cache('language-model/bahasa-combined2')

~/Documents/tf-1.15/env/lib/python3.7/site-packages/malaya_boilerplate-0.0.3-py3.7.egg/malaya_boilerplate/utils.py in delete_cache(location)
    188     if not os.path.exists(location):
    189         raise Exception(
--> 190             f'folder not exist, please check path from `{__package__}.utils.print_cache()`'
    191         )
    192     if not os.path.isdir(location):

Exception: folder not exist, please check path from `malaya-speech.utils.print_cache()`

Purge cache

You can simply delete all models, totally purge it. By simply,

malaya_speech.utils.delete_all_cache

<function malaya_boilerplate.utils.delete_all_cache()>

I am not gonna to run it, because I prefer to keep it for now?