Devices#

This tutorial is available as an IPython notebook at malaya-speech/example/devices.

List available devices supported to run Malaya-Speech model#

[1]:

import malaya_speech
import logging
logging.basicConfig(level = logging.INFO)

[2]:

malaya_speech.utils.available_device()

[2]:

[('CPU:0', '0.268 GB'),
 ('XLA_CPU:0', '17.18 GB'),
 ('XLA_GPU:0', '17.18 GB'),
 ('XLA_GPU:1', '17.18 GB'),
 ('XLA_GPU:2', '17.18 GB'),
 ('XLA_GPU:3', '17.18 GB'),
 ('GPU:0', '30.486 GB'),
 ('GPU:1', '30.489 GB'),
 ('GPU:2', '30.489 GB'),
 ('GPU:3', '30.489 GB')]

Use specific device for specific model#

To do that, pass device parameter to any load model function in Malaya, default is CPU:0.

malaya_speech.gender.deep_model(model = 'vggvox-v2', device = 'CPU:0')

Or if you want to use XLA,

malaya_speech.gender.deep_model(model = 'vggvox-v2', device = 'XLA_CPU:0')

By default, device will automatically set to a gpu with the most empty memory.

[3]:

gender = malaya_speech.gender.deep_model(model = 'vggvox-v2', device = 'CPU:0')

INFO:root:running gender/vggvox-v2 using device /device:GPU:1

Disable auto GPU#

Let say you do not want to use auto allocate to gpu, simply set auto_gpu to False, or set,

export CUDA_VISIBLE_DEVICES=''

[4]:

gender_cpu = malaya_speech.gender.deep_model(model = 'vggvox-v2', device = 'CPU:0', auto_gpu = False)

INFO:root:running gender/vggvox-v2 using device /device:CPU:0

[5]:

gender_xla_cpu = malaya_speech.gender.deep_model(model = 'vggvox-v2', device = 'XLA_CPU:0', auto_gpu = False)

INFO:root:running gender/vggvox-v2 using device /device:XLA_CPU:0

[7]:

y, sr = malaya_speech.load('speech/video/The-Singaporean-White-Boy.wav')
y = y[:int(sr * 0.5)]
len(y), sr

[7]:

(8000, 16000)

[8]:

%%time

gender_cpu.predict_proba([y])

CPU times: user 923 ms, sys: 429 ms, total: 1.35 s
Wall time: 1.22 s

[8]:

array([[0.21194158, 0.2119417 , 0.5761167 ]], dtype=float32)

[9]:

%%time

gender_xla_cpu.predict_proba([y])

CPU times: user 4.61 s, sys: 503 ms, total: 5.11 s
Wall time: 4.74 s

[9]:

array([[0.21194158, 0.2119417 , 0.5761167 ]], dtype=float32)

Again, not all Tensorflow operation support XLA.