腾讯云计算加速套件TACO KitTACO Infer 接口

优化接口

TACO Infer 为您提供了一套简单易用的模型优化接口。对于 CPU 计算设备，优化接口为 optimize_cpu：

def optimize_cpu(    input_model: Union[str, torch.nn.Module],    output_model_dir: str,    test_data: Optional[Dict[str, np.array]],    optimize_config: OptimizeConfig = OptimizeConfig(),    model_config: ModelConfig = ModelConfig(),) -> Dict:

对于GPU计算设备，优化接口为optimize_gpu:

def optimize_gpu(    input_model: Union[str, torch.nn.Module],    output_model_dir: str,    test_data: Optional[Dict[str, np.array]] = None,    optimize_config: OptimizeConfig = OptimizeConfig(),    model_config: ModelConfig = ModelConfig(),) -> Dict:

两种设备的优化接口输入参数完全一致。

输入参数说明

import numpy as np
def gen_test_data(batch_size = 1):    INPUT_NAME = "input:0"    image_size = 299    input_data = np.random.rand(batch_size, image_size, image_size, 3)    return {INPUT_NAME: input_data}

print(model_cfg.help())
How-to-assign-a-"model_config":tensorflow.inputs:type: <class 'list'>, default value: NoneInput tensor names as a list. Items may use node name as prefix, such as "Placeholder:0", or saved model signature.
tensorflow.outputs:type: <class 'list'>, default value: NoneOutput tensor names as a list.
tensorflow.saved_model.signature:type: <class 'str'>, default value: NoneTell TACO Inf which signature to use if more than 1 signature.
Example of updating a config:model_config.parse({"tensorflow.inputs": ['Placeholder:0']})



输出参数说明

优化模型后会产生一个 JSON 格式的优化报告，该报告包含了优化模型的硬件，软件以及一些总结信息。输出参数如下所示：

{    "hardware": {        "cpu": "AMD EPYC 7K62 48-Core Processor, family '23', model '49'",        "target device": "AMD EPYC 7K62 48-Core Processor, family '23', model '49'",        "reference": "https://en.wikichip.org/wiki/intel/cpuid"    },    "software": {        "framework": "tensorflow",        "framework version": "1.15.0"    },    "summary": {        "working_directory": "/root/taco_test/fast_transformer_encoder",        "input_model": "./model/fast-transformer-encoder.pb",        "output_model_dir": "./optimized_model",        "optimization time": "3min 46s 398ms",        "model format": "tensorflow frozen pb",        "status": "satisfactory",        "baseline latency": "49ms 517us",        "accelerated latency": "27ms 12us",        "speedup": "1.83"    }}

输出字段说明如下：hardware：硬件环境信息，包括设备类型、规格等。software：软件环境信息，包括框架以及框架版本。summary：模型优化的综合性信息，包括当前工作目录、输入模型路径、输出模型目录、优化时间、模型格式、运行状态、模型优化效果等。

Config 使用方式

OptimizeConfig 和 ModelConfig 使用方法是一致的。均支持通过 parse 接口设置相关属性的值，通过 “.” 获取属性值或者赋值。Config 使用示例如下所示：

from taco import ModelConfig

cfg = ModelConfig()
# assign by parse()cfg.parse({"tensorflow.inputs": ['Placeholder:0']})
print(cfg.tensorflow.inputs)
# assign by .cfg.tensorflow.saved_model.signature = "predictions"
print(cfg.tensorflow.saved_model.signature)



官网1折活动，限时活动，即将结束，速速收藏
同尘科技为腾讯云授权服务中心。
购买腾讯云产品享受折上折，更有现金返利。同意关联立享优惠