tsbenchmark package

tsbenchmark.api module

tsbenchmark.api.get_local_task(data_path, dataset_id='512754', random_state=2022, max_trials=3, reward_metric='smape') → TSTask

Get a TsTask from local for develop a new player and test.

TsTask is a unit task, which help Player get the data and metadata. It will get a TsTaskConfig locally and construct it to TSTask. Call TSTask.ready() method init start time and load data.

参数

  • data_path – str, default=’~/tmp/data_cache’. The path locally to cache data. TSLoader will download data and cache it in data_path.

  • dataset_id – str, default=’512754’. The unique id for a dataset task. You can get it from tests/dataset_desc.csv.

  • random_state – int, consts.GLOBAL_RANDOM_STATE. Determines random number for automl framework.

  • max_trials – int, default=3. Maximum number of tests for automl framework, optional.

  • reward_metric – str, default=’smape’. The optimize direction for model selection. Hypernets search reward metric name or callable. Possible values: ‘accuracy’, ‘auc’, ‘mse’, ‘mae’,’rmse’, ‘mape’, ‘smape’, and ‘msle’.

备注

  1. You can get attributes description from TSTask.

  2. In the report it support ‘smape’, ‘mape’, ‘mae’ and ‘rmse’.

参见

TSTask: Player will get the data and metadata from the TSTask then run algorithm for compete.

Returns: TSTask, The TsTask for player get the data and metadata.

tsbenchmark.api.get_task() → TSTask

Get a TsTask from benchmark server.

TsTask is a unit task, which help Player get the data and metadata. It will get TsTaskConfig from benchmark server and construct it to TSTask. Call TSTask.ready() method init start time and load data.

参见

TSTask : Player will get the data and metadata from the TSTask then run algorithm for compete.

备注

  1. You can get attributes description from TSTask.

  2. In the report it support ‘smape’, ‘mape’, ‘mae’ and ‘rmse’.

Returns: TSTask, The TsTask for player get the data and metadata.

tsbenchmark.api.report_task(report_data: Dict, bm_task_id=None, api_server_uri=None)

Report metrics or running information to api server.

参数

  • report_data – Dict. The report data generate by send_report_data.

  • bm_task_id – str, optional, BenchmarkTask id, if is None will get from current job

  • api_server_uri – str, optional, tsbenchmark api server uri, if is None will get from environment or use default value

tsbenchmark.api.send_report_data(task: TSTask, y_pred: DataFrame, key_params='', best_params='', sub_result=False)

Send report data.

This interface used for send report data to benchmark server. 1. Prepare the data which can be call be tsb.api.report_task. 2. Call method report_task, send the report data to the Benchmark Server.

参数

  • y_pred – pandas.DataFrame, The predicted values by the players. It should be a pandas.DataFrame, and it must have the headers name, which you can get from task.series_name.

  • key_params – str, default=’’ The params which user want to save to the report datas.

  • best_params – str, default=’’ The best model’s params, for automl, there are many models will be trained. If user want to save the best params, user may assign the best_params.

备注

When develop a new play locally, this method will help user validate the predicted and params.

tsbenchmark.tasks module

class tsbenchmark.tasks.TSTask(task_config, **kwargs)

基类：object

Player will get the data and metadata from the TSTask then run algorithm for compete.

参数

  • dataset_id – str, not None. The unique identification id.

  • date_name – str, not None. The name of the date column.

  • task – str, not None. The type of forecast. In time series task, it could be ‘univariate-forecast’ or ‘multivariate-forecast’.

  • horizon – int, not None. Number of periods of data to forecast ahead.

  • shape – str, not None. The dataset shape from the train dataframe. The result from pandas.DataFrame.shape().

  • series_name – str or arr. The names of the series columns. For ‘univariate-forecast’ task, it should not be None.For ‘multivariate-forecast’ task, it should be None. In the task from tsbenchmark.api.get_task() or tsbenchmark.api.get_local_task or called function TSTask.ready, series_name should not be None.

  • covariables_name – str or arr, may be None. The names of the covariables columns. It should be get after called function TSTask.ready(), or from task from tsbenchmark.api.get_task() or tsbenchmark.api.get_local_task.

  • dtformat – str, not None. The format of the date column.

  • random_state – int, consts.GLOBAL_RANDOM_STATE Determines random number for automl framework.

  • max_trials – int, default=3. Maximum number of tests for automl framework, optional.

  • reward_metric – str, default=’smape’. The optimize direction for model selection. Hypernets search reward metric name or callable. Possible values: ‘accuracy’, ‘auc’, ‘mse’, ‘mae’,’rmse’, ‘mape’, ‘smape’, and ‘msle’.

备注

In the report it support ‘smape’, ‘mape’, ‘mae’ and ‘rmse’.

get_data()

Get data contain train_data and test_data which will be used in the Player.

get_test()

Get a pandas.DadaFrame test data which will be used in the Player.

返回

The data for test.

返回类型

pandas.DataFrame

get_train()

Get a pandas.DadaFrame train data which will be used in the Player.

返回

The data for train.

返回类型

pandas.DataFrame

ready()

Init data download if the data have not been download yet.

to_dict()