tsbenchmark package
tsbenchmark.api module
- tsbenchmark.api.get_local_task(data_path, dataset_id='512754', random_state=2022, max_trials=3, reward_metric='smape') TSTask [source]
Get a TsTask from local for develop a new player and test.
TsTask is a unit task, which help Player get the data and metadata. It will get a TsTaskConfig locally and construct it to TSTask. Call TSTask.ready() method init start time and load data.
- Parameters
data_path – str, default=’~/tmp/data_cache’. The path locally to cache data. TSLoader will download data and cache it in data_path.
dataset_id – str, default=’512754’. The unique id for a dataset task. You can get it from tests/dataset_desc.csv.
random_state – int, consts.GLOBAL_RANDOM_STATE. Determines random number for automl framework.
max_trials – int, default=3. Maximum number of tests for automl framework, optional.
reward_metric – str, default=’smape’. The optimize direction for model selection. Hypernets search reward metric name or callable. Possible values: ‘accuracy’, ‘auc’, ‘mse’, ‘mae’,’rmse’, ‘mape’, ‘smape’, and ‘msle’.
Notes
You can get attributes description from TSTask.
In the report it support ‘smape’, ‘mape’, ‘mae’ and ‘rmse’.
See also
TSTask: Player will get the data and metadata from the TSTask then run algorithm for compete.
Returns: TSTask, The TsTask for player get the data and metadata.
- tsbenchmark.api.get_task() TSTask [source]
Get a TsTask from benchmark server.
TsTask is a unit task, which help Player get the data and metadata. It will get TsTaskConfig from benchmark server and construct it to TSTask. Call TSTask.ready() method init start time and load data.
See also
TSTask : Player will get the data and metadata from the TSTask then run algorithm for compete.
Notes
You can get attributes description from TSTask.
In the report it support ‘smape’, ‘mape’, ‘mae’ and ‘rmse’.
Returns: TSTask, The TsTask for player get the data and metadata.
- tsbenchmark.api.report_task(report_data: Dict, bm_task_id=None, api_server_uri=None)[source]
Report metrics or running information to api server.
- Parameters
report_data – Dict. The report data generate by send_report_data.
bm_task_id – str, optional, BenchmarkTask id, if is None will get from current job
api_server_uri – str, optional, tsbenchmark api server uri, if is None will get from environment or use default value
- tsbenchmark.api.send_report_data(task: TSTask, y_pred: DataFrame, key_params='', best_params='', sub_result=False)[source]
Send report data.
This interface used for send report data to benchmark server. 1. Prepare the data which can be call be tsb.api.report_task. 2. Call method report_task, send the report data to the Benchmark Server.
- Parameters
y_pred – pandas.DataFrame, The predicted values by the players. It should be a pandas.DataFrame, and it must have the headers name, which you can get from task.series_name.
key_params – str, default=’’ The params which user want to save to the report datas.
best_params – str, default=’’ The best model’s params, for automl, there are many models will be trained. If user want to save the best params, user may assign the best_params.
Notes
When develop a new play locally, this method will help user validate the predicted and params.
tsbenchmark.tasks module
- class tsbenchmark.tasks.TSTask(task_config, **kwargs)[source]
Bases:
object
Player will get the data and metadata from the TSTask then run algorithm for compete.
- Parameters
dataset_id – str, not None. The unique identification id.
date_name – str, not None. The name of the date column.
task – str, not None. The type of forecast. In time series task, it could be ‘univariate-forecast’ or ‘multivariate-forecast’.
horizon – int, not None. Number of periods of data to forecast ahead.
shape – str, not None. The dataset shape from the train dataframe. The result from pandas.DataFrame.shape().
series_name – str or arr. The names of the series columns. For ‘univariate-forecast’ task, it should not be None.For ‘multivariate-forecast’ task, it should be None. In the task from tsbenchmark.api.get_task() or tsbenchmark.api.get_local_task or called function TSTask.ready, series_name should not be None.
covariables_name – str or arr, may be None. The names of the covariables columns. It should be get after called function TSTask.ready(), or from task from tsbenchmark.api.get_task() or tsbenchmark.api.get_local_task.
dtformat – str, not None. The format of the date column.
random_state – int, consts.GLOBAL_RANDOM_STATE Determines random number for automl framework.
max_trials – int, default=3. Maximum number of tests for automl framework, optional.
reward_metric – str, default=’smape’. The optimize direction for model selection. Hypernets search reward metric name or callable. Possible values: ‘accuracy’, ‘auc’, ‘mse’, ‘mae’,’rmse’, ‘mape’, ‘smape’, and ‘msle’.
Notes
In the report it support ‘smape’, ‘mape’, ‘mae’ and ‘rmse’.
- get_test()[source]
Get a pandas.DadaFrame test data which will be used in the Player.
- Returns
The data for test.
- Return type
pandas.DataFrame