cybench.runs package
Submodules
cybench.runs.results_plots module
- cybench.runs.results_plots.box_plots_metrics(data, crop, countries, metric, metric_label, subplots_per_row=4)
- cybench.runs.results_plots.box_plots_residuals(data, crop, countries, residual_cols, residual_labels, ymin, ymax, subplots_per_row=4)
- cybench.runs.results_plots.plot_bars(df, metric, metric_label, title_label, file_name)
- cybench.runs.results_plots.plot_graph(df, x_col, hue_col, x_label, metric, metric_label, title, file_name, rotation=45)
- cybench.runs.results_plots.plot_metrics(df: DataFrame, metric: str | None = None)
- cybench.runs.results_plots.plot_yearly_metrics(data, crop, country, metric, metric_label)
- cybench.runs.results_plots.plot_yearly_residuals(data, crop, country, residual_cols, residual_labels)
- cybench.runs.results_plots.write_to_markdown(df, metrics, metrics_labels)
cybench.runs.run_benchmark module
- cybench.runs.run_benchmark.compute_metrics(run_name: str, model_names: list) DataFrame
- cybench.runs.run_benchmark.get_prediction_residuals(run_name: str, model_names: dict) DataFrame
- cybench.runs.run_benchmark.load_results(run_name: str) DataFrame
- cybench.runs.run_benchmark.run_benchmark(run_name: str, model_name: str | None = None, model_constructor: callable | None = None, model_init_kwargs: dict | None = None, model_fit_kwargs: dict | None = None, baseline_models: list | None = None, dataset_name: str = 'maize_NL') dict
Run the AgML benchmark. :param run_name: The name of the run. Will be used to store log files and model results :type run_name: str :param model_name: The name of the model. Will be used to store log files and model results :type model_name: str :param model_constructor: The constructor of the model. Will be used to construct the model :type model_constructor: Callable :param model_init_kwargs: The kwargs used when constructing the model. :type model_init_kwargs: dict :param model_fit_kwargs: The kwargs used to fit the model. :type model_fit_kwargs: dict :param baseline_models: A list of names of baseline models to run next to the provided model.
If unspecified, a default list of baseline models will be used.
- Parameters:
dataset_name (str) – The name of the dataset to load
- Returns:
a dictionary containing the results of the benchmark
- cybench.runs.run_benchmark.run_benchmark_on_all_data()
cybench.runs.validate_model module
- cybench.runs.validate_model.validate_single_model(run_name: str, model_name: str, model_constructor: callable, model_init_kwargs: dict | None = None, model_fit_kwargs: dict | None = None, baseline_models: list | None = None, dataset_name: str = 'test_maize_us', test_years_to_leave_out: list | None = None) dict
Run a single model on a single outer fold and return validation results. Test is is left out completely and not used for training or validation. Not used for benchmarking. Use run_benchmark instead. Hyperparameters should be optimized in each outer fold in the benchmark. This function should only be used for exploration of initial hyperparameter settings.
- Parameters:
run_name (str) – The name of the run. Will be used to store log files and model results
model_name (str) – The name of the model. Will be used to store log files and model results
model_constructor (Callable) – The constructor of the model. Will be used to construct the model
model_init_kwargs (dict) – The kwargs used when constructing the model.
model_fit_kwargs (dict) – The kwargs used to fit the model.
baseline_models (list) – A list of names of baseline models to run next to the provided model. If unspecified, a default list of baseline models will be used.
dataset_name (str) – The name of the dataset to load
- Returns:
a dictionary containing the results of the benchmark