API Reference
emhass.command_line module
- emhass.command_line.dayahead_forecast_optim(input_data_dict: dict, logger: Logger, save_data_to_file: bool | None = False, debug: bool | None = False) DataFrame
Perform a call to the day-ahead optimization routine.
- Parameters:
input_data_dict (dict) – A dictionnary with multiple data used by the action functions
logger (logging object) – The passed logger object
save_data_to_file (bool, optional) – Save optimization results to CSV file
debug (bool, optional) – A debug option useful for unittests
- Returns:
The output data of the optimization
- Return type:
pd.DataFrame
- emhass.command_line.forecast_model_fit(input_data_dict: dict, logger: Logger, debug: bool | None = False) Tuple[DataFrame, DataFrame, MLForecaster]
Perform a forecast model fit from training data retrieved from Home Assistant.
- Parameters:
input_data_dict (dict) – A dictionnary with multiple data used by the action functions
logger (logging.Logger) – The passed logger object
debug (Optional[bool], optional) – True to debug, useful for unit testing, defaults to False
- Returns:
The DataFrame containing the forecast data results without and with backtest and the mlforecaster object
- Return type:
Tuple[pd.DataFrame, pd.DataFrame, mlforecaster]
- emhass.command_line.forecast_model_predict(input_data_dict: dict, logger: Logger, use_last_window: bool | None = True, debug: bool | None = False, mlf: MLForecaster | None = None) DataFrame
Perform a forecast model predict using a previously trained skforecast model.
- Parameters:
input_data_dict (dict) – A dictionnary with multiple data used by the action functions
logger (logging.Logger) – The passed logger object
use_last_window (Optional[bool], optional) – True if the ‘last_window’ option should be used for the custom machine learning forecast model. The ‘last_window=True’ means that the data that will be used to generate the new forecast will be freshly retrieved from Home Assistant. This data is needed because the forecast model is an auto-regressive model with lags. If ‘False’ then the data using during the model train is used. Defaults to True
debug (Optional[bool], optional) – True to debug, useful for unit testing, defaults to False
mlf (Optional[mlforecaster], optional) – The ‘mlforecaster’ object previously trained. This is mainly used for debug and unit testing. In production the actual model will be read from a saved pickle file. Defaults to None
- Returns:
The DataFrame containing the forecast prediction data
- Return type:
pd.DataFrame
- emhass.command_line.forecast_model_tune(input_data_dict: dict, logger: Logger, debug: bool | None = False, mlf: MLForecaster | None = None) Tuple[DataFrame, MLForecaster]
Tune a forecast model hyperparameters using bayesian optimization.
- Parameters:
input_data_dict (dict) – A dictionnary with multiple data used by the action functions
logger (logging.Logger) – The passed logger object
debug (Optional[bool], optional) – True to debug, useful for unit testing, defaults to False
mlf (Optional[mlforecaster], optional) – The ‘mlforecaster’ object previously trained. This is mainly used for debug and unit testing. In production the actual model will be read from a saved pickle file. Defaults to None
- Returns:
The DataFrame containing the forecast data results using the optimized model
- Return type:
pd.DataFrame
- emhass.command_line.main()
Define the main command line entry function.
This function may take several arguments as inputs. You can type emhass –help to see the list of options:
action: Set the desired action, options are: perfect-optim, dayahead-optim, naive-mpc-optim, publish-data, forecast-model-fit, forecast-model-predict, forecast-model-tune
config: Define path to the config.yaml file
costfun: Define the type of cost function, options are: profit, cost, self-consumption
log2file: Define if we should log to a file or not
params: Configuration parameters passed from data/options.json if using the add-on
runtimeparams: Pass runtime optimization parameters as dictionnary
debug: Use True for testing purposes
- emhass.command_line.naive_mpc_optim(input_data_dict: dict, logger: Logger, save_data_to_file: bool | None = False, debug: bool | None = False) DataFrame
Perform a call to the naive Model Predictive Controller optimization routine.
- Parameters:
input_data_dict (dict) – A dictionnary with multiple data used by the action functions
logger (logging object) – The passed logger object
save_data_to_file (bool, optional) – Save optimization results to CSV file
debug (bool, optional) – A debug option useful for unittests
- Returns:
The output data of the optimization
- Return type:
pd.DataFrame
- emhass.command_line.perfect_forecast_optim(input_data_dict: dict, logger: Logger, save_data_to_file: bool | None = True, debug: bool | None = False) DataFrame
Perform a call to the perfect forecast optimization routine.
- Parameters:
input_data_dict (dict) – A dictionnary with multiple data used by the action functions
logger (logging object) – The passed logger object
save_data_to_file (bool, optional) – Save optimization results to CSV file
debug (bool, optional) – A debug option useful for unittests
- Returns:
The output data of the optimization
- Return type:
pd.DataFrame
- emhass.command_line.publish_data(input_data_dict: dict, logger: Logger, save_data_to_file: bool | None = False, opt_res_latest: DataFrame | None = None) DataFrame
Publish the data obtained from the optimization results.
- Parameters:
input_data_dict (dict) – A dictionnary with multiple data used by the action functions
logger (logging object) – The passed logger object
save_data_to_file (bool, optional) – If True we will read data from optimization results in dayahead CSV file
- Returns:
The output data of the optimization readed from a CSV file in the data folder
- Return type:
pd.DataFrame
- emhass.command_line.regressor_model_fit(input_data_dict: dict, logger: Logger, debug: bool | None = False) None
Perform a forecast model fit from training data retrieved from Home Assistant.
- Parameters:
input_data_dict (dict) – A dictionnary with multiple data used by the action functions
logger (logging.Logger) – The passed logger object
debug (Optional[bool], optional) – True to debug, useful for unit testing, defaults to False
- emhass.command_line.regressor_model_predict(input_data_dict: dict, logger: Logger, debug: bool | None = False, mlr: MLRegressor | None = None) None
Perform a prediction from csv file.
- Parameters:
input_data_dict (dict) – A dictionnary with multiple data used by the action functions
logger (logging.Logger) – The passed logger object
debug (Optional[bool], optional) – True to debug, useful for unit testing, defaults to False
- emhass.command_line.set_input_data_dict(emhass_conf: dict, costfun: str, params: str, runtimeparams: str, set_type: str, logger: Logger, get_data_from_file: bool | None = False) dict
Set up some of the data needed for the different actions.
- Parameters:
emhass_conf (dict) – Dictionary containing the needed emhass paths
costfun (str) – The type of cost function to use for optimization problem
params (str) – Configuration parameters passed from data/options.json
runtimeparams (str) – Runtime optimization parameters passed as a dictionary
set_type (str) – Set the type of setup based on following type of optimization
logger (logging object) – The passed logger object
get_data_from_file (bool, optional) – Use data from saved CSV file (useful for debug)
- Returns:
A dictionnary with multiple data used by the action functions
- Return type:
dict
emhass.forecast module
- class emhass.forecast.Forecast(retrieve_hass_conf: dict, optim_conf: dict, plant_conf: dict, params: str, emhass_conf: dict, logger: Logger, opt_time_delta: int | None = 24, get_data_from_file: bool | None = False)
Bases:
object
Generate weather, load and costs forecasts needed as inputs to the optimization.
In EMHASS we have basically 4 forecasts to deal with:
PV power production forecast (internally based on the weather forecast and the characteristics of your PV plant). This is given in Watts.
Load power forecast: how much power your house will demand on the next 24h. This is given in Watts.
PV production selling price forecast: at what price are you selling your excess PV production on the next 24h. This is given in EUR/kWh.
Load cost forecast: the price of the energy from the grid on the next 24h. This is given in EUR/kWh.
There are methods that are generalized to the 4 forecast needed. For all there forecasts it is possible to pass the data either as a passed list of values or by reading from a CSV file. With these methods it is then possible to use data from external forecast providers.
Then there are the methods that are specific to each type of forecast and that proposed forecast treated and generated internally by this EMHASS forecast class. For the weather forecast a first method (scrapper) uses a scrapping to the ClearOutside webpage which proposes detailed forecasts based on Lat/Lon locations. This method seems stable but as with any scrape method it will fail if any changes are made to the webpage API. Another method (solcast) is using the SolCast PV production forecast service. A final method (solar.forecast) is using another external service: Solar.Forecast, for which just the nominal PV peak installed power should be provided. Search the forecast section on the documentation for examples on how to implement these different methods.
The get_power_from_weather method is proposed here to convert from irradiance data to electrical power. The PVLib module is used to model the PV plant.
The specific methods for the load forecast are a first method (naive) that uses a naive approach, also called persistance. It simply assumes that the forecast for a future period will be equal to the observed values in a past period. The past period is controlled using parameter delta_forecast. A second method (mlforecaster) uses an internal custom forecasting model using machine learning. There is a section in the documentation explaining how to use this method.
Note
This custom machine learning model is introduced from v0.4.0. EMHASS proposed this new mlforecaster class with fit, predict and tune methods. Only the predict method is used here to generate new forecasts, but it is necessary to previously fit a forecaster model and it is a good idea to optimize the model hyperparameters using the tune method. See the dedicated section in the documentation for more help.
For the PV production selling price and Load cost forecasts the privileged method is a direct read from a user provided list of values. The list should be passed as a runtime parameter during the curl to the EMHASS API.
I reading from a CSV file, it should contain no header and the timestamped data should have the following format:
2021-04-29 00:00:00+00:00,287.07
2021-04-29 00:30:00+00:00,274.27
2021-04-29 01:00:00+00:00,243.38
…
The data columns in these files will correspond to the data in the units expected for each forecasting method.
- cloud_cover_to_irradiance(cloud_cover: Series, offset: int | None = 35) DataFrame
Estimates irradiance from cloud cover in the following steps.
Determine clear sky GHI using Ineichen model and climatological turbidity.
Estimate cloudy sky GHI using a function of cloud_cover
Estimate cloudy sky DNI using the DISC model.
Calculate DHI from DNI and GHI.
(This function was copied and modified from PVLib)
- Parameters:
cloud_cover (pd.Series) – Cloud cover in %.
offset (Optional[int], optional) – Determines the minimum GHI., defaults to 35
- Returns:
Estimated GHI, DNI, and DHI.
- Return type:
pd.DataFrame
- get_forecast_days_csv(timedelta_days: int | None = 1) date_range
Get the date range vector of forecast dates that will be used when loading a CSV file.
- Returns:
The forecast dates vector
- Return type:
pd.date_range
- get_forecast_out_from_csv_or_list(df_final: DataFrame, forecast_dates_csv: date_range, csv_path: str, data_list: list | None = None, list_and_perfect: bool | None = False) DataFrame
Get the forecast data as a DataFrame from a CSV file.
The data contained in the CSV file should be a 24h forecast with the same frequency as the main ‘freq’ parameter in the configuration file. The timestamp will not be used and a new DateTimeIndex is generated to fit the timestamp index of the input data in ‘df_final’.
- Parameters:
df_final (pd.DataFrame) – The DataFrame containing the input data.
forecast_dates_csv (pd.date_range) – The forecast dates vector
csv_path (str) – The path to the CSV file
- Returns:
The data from the CSV file
- Return type:
pd.DataFrame
- get_load_cost_forecast(df_final: DataFrame, method: str | None = 'hp_hc_periods', csv_path: str | None = 'data_load_cost_forecast.csv', list_and_perfect: bool | None = False) DataFrame
Get the unit cost for the load consumption based on multiple tariff periods. This is the cost of the energy from the utility in a vector sampled at the fixed freq value.
- Parameters:
df_final (pd.DataFrame) – The DataFrame containing the input data.
method (str, optional) – The method to be used to generate load cost forecast, the options are ‘hp_hc_periods’ for peak and non-peak hours contractsand ‘csv’ to load a CSV file, defaults to ‘hp_hc_periods’
csv_path (str, optional) – The path to the CSV file used when method = ‘csv’, defaults to “data_load_cost_forecast.csv”
- Returns:
The input DataFrame with one additionnal column appended containing the load cost for each time observation.
- Return type:
pd.DataFrame
- get_load_forecast(days_min_load_forecast: int | None = 3, method: str | None = 'naive', csv_path: str | None = 'data_load_forecast.csv', set_mix_forecast: bool | None = False, df_now: ~pandas.core.frame.DataFrame | None = Empty DataFrame Columns: [] Index: [], use_last_window: bool | None = True, mlf: ~emhass.machine_learning_forecaster.MLForecaster | None = None, debug: bool | None = False) Series
Get and generate the load forecast data.
- Parameters:
days_min_load_forecast (int, optional) – The number of last days to retrieve that will be used to generate a naive forecast, defaults to 3
method (str, optional) – The method to be used to generate load forecast, the options are ‘naive’ for a persistance model, ‘mlforecaster’ for using a custom previously fitted machine learning model, ‘csv’ to read the forecast from a CSV file and ‘list’ to use data directly passed at runtime as a list of values. Defaults to ‘naive’.
csv_path (str, optional) – The path to the CSV file used when method = ‘csv’, defaults to “/data/data_load_forecast.csv”
set_mix_forecast (Bool, optional) – Use a mixed forcast strategy to integra now/current values.
df_now (pd.DataFrame, optional) – The DataFrame containing the now/current data.
use_last_window (Bool, optional) – True if the ‘last_window’ option should be used for the custom machine learning forecast model. The ‘last_window=True’ means that the data that will be used to generate the new forecast will be freshly retrieved from Home Assistant. This data is needed because the forecast model is an auto-regressive model with lags. If ‘False’ then the data using during the model train is used.
mlf (mlforecaster, optional) – The ‘mlforecaster’ object previously trained. This is mainly used for debug and unit testing. In production the actual model will be read from a saved pickle file.
debug (Bool, optional) – The DataFrame containing the now/current data.
- Returns:
The DataFrame containing the electrical load power in Watts
- Return type:
pd.DataFrame
- static get_mix_forecast(df_now: DataFrame, df_forecast: DataFrame, alpha: float, beta: float, col: str) DataFrame
A simple correction method for forecasted data using the current real values of a variable.
- Parameters:
df_now (pd.DataFrame) – The DataFrame containing the current/real values
df_forecast (pd.DataFrame) – The DataFrame containing the forecast data
alpha (float) – A weight for the forecast data side
beta (float) – A weight for the current/real values sied
col (str) – The column variable name
- Returns:
The output DataFrame with the corrected values
- Return type:
pd.DataFrame
- get_power_from_weather(df_weather: ~pandas.core.frame.DataFrame, set_mix_forecast: bool | None = False, df_now: ~pandas.core.frame.DataFrame | None = Empty DataFrame Columns: [] Index: []) Series
Convert wheater forecast data into electrical power.
- Parameters:
df_weather (pd.DataFrame) – The DataFrame containing the weather forecasted data. This DF should be generated by the ‘get_weather_forecast’ method or at least contain the same columns names filled with proper data.
set_mix_forecast (Bool, optional) – Use a mixed forcast strategy to integra now/current values.
df_now (pd.DataFrame) – The DataFrame containing the now/current data.
- Returns:
The DataFrame containing the electrical power in Watts
- Return type:
pd.DataFrame
- get_prod_price_forecast(df_final: DataFrame, method: str | None = 'constant', csv_path: str | None = 'data_prod_price_forecast.csv', list_and_perfect: bool | None = False) DataFrame
Get the unit power production price for the energy injected to the grid.This is the price of the energy injected to the utility in a vector sampled at the fixed freq value.
- Parameters:
df_input_data (pd.DataFrame) – The DataFrame containing all the input data retrieved from hass
method (str, optional) – The method to be used to generate the production price forecast, the options are ‘constant’ for a fixed constant value and ‘csv’to load a CSV file, defaults to ‘constant’
csv_path (str, optional) – The path to the CSV file used when method = ‘csv’, defaults to “/data/data_load_cost_forecast.csv”
- Returns:
The input DataFrame with one additionnal column appended containing the power production price for each time observation.
- Return type:
pd.DataFrame
- get_weather_forecast(method: str | None = 'scrapper', csv_path: str | None = 'data_weather_forecast.csv') DataFrame
Get and generate weather forecast data.
- Parameters:
method (str, optional) – The desired method, options are ‘scrapper’, ‘csv’, ‘list’, ‘solcast’ and ‘solar.forecast’. Defaults to ‘scrapper’.
- Returns:
The DataFrame containing the forecasted data
- Return type:
pd.DataFrame
emhass.machine_learning_forecaster module
- class emhass.machine_learning_forecaster.MLForecaster(data: DataFrame, model_type: str, var_model: str, sklearn_model: str, num_lags: int, emhass_conf: dict, logger: Logger)
Bases:
object
A forecaster class using machine learning models with auto-regressive approach and featuresbased on timestamp information (hour, day, week, etc).
This class uses the skforecast module and the machine learning models are from scikit-learn.
It exposes three main methods:
fit: to train a model with the passed data.
predict: to obtain a forecast from a pre-trained model.
tune: to optimize the models hyperparameters using bayesian optimization.
- static add_date_features(data: DataFrame) DataFrame
Add date features from the input DataFrame timestamp
- Parameters:
data (pd.DataFrame) – The input DataFrame
- Returns:
The DataFrame with the added features
- Return type:
pd.DataFrame
- fit(split_date_delta: str | None = '48h', perform_backtest: bool | None = False) Tuple[DataFrame, DataFrame]
The fit method to train the ML model.
- Parameters:
split_date_delta (Optional[str], optional) – The delta from now to split_date_delta that will be used as the test period to evaluate the model, defaults to ‘48h’
perform_backtest (Optional[bool], optional) – If True then a back testing routine is performed to evaluate the performance of the model on the complete train set, defaults to False
- Returns:
The DataFrame containing the forecast data results without and with backtest
- Return type:
Tuple[pd.DataFrame, pd.DataFrame]
- static generate_exog(data_last_window, periods, var_name)
Generate the exogenous data for future timestamps.
- static neg_r2_score(y_true, y_pred)
The negative of the r2 score.
- predict(data_last_window: DataFrame | None = None) Series
The predict method to generate forecasts from a previously fitted ML model.
- Parameters:
data_last_window (Optional[pd.DataFrame], optional) – The data that will be used to generate the new forecast, this will be freshly retrieved from Home Assistant. This data is needed because the forecast model is an auto-regressive model with lags. If not passed then the data used during the model train is used, defaults to None
- Returns:
A pandas series containing the generated forecasts.
- Return type:
pd.Series
- tune(debug: bool | None = False) DataFrame
Tuning a previously fitted model using bayesian optimization.
- Parameters:
debug (Optional[bool], optional) – Set to True for testing and faster optimizations, defaults to False
- Returns:
The DataFrame with the forecasts using the optimized model.
- Return type:
pd.DataFrame
emhass.optimization module
- class emhass.optimization.Optimization(retrieve_hass_conf: dict, optim_conf: dict, plant_conf: dict, var_load_cost: str, var_prod_price: str, costfun: str, emhass_conf: dict, logger: Logger, opt_time_delta: int | None = 24)
Bases:
object
Optimize the deferrable load and battery energy dispatch problem using the linear programming optimization technique. All equipement equations, including the battery equations are hence transformed in a linear form.
This class methods are:
perform_optimization
perform_perfect_forecast_optim
perform_dayahead_forecast_optim
perform_naive_mpc_optim
- perform_dayahead_forecast_optim(df_input_data: DataFrame, P_PV: Series, P_load: Series) DataFrame
Perform a day-ahead optimization task using real forecast data. This type of optimization is intented to be launched once a day.
- Parameters:
df_input_data (pandas.DataFrame) – A DataFrame containing all the input data used for the optimization, notably the unit load cost for power consumption.
P_PV (pandas.DataFrame) – The forecasted PV power production.
P_load (pandas.DataFrame) – The forecasted Load power consumption. This power should not include the power from the deferrable load that we want to find.
- Returns:
opt_res: A DataFrame containing the optimization results
- Return type:
pandas.DataFrame
- perform_naive_mpc_optim(df_input_data: DataFrame, P_PV: Series, P_load: Series, prediction_horizon: int, soc_init: float | None = None, soc_final: float | None = None, def_total_hours: list | None = None, def_start_timestep: list | None = None, def_end_timestep: list | None = None) DataFrame
Perform a naive approach to a Model Predictive Control (MPC). This implementaion is naive because we are not using the formal formulation of a MPC. Only the sense of a receiding horizon is considered here. This optimization is more suitable for higher optimization frequency, ex: 5min.
- Parameters:
df_input_data (pandas.DataFrame) – A DataFrame containing all the input data used for the optimization, notably the unit load cost for power consumption.
P_PV (pandas.DataFrame) – The forecasted PV power production.
P_load (pandas.DataFrame) – The forecasted Load power consumption. This power should not include the power from the deferrable load that we want to find.
prediction_horizon (int) – The prediction horizon of the MPC controller in number of optimization time steps.
soc_init (float) – The initial battery SOC for the optimization. This parameter is optional, if not given soc_init = soc_final = soc_target from the configuration file.
soc_final – The final battery SOC for the optimization. This parameter is optional, if not given soc_init = soc_final = soc_target from the configuration file.
def_total_hours (list) – The functioning hours for this iteration for each deferrable load. (For continuous deferrable loads: functioning hours at nominal power)
def_start_timestep (list) – The timestep as from which each deferrable load is allowed to operate.
def_end_timestep (list) – The timestep before which each deferrable load should operate.
- Returns:
opt_res: A DataFrame containing the optimization results
- Return type:
pandas.DataFrame
- perform_optimization(data_opt: DataFrame, P_PV: array, P_load: array, unit_load_cost: array, unit_prod_price: array, soc_init: float | None = None, soc_final: float | None = None, def_total_hours: list | None = None, def_start_timestep: list | None = None, def_end_timestep: list | None = None, debug: bool | None = False) DataFrame
Perform the actual optimization using linear programming (LP).
- Parameters:
data_tp (pd.DataFrame) – A DataFrame containing the input data. The results of the optimization will be appended (decision variables, cost function values, etc)
P_PV (numpy.array) – The photovoltaic power values. This can be real historical values or forecasted values.
P_load (np.array) – The load power consumption values
unit_load_cost (np.array) – The cost of power consumption for each unit of time. This is the cost of the energy from the utility in a vector sampled at the fixed freq value
unit_prod_price (np.array) – The price of power injected to the grid each unit of time. This is the price of the energy injected to the utility in a vector sampled at the fixed freq value.
soc_init (float) – The initial battery SOC for the optimization. This parameter is optional, if not given soc_init = soc_final = soc_target from the configuration file.
soc_final – The final battery SOC for the optimization. This parameter is optional, if not given soc_init = soc_final = soc_target from the configuration file.
def_total_hours (list) – The functioning hours for this iteration for each deferrable load. (For continuous deferrable loads: functioning hours at nominal power)
def_start_timestep (list) – The timestep as from which each deferrable load is allowed to operate.
def_end_timestep (list) – The timestep before which each deferrable load should operate.
- Returns:
The input DataFrame with all the different results from the optimization appended
- Return type:
pd.DataFrame
- perform_perfect_forecast_optim(df_input_data: DataFrame, days_list: date_range) DataFrame
Perform an optimization on historical data (perfectly known PV production).
- Parameters:
df_input_data (pandas.DataFrame) – A DataFrame containing all the input data used for the optimization, notably photovoltaics and load consumption powers.
days_list (list) – A list of the days of data that will be retrieved from hass and used for the optimization task. We will retrieve data from now and up to days_to_retrieve days
- Returns:
opt_res: A DataFrame containing the optimization results
- Return type:
pandas.DataFrame
- static validate_def_timewindow(start: int, end: int, min_steps: int, window: int) Tuple[int, int, str]
Helper function to validate (and if necessary: correct) the defined optimization window of a deferrable load.
- Parameters:
start (int) – Start timestep of the optimization window of the deferrable load
end (int) – End timestep of the optimization window of the deferrable load
min_steps (int) – Minimal timesteps during which the load should operate (at nominal power)
window (int) – Total number of timesteps in the optimization window
- Returns:
start_validated: Validated start timestep of the optimization window of the deferrable load
- Return type:
int
- Returns:
end_validated: Validated end timestep of the optimization window of the deferrable load
- Return type:
int
- Returns:
warning: Any warning information to be returned from the validation steps
- Return type:
string
emhass.retrieve_hass module
- class emhass.retrieve_hass.RetrieveHass(hass_url: str, long_lived_token: str, freq: Timedelta, time_zone: timezone, params: str, emhass_conf: dict, logger: Logger, get_data_from_file: bool | None = False)
Bases:
object
Retrieve data from Home Assistant using the restful API.
This class allows the user to retrieve data from a Home Assistant instance using the provided restful API (https://developers.home-assistant.io/docs/api/rest/)
This class methods are:
get_data: to retrieve the actual data from hass
prepare_data: to apply some data treatment in preparation for the optimization task
post_data: Post passed data to hass
- static get_attr_data_dict(data_df: DataFrame, idx: int, entity_id: str, unit_of_measurement: str, friendly_name: str, list_name: str, state: float) dict
- get_data(days_list: date_range, var_list: list, minimal_response: bool | None = False, significant_changes_only: bool | None = False, test_url: str | None = 'empty') None
Retrieve the actual data from hass.
- Parameters:
days_list (pandas.date_range) – A list of days to retrieve. The ISO format should be used and the timezone is UTC. The frequency of the data_range should be freq=’D’
var_list (list) – The list of variables to retrive from hass. These should be the exact name of the sensor in Home Assistant. For example: [‘sensor.home_load’, ‘sensor.home_pv’]
minimal_response (bool, optional) – Retrieve a minimal response using the hass restful API, defaults to False
significant_changes_only (bool, optional) – Retrieve significant changes only using the hass restful API, defaults to False
- Returns:
The DataFrame populated with the retrieved data from hass
- Return type:
pandas.DataFrame
Warning
The minimal_response and significant_changes_only options are experimental
- post_data(data_df: DataFrame, idx: int, entity_id: str, unit_of_measurement: str, friendly_name: str, type_var: str, from_mlforecaster: bool | None = False, publish_prefix: str | None = '') None
Post passed data to hass.
- Parameters:
data_df (pd.DataFrame) – The DataFrame containing the data that will be posted to hass. This should be a one columns DF or a series.
idx (int) – The int index of the location of the data within the passed DataFrame. We will post just one value at a time.
entity_id (str) – The unique entity_id of the sensor in hass.
unit_of_measurement (str) – The units of the sensor.
friendly_name (str) – The friendly name that will be used in the hass frontend.
type_var (str) – A variable to indicate the type of variable: power, SOC, etc.
publish_prefix (str, optional) – A common prefix for all published data entity_id.
- prepare_data(var_load: str, load_negative: bool | None = False, set_zero_min: bool | None = True, var_replace_zero: list | None = None, var_interp: list | None = None) None
Apply some data treatment in preparation for the optimization task.
- Parameters:
var_load (str) – The name of the variable for the household load consumption.
load_negative (bool, optional) – Set to True if the retrived load variable is negative by convention, defaults to False
set_zero_min (bool, optional) – A special treatment for a minimum value saturation to zero. Values below zero are replaced by nans, defaults to True
var_replace_zero (list, optional) – A list of retrived variables that we would want to replace nans with zeros, defaults to None
var_interp (list, optional) – A list of retrived variables that we would want to interpolate nan values using linear interpolation, defaults to None
- Returns:
The DataFrame populated with the retrieved data from hass and after the data treatment
- Return type:
pandas.DataFrame
emhass.utils module
- emhass.utils.build_params(params: dict, params_secrets: dict, options: dict, addon: int, logger: Logger) dict
Build the main params dictionary from the loaded options.json when using the add-on.
- Parameters:
params (dict) – The main params dictionary
params_secrets (dict) – The dictionary containing the secret protected variables
options (dict) – The load dictionary from options.json
addon (int) – A “bool” to select if we are using the add-on
logger (logging.Logger) – The logger object
- Returns:
The builded dictionary
- Return type:
dict
- emhass.utils.get_days_list(days_to_retrieve: int) date_range
Get list of past days from today to days_to_retrieve.
- Parameters:
days_to_retrieve (int) – Total number of days to retrieve from the past
- Returns:
The list of days
- Return type:
pd.date_range
- emhass.utils.get_forecast_dates(freq: int, delta_forecast: int, timedelta_days: int | None = 0) DatetimeIndex
Get the date_range list of the needed future dates using the delta_forecast parameter.
- Parameters:
freq (int) – Optimization time step.
delta_forecast (int) – Number of days to forecast in the future to be used for the optimization.
timedelta_days (Optional[int], optional) – Number of truncated days needed for each optimization iteration, defaults to 0
- Returns:
A list of future forecast dates.
- Return type:
pd.core.indexes.datetimes.DatetimeIndex
- emhass.utils.get_injection_dict(df: DataFrame, plot_size: int | None = 1366) dict
Build a dictionary with graphs and tables for the webui.
- Parameters:
df (pd.DataFrame) – The optimization result DataFrame
plot_size (Optional[int], optional) – Size of the plot figure in pixels, defaults to 1366
- Returns:
A dictionary containing the graphs and tables in html format
- Return type:
dict
- emhass.utils.get_injection_dict_forecast_model_fit(df_fit_pred: DataFrame, mlf: MLForecaster) dict
Build a dictionary with graphs and tables for the webui for special MLF fit case.
- Parameters:
df_fit_pred (pd.DataFrame) – The fit result DataFrame
mlf (MLForecaster) – The MLForecaster object
- Returns:
A dictionary containing the graphs and tables in html format
- Return type:
dict
- emhass.utils.get_injection_dict_forecast_model_tune(df_pred_optim: DataFrame, mlf: MLForecaster) dict
Build a dictionary with graphs and tables for the webui for special MLF tune case.
- Parameters:
df_pred_optim (pd.DataFrame) – The tune result DataFrame
mlf (MLForecaster) – The MLForecaster object
- Returns:
A dictionary containing the graphs and tables in html format
- Return type:
dict
- emhass.utils.get_logger(fun_name: str, emhass_conf: dict, save_to_file: bool | None = True, logging_level: str | None = 'DEBUG') Tuple[Logger, StreamHandler]
Create a simple logger object.
- Parameters:
fun_name (str) – The Python function object name where the logger will be used
emhass_conf (dict) – Dictionary containing the needed emhass paths
save_to_file (bool, optional) – Write log to a file, defaults to True
- Returns:
The logger object and the handler
- Return type:
object
- emhass.utils.get_root(file: str, num_parent: int | None = 3) str
Get the root absolute path of the working directory.
- Parameters:
file – The passed file path with __file__
num_parent (int, optional) – The number of parents levels up to desired root folder
- Returns:
The root path
- Return type:
str
- emhass.utils.get_yaml_parse(emhass_conf: dict, use_secrets: bool | None = True, params: str | None = None) Tuple[dict, dict, dict]
Perform parsing of the config.yaml file.
- Parameters:
emhass_conf (dict) – Dictionary containing the needed emhass paths
use_secrets (bool, optional) – Indicate if we should use a secrets file or not. Set to False for unit tests.
params (str) – Configuration parameters passed from data/options.json
- Returns:
A tuple with the dictionaries containing the parsed data
- Return type:
tuple(dict)
- emhass.utils.set_df_index_freq(df: DataFrame) DataFrame
Set the freq of a DataFrame DateTimeIndex.
- Parameters:
df (pd.DataFrame) – Input DataFrame
- Returns:
Input DataFrame with freq defined
- Return type:
pd.DataFrame
- emhass.utils.treat_runtimeparams(runtimeparams: str, params: str, retrieve_hass_conf: dict, optim_conf: dict, plant_conf: dict, set_type: str, logger: Logger) Tuple[str, dict]
Treat the passed optimization runtime parameters.
- Parameters:
runtimeparams (str) – Json string containing the runtime parameters dict.
params (str) – Configuration parameters passed from data/options.json
retrieve_hass_conf (dict) – Container for data retrieving parameters.
optim_conf (dict) – Container for optimization parameters.
plant_conf (dict) – Container for technical plant parameters.
set_type (str) – The type of action to be performed.
logger (logging.Logger) – The logger object.
- Returns:
Returning the params and optimization parameter container.
- Return type:
Tuple[str, dict]