Automatic Error Handling

Sometimes there are parts of the design space that we want to explore that will cause the EnergyPlus simulation to fail, such as invalid combinations of parameter values.

from besos import eppy_funcs as ef, sampling
from besos.evaluator import EvaluatorEP
from besos.parameters import (
    CategoryParameter,
    FieldSelector,
    Parameter,
    RangeParameter,
)
from besos.problem import EPProblem

In this example, we load the emaple model and try to use an undefined material Invalid Material.

building = ef.get_building()

problem = EPProblem(
    [
        Parameter(
            FieldSelector(
                object_name="Mass NonRes Wall Insulation", field_name="Thickness"
            ),
            RangeParameter(min_val=0.01, max_val=0.99),
        ),
        Parameter(
            FieldSelector(
                class_name="Construction",
                object_name="ext-slab",
                field_name="Outside Layer",
            ),
            CategoryParameter(options=("HW CONCRETE", "Invalid Material")),
        ),
    ]
)

samples = sampling.dist_sampler(sampling.lhs, problem, 5)
samples
RangeParameter [0.01, 0.99] Category['HW CONCRETE', 'Invalid Material']
0 0.262924 HW CONCRETE
1 0.870820 HW CONCRETE
2 0.735874 Invalid Material
3 0.580238 Invalid Material
4 0.073492 Invalid Material

By default, evaluation of a DataFrame of parameters will end when an invalid combination is encountered.

try:
    EvaluatorEP(
        problem,
        building,
        error_mode="Failfast",
        out_dir="outputdir",
        err_dir="outputdir",
    ).df_apply(samples)
except Exception as e:
    print("caught", e)
HBox(children=(FloatProgress(value=0.0, description='Executing', max=5.0, style=ProgressStyle(description_widt…
Program Version,EnergyPlus, Version 9.0.1-bb7ca4f0da, YMD=2021.02.10 00:32,

   ********* Beginning Zone Sizing Calculations

   ** Severe  ** Did not find matching material for Construction EXT-SLAB, missing material = INVALID MATERIAL

   **  Fatal  ** GetSurfaceData: Errors discovered, program terminates.

   ...Summary of Errors that led to program termination:

   ..... Reference severe error count=1

   ..... Last severe error=Did not find matching material for Construction EXT-SLAB, missing material = INVALID MATERIAL

   ********* Warning:  Node connection errors not checked - most system input has not been read (see previous warning).

   ********* Fatal error -- final processing.  Program exited before simulations began.  See previous error messages.

   ********* EnergyPlus Warmup Error Summary. During Warmup: 0 Warning; 0 Severe Errors.

   ********* EnergyPlus Sizing Error Summary. During Sizing: 0 Warning; 1 Severe Errors.

   ********* EnergyPlus Terminated--Fatal Error Detected. 0 Warning; 1 Severe Errors; Elapsed Time=00hr 00min  0.30sec


caught Command '['/usr/local/EnergyPlus-9-0-1/energyplus', '--idd', PosixPath('/usr/local/EnergyPlus-9-0-1/Energy+.idd'), '--weather', '/home/user/.local/lib/python3.7/site-packages/besos/data/example_epw.epw', '--output-directory', '/home/user/.besos_01t5vs48', PosixPath('/home/user/.besos_01t5vs48/in.epJSON')]' returned non-zero exit status 1.
/home/user/.local/lib/python3.7/site-packages/besos/evaluator.py:191: UserWarning: for inputs: ['RangeParameter [0.01, 0.99]', "Category['HW CONCRETE', 'Invalid Material']"] problematic values were: RangeParameter [0.01, 0.99]                            0.735874
Category['HW CONCRETE', 'Invalid Material']    Invalid Material
Name: 2, dtype: object
  warn(msg)

However sometimes we want to have a fallback value for these invalid states. For example, with an optimization algorithm that is minimizing, we can set a very high output value. This can be specified with the error_value argument for evaluators. It must be of the form (objective_values, constraint_values) where objective_values and constraint_values are tuples of the same length as the number of objectives/constraints. Since we have 1 objective and no constraints, we use a tuple with one item for objective_values, and an empty tuple for the constraint values.

error_value = ((10.0 ** 20,), ())

EvaluatorEP(problem, building, error_mode="Print", error_value=error_value).df_apply(
    samples
)
/home/user/.local/lib/python3.7/site-packages/besos/evaluator.py:394: FutureWarning: Evaluators have changed the format used for error values. The new format accepts either None or a tuple of values. See the evaluator docstring for details on the new format.
Your error mode has automatically been converted to a new format equivalent.
  warnings.warn(warning, FutureWarning)
HBox(children=(FloatProgress(value=0.0, description='Executing', max=5.0, style=ProgressStyle(description_widt…
Program Version,EnergyPlus, Version 9.0.1-bb7ca4f0da, YMD=2021.02.10 00:32,

   ********* Beginning Zone Sizing Calculations

   ** Severe  ** Did not find matching material for Construction EXT-SLAB, missing material = INVALID MATERIAL

   **  Fatal  ** GetSurfaceData: Errors discovered, program terminates.

   ...Summary of Errors that led to program termination:

   ..... Reference severe error count=1

   ..... Last severe error=Did not find matching material for Construction EXT-SLAB, missing material = INVALID MATERIAL

   ********* Warning:  Node connection errors not checked - most system input has not been read (see previous warning).

   ********* Fatal error -- final processing.  Program exited before simulations began.  See previous error messages.

   ********* EnergyPlus Warmup Error Summary. During Warmup: 0 Warning; 0 Severe Errors.

   ********* EnergyPlus Sizing Error Summary. During Sizing: 0 Warning; 1 Severe Errors.

   ********* EnergyPlus Terminated--Fatal Error Detected. 0 Warning; 1 Severe Errors; Elapsed Time=00hr 00min  0.28sec


Program Version,EnergyPlus, Version 9.0.1-bb7ca4f0da, YMD=2021.02.10 00:32,

   ********* Beginning Zone Sizing Calculations

   ** Severe  ** Did not find matching material for Construction EXT-SLAB, missing material = INVALID MATERIAL

   **  Fatal  ** GetSurfaceData: Errors discovered, program terminates.

   ...Summary of Errors that led to program termination:

   ..... Reference severe error count=1

   ..... Last severe error=Did not find matching material for Construction EXT-SLAB, missing material = INVALID MATERIAL

   ********* Warning:  Node connection errors not checked - most system input has not been read (see previous warning).

   ********* Fatal error -- final processing.  Program exited before simulations began.  See previous error messages.

   ********* EnergyPlus Warmup Error Summary. During Warmup: 0 Warning; 0 Severe Errors.

   ********* EnergyPlus Sizing Error Summary. During Sizing: 0 Warning; 1 Severe Errors.

   ********* EnergyPlus Terminated--Fatal Error Detected. 0 Warning; 1 Severe Errors; Elapsed Time=00hr 00min  0.29sec
/home/user/.local/lib/python3.7/site-packages/besos/evaluator.py:191: UserWarning: for inputs: ['RangeParameter [0.01, 0.99]', "Category['HW CONCRETE', 'Invalid Material']"] problematic values were: RangeParameter [0.01, 0.99]                            0.580238
Category['HW CONCRETE', 'Invalid Material']    Invalid Material
Name: 3, dtype: object
  warn(msg)
Program Version,EnergyPlus, Version 9.0.1-bb7ca4f0da, YMD=2021.02.10 00:32,

   ********* Beginning Zone Sizing Calculations

   ** Severe  ** Did not find matching material for Construction EXT-SLAB, missing material = INVALID MATERIAL

   **  Fatal  ** GetSurfaceData: Errors discovered, program terminates.

   ...Summary of Errors that led to program termination:

   ..... Reference severe error count=1

   ..... Last severe error=Did not find matching material for Construction EXT-SLAB, missing material = INVALID MATERIAL

   ********* Warning:  Node connection errors not checked - most system input has not been read (see previous warning).

   ********* Fatal error -- final processing.  Program exited before simulations began.  See previous error messages.

   ********* EnergyPlus Warmup Error Summary. During Warmup: 0 Warning; 0 Severe Errors.

   ********* EnergyPlus Sizing Error Summary. During Sizing: 0 Warning; 1 Severe Errors.

   ********* EnergyPlus Terminated--Fatal Error Detected. 0 Warning; 1 Severe Errors; Elapsed Time=00hr 00min  0.29sec
/home/user/.local/lib/python3.7/site-packages/besos/evaluator.py:191: UserWarning: for inputs: ['RangeParameter [0.01, 0.99]', "Category['HW CONCRETE', 'Invalid Material']"] problematic values were: RangeParameter [0.01, 0.99]                           0.0734916
Category['HW CONCRETE', 'Invalid Material']    Invalid Material
Name: 4, dtype: object
  warn(msg)
Electricity:Facility
0 1.826067e+09
1 1.812593e+09
2 1.000000e+20
3 1.000000e+20
4 1.000000e+20

This time, we got a warning for the invalid states, and our error value was used as the result. If we do not want to display these warnings, we can set the error_mode='Silent'. Omiting the error value will use a reasonable default, set to the opposite of what we are optimizing each objective towards. (This does not work for problems with constraints).

evaluator = EvaluatorEP(problem, building, error_mode="Silent")
print("Error value defaulted to:", evaluator.error_value)
Error value defaulted to: [inf]
evaluator.df_apply(samples)
HBox(children=(FloatProgress(value=0.0, description='Executing', max=5.0, style=ProgressStyle(description_widt…
Electricity:Facility
0 1.826067e+09
1 1.812593e+09
2 inf
3 inf
4 inf