Rosenbrock Function

The Rosenbrock function is a non-convex function defined as below.

\[ f(\mathbf{x}) = \sum_{i=1}^{n-1} \left[ (1 - x_i)^2 + 100 (x_{i+1} - x_i^2)^2 \right] \]

The function yields the global minimum \(f(x_1, \cdots, x_n) = f(1, \cdots, 1) = 0\), and the two-dimensional landscape in log scale is shown below.

The Rosenbrock function is often used to evaluate optimization algorithm due to:

The function is unimodal, and the global minimum lies in a narrow, parabolic valley. However, even though this valley is easy to find, convergence to the minimum is difficult. Picheny, V., Wagner, T. & Ginsbourger, D. A benchmark of kriging-based infill criteria for noisy optimization. Struct Multidisc Optim 48, 607–626 (2013).

Here is an example of black-box optimization for the Rosenbrock function (this function is considered a black-box for evaluation purpose). In this example, we consider \(n=5\).

The Rosenbrock function is prepared in Amplify-BBOpt as rosenbrock_function for an arbitrary input size.

Objective function and variables

from datetime import timedelta

from amplify import FixstarsClient
from amplify_bbopt import (
    DatasetGenerator,
    KernelQAOptimizer,
    RealVariableList,
    blackbox,
    plot_history,
)
from amplify_bbopt.objective_example import rosenbrock_function


@blackbox
def blackbox_func(
    x: list[float] = RealVariableList(bounds=(-3, 3), length=5, nbins=301),  # noqa: B008
) -> float:
    ret = rosenbrock_function(x)
    print(f"{x=}, {ret=:.3e}")
    return ret

Initial training data

# Generate initial training dataset with 10 samples
data = DatasetGenerator(objective=blackbox_func).generate(num_samples=10)
Hide code cell output
amplify-bbopt | 2024/10/04 05:51:29 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 05:51:29 | INFO | #0/10 initial data for blackbox_func
amplify-bbopt | 2024/10/04 05:51:29 | INFO | - [obj]: x=[2.12, 0.8200000000000003, 0.06000000000000005, -1.38, -1.16], ret=2.526e+03
amplify-bbopt | 2024/10/04 05:51:29 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 05:51:29 | INFO | #1/10 initial data for blackbox_func
amplify-bbopt | 2024/10/04 05:51:29 | INFO | - [obj]: x=[-2.76, -2.56, -2.92, -1.96, 1.88], ret=3.077e+04
amplify-bbopt | 2024/10/04 05:51:29 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 05:51:29 | INFO | #2/10 initial data for blackbox_func
amplify-bbopt | 2024/10/04 05:51:29 | INFO | - [obj]: x=[0.8999999999999999, 2.4800000000000004, 0.020000000000000018, 0.6400000000000001, 2.84], ret=4.672e+03
amplify-bbopt | 2024/10/04 05:51:29 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 05:51:29 | INFO | #3/10 initial data for blackbox_func
amplify-bbopt | 2024/10/04 05:51:29 | INFO | - [obj]: x=[1.38, 0.8000000000000003, 0.26000000000000023, 0.3599999999999999, 2.62], ret=7.663e+02
amplify-bbopt | 2024/10/04 05:51:29 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 05:51:29 | INFO | #4/10 initial data for blackbox_func
amplify-bbopt | 2024/10/04 05:51:29 | INFO | - [obj]: x=[-1.3399999999999999, 1.9000000000000004, 1.0200000000000005, -3.0, -0.6400000000000001], ret=1.162e+04
amplify-bbopt | 2024/10/04 05:51:29 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 05:51:29 | INFO | #5/10 initial data for blackbox_func
amplify-bbopt | 2024/10/04 05:51:29 | INFO | - [obj]: x=[2.16, 0.3200000000000003, -2.8, 1.6000000000000005, 1.38], ret=6.780e+03
amplify-bbopt | 2024/10/04 05:51:29 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 05:51:29 | INFO | #6/10 initial data for blackbox_func
amplify-bbopt | 2024/10/04 05:51:29 | INFO | - [obj]: x=[2.08, -1.96, -2.48, 2.1799999999999997, -2.88], ret=1.537e+04
amplify-bbopt | 2024/10/04 05:51:29 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 05:51:29 | INFO | #7/10 initial data for blackbox_func
amplify-bbopt | 2024/10/04 05:51:29 | INFO | - [obj]: x=[0.2400000000000002, -2.52, -1.2, -0.1200000000000001, -0.45999999999999996], ret=6.650e+03
amplify-bbopt | 2024/10/04 05:51:29 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 05:51:29 | INFO | #8/10 initial data for blackbox_func
amplify-bbopt | 2024/10/04 05:51:29 | INFO | - [obj]: x=[-0.5800000000000001, -2.84, -2.98, -2.26, -2.96], ret=3.217e+04
amplify-bbopt | 2024/10/04 05:51:29 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 05:51:29 | INFO | #9/10 initial data for blackbox_func
amplify-bbopt | 2024/10/04 05:51:29 | INFO | - [obj]: x=[1.0200000000000005, 0.16000000000000014, 0.8799999999999999, -1.46, 0.7000000000000002], ret=8.615e+02

Optimizer

# Set up solver client
client = FixstarsClient()
client.parameters.timeout = timedelta(milliseconds=2000)  # 2 seconds
# client.token = "xxxxxxxxxxx"  # Enter your Amplify AE API token.

# Instantiate an optimizer
optimizer = KernelQAOptimizer(data=data, objective=blackbox_func, client=client)

print(optimizer)
Hide code cell output
num variables: 1
num elemental variables: 5
num amplify variables: 1500
optimizer client: FixstarsClient
objective weight: 1.0
--------------------
trainer class: ModelKernelTrainer
model class: ModelKernel
model params: {beta: 0.0, gamma: 0.0}
reg_param: 1
# Perform optimization for [num_cycles] optimization cycles
optimizer.optimize(num_cycles=20)
Hide code cell output
amplify-bbopt | 2024/10/04 05:51:29 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 05:51:29 | INFO | #1/20 optimization cycle, constraint wt: 6.43e+04
amplify-bbopt | 2024/10/04 05:51:29 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 05:51:39 | INFO | num_iterations: 20
amplify-bbopt | 2024/10/04 05:51:39 | INFO | - [obj]: x=[2.5200000000000005, 0.16000000000000014, 0.8799999999999999, 1.6000000000000005, 0.2400000000000002], ret=4.515e+03
amplify-bbopt | 2024/10/04 05:51:39 | INFO | y_hat=4.515e+03, best objective=7.663e+02
amplify-bbopt | 2024/10/04 05:51:39 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 05:51:39 | INFO | #2/20 optimization cycle, constraint wt: 6.43e+04
amplify-bbopt | 2024/10/04 05:51:39 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 05:51:47 | INFO | num_iterations: 20
amplify-bbopt | 2024/10/04 05:51:47 | INFO | - [obj]: x=[-3.0, 0.3200000000000003, -1.2, -0.1200000000000001, -1.44], ret=8.181e+03
amplify-bbopt | 2024/10/04 05:51:47 | INFO | y_hat=8.181e+03, best objective=7.663e+02
amplify-bbopt | 2024/10/04 05:51:47 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 05:51:47 | INFO | #3/20 optimization cycle, constraint wt: 6.43e+04
amplify-bbopt | 2024/10/04 05:51:47 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 05:51:54 | INFO | num_iterations: 23
amplify-bbopt | 2024/10/04 05:51:54 | INFO | - [obj]: x=[2.12, 0.8000000000000003, 0.06000000000000005, -3.0, 0.7000000000000002], ret=9.208e+03
amplify-bbopt | 2024/10/04 05:51:54 | INFO | y_hat=9.208e+03, best objective=7.663e+02
amplify-bbopt | 2024/10/04 05:51:54 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 05:51:54 | INFO | #4/20 optimization cycle, constraint wt: 6.43e+04
amplify-bbopt | 2024/10/04 05:51:54 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 05:51:58 | INFO | num_iterations: 20
amplify-bbopt | 2024/10/04 05:51:58 | INFO | - [obj]: x=[1.0200000000000005, 0.16000000000000014, 0.8799999999999999, 0.3599999999999999, -0.45999999999999996], ret=2.036e+02
amplify-bbopt | 2024/10/04 05:51:58 | INFO | y_hat=2.036e+02, best objective=2.036e+02
amplify-bbopt | 2024/10/04 05:51:58 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 05:51:58 | INFO | #5/20 optimization cycle, constraint wt: 6.43e+04
amplify-bbopt | 2024/10/04 05:51:58 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 05:52:02 | INFO | num_iterations: 24
amplify-bbopt | 2024/10/04 05:52:02 | INFO | - [obj]: x=[1.38, 0.16000000000000014, 0.8799999999999999, -0.1200000000000001, -0.45999999999999996], ret=4.819e+02
amplify-bbopt | 2024/10/04 05:52:02 | INFO | y_hat=4.819e+02, best objective=2.036e+02
amplify-bbopt | 2024/10/04 05:52:02 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 05:52:02 | INFO | #6/20 optimization cycle, constraint wt: 6.43e+04
amplify-bbopt | 2024/10/04 05:52:02 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 05:52:06 | INFO | num_iterations: 23
amplify-bbopt | 2024/10/04 05:52:06 | INFO | - [obj]: x=[-3.0, -3.0, 0.8799999999999999, -1.38, -3.0], ret=2.390e+04
amplify-bbopt | 2024/10/04 05:52:06 | INFO | y_hat=2.390e+04, best objective=2.036e+02
amplify-bbopt | 2024/10/04 05:52:06 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 05:52:06 | INFO | #7/20 optimization cycle, constraint wt: 6.43e+04
amplify-bbopt | 2024/10/04 05:52:06 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 05:52:11 | INFO | num_iterations: 17
amplify-bbopt | 2024/10/04 05:52:11 | INFO | - [obj]: x=[1.0200000000000005, 0.16000000000000014, -1.2, 1.6000000000000005, -0.45999999999999996], ret=1.148e+03
amplify-bbopt | 2024/10/04 05:52:11 | INFO | y_hat=1.148e+03, best objective=2.036e+02
amplify-bbopt | 2024/10/04 05:52:11 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 05:52:11 | INFO | #8/20 optimization cycle, constraint wt: 6.43e+04
amplify-bbopt | 2024/10/04 05:52:11 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 05:52:15 | INFO | num_iterations: 21
amplify-bbopt | 2024/10/04 05:52:15 | INFO | - [obj]: x=[1.0200000000000005, 0.16000000000000014, -1.08, -0.1200000000000001, -0.1200000000000001], ret=3.733e+02
amplify-bbopt | 2024/10/04 05:52:15 | INFO | y_hat=3.733e+02, best objective=2.036e+02
amplify-bbopt | 2024/10/04 05:52:15 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 05:52:15 | INFO | #9/20 optimization cycle, constraint wt: 6.43e+04
amplify-bbopt | 2024/10/04 05:52:15 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 05:52:19 | INFO | num_iterations: 28
amplify-bbopt | 2024/10/04 05:52:19 | INFO | - [obj]: x=[1.0200000000000005, 0.16000000000000014, 0.8799999999999999, -3.0, -1.16], ret=1.191e+04
amplify-bbopt | 2024/10/04 05:52:19 | INFO | y_hat=1.191e+04, best objective=2.036e+02
amplify-bbopt | 2024/10/04 05:52:19 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 05:52:19 | INFO | #10/20 optimization cycle, constraint wt: 6.43e+04
amplify-bbopt | 2024/10/04 05:52:19 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 05:52:23 | INFO | num_iterations: 20
amplify-bbopt | 2024/10/04 05:52:23 | INFO | - [obj]: x=[1.38, 1.9000000000000004, 0.26000000000000023, 0.3599999999999999, 0.7000000000000002], ret=1.165e+03
amplify-bbopt | 2024/10/04 05:52:23 | INFO | y_hat=1.165e+03, best objective=2.036e+02
amplify-bbopt | 2024/10/04 05:52:23 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 05:52:23 | INFO | #11/20 optimization cycle, constraint wt: 6.43e+04
amplify-bbopt | 2024/10/04 05:52:23 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 05:52:28 | INFO | num_iterations: 27
amplify-bbopt | 2024/10/04 05:52:28 | INFO | - [obj]: x=[2.16, 0.8200000000000003, -1.64, 0.3599999999999999, -0.45999999999999996], ret=2.600e+03
amplify-bbopt | 2024/10/04 05:52:28 | INFO | y_hat=2.600e+03, best objective=2.036e+02
amplify-bbopt | 2024/10/04 05:52:28 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 05:52:28 | INFO | #12/20 optimization cycle, constraint wt: 6.43e+04
amplify-bbopt | 2024/10/04 05:52:28 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 05:52:32 | INFO | num_iterations: 20
amplify-bbopt | 2024/10/04 05:52:32 | INFO | - [obj]: x=[1.0200000000000005, 0.16000000000000014, 0.06000000000000005, 0.3599999999999999, 0.7000000000000002], ret=1.249e+02
amplify-bbopt | 2024/10/04 05:52:32 | INFO | y_hat=1.249e+02, best objective=1.249e+02
amplify-bbopt | 2024/10/04 05:52:32 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 05:52:32 | INFO | #13/20 optimization cycle, constraint wt: 6.43e+04
amplify-bbopt | 2024/10/04 05:52:32 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 05:52:36 | INFO | num_iterations: 20
amplify-bbopt | 2024/10/04 05:52:36 | INFO | - [obj]: x=[1.0200000000000005, 0.8200000000000003, 0.26000000000000023, -0.1200000000000001, -0.45999999999999996], ret=4.972e+01
amplify-bbopt | 2024/10/04 05:52:36 | INFO | y_hat=4.972e+01, best objective=4.972e+01
amplify-bbopt | 2024/10/04 05:52:36 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 05:52:36 | INFO | #14/20 optimization cycle, constraint wt: 6.43e+04
amplify-bbopt | 2024/10/04 05:52:36 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 05:52:40 | INFO | num_iterations: 22
amplify-bbopt | 2024/10/04 05:52:40 | INFO | - [obj]: x=[1.38, 0.16000000000000014, -0.43999999999999995, 0.3599999999999999, -0.45999999999999996], ret=3.668e+02
amplify-bbopt | 2024/10/04 05:52:40 | INFO | y_hat=3.668e+02, best objective=4.972e+01
amplify-bbopt | 2024/10/04 05:52:40 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 05:52:40 | INFO | #15/20 optimization cycle, constraint wt: 6.43e+04
amplify-bbopt | 2024/10/04 05:52:40 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 05:52:44 | INFO | num_iterations: 19
amplify-bbopt | 2024/10/04 05:52:44 | INFO | - [obj]: x=[-3.0, 0.8200000000000003, -3.0, -1.46, -2.74], ret=2.139e+04
amplify-bbopt | 2024/10/04 05:52:44 | INFO | y_hat=2.139e+04, best objective=4.972e+01
amplify-bbopt | 2024/10/04 05:52:44 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 05:52:44 | INFO | #16/20 optimization cycle, constraint wt: 6.43e+04
amplify-bbopt | 2024/10/04 05:52:44 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 05:52:50 | INFO | num_iterations: 18
amplify-bbopt | 2024/10/04 05:52:50 | INFO | - [obj]: x=[2.12, 0.3200000000000003, 0.26000000000000023, -0.1200000000000001, -0.45999999999999996], ret=1.775e+03
amplify-bbopt | 2024/10/04 05:52:50 | INFO | y_hat=1.775e+03, best objective=4.972e+01
amplify-bbopt | 2024/10/04 05:52:50 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 05:52:50 | INFO | #17/20 optimization cycle, constraint wt: 6.43e+04
amplify-bbopt | 2024/10/04 05:52:50 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 05:52:54 | INFO | num_iterations: 19
amplify-bbopt | 2024/10/04 05:52:54 | INFO | - [obj]: x=[1.0200000000000005, 0.8200000000000003, 2.4000000000000004, 0.3599999999999999, 0.7000000000000002], ret=3.254e+03
amplify-bbopt | 2024/10/04 05:52:54 | INFO | y_hat=3.254e+03, best objective=4.972e+01
amplify-bbopt | 2024/10/04 05:52:54 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 05:52:54 | INFO | #18/20 optimization cycle, constraint wt: 6.43e+04
amplify-bbopt | 2024/10/04 05:52:54 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 05:52:59 | INFO | num_iterations: 20
amplify-bbopt | 2024/10/04 05:52:59 | INFO | - [obj]: x=[1.0200000000000005, 0.16000000000000014, 0.8799999999999999, 0.3599999999999999, 0.7000000000000002], ret=2.013e+02
amplify-bbopt | 2024/10/04 05:52:59 | INFO | y_hat=2.013e+02, best objective=4.972e+01
amplify-bbopt | 2024/10/04 05:52:59 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 05:52:59 | INFO | #19/20 optimization cycle, constraint wt: 6.43e+04
amplify-bbopt | 2024/10/04 05:52:59 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 05:53:03 | INFO | num_iterations: 27
amplify-bbopt | 2024/10/04 05:53:03 | INFO | - [obj]: x=[1.0200000000000005, -3.0, -1.2, -0.1200000000000001, -0.45999999999999996], ret=1.232e+04
amplify-bbopt | 2024/10/04 05:53:03 | INFO | y_hat=1.232e+04, best objective=4.972e+01
amplify-bbopt | 2024/10/04 05:53:03 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 05:53:03 | INFO | #20/20 optimization cycle, constraint wt: 6.43e+04
amplify-bbopt | 2024/10/04 05:53:03 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 05:53:08 | INFO | num_iterations: 21
amplify-bbopt | 2024/10/04 05:53:08 | INFO | - [obj]: x=[0.2400000000000002, 0.16000000000000014, 0.06000000000000005, 0.3599999999999999, -0.45999999999999996], ret=5.121e+01
amplify-bbopt | 2024/10/04 05:53:08 | INFO | y_hat=5.121e+01, best objective=4.972e+01

Result

# Solution (optimal input to the black-box objective function)
print(f"{optimizer.best_solution=}")

# Corresponding objective function value
print(f"{optimizer.best_objective=:.3e}")

# Optimization history
plot_history(optimizer.fetch_history(), log_scale=True)
optimizer.best_solution={'x': [1.0200000000000005, 0.8200000000000003, 0.26000000000000023, -0.1200000000000001, -0.45999999999999996]}
optimizer.best_objective=4.972e+01