Rastrigin Function

The Rastrigin function is a non-convex function defined as below.

\[ f(\mathbf{x}) = 10n + \sum_{i=1}^{n} \left[ x_i^2 - 10 \cos(2 \pi x_i) \right] \]

The function yields the global minimum \(f(x_1, \cdots, x_n) = f(0, \cdots, 0) = 0\), and the two-dimensional landscape in log scale is shown below.

As you can see, the Rastrigin function is a typical example of non-linear multimodal function and often used to evaluate optimization algorithms.

Here is an example of black-box optimization for the Rastrigin function (this function is considered a black-box for evaluation purpose). In this example, we consider \(n=5\).

The Rastrigin function is prepared in Amplify-BBOpt as rastrigin_function for an arbitrary input size.

Objective function and variables

from datetime import timedelta

from amplify import FixstarsClient
from amplify_bbopt import (
    DatasetGenerator,
    KernelQAOptimizer,
    RealVariableList,
    blackbox,
    plot_history,
)
from amplify_bbopt.objective_example import rastrigin_function


@blackbox
def blackbox_func(
    x: list[float] = RealVariableList(bounds=(-3, 3), length=5, nbins=301),  # noqa: B008
) -> float:
    ret = rastrigin_function(x)
    print(f"{x=}, {ret=:.3e}")
    return ret

Initial training data

# Generate initial training dataset with 10 samples
data = DatasetGenerator(objective=blackbox_func).generate(num_samples=10)

Show code cell output Hide code cell output

amplify-bbopt | 2024/10/04 06:51:46 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:51:46 | INFO | #0/10 initial data for blackbox_func
amplify-bbopt | 2024/10/04 06:51:46 | INFO | - [obj]: x=[2.12, 0.8200000000000003, 0.06000000000000005, -1.38, -1.16], ret=3.951e+01
amplify-bbopt | 2024/10/04 06:51:46 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:51:46 | INFO | #1/10 initial data for blackbox_func
amplify-bbopt | 2024/10/04 06:51:46 | INFO | - [obj]: x=[-2.76, -2.56, -2.92, -1.96, 1.88], ret=6.300e+01
amplify-bbopt | 2024/10/04 06:51:46 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:51:46 | INFO | #2/10 initial data for blackbox_func
amplify-bbopt | 2024/10/04 06:51:46 | INFO | - [obj]: x=[0.8999999999999999, 2.4800000000000004, 0.020000000000000018, 0.6400000000000001, 2.84], ret=5.836e+01
amplify-bbopt | 2024/10/04 06:51:46 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:51:46 | INFO | #3/10 initial data for blackbox_func
amplify-bbopt | 2024/10/04 06:51:46 | INFO | - [obj]: x=[1.38, 0.8000000000000003, 0.26000000000000023, 0.3599999999999999, 2.62], ret=7.810e+01
amplify-bbopt | 2024/10/04 06:51:46 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:51:46 | INFO | #4/10 initial data for blackbox_func
amplify-bbopt | 2024/10/04 06:51:46 | INFO | - [obj]: x=[-1.3399999999999999, 1.9000000000000004, 1.0200000000000005, -3.0, -0.6400000000000001], ret=4.958e+01
amplify-bbopt | 2024/10/04 06:51:46 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:51:46 | INFO | #5/10 initial data for blackbox_func
amplify-bbopt | 2024/10/04 06:51:46 | INFO | - [obj]: x=[2.16, 0.3200000000000003, -2.8, 1.6000000000000005, 1.38], ret=7.826e+01
amplify-bbopt | 2024/10/04 06:51:46 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:51:46 | INFO | #6/10 initial data for blackbox_func
amplify-bbopt | 2024/10/04 06:51:46 | INFO | - [obj]: x=[2.08, -1.96, -2.48, 2.1799999999999997, -2.88], ret=5.729e+01
amplify-bbopt | 2024/10/04 06:51:46 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:51:46 | INFO | #7/10 initial data for blackbox_func
amplify-bbopt | 2024/10/04 06:51:46 | INFO | - [obj]: x=[0.2400000000000002, -2.52, -1.2, -0.1200000000000001, -0.45999999999999996], ret=6.667e+01
amplify-bbopt | 2024/10/04 06:51:46 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:51:46 | INFO | #8/10 initial data for blackbox_func
amplify-bbopt | 2024/10/04 06:51:46 | INFO | - [obj]: x=[-0.5800000000000001, -2.84, -2.98, -2.26, -2.96], ret=6.558e+01
amplify-bbopt | 2024/10/04 06:51:46 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:51:46 | INFO | #9/10 initial data for blackbox_func
amplify-bbopt | 2024/10/04 06:51:46 | INFO | - [obj]: x=[1.0200000000000005, 0.16000000000000014, 0.8799999999999999, -1.46, 0.7000000000000002], ret=4.467e+01

Optimizer

# Set up solver client
client = FixstarsClient()
client.parameters.timeout = timedelta(milliseconds=2000)  # 2 seconds
# client.token = "xxxxxxxxxxx"  # Enter your Amplify AE API token.

# Instantiate an optimizer
optimizer = KernelQAOptimizer(data=data, objective=blackbox_func, client=client)

print(optimizer)

Show code cell output Hide code cell output

num variables: 1
num elemental variables: 5
num amplify variables: 1500
optimizer client: FixstarsClient
objective weight: 1.0
--------------------
trainer class: ModelKernelTrainer
model class: ModelKernel
model params: {beta: 0.0, gamma: 0.0}
reg_param: 1

# Perform optimization for [num_cycles] optimization cycles
optimizer.optimize(num_cycles=20)

Show code cell output Hide code cell output

amplify-bbopt | 2024/10/04 06:51:46 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:51:46 | INFO | #1/20 optimization cycle, constraint wt: 1.57e+02
amplify-bbopt | 2024/10/04 06:51:46 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 06:51:51 | INFO | num_iterations: 24
amplify-bbopt | 2024/10/04 06:51:51 | INFO | - [obj]: x=[-3.0, 2.5, 0.020000000000000018, 0.6400000000000001, -3.0], ret=6.111e+01
amplify-bbopt | 2024/10/04 06:51:51 | INFO | y_hat=6.111e+01, best objective=3.951e+01
amplify-bbopt | 2024/10/04 06:51:51 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:51:51 | INFO | #2/20 optimization cycle, constraint wt: 1.57e+02
amplify-bbopt | 2024/10/04 06:51:51 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 06:51:55 | INFO | num_iterations: 24
amplify-bbopt | 2024/10/04 06:51:55 | INFO | - [obj]: x=[-3.0, -3.0, -3.0, -3.0, -3.0], ret=4.500e+01
amplify-bbopt | 2024/10/04 06:51:55 | INFO | y_hat=4.500e+01, best objective=3.951e+01
amplify-bbopt | 2024/10/04 06:51:55 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:51:55 | INFO | #3/20 optimization cycle, constraint wt: 1.57e+02
amplify-bbopt | 2024/10/04 06:51:55 | INFO | model corrcoef: 0.815, beta: 0.0
amplify-bbopt | 2024/10/04 06:52:01 | INFO | num_iterations: 27
amplify-bbopt | 2024/10/04 06:52:01 | INFO | - [obj]: x=[2.12, -3.0, 0.020000000000000018, -3.0, -1.16], ret=3.127e+01
amplify-bbopt | 2024/10/04 06:52:01 | INFO | y_hat=3.127e+01, best objective=3.127e+01
amplify-bbopt | 2024/10/04 06:52:01 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:52:01 | INFO | #4/20 optimization cycle, constraint wt: 1.57e+02
amplify-bbopt | 2024/10/04 06:52:01 | INFO | model corrcoef: 0.821, beta: 0.0
amplify-bbopt | 2024/10/04 06:52:05 | INFO | num_iterations: 34
amplify-bbopt | 2024/10/04 06:52:05 | INFO | modifying solution (11, is_frequent=False), {'x': [-3.0, -3.0, -3.0, -3.0, -3.0]} --> {'x': [1.58, -0.6999999999999997, -0.23999999999999977, 3.0, 1.8399999999999999]}.
amplify-bbopt | 2024/10/04 06:52:05 | INFO | - [obj]: x=[1.58, -0.6999999999999997, -0.23999999999999977, 3.0, 1.8399999999999999], ret=6.130e+01
amplify-bbopt | 2024/10/04 06:52:05 | INFO | y_hat=6.130e+01, best objective=3.127e+01
amplify-bbopt | 2024/10/04 06:52:05 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:52:05 | INFO | #5/20 optimization cycle, constraint wt: 1.57e+02
amplify-bbopt | 2024/10/04 06:52:05 | INFO | model corrcoef: 0.822, beta: 0.0
amplify-bbopt | 2024/10/04 06:52:09 | INFO | num_iterations: 24
amplify-bbopt | 2024/10/04 06:52:09 | INFO | modifying solution (1, is_frequent=False), {'x': [-3.0, -3.0, -3.0, -3.0, -3.0]} --> {'x': [2.9000000000000004, -0.7199999999999998, 1.12, 2.7, 0.8999999999999999]}.
amplify-bbopt | 2024/10/04 06:52:09 | INFO | - [obj]: x=[2.9000000000000004, -0.7199999999999998, 1.12, 2.7, 0.8999999999999999], ret=4.978e+01
amplify-bbopt | 2024/10/04 06:52:09 | INFO | y_hat=4.978e+01, best objective=3.127e+01
amplify-bbopt | 2024/10/04 06:52:09 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:52:09 | INFO | #6/20 optimization cycle, constraint wt: 1.57e+02
amplify-bbopt | 2024/10/04 06:52:09 | INFO | model corrcoef: 0.820, beta: 0.0
amplify-bbopt | 2024/10/04 06:52:13 | INFO | num_iterations: 32
amplify-bbopt | 2024/10/04 06:52:13 | INFO | modifying solution (1, is_frequent=False), {'x': [-3.0, -3.0, -3.0, -3.0, -3.0]} --> {'x': [2.04, 1.1399999999999997, 1.2199999999999998, -0.6600000000000001, 2.26]}.
amplify-bbopt | 2024/10/04 06:52:13 | INFO | - [obj]: x=[2.04, 1.1399999999999997, 1.2199999999999998, -0.6600000000000001, 2.26], ret=5.055e+01
amplify-bbopt | 2024/10/04 06:52:13 | INFO | y_hat=5.055e+01, best objective=3.127e+01
amplify-bbopt | 2024/10/04 06:52:13 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:52:13 | INFO | #7/20 optimization cycle, constraint wt: 1.57e+02
amplify-bbopt | 2024/10/04 06:52:13 | INFO | model corrcoef: 0.819, beta: 0.0
amplify-bbopt | 2024/10/04 06:52:17 | INFO | num_iterations: 28
amplify-bbopt | 2024/10/04 06:52:17 | INFO | modifying solution (1, is_frequent=False), {'x': [-3.0, -3.0, -3.0, -3.0, -3.0]} --> {'x': [-2.2, 0.48, 1.3399999999999999, 2.08, 0.16000000000000014]}.
amplify-bbopt | 2024/10/04 06:52:17 | INFO | - [obj]: x=[-2.2, 0.48, 1.3399999999999999, 2.08, 0.16000000000000014], ret=5.929e+01
amplify-bbopt | 2024/10/04 06:52:17 | INFO | y_hat=5.929e+01, best objective=3.127e+01
amplify-bbopt | 2024/10/04 06:52:17 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:52:17 | INFO | #8/20 optimization cycle, constraint wt: 1.57e+02
amplify-bbopt | 2024/10/04 06:52:17 | INFO | model corrcoef: 0.819, beta: 0.0
amplify-bbopt | 2024/10/04 06:52:22 | INFO | num_iterations: 29
amplify-bbopt | 2024/10/04 06:52:22 | INFO | modifying solution (1, is_frequent=False), {'x': [-3.0, -3.0, -3.0, -3.0, -3.0]} --> {'x': [-0.7399999999999998, -1.14, -0.45999999999999996, -0.08000000000000007, 1.3200000000000003]}.
amplify-bbopt | 2024/10/04 06:52:22 | INFO | - [obj]: x=[-0.7399999999999998, -1.14, -0.45999999999999996, -0.08000000000000007, 1.3200000000000003], ret=5.324e+01
amplify-bbopt | 2024/10/04 06:52:22 | INFO | y_hat=5.324e+01, best objective=3.127e+01
amplify-bbopt | 2024/10/04 06:52:22 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:52:22 | INFO | #9/20 optimization cycle, constraint wt: 1.57e+02
amplify-bbopt | 2024/10/04 06:52:22 | INFO | model corrcoef: 0.818, beta: 0.0
amplify-bbopt | 2024/10/04 06:52:26 | INFO | num_iterations: 29
amplify-bbopt | 2024/10/04 06:52:26 | INFO | modifying solution (1, is_frequent=True), {'x': [-3.0, -3.0, -3.0, -3.0, -3.0]} --> {'x': [-3.0, -3.0, -3.0, -3.0, -2.98]}.
amplify-bbopt | 2024/10/04 06:52:26 | INFO | - [obj]: x=[-3.0, -3.0, -3.0, -3.0, -2.98], ret=4.496e+01
amplify-bbopt | 2024/10/04 06:52:26 | INFO | y_hat=4.496e+01, best objective=3.127e+01
amplify-bbopt | 2024/10/04 06:52:26 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:52:26 | INFO | #10/20 optimization cycle, constraint wt: 1.57e+02
amplify-bbopt | 2024/10/04 06:52:26 | INFO | model corrcoef: 0.820, beta: 0.0
amplify-bbopt | 2024/10/04 06:52:35 | INFO | num_iterations: 26
amplify-bbopt | 2024/10/04 06:52:35 | INFO | - [obj]: x=[2.9000000000000004, -1.14, -0.45999999999999996, -3.0, -3.0], ret=5.314e+01
amplify-bbopt | 2024/10/04 06:52:35 | INFO | y_hat=5.314e+01, best objective=3.127e+01
amplify-bbopt | 2024/10/04 06:52:35 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:52:35 | INFO | #11/20 optimization cycle, constraint wt: 1.57e+02
amplify-bbopt | 2024/10/04 06:52:35 | INFO | model corrcoef: 0.819, beta: 0.0
amplify-bbopt | 2024/10/04 06:52:39 | INFO | num_iterations: 33
amplify-bbopt | 2024/10/04 06:52:39 | INFO | modifying solution (1, is_frequent=True), {'x': [-3.0, -3.0, -3.0, -3.0, -3.0]} --> {'x': [-3.0, -3.0, -2.98, -3.0, -3.0]}.
amplify-bbopt | 2024/10/04 06:52:39 | INFO | - [obj]: x=[-3.0, -3.0, -2.98, -3.0, -3.0], ret=4.496e+01
amplify-bbopt | 2024/10/04 06:52:39 | INFO | y_hat=4.496e+01, best objective=3.127e+01
amplify-bbopt | 2024/10/04 06:52:39 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:52:39 | INFO | #12/20 optimization cycle, constraint wt: 1.57e+02
amplify-bbopt | 2024/10/04 06:52:39 | INFO | model corrcoef: 0.822, beta: 0.0
amplify-bbopt | 2024/10/04 06:52:44 | INFO | num_iterations: 30
amplify-bbopt | 2024/10/04 06:52:44 | INFO | - [obj]: x=[2.12, -3.0, 2.62, 2.7, -3.0], ret=6.974e+01
amplify-bbopt | 2024/10/04 06:52:44 | INFO | y_hat=6.974e+01, best objective=3.127e+01
amplify-bbopt | 2024/10/04 06:52:44 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:52:44 | INFO | #13/20 optimization cycle, constraint wt: 1.57e+02
amplify-bbopt | 2024/10/04 06:52:44 | INFO | model corrcoef: 0.830, beta: 0.0
amplify-bbopt | 2024/10/04 06:52:48 | INFO | num_iterations: 28
amplify-bbopt | 2024/10/04 06:52:48 | INFO | modifying solution (2, is_frequent=True), {'x': [-3.0, -3.0, -3.0, -3.0, -3.0]} --> {'x': [-3.0, -3.0, -2.96, -3.0, -3.0]}.
amplify-bbopt | 2024/10/04 06:52:48 | INFO | - [obj]: x=[-3.0, -3.0, -2.96, -3.0, -3.0], ret=4.508e+01
amplify-bbopt | 2024/10/04 06:52:48 | INFO | y_hat=4.508e+01, best objective=3.127e+01
amplify-bbopt | 2024/10/04 06:52:48 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:52:48 | INFO | #14/20 optimization cycle, constraint wt: 1.57e+02
amplify-bbopt | 2024/10/04 06:52:48 | INFO | model corrcoef: 0.835, beta: 0.0
amplify-bbopt | 2024/10/04 06:52:52 | INFO | num_iterations: 20
amplify-bbopt | 2024/10/04 06:52:52 | INFO | - [obj]: x=[2.12, -1.96, 0.020000000000000018, -3.0, 0.8999999999999999], ret=2.316e+01
amplify-bbopt | 2024/10/04 06:52:52 | INFO | y_hat=2.316e+01, best objective=2.316e+01
amplify-bbopt | 2024/10/04 06:52:52 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:52:52 | INFO | #15/20 optimization cycle, constraint wt: 1.57e+02
amplify-bbopt | 2024/10/04 06:52:52 | INFO | model corrcoef: 0.842, beta: 0.0
amplify-bbopt | 2024/10/04 06:52:56 | INFO | num_iterations: 22
amplify-bbopt | 2024/10/04 06:52:56 | INFO | modifying solution (1, is_frequent=False), {'x': [2.12, -1.96, 0.020000000000000018, -3.0, 0.8999999999999999]} --> {'x': [-0.6600000000000001, -1.04, 2.34, -1.42, -1.64]}.
amplify-bbopt | 2024/10/04 06:52:56 | INFO | - [obj]: x=[-0.6600000000000001, -1.04, 2.34, -1.42, -1.64], ret=7.787e+01
amplify-bbopt | 2024/10/04 06:52:56 | INFO | y_hat=7.787e+01, best objective=2.316e+01
amplify-bbopt | 2024/10/04 06:52:56 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:52:56 | INFO | #16/20 optimization cycle, constraint wt: 1.57e+02
amplify-bbopt | 2024/10/04 06:52:56 | INFO | model corrcoef: 0.856, beta: 0.0
amplify-bbopt | 2024/10/04 06:53:01 | INFO | num_iterations: 30
amplify-bbopt | 2024/10/04 06:53:01 | INFO | - [obj]: x=[2.12, -1.96, -3.0, -3.0, 0.7000000000000002], ret=4.294e+01
amplify-bbopt | 2024/10/04 06:53:01 | INFO | y_hat=4.294e+01, best objective=2.316e+01
amplify-bbopt | 2024/10/04 06:53:01 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:53:01 | INFO | #17/20 optimization cycle, constraint wt: 1.57e+02
amplify-bbopt | 2024/10/04 06:53:01 | INFO | model corrcoef: 0.855, beta: 0.0
amplify-bbopt | 2024/10/04 06:53:05 | INFO | num_iterations: 20
amplify-bbopt | 2024/10/04 06:53:05 | INFO | - [obj]: x=[2.08, -1.96, 0.020000000000000018, -1.38, 0.8999999999999999], ret=3.171e+01
amplify-bbopt | 2024/10/04 06:53:05 | INFO | y_hat=3.171e+01, best objective=2.316e+01
amplify-bbopt | 2024/10/04 06:53:05 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:53:05 | INFO | #18/20 optimization cycle, constraint wt: 1.57e+02
amplify-bbopt | 2024/10/04 06:53:05 | INFO | model corrcoef: 0.857, beta: 0.0
amplify-bbopt | 2024/10/04 06:53:10 | INFO | num_iterations: 19
amplify-bbopt | 2024/10/04 06:53:10 | INFO | - [obj]: x=[2.12, 2.4800000000000004, 0.020000000000000018, -3.0, 0.8999999999999999], ret=4.508e+01
amplify-bbopt | 2024/10/04 06:53:10 | INFO | y_hat=4.508e+01, best objective=2.316e+01
amplify-bbopt | 2024/10/04 06:53:10 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:53:10 | INFO | #19/20 optimization cycle, constraint wt: 1.57e+02
amplify-bbopt | 2024/10/04 06:53:10 | INFO | model corrcoef: 0.856, beta: 0.0
amplify-bbopt | 2024/10/04 06:53:17 | INFO | num_iterations: 24
amplify-bbopt | 2024/10/04 06:53:17 | INFO | - [obj]: x=[-3.0, -1.96, -3.0, 2.1799999999999997, -3.0], ret=4.165e+01
amplify-bbopt | 2024/10/04 06:53:17 | INFO | y_hat=4.165e+01, best objective=2.316e+01
amplify-bbopt | 2024/10/04 06:53:17 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:53:17 | INFO | #20/20 optimization cycle, constraint wt: 1.57e+02
amplify-bbopt | 2024/10/04 06:53:17 | INFO | model corrcoef: 0.856, beta: 0.0
amplify-bbopt | 2024/10/04 06:53:21 | INFO | num_iterations: 20
amplify-bbopt | 2024/10/04 06:53:21 | INFO | - [obj]: x=[-2.2800000000000002, -1.96, 1.12, -3.0, 1.3200000000000003], ret=5.019e+01
amplify-bbopt | 2024/10/04 06:53:21 | INFO | y_hat=5.019e+01, best objective=2.316e+01

Result

# Print results
print(
    f"{optimizer.best_solution=}"
)  # Solution (optimal input to the black-box objective function)
print(
    f"{optimizer.best_objective=:.3e}"
)  # Corresponding objective function value
plot_history(optimizer.fetch_history(), log_scale=True)  # Optimization history

optimizer.best_solution={'x': [2.12, -1.96, 0.020000000000000018, -3.0, 0.8999999999999999]}
optimizer.best_objective=2.316e+01