Sphere Function¶
The sphere function is a convex function defined as below.
\[
f(\mathbf{x}) = \sum_{i=1}^n x_i^2.
\]
The function yields the global minimum \(f(x_1, \cdots, x_n) = f(0, \cdots, 0) = 0\), and its two-dimensional landscape in log scale is shown below.
The sphere function is prepared in Amplify-BBOpt as sphere_function for an arbitrary input size. Here, let us see this function as a black-box function and test our black-box optimization for the sphere function. In this example, we consider \(n=5\).
Objective function and variables¶
from datetime import timedelta
from amplify import FixstarsClient
from amplify_bbopt import (
DatasetGenerator,
KernelQAOptimizer,
RealVariableList,
blackbox,
plot_history,
)
from amplify_bbopt.objective_example import sphere_function
@blackbox
def blackbox_func(
x: list[float] = RealVariableList(bounds=(-3, 3), length=5, nbins=301),
) -> float:
ret = sphere_function(x)
print(f"{x=}, {ret=:.3e}")
return ret
Initial training data¶
# Generate initial training dataset with 10 samples
data = DatasetGenerator(objective=blackbox_func).generate(num_samples=10)
Show code cell output
amplify-bbopt | 2024/10/04 06:03:37 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:03:37 | INFO | #0/10 initial data for blackbox_func
amplify-bbopt | 2024/10/04 06:03:37 | INFO | - [obj]: x=[2.12, 0.8200000000000003, 0.06000000000000005, -1.38, -1.16], ret=8.420e+00
amplify-bbopt | 2024/10/04 06:03:37 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:03:37 | INFO | #1/10 initial data for blackbox_func
amplify-bbopt | 2024/10/04 06:03:37 | INFO | - [obj]: x=[-2.76, -2.56, -2.92, -1.96, 1.88], ret=3.007e+01
amplify-bbopt | 2024/10/04 06:03:37 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:03:37 | INFO | #2/10 initial data for blackbox_func
amplify-bbopt | 2024/10/04 06:03:37 | INFO | - [obj]: x=[0.8999999999999999, 2.4800000000000004, 0.020000000000000018, 0.6400000000000001, 2.84], ret=1.544e+01
amplify-bbopt | 2024/10/04 06:03:37 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:03:37 | INFO | #3/10 initial data for blackbox_func
amplify-bbopt | 2024/10/04 06:03:37 | INFO | - [obj]: x=[1.38, 0.8000000000000003, 0.26000000000000023, 0.3599999999999999, 2.62], ret=9.606e+00
amplify-bbopt | 2024/10/04 06:03:37 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:03:37 | INFO | #4/10 initial data for blackbox_func
amplify-bbopt | 2024/10/04 06:03:37 | INFO | - [obj]: x=[-1.3399999999999999, 1.9000000000000004, 1.0200000000000005, -3.0, -0.6400000000000001], ret=1.586e+01
amplify-bbopt | 2024/10/04 06:03:37 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:03:37 | INFO | #5/10 initial data for blackbox_func
amplify-bbopt | 2024/10/04 06:03:37 | INFO | - [obj]: x=[2.16, 0.3200000000000003, -2.8, 1.6000000000000005, 1.38], ret=1.707e+01
amplify-bbopt | 2024/10/04 06:03:37 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:03:37 | INFO | #6/10 initial data for blackbox_func
amplify-bbopt | 2024/10/04 06:03:37 | INFO | - [obj]: x=[2.08, -1.96, -2.48, 2.1799999999999997, -2.88], ret=2.737e+01
amplify-bbopt | 2024/10/04 06:03:37 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:03:37 | INFO | #7/10 initial data for blackbox_func
amplify-bbopt | 2024/10/04 06:03:37 | INFO | - [obj]: x=[0.2400000000000002, -2.52, -1.2, -0.1200000000000001, -0.45999999999999996], ret=8.074e+00
amplify-bbopt | 2024/10/04 06:03:37 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:03:37 | INFO | #8/10 initial data for blackbox_func
amplify-bbopt | 2024/10/04 06:03:37 | INFO | - [obj]: x=[-0.5800000000000001, -2.84, -2.98, -2.26, -2.96], ret=3.115e+01
amplify-bbopt | 2024/10/04 06:03:37 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:03:37 | INFO | #9/10 initial data for blackbox_func
amplify-bbopt | 2024/10/04 06:03:37 | INFO | - [obj]: x=[1.0200000000000005, 0.16000000000000014, 0.8799999999999999, -1.46, 0.7000000000000002], ret=4.462e+00
Optimizer¶
# Set up solver client
client = FixstarsClient()
client.parameters.timeout = timedelta(milliseconds=2000) # 2 seconds
# client.token = "xxxxxxxxxxx" # Enter your Amplify AE API token.
# Instantiate an optimizer
optimizer = KernelQAOptimizer(data=data, objective=blackbox_func, client=client)
print(optimizer)
Show code cell output
num variables: 1
num elemental variables: 5
num amplify variables: 1500
optimizer client: FixstarsClient
objective weight: 1.0
--------------------
trainer class: ModelKernelTrainer
model class: ModelKernel
model params: {beta: 0.0, gamma: 0.0}
reg_param: 1
# Perform optimization for [num_cycles] optimization cycles
optimizer.optimize(num_cycles=20)
Show code cell output
amplify-bbopt | 2024/10/04 06:03:37 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:03:37 | INFO | #1/20 optimization cycle, constraint wt: 6.23e+01
amplify-bbopt | 2024/10/04 06:03:37 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 06:03:43 | INFO | num_iterations: 23
amplify-bbopt | 2024/10/04 06:03:43 | INFO | - [obj]: x=[-3.0, 0.16000000000000014, 0.8799999999999999, 0.3599999999999999, -0.45999999999999996], ret=1.014e+01
amplify-bbopt | 2024/10/04 06:03:43 | INFO | y_hat=1.014e+01, best objective=4.462e+00
amplify-bbopt | 2024/10/04 06:03:43 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:03:43 | INFO | #2/20 optimization cycle, constraint wt: 6.23e+01
amplify-bbopt | 2024/10/04 06:03:43 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 06:03:50 | INFO | num_iterations: 20
amplify-bbopt | 2024/10/04 06:03:50 | INFO | - [obj]: x=[1.38, 0.16000000000000014, 0.26000000000000023, -3.0, 0.7000000000000002], ret=1.149e+01
amplify-bbopt | 2024/10/04 06:03:50 | INFO | y_hat=1.149e+01, best objective=4.462e+00
amplify-bbopt | 2024/10/04 06:03:50 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:03:50 | INFO | #3/20 optimization cycle, constraint wt: 6.23e+01
amplify-bbopt | 2024/10/04 06:03:50 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 06:03:55 | INFO | num_iterations: 25
amplify-bbopt | 2024/10/04 06:03:55 | INFO | - [obj]: x=[-3.0, -3.0, -1.2, -0.1200000000000001, -2.16], ret=2.412e+01
amplify-bbopt | 2024/10/04 06:03:55 | INFO | y_hat=2.412e+01, best objective=4.462e+00
amplify-bbopt | 2024/10/04 06:03:55 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:03:55 | INFO | #4/20 optimization cycle, constraint wt: 6.23e+01
amplify-bbopt | 2024/10/04 06:03:55 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 06:03:59 | INFO | num_iterations: 20
amplify-bbopt | 2024/10/04 06:03:59 | INFO | - [obj]: x=[1.0200000000000005, 0.8000000000000003, 0.8799999999999999, 0.3599999999999999, -0.45999999999999996], ret=2.796e+00
amplify-bbopt | 2024/10/04 06:03:59 | INFO | y_hat=2.796e+00, best objective=2.796e+00
amplify-bbopt | 2024/10/04 06:03:59 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:03:59 | INFO | #5/20 optimization cycle, constraint wt: 6.23e+01
amplify-bbopt | 2024/10/04 06:03:59 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 06:04:04 | INFO | num_iterations: 22
amplify-bbopt | 2024/10/04 06:04:04 | INFO | - [obj]: x=[1.0200000000000005, 0.3200000000000003, 0.8799999999999999, -1.0999999999999999, -0.45999999999999996], ret=3.339e+00
amplify-bbopt | 2024/10/04 06:04:04 | INFO | y_hat=3.339e+00, best objective=2.796e+00
amplify-bbopt | 2024/10/04 06:04:04 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:04:04 | INFO | #6/20 optimization cycle, constraint wt: 6.23e+01
amplify-bbopt | 2024/10/04 06:04:04 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 06:04:08 | INFO | num_iterations: 28
amplify-bbopt | 2024/10/04 06:04:08 | INFO | - [obj]: x=[1.0200000000000005, -3.0, 0.8799999999999999, -3.0, -0.45999999999999996], ret=2.003e+01
amplify-bbopt | 2024/10/04 06:04:08 | INFO | y_hat=2.003e+01, best objective=2.796e+00
amplify-bbopt | 2024/10/04 06:04:08 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:04:08 | INFO | #7/20 optimization cycle, constraint wt: 6.23e+01
amplify-bbopt | 2024/10/04 06:04:08 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 06:04:12 | INFO | num_iterations: 20
amplify-bbopt | 2024/10/04 06:04:12 | INFO | - [obj]: x=[0.2400000000000002, 0.8000000000000003, -1.32, -0.1200000000000001, -0.45999999999999996], ret=2.666e+00
amplify-bbopt | 2024/10/04 06:04:12 | INFO | y_hat=2.666e+00, best objective=2.666e+00
amplify-bbopt | 2024/10/04 06:04:12 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:04:12 | INFO | #8/20 optimization cycle, constraint wt: 6.23e+01
amplify-bbopt | 2024/10/04 06:04:13 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 06:04:16 | INFO | num_iterations: 23
amplify-bbopt | 2024/10/04 06:04:16 | INFO | - [obj]: x=[-3.0, 0.8000000000000003, 0.8799999999999999, -1.46, -3.0], ret=2.155e+01
amplify-bbopt | 2024/10/04 06:04:16 | INFO | y_hat=2.155e+01, best objective=2.666e+00
amplify-bbopt | 2024/10/04 06:04:16 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:04:16 | INFO | #9/20 optimization cycle, constraint wt: 6.23e+01
amplify-bbopt | 2024/10/04 06:04:16 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 06:04:21 | INFO | num_iterations: 21
amplify-bbopt | 2024/10/04 06:04:21 | INFO | - [obj]: x=[1.0200000000000005, 0.16000000000000014, -3.0, 0.3599999999999999, -0.45999999999999996], ret=1.041e+01
amplify-bbopt | 2024/10/04 06:04:21 | INFO | y_hat=1.041e+01, best objective=2.666e+00
amplify-bbopt | 2024/10/04 06:04:21 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:04:21 | INFO | #10/20 optimization cycle, constraint wt: 6.23e+01
amplify-bbopt | 2024/10/04 06:04:21 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 06:04:25 | INFO | num_iterations: 23
amplify-bbopt | 2024/10/04 06:04:25 | INFO | - [obj]: x=[0.2400000000000002, 0.3200000000000003, 0.8799999999999999, 0.3599999999999999, 0.7000000000000002], ret=1.554e+00
amplify-bbopt | 2024/10/04 06:04:25 | INFO | y_hat=1.554e+00, best objective=1.554e+00
amplify-bbopt | 2024/10/04 06:04:25 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:04:25 | INFO | #11/20 optimization cycle, constraint wt: 6.23e+01
amplify-bbopt | 2024/10/04 06:04:25 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 06:04:30 | INFO | num_iterations: 22
amplify-bbopt | 2024/10/04 06:04:30 | INFO | - [obj]: x=[2.66, 0.6200000000000001, 0.8799999999999999, -0.1200000000000001, 0.7000000000000002], ret=8.739e+00
amplify-bbopt | 2024/10/04 06:04:30 | INFO | y_hat=8.739e+00, best objective=1.554e+00
amplify-bbopt | 2024/10/04 06:04:30 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:04:30 | INFO | #12/20 optimization cycle, constraint wt: 6.23e+01
amplify-bbopt | 2024/10/04 06:04:30 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 06:04:34 | INFO | num_iterations: 20
amplify-bbopt | 2024/10/04 06:04:34 | INFO | - [obj]: x=[0.2400000000000002, 0.3200000000000003, -1.2, -1.0999999999999999, -0.45999999999999996], ret=3.022e+00
amplify-bbopt | 2024/10/04 06:04:34 | INFO | y_hat=3.022e+00, best objective=1.554e+00
amplify-bbopt | 2024/10/04 06:04:34 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:04:34 | INFO | #13/20 optimization cycle, constraint wt: 6.23e+01
amplify-bbopt | 2024/10/04 06:04:34 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 06:04:38 | INFO | num_iterations: 21
amplify-bbopt | 2024/10/04 06:04:38 | INFO | - [obj]: x=[0.2400000000000002, 0.3200000000000003, 0.8799999999999999, 0.3599999999999999, 0.3799999999999999], ret=1.208e+00
amplify-bbopt | 2024/10/04 06:04:38 | INFO | y_hat=1.208e+00, best objective=1.208e+00
amplify-bbopt | 2024/10/04 06:04:38 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:04:38 | INFO | #14/20 optimization cycle, constraint wt: 6.23e+01
amplify-bbopt | 2024/10/04 06:04:38 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 06:04:43 | INFO | num_iterations: 20
amplify-bbopt | 2024/10/04 06:04:43 | INFO | modifying solution (11, is_frequent=False), {'x': [0.2400000000000002, 0.3200000000000003, 0.8799999999999999, 0.3599999999999999, 0.3799999999999999]} --> {'x': [1.58, -0.6999999999999997, -0.23999999999999977, 3.0, 1.8399999999999999]}.
amplify-bbopt | 2024/10/04 06:04:43 | INFO | - [obj]: x=[1.58, -0.6999999999999997, -0.23999999999999977, 3.0, 1.8399999999999999], ret=1.543e+01
amplify-bbopt | 2024/10/04 06:04:43 | INFO | y_hat=1.543e+01, best objective=1.208e+00
amplify-bbopt | 2024/10/04 06:04:43 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:04:43 | INFO | #15/20 optimization cycle, constraint wt: 6.23e+01
amplify-bbopt | 2024/10/04 06:04:43 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 06:04:47 | INFO | num_iterations: 21
amplify-bbopt | 2024/10/04 06:04:47 | INFO | modifying solution (1, is_frequent=False), {'x': [0.2400000000000002, 0.3200000000000003, 0.8799999999999999, 0.3599999999999999, 0.3799999999999999]} --> {'x': [2.9000000000000004, -0.7199999999999998, 1.12, 2.7, 0.8999999999999999]}.
amplify-bbopt | 2024/10/04 06:04:47 | INFO | - [obj]: x=[2.9000000000000004, -0.7199999999999998, 1.12, 2.7, 0.8999999999999999], ret=1.828e+01
amplify-bbopt | 2024/10/04 06:04:47 | INFO | y_hat=1.828e+01, best objective=1.208e+00
amplify-bbopt | 2024/10/04 06:04:47 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:04:47 | INFO | #16/20 optimization cycle, constraint wt: 6.23e+01
amplify-bbopt | 2024/10/04 06:04:47 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 06:04:51 | INFO | num_iterations: 20
amplify-bbopt | 2024/10/04 06:04:51 | INFO | modifying solution (1, is_frequent=False), {'x': [0.2400000000000002, 0.3200000000000003, 0.8799999999999999, 0.3599999999999999, 0.3799999999999999]} --> {'x': [2.04, 1.1399999999999997, 1.2199999999999998, -0.6600000000000001, 2.26]}.
amplify-bbopt | 2024/10/04 06:04:51 | INFO | - [obj]: x=[2.04, 1.1399999999999997, 1.2199999999999998, -0.6600000000000001, 2.26], ret=1.249e+01
amplify-bbopt | 2024/10/04 06:04:51 | INFO | y_hat=1.249e+01, best objective=1.208e+00
amplify-bbopt | 2024/10/04 06:04:51 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:04:51 | INFO | #17/20 optimization cycle, constraint wt: 6.23e+01
amplify-bbopt | 2024/10/04 06:04:52 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 06:04:56 | INFO | num_iterations: 21
amplify-bbopt | 2024/10/04 06:04:56 | INFO | modifying solution (1, is_frequent=False), {'x': [0.2400000000000002, 0.3200000000000003, 0.8799999999999999, 0.3599999999999999, 0.3799999999999999]} --> {'x': [-2.2, 0.48, 1.3399999999999999, 2.08, 0.16000000000000014]}.
amplify-bbopt | 2024/10/04 06:04:56 | INFO | - [obj]: x=[-2.2, 0.48, 1.3399999999999999, 2.08, 0.16000000000000014], ret=1.122e+01
amplify-bbopt | 2024/10/04 06:04:56 | INFO | y_hat=1.122e+01, best objective=1.208e+00
amplify-bbopt | 2024/10/04 06:04:56 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:04:56 | INFO | #18/20 optimization cycle, constraint wt: 6.23e+01
amplify-bbopt | 2024/10/04 06:04:56 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 06:05:04 | INFO | num_iterations: 24
amplify-bbopt | 2024/10/04 06:05:04 | INFO | modifying solution (1, is_frequent=True), {'x': [0.2400000000000002, 0.3200000000000003, 0.8799999999999999, 0.3599999999999999, 0.3799999999999999]} --> {'x': [0.2400000000000002, 0.33999999999999986, 0.8799999999999999, 0.3599999999999999, 0.3799999999999999]}.
amplify-bbopt | 2024/10/04 06:05:04 | INFO | - [obj]: x=[0.2400000000000002, 0.33999999999999986, 0.8799999999999999, 0.3599999999999999, 0.3799999999999999], ret=1.222e+00
amplify-bbopt | 2024/10/04 06:05:04 | INFO | y_hat=1.222e+00, best objective=1.208e+00
amplify-bbopt | 2024/10/04 06:05:04 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:05:04 | INFO | #19/20 optimization cycle, constraint wt: 6.23e+01
amplify-bbopt | 2024/10/04 06:05:04 | INFO | model corrcoef: 1.000, beta: 0.0
amplify-bbopt | 2024/10/04 06:05:08 | INFO | num_iterations: 22
amplify-bbopt | 2024/10/04 06:05:08 | INFO | - [obj]: x=[-3.0, -3.0, -3.0, -3.0, -3.0], ret=4.500e+01
amplify-bbopt | 2024/10/04 06:05:08 | INFO | y_hat=4.500e+01, best objective=1.208e+00
amplify-bbopt | 2024/10/04 06:05:08 | INFO | ----------------------------------------
amplify-bbopt | 2024/10/04 06:05:08 | INFO | #20/20 optimization cycle, constraint wt: 9.00e+01
amplify-bbopt | 2024/10/04 06:05:08 | INFO | model corrcoef: 0.646, beta: 0.0
amplify-bbopt | 2024/10/04 06:05:13 | INFO | num_iterations: 19
amplify-bbopt | 2024/10/04 06:05:13 | INFO | modifying solution (1, is_frequent=True), {'x': [0.2400000000000002, 0.3200000000000003, 0.8799999999999999, 0.3599999999999999, 0.3799999999999999]} --> {'x': [0.2400000000000002, 0.3200000000000003, 0.8799999999999999, 0.3599999999999999, 0.3999999999999999]}.
amplify-bbopt | 2024/10/04 06:05:13 | INFO | - [obj]: x=[0.2400000000000002, 0.3200000000000003, 0.8799999999999999, 0.3599999999999999, 0.3999999999999999], ret=1.224e+00
amplify-bbopt | 2024/10/04 06:05:13 | INFO | y_hat=1.224e+00, best objective=1.208e+00
Result¶
# Solution (optimal input to the black-box objective function)
print(f"{optimizer.best_solution=}")
# Corresponding objective function value
print(f"{optimizer.best_objective=:.3e}")
# Optimization history
plot_history(optimizer.fetch_history(), log_scale=True)
optimizer.best_solution={'x': [0.2400000000000002, 0.3200000000000003, 0.8799999999999999, 0.3599999999999999, 0.3799999999999999]}
optimizer.best_objective=1.208e+00