Performance of Amplify AE¶
To evaluate the performance of Amplify AE, we are obtaining benchmark results from two perspectives: (1) solution performance and (2) annealing speed.
Solution Performance Benchmark¶
We obtained the objective function values at different execution times using publicly available benchmark problems and real-world optimization problems as our benchmark set. For comparison, we also obtained benchmarks for two types of Ising machines from other companies. For these solvers, we tested multiple configurable parameter settings and present the case that yielded the best results.
Since the internal algorithms and solution performance of Amplify AE are not dependent on the execution time (time_limit_ms), we plot multiple solutions (elapsed time and objective function value) included in the response of a single execution.
On the other hand, for the solvers from other companies, we only obtained the best result per execution, so the data with different execution times are independent.
Each data point displays the median value from 11 trials. If a feasible solution could not be obtained within the specified execution time, that data point is not plotted.
Execution Environment¶
Amplify SDK v1.4.0
Amplify AE v1.0.0
1 NVIDIA Hopper
Constraint mode
Benchmark Problems¶
The following is a summary of the problems used for benchmarking. We have selected several benchmark problems with objective functions of degree 2 or higher, as well as operational challenges from existing companies.
Problem Set |
Number of Variables |
Objective Function |
Equality Constraints (Degree/Number) |
Inequality Constraints |
|---|---|---|---|---|
300 |
Quadratic |
None |
Up to linear/1 |
|
5,100 |
Quadratic |
Up to linear/160 |
Up to linear/5 |
|
Production Planning Problem |
7,400 |
Quadratic |
Up to linear/933 |
None |
Staff Scheduling Problem |
730 |
Quadratic |
Up to quadratic/365 |
Up to linear/720 |
Benchmark Results (Solving Performance)¶
2-Dimensional Quadratic Knapsack Problem (QKP, jeu_300_50_1)¶
The knapsack problem is to find the selection of items that maximizes the total value, given the value and weight of items, such that the total weight does not exceed a specific limit. The quadratic knapsack problem (QKP) is a more challenging problem that considers the additional value generated by pairs of items.
Solver Name |
Time to Obtain Feasible Solution[1] |
Time to Obtain Optimal Solution |
|---|---|---|
Amplify AE |
0.00176 s 🏆 |
0.233 s 🏆 |
Company A Solver |
3.09 s |
4.08 s |
Company B Solver |
0.12 s |
60.2 s |
Capacitated Vehicle Routing Problem (CVRP, E-n51-k5)¶
The Capacitated Vehicle Routing Problem (CVRP) is a problem of selecting the routes with the minimum distance or transportation cost among those that deliver goods to \(n\) destinations using \(k\) vehicles with limited carrying capacity.
Solver Name |
Time to Obtain Feasible Solution |
Objective Function Value of Best Solution[2] |
|---|---|---|
Amplify AE |
0.0780 s 🏆 |
589 🏆 |
Company A Solver |
1.12 s |
653 |
Company B Solver |
(Could not obtain feasible solution) |
(Could not obtain feasible solution) |
Production Planning Problem¶
We solved the business challenge of minimizing production costs in a factory’s production schedule using Amplify AE.
Staff Scheduling Problem¶
We solved the business challenge of creating the most efficient worker shift schedule under various constraints with Amplify AE.
Solver Name |
Time to Obtain Feasible Solution |
Objective Function Value of Best Solution [2] |
|---|---|---|
Amplify AE |
0.026 s 🏆 |
23.7 🏆 |
Company A Solver |
21.0 s |
24.5 |
Company B Solver |
(Could not obtain feasible solution) |
(Could not obtain feasible solution) |
Annealing Speed Benchmark¶
In GPU annealing, a higher number of flips per second increases the number of combinations that can be explored, thus improving the chances of finding a better solution within a given time limit.
Amplify’s cloud service offers higher-performance GPUs for its higher-tier plans. Therefore, we defined “annealing speed” as the number of variable flips per second and measured it in relation to GPU performance and the number of GPUs provided for each plan. The benchmarks used a set of problem sizes: small, medium, and large.
For information on the types of GPUs provided in each plan, please refer to the Pricing Page.
GPU Name |
Offered Plans |
|---|---|
NVIDIA Volta |
Basic / Standard |
NVIDIA Ampere |
Premium |
NVIDIA Hopper |
S Premium / Enterprise |
Execution Environment¶
Amplify SDK v1.4.0
Amplify AE 1.0.0 Constraint Mode
Number of GPUs: 1 - 4 (2 or more only for NVIDIA Hopper)
Benchmark Results (Annealing Speed)¶
We plotted the annealing speed as a ratio to the Basic plan and as absolute values. The differences between plans become more pronounced as the problem size increases.