Evaluation of Execution Results¶
The execution result returned by the solve()
function contains various information about the solutions, model conversion, and execution time.
This page explains how to obtain and use this information.
The following is an example of obtaining the optimization result for a model consisting of an objective function and a constraint using the FixstarsClient
.
from datetime import timedelta
from amplify import VariableGenerator, equal_to, FixstarsClient, solve
# Create an array of decision variables
gen = VariableGenerator()
q = gen.array("Binary", 5)
# Create an objective function and a constraint
objective = q[0] * q[1]  q[2]
constraint = equal_to(q[0] + q[1] + q[2], 1)
# Define a model
model = objective + constraint
# Create a solver client
client = FixstarsClient()
# client.token = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
client.parameters.timeout = timedelta(milliseconds=1000)
# Obtaining the result of the run
result = solve(model, client)
The Result class¶
The following explains how to obtain information about solutions, model conversion, and execution time from an instance of the Result
class returned by the solve()
function.
Information on the solutions¶
The solutions
attribute of the Result
class stores the solutions to the input model. The solutions
attribute is an instance of the SolutionList
class, which behaves like a list with the Solution
class as an element.
>>> type(result.solutions[0])
<class 'amplify.Result.Solution'>
The Solution
class represents a solution and has the following attributes.
Attribute name 
Type 
Summary 

Objective function value. 

Values of variables in the solution. 

Whether the solution meet the constraints. 

The timestamp at which the solution is obtained. 
The Values
class, which is the type of the values
attribute, is a class that represents solution values and acts like a dictionary with variables as keys and solution values as values.
>>> solution = result.solutions[0]
>>> solution.values
Values({Poly(q_0): 0, Poly(q_1): 0, Poly(q_2): 1})
>>> solution.values[q[0]]
0.0
>>> solution.values[q[2]]
1.0
You can also use the evaluate()
methods of the variable array and polynomial classes and the is_satisfied()
method of the constraint class to assign the solution values represented by the {py: class}~amplify.Values
class to the variable array, polynomial or constraint conditions. See decisionvariableevaluation
, polynomialevaluation
, and constraintevaluation
for more details.
The Result
class provides several shortcuts to accessing the solution besides the solutions
attribute. First, the best
attribute of the Result
class provides the best solution. Accessing the result directly from the Result
class by index is also possible.
>>> result.best.values
Values({Poly(q_0): 0, Poly(q_1): 0, Poly(q_2): 1})
>>> len(result)
1
>>> result[0].values
Values({Poly(q_0): 0, Poly(q_1): 0, Poly(q_2): 1})
Information on the model conversion¶
The following attributes can obtain information about the model conversion. See Model Conversions for details.
The information on the intermediate model 

The information on the graph embedding 
Information on the response of the solver¶
The solve
function calls the solve(...)
method of a solver client to run the solver after the model conversion and graph embedding have been performed. The object returned by the solver client can be obtained from the client_result
attribute.
>>> type(result.client_result)
<class 'amplify.FixstarsClient.Result'>
The type of the client_result
depends on the type of the solver client. See Client details and its subpages for more information about the result type of each solver client.
Information on the execution time¶
You can obtain information about various execution times through the following attributes. See Execution Time information for more information.
Time taken to 

Time between sending a request to the solver and receiving a response 

Time spent by the solver in optimization 
Decision variable evaluation¶
The solution values can be assigned to a variable array used in the formulation. The assigned result is returned as a NumPy array with the same shape as the variable array.
The following performs the assignment to evaluate a variable array. The values
attribute of the Solution
object is passed to the evaluate()
method of the PolyArray
class.
Tip
The solution in the result can be obtained by retrieving the best result of the run with the best
attribute or by accessing the elements in the same way as for a list.
>>> print(result.best.values)
{q_0: 0, q_1: 0, q_2: 1}
>>> q_values = q.evaluate(result.best.values)
>>> print(q_values)
[0. 0. 1. 0. 0.]
If a variable not used in the formulation is included, as in q[3]
or q[4]
above, it is assigned one of its possible values by default. In the above example, the Amplify SDK assigns 0
as the default value for the binary variable.
The default
keyword argument to the evaluate()
method can change the value used if no evaluation is performed; if the default
keyword argument is given as a number, variables not passed to the solver are assigned that value.
>>> q_values = q.evaluate(result.best.values, default=3)
Warning: Substituting variable q_3 with 3 is out of bounds.
Warning: Substituting variable q_4 with 3 is out of bounds.
>>> print(q_values)
[0. 0. 1. 3. 3.]
Note
A warning is printed if a value outside the bounds of the variable is given, such as default=3
, as shown above.
If the default
keyword argument is None
, variables not passed to the solver remain as they are. Only in this case the evaluate()
method return a PolyArray
.
>>> q_values = q.evaluate(result.best.values, default=None)
>>> print(q_values)
[ 0, 0, 1, q_3, q_4]
Attention
Some variables may not be passed to the solver even if they are included in the model. This is because model conversions such as penalty function generation or graph embedding can cause terms to cancel each other out.
Polynomial evaluation¶
The result of evaluating the objective function of the input model with the solution returned by the solve()
function can be obtained using the objective
attribute of the Solution
object.
>>> solution = result.best
>>> solution.objective
1.0
On the other hand, there are cases where you want to evaluate a polynomial other than the objective function with the solution returned by the solver, for example, when the objective function is expressed as the sum of several polynomials. In this case, we pass the values
attribute of the Solution
object to Poly
’s evaluate()
method, just as we would evaluate an array of variables.
>>> objective_1 = q[0] * q[1] # The first term of the objective function
>>> objective_1.evaluate(solution.values)
0.0
Hint
The behavior when the polynomial contains variables not passed to the solver is similar to the evaluate()
method of the PolyArray
class; the default
keyword argument can change this behavior.
Constraint evaluation¶
If you want to know if the resulting solution satisfies the constraints, you can check with the feasible
attribute of the Solution
class.
>>> solution.feasible
True
By default, this will always be True
because the solve()
function retrieves only solutions that satisfy all constraints in the model. To change this behavior and allow the solver to retrieve solutions that do not satisfy the constraints, pass a bool
to the filter_solution
keyword argument of the solve()
function upfront or set filter_solution
of the Result
class to False
afterward.
Let’s test this by adding a constraint to the model that cannot be satisfied inconsistently.
# Create an objective function and constraints
objective = q[0] * q[1]  q[2]
constraint1 = equal_to(q[0] + q[1] + q[2], 1, label="sum equals one")
constraint2 = equal_to(q[0] + q[1] + q[2], 2, label="sum equals two")
# Define a model (with conflicting constraints)
model = objective + constraint1 + constraint2
# Get the result of the run
result = solve(model, client)
You cannot retrieve the solution from Result
if the solution filter is enabled.
>>> result.best.feasible
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
RuntimeError: result has no feasible solution
Turning off the solution filter allows us to obtain solutions that do not satisfy the constraints.
>>> result.filter_solution = False
>>> result.best.feasible
False
Suppose you obtained a solution that does not satisfy a constraint for some reason, such as model configuration, penalty function weights, or solver settings. In that case, it may be desirable to determine which constraint was not satisfied.
You can check whether the constraint conditions are satisfied by passing the solution returned by the solve()
function to the is_satisfied()
method of the Constraint
class.
>>> constraint1.is_satisfied(result.best.values)
True
>>> constraint2.is_satisfied(result.best.values)
False
In the example above, we see that constraint2
could not be satisfied.
We can also mechanically identify constraints from the list of constraints in the model that are not satisfied, as follows. This method is useful when adjusting and rerunning the penalty function weights.
>>> list(c for c in model.constraints if not c.is_satisfied(result.best.values))
[Constraint({conditional: q_0 + q_1 + q_2 == 2, weight: 1, label: "sum equals two"})]