---
file_format: mystnb
kernelspec:
  name: python3
mystnb:
  execution_mode: 'inline'
---

# Execution Time information

The Amplify SDK provides an interface to obtain information of various execution times when solving combinatorial optimization problems.

```{image} ../_images/timing_light.drawio.svg
:width: 70%
:align: center
:class: only-light
```

```{image} ../_images/timing_dark.drawio.svg
:width: 70%
:align: center
:class: only-dark
```

## Types of available time information

The following table shows the types of execution time information that you can obtain as {py:class}`datetime.timedelta` objects.

```{list-table}
* - {py:attr}`amplify.Result.total_time`
  - The total time taken by {py:func}`amplify.solve`.
* - {py:attr}`amplify.Result.response_time`
  - The time between sending the query to the solver and receiving the response.
* - {py:attr}`amplify.Result.execution_time`
  - The time the solver used to solve the problem.
* - {py:attr}`amplify.Result.Solution.time`
  - For each solution the solver returns, this is the time from the start of the solver's search until the solver obtains the solution.
```

## Example of obtaining execution time

An example of obtaining execution time is shown below. First, we execute the {py:func}`~amplify.solve` function to obtain a {py:class}`~amplify.Result` object.

```{testcode}
from amplify import VariableGenerator, one_hot, FixstarsClient, solve
from datetime import timedelta

gen = VariableGenerator()
q = gen.array("Binary", 3)

objective = q[0] * q[1] - q[2]
constraint = one_hot(q)

model = objective + constraint

client = FixstarsClient()
# client.token = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
client.parameters.timeout = timedelta(milliseconds=1000)

result = solve(model, client)
```

The {py:attr}`~amplify.Result.total_time` attribute of {py:class}`~amplify.Result` represents the time from the start to the end of the {py:func}`~amplify.solve` function. This time includes the response time of the solver, plus the processing time taken by the Amplify SDK to perform various conversions and create the query data to pass to the solver.

```{doctest}
>>> result.total_time # doctest: +SKIP
datetime.timedelta(seconds=1, microseconds=870877)
```

The {py:attr}`~amplify.Result.response_time` attribute of {py:class}`~amplify.Result` represents the time between sending the request to the solver in the {py:func}`~amplify.solve` function and receiving the response.

```{doctest}
>>> result.response_time # doctest: +SKIP
datetime.timedelta(seconds=1, microseconds=865211)
```

The {py:attr}`~amplify.Result.execution_time` attribute of {py:class}`~amplify.Result` represents the time taken by the solver to find the solution. This time is usually the value contained in the response from the solver.

```{doctest}
>>> result.execution_time # doctest: +SKIP
datetime.timedelta(microseconds=980543)
```

For each solution ({py:class}`~amplify.Result.Solution`) in the {py:class}`~amplify.Result`, the time between starting the solver search and finding the solution can be retrieved using the {py:attr}`~amplify.Result.Solution.time` attribute. This is usually the value contained in the solver response, but if the solver does not return such information, it will be the same value as the {py:attr}`~amplify.Result.execution_time` attribute.

```{doctest}
>>> result.best.time # doctest: +SKIP
datetime.timedelta(microseconds=27925)
```

When the {py:func}`~amplify.solve` function is run with the `dry_run` option, {py:attr}`~amplify.Result.total_time` is approximately equal to the time it took the Amplify SDK to convert the model and generate the query data. Also, {py:attr}`~amplify.Result.response_time` and {py:attr}`~amplify.Result.execution_time` will be zero.

```{doctest}
>>> dry_run_result = solve(model, client, dry_run=True)
>>> dry_run_result.total_time # doctest: +SKIP
datetime.timedelta(microseconds=59)
```

## Plotting the time obtaining the solution

For a solver configured to return multiple solutions, we will show you how to plot the time the solver obtained each solution. Such a plot allows us to see the solver's solutions at a given time during the solver execution. You can use the convergence of the solution to determine if the run time is too short or too long.

For example, the following figure shows how Amplify Annealing Engine (AE) updates the solution objective values for a 50-city random traveling salesperson problem.

The first step is to create the model. See "[](tsp.ipynb)" for details on the formulation.

```{code-cell}
---
tags: [remove-input, remove-output]
---
import numpy as np
from amplify import VariableGenerator, einsum, one_hot

N = 50
np.random.seed(12345)
x = np.random.rand(N)
y = np.random.rand(N)
d = (
    (x[:, np.newaxis] - x[np.newaxis, :]) ** 2
    + (y[:, np.newaxis] - y[np.newaxis, :]) ** 2
) ** 0.5

gen = VariableGenerator()
q = gen.array("Binary", N + 1, N)
q[-1, :] = q[0, :]

objective = einsum("ij,ki,kj->", d, q[:-1], q[1:])
constraints = one_hot(q[:-1], axis=1) + one_hot(q[:-1], axis=0)

model = objective + 0.5 * d.max() * constraints
```

```{testcode}
import numpy as np
from amplify import VariableGenerator, einsum, one_hot

N = 50
x = np.random.rand(N)
y = np.random.rand(N)
d = (
    (x[:, np.newaxis] - x[np.newaxis, :]) ** 2
    + (y[:, np.newaxis] - y[np.newaxis, :]) ** 2
) ** 0.5

gen = VariableGenerator()
q = gen.array("Binary", N + 1, N)
q[-1, :] = q[0, :]

objective = einsum("ij,ki,kj->", d, q[:-1], q[1:])
constraints = one_hot(q[:-1], axis=1) + one_hot(q[:-1], axis=0)

model = objective + d.max() * constraints
```

Now, we set the solver client. By default, Amplify AE returns only the best solution found, so we configure it to return all solutions found during the search.

```{code-cell}
---
tags: [remove-input, remove-output]
---
from amplify import FixstarsClient

client = FixstarsClient()
client.parameters.timeout = 1000
client.parameters.outputs.num_outputs = 0
```

```{testcode}
from amplify import FixstarsClient

client = FixstarsClient()
client.parameters.timeout = 1000  # Timeout is 1000 ms
client.parameters.outputs.num_outputs = 0  # Return all solutions found
```

Run the solver and plot the times and objective function values of all solutions obtained.

```{code-cell}
---
tags: [remove-input, remove-output]
---
from amplify import solve
import matplotlib.pyplot as plt

while True:
  result = solve(model, client)
  if result.best.objective <= 5.55:
    break

fig = plt.figure(1)

times = [solution.time.total_seconds() for solution in result]
objective_values = [solution.objective for solution in result]

plt.scatter(times, objective_values)
plt.plot(times, objective_values)
plt.xlabel("elapsed time in seconds")
plt.ylabel("objective value")
plt.grid(True)
```

```{testcode}
from amplify import solve
import matplotlib.pyplot as plt

# Run the solver
result = solve(model, client)

# Get time and objective function values for each solution
times = [solution.time.total_seconds() for solution in result]
objective_values = [solution.objective for solution in result]

# Plot
plt.scatter(times, objective_values)
plt.plot(times, objective_values)
plt.xlabel("elapsed time in seconds")
plt.ylabel("objective value")
plt.grid(True)
```

```{eval:figure} fig
:width: 50%
History of solutions obtained by Amplify AE for the 50-city traveling salesperson problem
```

You can retrieve the initial and best solutions of the search and plot against each other as follows.

```{code-cell}
---
tags: [remove-input, remove-output]
---
def tsp_plot(q_values, x, y):
    route_x = q_values @ x
    route_y = q_values @ y

    fig = plt.figure()
    plt.scatter(x, y)
    plt.plot(route_x, route_y)
    return fig

# Plot of the initial solution
fig_initial = tsp_plot(q.evaluate(result[-1].values), x, y)
init_sec = round(result[-1].time.total_seconds(), 2)
# Plot of the best solution
fig_last = tsp_plot(q.evaluate(result[0].values), x, y)
last_sec = round(result[0].time.total_seconds(), 2)
```

```{testcode}
def tsp_plot(q_values, x, y):
    route_x = q_values @ x
    route_y = q_values @ y

    plt.scatter(x, y)
    plt.plot(route_x, route_y)
    plt.show()

# Plot of the initial solution
tsp_plot(q.evaluate(result[-1].values), x, y)
# Plot of the best solution
tsp_plot(q.evaluate(result[0].values), x, y)
```

`````{grid} 1 1 2 2
````{grid-item}
```{eval:figure} fig_initial
Path ({eval}`init_sec` seconds) of the initial solution
```
````
````{grid-item}
```{eval:figure} fig_last
Path ({eval}`last_sec` seconds) of the best solution
```
````
`````