ConQuER · Controlling Quantum Experiments with Reinforcement Learning

Controlling Quantum Experiments with Reinforcement Learning

Artificially intelligent systems are integrated with various technologies we use on a near-daily basis. Whether it is in route-planning or in recommendation systems for ads or TV shows, the underlying algorithm likely uses some form of “learning from data”. These types of learning algorithms are likely going to be ever more present in future technologies. Relevant for this project in particular is the future of quantum technologies. Quantum computers are the prime example of such quantum technology, in which a fundamentally new way of performing computations is done that relies on the quantum mechanical phenomena. This promises to outperform classical computers in certain tasks, speeding up algorithms and thereby enabling computations that are infeasible on our current classical hardware. Building a quantum computer requires exquisite control over its building blocks: quantum bits. These quantum analogs of classical bits are difficult to control, because the mere act of 'looking at them' (measuring them) destroys the quantum information stored in them. Quantum systems are very susceptible to noise, causing them to decohere. Once fully incoherent, the quantum mechanical nature of the system is lost. This project investigated the use of artificial intelligence (using "reinforcement learning" (RL)) for controlling quantum systems, directly integrating it with experiments, to try and preserve the coherence of a quantum system. In reinforcement learning approaches to controlling quantum systems, an AI needs to learn values of control parameters (e.g. a magnetic field). For example, the agent must learn to say: "For the next period, set the magnetic field to value X". Or, "For the next second, set this voltage to value Y". This project aimed at testing a different type of control, in which the agent says: "Next, add amount B to the magnetic field", and "Now, keep the magnetic field at this value for the next second". The objectives for this project were to: * Introducing a novel reinforcement learning agent, training it on a quantum memory problem. * Implementing a noisy variant of the quantum memory problem, and investigating how the agent learns to perform in such a scenario. * The explicit integration of a reinforcement learning algorithm with ongoing spin-qubit experiments at Copenhagen university. The conclusion for this project is twofold. An agent that is replaced by an optimization algorithm (called "CMA-ES") works just as good. This was initially meant as a benchmark problem, though it seems that this method by itself may be enhanced still. On the other hand, the integration with experiments is still ongoing and will be feasible in the near future. The integration of an optimization routine directly with the experiment was completed, and what is left now is to swap the CMA-ES with RL.

Data: CORDIS, © European Union