Inho Choi

This review is written by non-expert and used for personal study only.

Creation of an ultracold gas of triatomic molecules from an atom-diatomic molecule mixture

  • Ultracold Chemistry

Ultracold triatomic molecules present exciting new research possibilities, including the study of complex quantum interactions involving three particles and the behavior of large molecules at extremely low temperatures. However, creating and controlling these ultracold triatomic molecular gases is a highly challenging task.

There are two main techniques for producing ultracold molecules: direct cooling, which involves lowering the temperature of the molecules, and forming molecules from ultracold atomic gases, which are easier to cool. While crafting triatomic molecules from ultracold atom-diatomic molecule mixtures is difficult due to their complex nature, recent discoveries of resonances hint at the possibility of creating them.

Scientists were able to produce a weakly-bound ultracold gas of $^{23}Na^{40}K_2$ triatomic molecules by mixing $^{23}Na^{40}K$ ground-state molecules and $^{40}K$ atoms. They created these triatomic molecules by carefully adjusting the magnetic field to cause a Feshbach resonance, which is a phenomenon that occurs when two particles (in this case, $^{23}Na^{40}K$ molecules and $^{40}K$ atoms) interact at a specific energy level. The scientists then used radio-frequency (rf) dissociation, a technique that involves breaking apart the molecules using electromagnetic radiation, to directly detect the triatomic molecules. They found that the peak density of the triatomic molecular gas was much higher than previously reported for cold polyatomic molecules.

To better understand the Feshbach resonance, the researchers measured the binding energies of the triatomic molecules using the rf loss spectrum, which provides information about the energy levels at which molecules are lost from the system. They also studied the process of adiabatic magneto-association, which involves gradually changing the magnetic field to form triatomic molecules, by moving the magnetic field from a position above the resonance to the side associated with triatomic molecules.

The researchers observed a sharp decrease in the number of $^{23}Na^{40}K$ molecules when they adjusted the magnetic field across the resonance. This extra loss might be due to the formation of triatomic molecules. To obtain clear evidence for the creation of triatomic molecules, they probed the molecules by dissociating them into free diatomic molecules and atoms. They discovered that the short lifetime of the triatomic molecules is mainly limited by three factors: predissociation (spontaneous breaking apart of the molecules), inelastic collisions (interactions that cause energy loss) with atoms and diatomic molecules, and the excitation of molecules by the trap laser, which can cause them to break apart.

The researchers successfully created ultracold triatomic molecules from a mixture of $NaK$ molecules and $K$ atoms using a technique called adiabatic magneto-association. These triatomic molecules were in an excited vibrational state, which means that their internal motion can be controlled and studied using lasers. Further improvements in this area could lead to the creation of new quantum states of matter, such as Bose-Einstein condensates, which are formed when particles reach extremely low temperatures and begin to behave collectively.

Required Additional Study Materials

  • L. D. Carr, D. DeMille, R. V. Krems, J. Ye, Cold and ultracold molecules: Science, technology and applications. New J. Phys. 11, 055049 (2009).
  • G. Quéméner, P. S. Julienne, Ultracold molecules under control! Chem. Rev. 112, 4949–5011 (2012).
  • C. Chin, R. Grimm, P. Julienne, E. Tiesinga, Feshbach resonances in ultracold gases. Rev. Mod. Phys. 82, 1225–1286 (2010).
  • T. Köhler, K. Góral, P. S. Julienne, Production of cold molecules via magnetically tunable Feshbach resonances. Rev. Mod. Phys. 78, 1311–1361 (2006).
Introductory material
  • “An Introduction to Cold and Ultracold Chemistry: Atoms, Molecules, Ions and Rydbergs” by Jesús Pérez Ríos.

Reference

Yang, H. et al. Creation of an ultracold gas of triatomic molecules from an atom-diatomic molecule mixture. Science 378, 1009-1013 (2023).

Exceptional fracture toughness of CrCoNi-based medium- and high-entropy alloys at 20 kelvin

  • Metallurgy

High-entropy alloys (HEAs) are a unique type of metallic materials that have multiple primary elements in their composition. Equiatomic $CrCoNi$-based alloys, which include the $CrMnFeCoNi$ alloy and the $CrCoNi$ medium-entropy alloy (MEA), are known for their exceptional strength and toughness. These properties make them ideal for potential applications in extreme environments, such as those with very high strain rates and cryogenic temperatures (extremely low temperatures).

Researchers studied the mechanical properties of $CrCoNi$ and $CrMnFeCoNi$ alloys at low temperatures by performing uniaxial tensile tests and nonlinear elastic J-based fracture toughness tests. Fracture toughness is a measure of a material’s resistance to crack propagation. The $CrCoNi$ alloy displayed one of the highest fracture toughness values ever reported, particularly at $20 K (-253°C)$.

In addition to measuring fracture toughness, the researchers used in situ neutron diffraction measurements and postfracture electron backscatter diffraction analysis to investigate how the material deforms plastically and how defects in the material behave. Interestingly, the exceptional fracture resistance of these alloys increased as the temperature decreased, which is not a common trait among metallic materials.

Both $CrMnFeCoNi$ and $CrCoNi$ alloys showed improved fracture toughness with decreasing temperature, exhibiting exceptionally high values at $20 K$. At this temperature, the fracture surfaces displayed no signs of brittle fracture features, meaning the material failed entirely due to ductile mechanisms, such as the formation and coalescence of microvoids.

Microvoid

Microvoids are small, empty spaces or cavities that can form within a material, typically as a result of defects or impurities during manufacturing or processing. In the context of material science and engineering, microvoids can influence a material’s mechanical properties, such as strength, ductility, and toughness. The presence of microvoids can lead to a reduction in material performance and, in some cases, premature failure.

In ductile materials, microvoids play a crucial role in the fracture process. When a material is subjected to stress, microvoids can form around inclusions, second-phase particles, or other defects within the material. As the stress increases, these microvoids grow in size and eventually coalesce with neighboring microvoids, creating larger voids. This process continues until a crack forms, and the material ultimately fails due to ductile fracture. This type of fracture is characterized by a significant amount of plastic deformation before failure and is often referred to as “microvoid coalescence.”

The remarkable fracture resistance of these single-phase solid solution alloys at cryogenic temperatures can be attributed to the cooperative behavior of defects in the material, responsible for plastic deformation. This cooperation leads to a complex mixture of phases and defect structures in the microstructure.

The researchers examined the microscopic deformation mechanisms in the $CrCoNi$ alloy using fracture toughness tests and electron backscatter diffraction (EBSD). They observed deformation-induced twinning (the formation of mirror-image crystal structures) in highly deformed grains under high-stress conditions near the crack tip. High-resolution transmission electron microscopy (HRTEM) imaging revealed planar deformation features, such as nanotwins and stacking faults, at room temperature. At $20 K$, they observed deformation bands full of stacking faults and laths (elongated structures) of the hexagonal close-packed (hcp) phase. The change in deformation modes at 20 K is primarily responsible for the outstanding fracture toughness.

Stacking-fault energies, which influence the formation of defects, decreased at lower temperatures for both alloys. This finding is consistent with theoretical predictions and suggests that multiple alloying elements may be necessary to activate all relevant mechanisms in these alloys.

At low deformation temperatures, dislocation motion and twin growth are limited, but flow stress (resistance to deformation) and the formation of twins and hcp phases increase at $20 K$. Deformation mechanisms like dislocation glide, stacking fault formation, nanotwinning, and phase transformation cause strain hardening (the material becomes stronger as it deforms), which enhances toughness and delays necking (localized thinning). Strain hardening intensifies at lower temperatures due to lower stacking-fault energy, promotion of nanotwinning, and the onset of deformation-induced transformation to the hcp phase.

$CrCoNi$-based MEAs and HEAs exhibit improved fracture toughness at cryogenic temperatures, making them unique as cryogenic structural materials. The enhanced mechanical properties of these single-phase face-centered cubic (fcc) $CrCoNi$-based alloys result from their effective strain-hardening capacity, generated by various deformation mechanisms. Insights from this research can help guide the design of damage-resistant materials for potential applications such as long-distance transportation of liquid hydrogen and liquefied natural gas. HEAs and MEAs are uniquely qualified for practical use across a wide range of cryogenic temperatures due to their ability to activate multiple strain-hardening mechanisms in the correct sequence. This harmonious activation leads to simultaneous increases in strength, ductility, and toughness, making them ideal materials for various demanding applications.

Required Additional Study Materials

  • E. P. George, W. A. Curtin, C. C. Tasan, High entropy alloys: A focused review of mechanical properties and deformation mechanisms. Acta Mater. 188, 435–474 (2020).
  • E. J. Pickering, N. G. Jones, High-entropy alloys: A critical assessment of their founding principles and future prospects. Int. Mater. Rev. 61, 183–202 (2016).
  • D. B. Miracle, O. N. Senkov, A critical review of high entropy alloys and related concepts. Acta Mater. 122, 448–511 (2017).
  • Z. Li, S. Zhao, R. O. Ritchie, M. A. Meyers, Mechanical properties of high-entropy alloys with emphasis on face-centered cubic alloys. Prog. Mater. Sci. 102, 296–345 (2019).
  • E. P. George, D. Raabe, R. O. Ritchie, High-entropy alloys. Nat. Rev. Mater. 4, 515–534 (2019).
  • E. P. George, R. O. Ritchie, High-entropy materials. MRS Bull. 47, 145–150 (2022).
Introductory material
  • “A Textbook of Material Science and Metallurgy” by O.P. Khanna
  • “Concepts in Physical Metallurgy” by A Lavakumar

Reference

Liu, D. et al. Exceptional fracture toughness of CrCoNi-based medium- and high-entropy alloys at 20 kelvin. Science 378, 978-983 (2023).

Mastering the game of Stratego with model-free multiagent reinforcement learning

  • Machine Learning

AI advancements have often been gauged using board games, and Stratego is particularly challenging due to its intricate design and private setup phase. With more possible states than both no-limit Texas Hold’em poker and the game of Go, Stratego demands reasoning over an enormous number of potential deployment combinations. Artificial agents have only managed to achieve a skill level similar to human amateurs so far.

A new game-theoretic method, Regularized Nash Dynamics (R-NaD), was developed to help AI learn Stratego via self-play without the need for human demonstrations. By integrating R-NaD with a deep neural network architecture, the resulting bot, DeepNash, found a strategy that couldn’t be exploited and reached human expert-level proficiency in Stratego Classic. Impressively, DeepNash outperformed all existing state-of-the-art bots and showcased highly competitive gameplay against expert human Stratego players without using any search techniques.

R-NaD uses an end-to-end learning approach to tackle Stratego, combining the learning of the deployment phase with the gameplay phase using deep reinforcement learning (RL) and a game-theoretic approach. The objective is to approximate a Nash equilibrium—a stable state where no player can benefit by changing their strategy—through self-play. This ensures the agent performs well, even when faced with the most challenging opponents.

The R-NaD method fuses model-free RL in self-play with a game-theoretic algorithmic concept, guiding the agent’s learning behavior toward the Nash equilibrium without directly modeling private states from public ones. The primary challenge is scaling this model-free RL technique with R-NaD to make self-play competitive against human expert players in Stratego.

Nash equilibrium

Nash equilibrium is a fundamental concept in game theory. It is a solution concept that describes a stable state in a game where no player can improve their outcome by unilaterally changing their strategy, given that the other players maintain their current strategies. In other words, when a game reaches a Nash equilibrium, every player has chosen their optimal strategy, considering the strategies chosen by the other players.

DeepNash uses self-play and model-free RL to learn a Nash equilibrium in Stratego, which has been difficult to stabilize in complex games. The R-NaD algorithm applies a learning update rule with a Lyapunov function—a mathematical tool ensuring stability—to guarantee convergence to a fixed point. This is the key to DeepNash’s success.

DeepNash employs the R-NaD algorithm for learning in normal form games, which are simplified representations of strategic situations. The learning process consists of three critical stages: reward transformation, dynamics, and final update. The algorithm generates a sequence of fixed points converging to a Nash equilibrium in zero-sum two-player games. Convergence is guaranteed by the Lyapunov function.

Lyapunov function

A Lyapunov function is a scalar function $V(x)$ defined over the state space of the dynamical system. It is often designed to have properties similar to an energy function or a potential function. For a given equilibrium point $x*$, the Lyapunov function must meet the following conditions:

  1. V(x) is continuous and positive definite, meaning $V(x) > 0$ for all $x ≠ x* $ and $V(x*) = 0$.
  2. The time derivative of $V(x)$ along the trajectories of the system, denoted as $dV(x)/dt$ or $V’(x)$, is negative semi-definite, meaning $V’(x) ≤ 0$ for all $x ≠ x* $. In some cases, it can be negative definite, meaning $V’(x) < 0$ for all $x ≠ x* $.

If a Lyapunov function can be found that satisfies these conditions, it can be concluded that the equilibrium point $x* $ is stable. In the case where $V’(x)$ is negative definite, $x* $ is asymptotically stable, meaning that trajectories in the state space converge to $x* $ over time.

The R-NaD method for model-free RL in imperfect information games includes three components: a core training component using a deep convolutional network, a component for refining the learned policy, and a test-time postprocessing component that filters out low-probability actions and errors using game-specific knowledge.

In imperfect information games, players take turns and receive rewards based on their actions and observations of the game state. These action and observation trajectories are used to learn a parameterized policy in model-free RL.

DeepNash’s observation consists of a spatial tensor with 82 stacked frames that encode its own pieces, both players’ pieces’ public information, and the last 40 moves’ encoding. The reward transform used in DeepNash is updated via the Neural Replicator Dynamics (NeuRD) update. After a predetermined number of learning steps, an approximate fixed point policy is obtained and used as DeepNash’s next regularization policy.

During training, DeepNash’s model policy is fine-tuned to eliminate low-probability errors. Additional heuristics are applied at test time to remove obvious mistakes. Although these heuristics do not significantly improve self-play quantitatively, DeepNash performs well against human expert players and cutting-edge Stratego bots.

DeepNash was evaluated against human expert players and top-tier Stratego bots, winning $84\%$ of its matches against human players on the Gravon platform and achieving a rating of 1799. These results demonstrate that DeepNash can reach human expert competency in Stratego through self-play learning without relying on pre-existing human data.

DeepNash was also evaluated against various existing Stratego computer programs, such as Probe, Master of the Flag, and Demon of Ignorance, among others. Despite not being trained against any of these bots, DeepNash won the majority of games, indicating its effectiveness in playing Stratego.

Designed to learn a Nash equilibrium policy, DeepNash can generate diverse strategies, execute bluffs, and take calculated risks, making it difficult for human players to discern exploitable patterns.

In Stratego, players must strike a balance between capturing an opponent’s piece and revealing information about their own piece, as opposed to concealing a piece’s identity. DeepNash demonstrated exceptional skill in making these trade-offs, valuing information and material in a sophisticated manner.

Bluffing is a strategy employed by agents to deceive opponents and gain an upper hand. Positive and negative bluffing involve feigning a piece’s higher or lower value, while a more intricate bluff involves moving an unrevealed scout near the opponent’s 10 to acquire material and avoid the scout’s capture.

The game-theoretic method DeepNash enables AI to play Stratego from scratch in self-play up to a human expert level. Combining a deep residual neural network with the game-theoretical R-NaD multiagent learning algorithm, DeepNash does not require search or explicit opponent modeling. With an overall win rate of $84\%$ against human expert players, DeepNash’s extraordinary performance has surpassed the Stratego community’s expectations of what could be achieved with current techniques. This innovation in AI makes complex board games more accessible.

Required Additional Study Materials

  • M. Campbell, A. J. Hoane Jr., F. Hse, Deep blue. Artif. Intell. 134, 57–83 (2002).
  • M. Schmid, M. Moravcik, N. Burch, R. Kadlec, J. Davidson, K. Waugh, N. Bard, F. Timbers, M. Lanctot, Z. Holland, E. Davoodi, A. Christianson, M. Bowling Player of games. arXiv:2112.03178 [cs.AI] (2021). J. von Neumann, O. Morgenstern, Theory of Games and Economic Behavior (Princeton Univ. Press, 1947).
Introductory material
  • “Reinforcement Learning: An Introduction” by Richard S. Sutton and Andrew G. Barto
  • “Handbook of Reinforcement Learning and Control” edited by Kyriakos G. Vamvoudakis, Yan Wan, Frank L. Lewis, and Derya Cansever

Reference

Perolat, J. et al. Mastering the game of Stratego with model-free multiagent reinforcement learning. Science 378, 990-996 (2023).

Observing the quantum topology of light

  • Quantum Simulation

The quantum Hall effect uncovers new phases of matter, which are defined by unique properties of energy bands known as topological invariants. Chiral edge states found between Landau levels contribute to a special type of electrical conductivity that is not affected by local defects. The Haldane model forms the basis for materials called topological insulators, while optical simulation of these quantum Hall edge states leads to an emerging research area, topological photonics. Recent advancements in circuit quantum electrodynamics enable the development of intrinsic quantum topological states of light, which offer new possibilities in designing photonic topology and controlling bosonic quantum information processing.

The Hamiltonian, which describes the energy of the system involving $R_j$’s and $Q_0$, is represented by a multimode JC model within an approximation called the rotating-wave approximation. This Hamiltonian conserves the total excitation number $N$ and includes various mathematical operators associated with transitions and energy levels. The SSH model, which consists of a bipartite tight-binding lattice and spin states, can exhibit two different topological phases. By gradually adjusting the parameters, the topological zero-energy state can be moved from one end of the lattice to the other. This zero-energy state is protected by the energy gap and maintains coherence during transport, as confirmed by a technique called quantum state tomography.

Fock states in a subspace with $N$ excitations form a two-dimensional honeycomb lattice where the coupling strengths depend on the lattice sites, introducing a strain that results in something called pseudo-Landau levels. These Landau levels are characterized by their chiralities, which act as the lattice momentum in conventional lattices and correspond to the two corners of a region called the Brillouin zone, denoted as K and K’ valleys. A Lifshitz topological edge separates the Fock-state lattice into two phases—a semimetallic phase within the incircle and a band insulator phase outside of it. The qubit remains in the ground state during the evolution, highlighting a fundamental difference between classical and quantum predictions.

Valleys in FSLs can be identified by their chiralities, and a phenomenon called VHE can be used to transport the wave function between two valleys, controlling the chirality of the quantum states of multiple resonators. The Haldane model is an important model in topological physics. It introduces complex hopping between next-nearest-neighbor lattice sites in the Fock-state lattice, transforming flat Landau levels into a two-band structure with gapless chiral edge states. In the experiment, photons are directly excited to obtain an initial state, and the coupling strengths are periodically modulated to realize the Haldane Hamiltonian. The chiral rotating wave function moves toward the center of the Fock-state lattice due to decoherence, nonlinearity, and imperfect control pulses.

This study demonstrates the coherent control of topological zero-energy states in 1D and 2D FSLs using small energy perturbations. The techniques developed here can be applied to investigate topological states of more complex systems involving qubits and resonators, paving the way for new control methods in quantum state engineering of bosonic modes.

Required Additional Study Materials

  • L. Lu, J. D. Joannopoulos, M. Soljačić, Topological photonics. Nat. Photonics 8, 821–829 (2014).
  • A. B. Khanikaev, G. Shvets, Two-dimensional topological photonics. Nat. Photonics 11, 763–773 (2017).
  • T. Ozawa, H. M. Price, A. Amo, N. Goldman, M. Hafezi, L. Lu, M. C. Rechtsman, D. Schuster, J. Simon, O. Zilberberg, I. Carusotto, Topological photonics. Rev. Mod. Phys. 91, 015006 (2019).
  • I. Carusotto, A. A. Houck, A. J. Kollár, P. Roushan, D. I. Schuster, J. Simon, Photonic materials in circuit quantum electrodynamics. Nat. Phys. 16, 268–279 (2020).
  • W.-L. Ma, S. Puri, R. J. Schoelkopf, M. H. Devoret, S. M. Girvin, L. Jiang, Quantum control of bosonic modes with superconducting circuits. Sci. Bull. 66, 1789–1805 (2021).
  • A. H. Castro Neto, F. Guinea, N. M. R. Peres, K. S. Novoselov, A. K. Geim, The electronic properties of graphene. Rev. Mod. Phys. 81, 109–162 (2009).
  • M. O. Goerbig, Electronic properties of graphene in a strong magnetic field. Rev. Mod. Phys. 83, 1193–1243 (2011).
Introductory material
  • “Topological Photonics” by Tomoki Ozawa et al.
  • “Topological Photonics: Where do we go from here?” by Mordechai Segev and Miguel A. Bandres

Leave a comment

You may also enjoy