Journal Club

The IAIFI Journal Club is open to IAIFI members and affiliates.

Sign up to lead a discussion!

Upcoming Journal Clubs

  • Jeffrey Lazar, Graduate Student, Harvard
    • Tuesday, September 26, 2023, 11:00 am–12:00 pm, MIT LNS Conference Room (26-528)
    • Open-Source Simulation and Machine Learning for Neutrino Telescopes
    • In the last decade, the filed of neutrino astronomy has made major strides, culminating of the definitive detection of galactic and extragalactic components of the astrophysical neutrino flux. We can now begin characterizing these astrophysical beams and pursuing new physics through them. Machine learning techniques have played an integral part in these recent advances, and while these current efforts have been impressive, it is clear that there is room to improve. This face, along with the growing, global network of neutrino telescopes, drives the need for open-source tools to use all person power and avoid reduplicating effort. In this talk I will present Prometheus, the first open-source, end-to-end simulation for neutrino telescopes. Furthermore, I will show a recent example of Prometheus to develop machine learning techniques capable of running at typical neutrino telescope trigger rates.

  • Manos Theodosis, Graduate Student, Harvard
    • Tuesday, October 3, 2023, 11:00 am–12:00 pm, MIT LNS Conference Room (26-528)
    • Title to come
    • Abstract to come

  • Akshunna Dogra, Graduate Student, Imperial College London
    • Tuesday, October 31, 2023, 11:00 am–12:00 pm, MIT LNS Conference Room (26-528)
    • Title to come
    • Abstract to come

  • Zeviel Imani, Graduate Student, Tufts
    • Tuesday, November 14, 2023, 11:00 am–12:00 pm, MIT LNS Conference Room (26-528)
    • Title to come
    • Abstract to come

Past Journal Clubs

Spring 2023 Journal Clubs

  • Di Luo, IAIFI Fellow
    • April 25, 2023, 11:00am-12:00pm
    • Multi-legged Robot Locomotion via Spin Models Duality
    • Contact planning is crucial in locomoting systems.Specifically, appropriate contact planning can enable versatile behaviors (e.g., sidewinding in limbless locomotors) and facilitate speed-dependent gait transitions (e.g., walk-trot-gallop in quadrupedal locomotors). The challenges of contact planning include determining not only the sequence by which contact is made and broken between the locomotor and the environments, but also the sequence of internal shape changes (e.g., body bending and limb shoulder joint oscillation). Most state-of-art contact planning algorithms focused on conventional robots (e.g.biped and quadruped) and conventional tasks (e.g. forward locomotion), and there is a lack of study on general contact planning in multi-legged robots. In this talk, I am going to discuss that using geometric mechanics framework, we can obtain the global optimal contact sequence given the internal shape changes sequence. Therefore, we simplify the contact planning problem to a graph optimization problem to identify the internal shape changes. Taking advantages of the spatio-temporal symmetry in locomotion, we map the graph optimization problem to special cases of spin models, which allows us to obtain the global optima in polynomial time. We apply our approach to develop new forward and sidewinding behaviors in a hexapod and a 12-legged centipede. We verify our predictions using numerical and robophysical models, and obtain novel and effective locomotion behaviors.
    • Slides (For IAIFI members only)
  • Ziming Liu, Grad Student, MIT
    • April 25, 2023, 11:00am-12:00pm
    • Physics-inspired generative models
    • It might be surprising and delightful to physicists that physics has been playing a huge role in diffusion models. In fact, the evolution of our physical world can be viewed as a generation process. In this journal club, I will first review diffusion models, the more recent PFGM/PFGM++ inspired from electrostatics, and then introduce the GenPhys framework which manages to convert even more physical processes to generative models.
    • Slides (For IAIFI members only)
  • Asem Wardak, Research Fellow, Harvard
    • April 11, 2023, 11:00am-12:00pm
    • Extended Anderson Criticality in Heavy-Tailed Neural Networks
    • This talk focuses on nonlinearly interacting systems with random, heavy-tailed connectivity. We show how heavy-tailed connectivity gives rise to an extended critical regime of spatially multifractal fluctuations between the quiescent and active phases. This phase differs from the edge of chaos in classical networks by the appearance of universal hallmarks of the Anderson transition in condensed matter physics over an extended region in phase space. We then investigate some consequences of the multifractal Anderson regime for performing persistent computations.
    • Slides (For IAIFI members only)
  • Joshua Villarreal, Grad Student, MIT
    • April 4, 2023, 11:00am-12:00pm
    • Surrogate Modeling of Particle Accelerators
    • Abstract: The design, construction, and fine-tuning of particle accelerators has never been easy. Each is a technical challenge in and of itself, and the need to repeatedly run accurate, high-fidelity simulations of the beam traversing the device can slow development. This is especially true for many modern-day particle accelerators, whose beam dynamics tend to observe more nonlinear effects like those arising from space charge, making their simulation more computationally expensive. Thus, there is demand to develop machine learning and statistical learning models that can reproduce these beam dynamic simulations with orders-of-magnitude improvements in runtime. In this talk, I present an overview of recent efforts to build such accelerator surrogate models, which can be used for the design optimization and real-time commissioning, tuning, and running of the accelerator they aim to replicate. As an example, I also present the status of IsoDAR’s work to build a surrogate model for a Radio-Frequency Quadrupole accelerator, a vital component to IsoDAR’s groundbreaking design. I outline challenges of these and other virtual accelerators, and present future plans to make these surrogate models ubiquitous in future development of accelerator experiments of all kinds.
    • Slides (For IAIFI members only)
  • Daniel Murnane, Postdoc Researcher, Berkeley Lab
    • March 21, 2023, 11:00am-12:00pm
    • Multi-Tasking ML for Point Clouds at the LHC
    • Abstract: The Large Hadron Collider is one of the world’s most data-intensive experiments. Every second, millions of collisions are processed, each one resembling a jigsaw puzzle with thousands of pieces. With the upcoming upgrade to the High Luminosity LHC, this problem will only become more complex. To make sense of this data, deep learning techniques are increasingly being used. For example, graph neural networks and transformers have proven effective at handling point cloud tasks such as track reconstruction and jet tagging. In this talk, I will review the point cloud problems in collider physics and recent deep learning solutions investigated by the Exatrkx project - an initiative to implement innovative algorithms for HEP at exascale. These architectures can accurately perform tracking and tagging with low latency, even in the high luminosity regime. Additionally, I will explore how multi-tasking and multi-modal networks can combine several of these different tasks.
    • Slides (For IAIFI members only)
  • Manuel Szewc, Postdoc, University of Cincinnati
    • March 14, 2023, 11:00am-12:00pm
    • Modeling Hadronization with Machine Learning
    • Abstract: A fundamental part of event generation, hadronization is currently simulated with the help of fine-tuned empirical models. In this talk, I’ll present MLHAD, a proposed alternative for hadronization where the empirical model is replaced by a surrogate Machine Learning-based model to be ultimately data-trainable. I’ll detail the current stage of development and discuss possible ways forward.
    • Slides (For IAIFI members only)
  • Max Tegmark, Professor, MIT
    • February 28, 2023, 11:00am-12:00pm
    • Mechanistic interpretability
    • Abstract: Mechanistic interpretability aims to reverse-engineer trained neural networks to distill out the algorithms they have discovered for performing various tasks. Although such “artificial neuroscience” is hard and fun, it’s easier than conventional neuroscience since you have complete knowledge of what every neuron and synapse is doing.
    • Slides (For IAIFI members only)
  • Liping Liu, Assistant Professor, Tufts University
    • February 14, 2023, 11:00am-12:00pm
    • Address combinatorial graph problems with learning methods
    • Abstract: There are plenty of hard combinatorial problems defined on graphs. Recently learning algorithms have been used to speed up the search for approximate solutions to these problems. This talk will start with an introduction to hard problems on graphs and traditional algorithms, then it will give an overview of learning algorithms for solving combinatorial problems on graphs. The second part of the talk will focus on two specific problems, graph matching and subgraph distance calculation, and discuss neural methods for these two problems. Finally, it will conclude with open questions: why and when can neural networks help to solve combinatorial problems?
    • References:
    • Slides (For IAIFI members only)

Fall 2022 Journal Clubs

  • Anna Golubeva, IAIFI Fellow and Matt Schwartz, Professor, Harvard
  • Michael Toomey, PhD Student, Brown University
    • November 15, 2022, 11:00am-12:00pm
    • Deep Learning the Dark Sector
    • Abstract: One of the most pressing questions in physics today is the microphysical origin of dark matter. While there have been numerous experimental programs aimed at detecting its interactions with the Standard Model, all efforts to-date have come up empty. An alternative method to constrain dark matter is purely based on its gravitational interactions. In particular, gravitational lensing can be very sensitive to the distribution and morphology of dark matter substructure which can vary appreciably between different models. However, the complexity of data sets, systematics, and large volumes of data make the dimensionality of this problem difficult to approach from more traditional methods. Thankfully, this is a task ideally suited for machine learning. In this talk we will demonstrate how machine learning will play a critical role in distinguishing between models of dark matter and constraining model parameters in lensing data. We will additionally discuss techniques unique to ML for transferring the knowledge accumulated by models in the controlled setting of simulation to real data sets utilizing unsupervised domain adaptation.
    • Slides (For IAIFI members only)
  • Ziming Liu, PhD Student, MIT
    • November 8, 2022, 11:00am-12:00pm
    • Toy Models of Superposition
    • Abstract: It would be very convenient if the individual neurons of artificial neural networks corresponded to cleanly interpretable features of the input. For example, in an “ideal” ImageNet classifier, each neuron would fire only in the presence of a specific visual feature, such as the color red, a left-facing curve, or a dog snout. But it isn’t always the case that features correspond so cleanly to neurons, especially in large language models where it actually seems rare for neurons to correspond to clean features. I will present a recent paper “Toy Models of Superposition” from Anthropic, aiming to answer these questions: Why is it that neurons sometimes align with features and sometimes don’t? Why do some models and tasks have many of these clean neurons, while they’re vanishingly rare in others?
    • Slides (For IAIFI members only)
  • Sona Najafi, Researcher, IBM
    • October 25, 2022, 11:00am-12:00pm
    • Quantum machine learning from algorithms to hardware
    • Abstract: The rapid progress of technology over the past few decades has led to the emergence of two powerful computational paradigms known as quantum computing and machine learning. While machine learning tries to learn the solutions from data, quantum computing harnesses the quantum laws for more powerful computation compared to classical computers. In this talk, I will discuss three domains of quantum machine learning, each harnessing a particular aspect of quantum computers and targeting specific problems. The first domain scrutinizes the power of quantum computers to work with high-dimensional data and speed-up algebra, but raises the caveat of input/output due to the quantum measurement rules. The second domain circumvents this problem by using a hybrid architecture, performing optimization on a classical computer while evaluating parameterized states on a quantum circuit, chosen based on a particular issue. Finally, the third domain is inspired by brain-like computation and uses a given quantum system’s natural interaction and unitary dynamic as a source for learning
  • Kim Nicoli, Grad Student, Technical University of Berlin
    • October 18, 2022, 11:00am-12:00pm
    • Deep Learning approaches in lattice quantum field theory: recent advances and future challenges**
    • Abstract: Normalizing flows are deep generative models that leverage the change of variable formula to map simple base densities to arbitrary complex target distributions. Recent works have shown the potential of such methods in learning normalized Boltzmann densities in many fields ranging from condensed matter physics to molecular science to lattice field theory. Though sampling from a flow-based density comes with many advantages over standard MCMC sampling, it is known that these methods still suffer from several limitations. In my talk, I will start to give an overview on how to deploy deep generative models to learn Boltzmann densities in the context of a phi^4 lattice field theory. Specifically, I’ll focus on how these methods open up the possibility to estimate thermodynamic observables, i.e., physical observables which depend on the partition function and hence are not straightforward to estimate using standard MCMC methods. In the second part of my talk, I will present two ideas that have been proposed to mitigate the well-known problem of mode-collapse which often occurs when normalizing flows are trained to learn a multimodal target density. More specifically I’ll talk about a novel “mode-dropping estimator” and path gradients. In the last part of my talk, I’ll present a new idea which aims at using flow-based methods to mitigate the sign problem.
    • Slides (For IAIFI members only)
  • Adriana Dropulic, Grad Student, Princeton
    • October 4, 2022, 11:00am-12:00pm
    • Machine Learning the 6th Dimension: Stellar Radial Velocities from 5D Phase-Space Correlations
    • Abstract: The Gaia satellite will observe the positions and velocities of over a billion Milky Way stars. In the early data releases, most observed stars do not have complete 6D phase-space information. We demonstrate the ability to infer the missing line-of-sight velocities until more spectroscopic observations become available. We utilize a novel neural network architecture that, after being trained on a subset of data with complete phase-space information, takes in a star’s 5D astrometry (angular coordinates, proper motions, and parallax) and outputs a predicted line-of-sight velocity with an associated uncertainty. Working with a mock Gaia catalog, we show that the network can successfully recover the distributions and correlations of each velocity component for stars that fall within ~5 kpc of the Sun. We also demonstrate that the network can accurately reconstruct the velocity distribution of a kinematic substructure in the stellar halo that is spatially uniform, even when it comprises a small fraction of the total star count. We apply the neural network to real Gaia data and discuss how the inferred information augments our understanding of the Milky Way’s formation history.
    • Slides (For IAIFI members only)
  • Iris Cong, Grad Student, Harvard
    • September 27, 2022, 11:00am-12:00pm
    • Quantum Convolutional Neural Networks
    • Abstract: Convolutional neural networks (CNNs) have recently proven successful for many complex applications ranging from image recognition to precision medicine. In the first part of my talk, motivated by recent advances in realizing quantum information processors, I introduce and analyze a quantum circuit-based algorithm inspired by CNNs. Our quantum convolutional neural network (QCNN) uses only O(log(N)) variational parameters for input sizes of N qubits, allowing for its efficient training and implementation on realistic, near-term quantum devices. To explicitly illustrate its capabilities, I show that QCNN can accurately recognize quantum states associated with a one-dimensional symmetry-protected topological phase, with performance surpassing existing approaches. I further demonstrate that QCNN can be used to devise a quantum error correction (QEC) scheme optimized for a given, unknown error model that substantially outperforms known quantum codes of comparable complexity. The design of such error correction codes is particularly important for near-term experiments, whose error models may be different from those addressed by general-purpose QEC schemes. If time permits, I will also present our latest results on generalizing the QCNN framework to more accurately and efficiently identify two-dimensional topological phases of matter.
    • Slides (For IAIFI members only)
  • Miles Cranmer, Grad Student, Princeton
    • September 20, 2022, 11:00am–12:00pm
    • Interpretable Machine Learning for Physics
    • Abstract: Would Kepler have discovered his laws if machine learning had been around in 1609? Or would he have been satisfied with the accuracy of some black box regression model, leaving Newton without the inspiration to find the law of gravitation? In this talk I will present a review of some industry-oriented machine learning algorithms, and discuss a major issue facing their use in the natural sciences: a lack of interpretability. I will then outline several approaches I have created with collaborators to help address these problems, based largely on a mix of structured deep learning and symbolic methods. This will include an introduction to the PySR software (https://astroautomata.com/PySR), a Python/Julia package for high-performance symbolic regression. I will conclude by demonstrating applications of such techniques and how we may gain new insights from such results.
    • Resources: https://arxiv.org/abs/2207.12409; https://arxiv.org/abs/2202.02306; https://arxiv.org/abs/2006.11287
    • Slides (For IAIFI members only)
  • Anindita Maiti, Grad Student, Northeastern
    • September 13, 2022, 11:00am-12:00pm
    • A Study of Neural Network Field Theories
    • Abstract: I will present a systematic exploration of field theories arising in Neural Networks, using a dual framework given by Neural Network parameters. The infinite width limit of NN architectures, combined with i.i.d. parameters, lead to Gaussian Processes in Neural Networks by the Central Limit Theorem (CLT), corresponding to generalized free field theories. Small and large violations of the CLT respectively lead to weakly coupled and non-perturbative non-Lagrangian field theories in Neural Networks. Non-Gaussianity, locality (via cluster decomposition), and symmetries of Neural Network field theories are examined via NN parameter space, without necessitating the knowledge of field theoretic actions. Thus, Neural Network field theories, in conjunction to this duality via parameters, may have potential implications for Physics and Machine Learning both.
    • Resources: https://arxiv.org/abs/2106.00694
    • Slides (For IAIFI members only)

Spring 2022 Journal Clubs

Fall 2021 Journal Clubs

  • Michael Douglas
  • Slides (For IAIFI members only)

  • Ziming Liu
  • Slides (For IAIFI members only)

  • Ge Yang
    • Thursday, October 21, 2021, 11:00am-12:00pm
    • “Learning and Generalization: Revisiting Neural Representations”
    • Abstract/Resources: Understanding how deep neural networks learn and generalize has been a central pursuit of intelligence research. This is because we want to build agents that can learn quickly from a small amount of data, that also generalizes to a wider set of scenarios. In this talk, we take a systems approach by identifying key bottleneck components that limits learning and generalization. We will present two key results — overcoming the simplicity bias of neural value approximation via random Fourier features and going beyond the training distribution via invariance through inference.
  • Eric Michaud, PhD Student, MIT
    • Thursday, November 18, 2021 11:00am-12:00pm
    • “Curious Properties of Neural Networks”
    • Abstract/Resources: In this informal talk/discussion, I will highlight some facts about neural networks which I find to be particularly fun and surprising. Possible topics could include the Lottery Ticket Hypothesis (https://arxiv.org/abs/1803.03635), Double Descent (https://arxiv.org/abs/1912.02292), and “grokking” (https://mathai-iclr.github.io/papers/papers/MATHAI_29_paper.pdf). There will be time for discussion and for attendees to bring up their own favorite surprising facts about deep learning.
  • Murphy Niu, Google Quantum AI

Spring 2021 Journal Clubs

Fall 2020 Journal Clubs