BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Biomedical Mathematics Group - ECPv6.16.3//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:Biomedical Mathematics Group
X-ORIGINAL-URL:https://www.ibs.re.kr/bimag
X-WR-CALDESC:Events for Biomedical Mathematics Group
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:Asia/Seoul
BEGIN:STANDARD
TZOFFSETFROM:+0900
TZOFFSETTO:+0900
TZNAME:KST
DTSTART:20250101T000000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=Asia/Seoul:20260605T100000
DTEND;TZID=Asia/Seoul:20260605T120000
DTSTAMP:20260605T023123
CREATED:20260429T070216Z
LAST-MODIFIED:20260527T003414Z
UID:12396-1780653600-1780660800@www.ibs.re.kr
SUMMARY:Chaos Meets Attention: Transformers for Large-Scale Dynamical Prediction- Gyuyoung Hwang
DESCRIPTION:In this talk\, we discuss the paper “Chaos Meets Attention: Transformers for Large-Scale Dynamical Prediction” by Yi He et al.\, ICML Poster\, 2025. \nAbstract: \nGenerating long-term trajectories of dissipative chaotic systems autoregressively is a highly challenging task. The inherent positive Lyapunov exponents amplify prediction errors over time. Many chaotic systems possess a crucial property — ergodicity on their attractors\, which makes long-term prediction possible. State-of-the-art methods address ergodicity by preserving statistical properties using optimal transport techniques. However\, these methods face scalability challenges due to the curse of dimensionality when matching distributions. To overcome this bottleneck\, we propose a scalable transformer-based framework capable of stably generating long-term high-dimensional and high-resolution chaotic dynamics while preserving ergodicity. Our method is grounded in a physical perspective\, revisiting the Von Neumann mean ergodic theorem to ensure the preservation of long-term statistics in the L2 space. We introduce novel modifications to the attention mechanism\, making the transformer architecture well-suited for learning large-scale chaotic systems. Compared to operator-based and transformer-based methods\, our model achieves better performances across five metrics\, from short-term prediction accuracy to long-term statistics. In addition to our methodological contributions\, we introduce new chaotic system benchmarks: a machine learning dataset of 140 snapshots of turbulent channel flow and a processed high-dimensional Kolmogorov Flow dataset\, along with various evaluation metrics for both short- and long-term performances. Both are well-suited for machine learning research on chaotic systems.
URL:https://www.ibs.re.kr/bimag/event/bridging-known-and-unknown-dynamics-by-transformer-based-machine-learning-inference-from-sparse-observations-gyuyoung-hwang/
LOCATION:B232 Seminar Room\, IBS\, 55 Expo-ro Yuseong-gu\, Daejeon\, Daejeon\, 34126\, Korea\, Republic of
CATEGORIES:Journal Club
ORGANIZER;CN="Jae Kyoung Kim":MAILTO:jaekkim@kaist.ac.kr
END:VEVENT
END:VCALENDAR