{"id":7278,"date":"2023-01-27T10:15:41","date_gmt":"2023-01-27T01:15:41","guid":{"rendered":"https:\/\/www.ibs.re.kr\/bimag\/?post_type=tribe_events&p=7278"},"modified":"2023-02-14T23:46:10","modified_gmt":"2023-02-14T14:46:10","slug":"2023-02-17-jc","status":"publish","type":"tribe_events","link":"https:\/\/www.ibs.re.kr\/bimag\/event\/2023-02-17-jc\/","title":{"rendered":"Hyeontae Jo, Characterizing possible failure modes in physics-informed neural networks"},"content":{"rendered":"
\n
\n

We will discuss about \u201cCharacterizing possible failure modes in physics-informed neural networks\u201d, Krishnapriyan, Aditi, et al., Advances in Neural Information Processing Systems<\/i>\u00a034 (2021): 26548-26560.<\/p>\n

Abstract<\/p>\n

\n
\n
\n
\n
\n
\n

Recent work in scientific machine learning has developed so-called physics-informed neural network (PINN) models. The typical approach is to incorporate physical domain knowledge as soft constraints on an empirical loss function and use existing machine learning methodologies to train the model. We demonstrate that, while existing PINN methodologies can learn good models for relatively trivial problems, they can easily fail to learn relevant physical phenomena for even slightly more complex problems. In particular, we analyze several distinct situations of widespread physical interest, including learning differential equations with convection, reaction, and diffusion operators. We provide evidence that the soft regularization in PINNs, which involves PDE-based differential operators, can introduce a number of subtle problems, including making the problem more ill-conditioned. Importantly, we show that these possible failure modes are not due to the lack of expressivity in the NN architecture, but that the PINN’s setup makes the loss landscape very hard to optimize. We then describe two promising solutions to address these failure modes. The first approach is to use curriculum regularization, where the PINN’s loss term starts from a simple PDE regularization, and becomes progressively more complex as the NN gets trained. The second approach is to pose the problem as a sequence-to-sequence learning task, rather than learning to predict the entire space-time at once. Extensive testing shows that we can achieve up to 1-2 orders of magnitude lower error with these methods as compared to regular PINN training.<\/p>\n

 <\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/section>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

We will discuss about \u201cCharacterizing possible failure modes in physics-informed neural networks\u201d, Krishnapriyan, Aditi, et al., Advances in Neural Information Processing Systems\u00a034 (2021): 26548-26560. Abstract Recent work in scientific machine … <\/p>\n

Continue reading “Hyeontae Jo, Characterizing possible failure modes in physics-informed neural networks”<\/span><\/a><\/p>\n","protected":false},"author":4,"featured_media":0,"template":"","meta":{"_editorskit_title_hidden":false,"_editorskit_reading_time":0,"_editorskit_is_block_options_detached":false,"_editorskit_block_options_position":"{}","_uag_custom_page_level_css":"","_tribe_events_status":"","_tribe_events_status_reason":"","footnotes":""},"tags":[],"tribe_events_cat":[219],"class_list":["post-7278","tribe_events","type-tribe_events","status-publish","hentry","tribe_events_cat-journal-club","cat_journal-club"],"yoast_head":"\nHyeontae Jo, Characterizing possible failure modes in physics-informed neural networks - Biomedical Mathematics Group<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.ibs.re.kr\/bimag\/event\/2023-02-17-jc\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Hyeontae Jo, Characterizing possible failure modes in physics-informed neural networks - Biomedical Mathematics Group\" \/>\n<meta property=\"og:description\" content=\"We will discuss about \u201cCharacterizing possible failure modes in physics-informed neural networks\u201d, Krishnapriyan, Aditi, et al., Advances in Neural Information Processing Systems\u00a034 (2021): 26548-26560. Abstract Recent work in scientific machine … Continue reading "Hyeontae Jo, Characterizing possible failure modes in physics-informed neural networks"\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.ibs.re.kr\/bimag\/event\/2023-02-17-jc\/\" \/>\n<meta property=\"og:site_name\" content=\"Biomedical Mathematics Group\" \/>\n<meta property=\"article:modified_time\" content=\"2023-02-14T14:46:10+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.ibs.re.kr\\\/bimag\\\/event\\\/2023-02-17-jc\\\/\",\"url\":\"https:\\\/\\\/www.ibs.re.kr\\\/bimag\\\/event\\\/2023-02-17-jc\\\/\",\"name\":\"Hyeontae Jo, Characterizing possible failure modes in physics-informed neural networks - Biomedical Mathematics Group\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.ibs.re.kr\\\/bimag\\\/#website\"},\"datePublished\":\"2023-01-27T01:15:41+00:00\",\"dateModified\":\"2023-02-14T14:46:10+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.ibs.re.kr\\\/bimag\\\/event\\\/2023-02-17-jc\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.ibs.re.kr\\\/bimag\\\/event\\\/2023-02-17-jc\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.ibs.re.kr\\\/bimag\\\/event\\\/2023-02-17-jc\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.ibs.re.kr\\\/bimag\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Events\",\"item\":\"https:\\\/\\\/www.ibs.re.kr\\\/bimag\\\/events\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Hyeontae Jo, Characterizing possible failure modes in physics-informed neural networks\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.ibs.re.kr\\\/bimag\\\/#website\",\"url\":\"https:\\\/\\\/www.ibs.re.kr\\\/bimag\\\/\",\"name\":\"Biomedical Mathematics Group\",\"description\":\"\uae30\ucd08\uacfc\ud559\uc5f0\uad6c\uc6d0 \uc758\uc0dd\uba85\uc218\ud559\uadf8\ub8f9\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.ibs.re.kr\\\/bimag\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.ibs.re.kr\\\/bimag\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.ibs.re.kr\\\/bimag\\\/#organization\",\"name\":\"IBS Biomedical Mathematics Group\",\"url\":\"https:\\\/\\\/www.ibs.re.kr\\\/bimag\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.ibs.re.kr\\\/bimag\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/www.ibs.re.kr\\\/bimag\\\/cms\\\/wp-content\\\/uploads\\\/2021\\\/02\\\/ibs-circle-1.png\",\"contentUrl\":\"https:\\\/\\\/www.ibs.re.kr\\\/bimag\\\/cms\\\/wp-content\\\/uploads\\\/2021\\\/02\\\/ibs-circle-1.png\",\"width\":250,\"height\":250,\"caption\":\"IBS Biomedical Mathematics Group\"},\"image\":{\"@id\":\"https:\\\/\\\/www.ibs.re.kr\\\/bimag\\\/#\\\/schema\\\/logo\\\/image\\\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Hyeontae Jo, Characterizing possible failure modes in physics-informed neural networks - Biomedical Mathematics Group","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.ibs.re.kr\/bimag\/event\/2023-02-17-jc\/","og_locale":"en_US","og_type":"article","og_title":"Hyeontae Jo, Characterizing possible failure modes in physics-informed neural networks - Biomedical Mathematics Group","og_description":"We will discuss about \u201cCharacterizing possible failure modes in physics-informed neural networks\u201d, Krishnapriyan, Aditi, et al., Advances in Neural Information Processing Systems\u00a034 (2021): 26548-26560. Abstract Recent work in scientific machine … Continue reading \"Hyeontae Jo, Characterizing possible failure modes in physics-informed neural networks\"","og_url":"https:\/\/www.ibs.re.kr\/bimag\/event\/2023-02-17-jc\/","og_site_name":"Biomedical Mathematics Group","article_modified_time":"2023-02-14T14:46:10+00:00","twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.ibs.re.kr\/bimag\/event\/2023-02-17-jc\/","url":"https:\/\/www.ibs.re.kr\/bimag\/event\/2023-02-17-jc\/","name":"Hyeontae Jo, Characterizing possible failure modes in physics-informed neural networks - Biomedical Mathematics Group","isPartOf":{"@id":"https:\/\/www.ibs.re.kr\/bimag\/#website"},"datePublished":"2023-01-27T01:15:41+00:00","dateModified":"2023-02-14T14:46:10+00:00","breadcrumb":{"@id":"https:\/\/www.ibs.re.kr\/bimag\/event\/2023-02-17-jc\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.ibs.re.kr\/bimag\/event\/2023-02-17-jc\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.ibs.re.kr\/bimag\/event\/2023-02-17-jc\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.ibs.re.kr\/bimag\/"},{"@type":"ListItem","position":2,"name":"Events","item":"https:\/\/www.ibs.re.kr\/bimag\/events\/"},{"@type":"ListItem","position":3,"name":"Hyeontae Jo, Characterizing possible failure modes in physics-informed neural networks"}]},{"@type":"WebSite","@id":"https:\/\/www.ibs.re.kr\/bimag\/#website","url":"https:\/\/www.ibs.re.kr\/bimag\/","name":"Biomedical Mathematics Group","description":"\uae30\ucd08\uacfc\ud559\uc5f0\uad6c\uc6d0 \uc758\uc0dd\uba85\uc218\ud559\uadf8\ub8f9","publisher":{"@id":"https:\/\/www.ibs.re.kr\/bimag\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.ibs.re.kr\/bimag\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.ibs.re.kr\/bimag\/#organization","name":"IBS Biomedical Mathematics Group","url":"https:\/\/www.ibs.re.kr\/bimag\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.ibs.re.kr\/bimag\/#\/schema\/logo\/image\/","url":"https:\/\/www.ibs.re.kr\/bimag\/cms\/wp-content\/uploads\/2021\/02\/ibs-circle-1.png","contentUrl":"https:\/\/www.ibs.re.kr\/bimag\/cms\/wp-content\/uploads\/2021\/02\/ibs-circle-1.png","width":250,"height":250,"caption":"IBS Biomedical Mathematics Group"},"image":{"@id":"https:\/\/www.ibs.re.kr\/bimag\/#\/schema\/logo\/image\/"}}]}},"uagb_featured_image_src":{"full":false,"thumbnail":false,"medium":false,"medium_large":false,"large":false,"1536x1536":false,"2048x2048":false,"dimag-thumbnail":false,"twentyseventeen-featured-image":false,"twentyseventeen-thumbnail-avatar":false},"uagb_author_info":{"display_name":"Hyeontae Jo","author_link":"https:\/\/www.ibs.re.kr\/bimag\/author\/ibs-htj\/"},"uagb_comment_info":0,"uagb_excerpt":"We will discuss about \u201cCharacterizing possible failure modes in physics-informed neural networks\u201d, Krishnapriyan, Aditi, et al., Advances in Neural Information Processing Systems\u00a034 (2021): 26548-26560. Abstract Recent work in scientific machine … Continue reading \"Hyeontae Jo, Characterizing possible failure modes in physics-informed neural networks\"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.ibs.re.kr\/bimag\/wp-json\/wp\/v2\/tribe_events\/7278","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.ibs.re.kr\/bimag\/wp-json\/wp\/v2\/tribe_events"}],"about":[{"href":"https:\/\/www.ibs.re.kr\/bimag\/wp-json\/wp\/v2\/types\/tribe_events"}],"author":[{"embeddable":true,"href":"https:\/\/www.ibs.re.kr\/bimag\/wp-json\/wp\/v2\/users\/4"}],"version-history":[{"count":3,"href":"https:\/\/www.ibs.re.kr\/bimag\/wp-json\/wp\/v2\/tribe_events\/7278\/revisions"}],"predecessor-version":[{"id":7349,"href":"https:\/\/www.ibs.re.kr\/bimag\/wp-json\/wp\/v2\/tribe_events\/7278\/revisions\/7349"}],"wp:attachment":[{"href":"https:\/\/www.ibs.re.kr\/bimag\/wp-json\/wp\/v2\/media?parent=7278"}],"wp:term":[{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.ibs.re.kr\/bimag\/wp-json\/wp\/v2\/tags?post=7278"},{"taxonomy":"tribe_events_cat","embeddable":true,"href":"https:\/\/www.ibs.re.kr\/bimag\/wp-json\/wp\/v2\/tribe_events_cat?post=7278"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}