9/26/2021

科技部半導體射月計畫 109 年度產學技術交流

 科技部半導體射月計畫 109 年度產學技術交流

  • Boris Murmann, tinyML: The Perfect Storm for Innovation in Ultra-Low-Power System Design 
  • 梁伯嵩,IC 運算平台趨勢: 數位運算、人工智慧與量子運算
  • Tetsu Ohtou, Semiconductor Process and Equipment Technology for Advanced Logic Devices 
  • 陳俊雄,汽車產業及感測元件發展趨勢

9/25/2021

疫情中從海外看台灣

孟買春秋,疫情中從海外看台灣,思想坦克,2021 年 9 月 24 日

經過兩年,我和丈夫終於回到普羅旺斯的家,離開台北時對台灣之外的疫情世界忐忑不安,畢竟過去一年多台灣彷彿世外桃源,不知疫情為何物。然而抵達南法十多天之後,緊張的心情似乎已經消失了,取而代之的是令我無比驕傲的台灣人身分。

9/23/2021

一流的人讀書,都在哪裡畫線?

Eiji Doi 著,歐凱寧譯,一流的人讀書,都在哪裡畫線?:菁英閱讀的深思考技術,天下雜誌,2021

進入社會後,讀書,有個重要的任務,就是投資自己的生涯,從龐雜、陌生的領域中建立起讓自己成長的知識基礎。

9/19/2021

Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategies

Paul Vicol, Luke Metz, and Jascha Sohl-Dickstein, Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategies, ICML 2021. (paperOutstanding Paper Awards)

Unrolled computation graphs arise in many scenarios, including training RNNs, tuning hyperparameters through unrolled optimization, and training learned optimizers. Current approaches to optimizing parameters in such computation graphs suffer from high variance gradients, bias, slow updates, or large memory usage. We introduce a method called Persistent Evolution Strategies (PES), which divides the computation graph into a series of truncated unrolls, and performs an evolution strategies-based update step after each unroll. PES eliminates bias from these truncations by accumulating correction terms over the entire sequence of unrolls. PES allows for rapid parameter updates, has low memory usage, is unbiased, and has reasonable variance characteristics. We experimentally demonstrate the advantages of PES compared to several other methods for gradient estimation on synthetic tasks, and show its applicability to training learned optimizers and tuning hyperparameters.