Data-Driven Science and Engineering

Steven L. Brunton and J. Nathan Kutz, Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control, 2nd edition, Cambridge University Press (pdf, amazing videos, connection with dynamical systems in engineering)

作業研究和機器學習

課程

• 作業研究 (上): 3.8 迴歸，11.2 迴歸，12.3.1 類神經
• Google Scholar: generative ai optimization
• 作業研究 (下): 2 Robust Optimization (穩健最佳化)，3 適應穩健最佳化，5 資料驅動的報童模型
• Applications in generative AI: diffusion probabilistic models 
• Dmytro Kuzmenko, Denoising diffusion probabilistic models
• Lilian Weng, What are Diffusion Models?
• Dimitris Bertsimas and Georgios Margaritis, Robust and Adaptive Optimization under a Large Language Model Lens, arXiv:2501.00568.
• 最佳化和機器學習：

  機器學習和作業研究的奇妙結合

研究 (Research)

Conference papers:

• C.-H. Hsu and T.-Y. Liao,  Enhanced holistic regression for multicollinearity detection and feature selection, Available at SSRN, 2025. (code)
• 廖庭煜、維琪、許志華、饒忻，增強不確定性下的決策：結合 TRIZ 和機器學習方法的穩健優化框架，2025 系統性創新研討會暨專案競賽，論文競賽獎第一名 (code)

Journal articles:

• C.-H. Hsu and H.-C. Yang, Suboptimal Explainable Scheme for Machining Outcome Estimation, IEEE Robotics and Automation Letters, Vol. 7, No. 3, July 2022, pp. 7834 - 7841.

Guides for students in Business Analytics Laboratory (商業分析實驗室學生指引)

Knowledge to master for a better foundation (and future)

Tools and general: 

• Antonio Torralba, Phillip Isola, and William Freeman, Foundations of Computer Vision, The MIT Press, 2024. (On Research, Writing and Speaking) (new)
• Takeo Kanade, Think Like an Amateur, Do As an Expert (new)
• AI in education: Geoffrey Hinton’s and Yann LeCun’s vision of the future, The Buzz Business, 2023. 
• The role of AI in education extends to fostering creativity and critical thinking.
• Another significant impact of AI in education is its potential to democratize access to quality learning.

Google's quantum chip Willow

 Hartmut Neven, Meet Willow, our state-of-the-art quantum chip, Google Research, Dec 09, 2024.

Neural Coordination and Capacity Control for Inventory Management

C Eisenach, U Ghai, D Madeka, K Torkkola, D Foster, S Kakade, Neural Coordination and Capacity Control for Inventory Management, 2024, arXiv preprint arXiv:2410.02817. (SCOT, Amazon)

Google DeepMind 科學家公布具備人類玩家業餘水準的桌球機器人

陳曉莉，DeepMind的桌球機器人已達業餘的人類選手水準，iThome，2024-08-09 

Data-Driven Performance Guarantees for Classical and Learned Optimizers

R. Sambharya and B. Stellato, Data-Driven Performance Guarantees for Classical and Learned Optimizers, arXiv e-prints:2404.13831,2024. (Python code)

Optimization of Optimal Sparse Decision Trees

Cynthia Rudin, Optimization of Optimal Sparse Decision Trees

Fluid approximations for stochastic optimization

When one encounters a stochastic optimization/control problem, one popular approach is to transform it into a deterministic problem by fluid approximation. The following highly-cited classic papers illustrate the applications of this approach:   

10 個改變世界的統計觀念

Leda Liang, Ten Statistical Ideas that Changed the World, (YouTube, Stanford course, Tibshirani's GitHub)

1 兆電晶體 GPU 的到來

鉅亨網新聞中心，台積電董事長劉德音撰文 談1兆電晶體GPU的到來，2024-03-29

文中指出，從 1997 年擊敗西洋棋人類冠軍的「深藍」，到 2023 年爆火的 ChatGPT，再過 15 年，人工智慧已經發展到可以「合成知識」(synthesize knowledge) 的地步，可以創作詩歌、創作藝術品、診斷疾病、編寫總結報告和電腦程式碼，甚至可以設計與人類製造的積體電路相媲美的積體電路。

End-to-End Predict-then-Optimize Library for Linear and Integer Programming

Bo Tang and Elias B. Khalil, PyEPO: A PyTorch-based End-to-End Predict-then-Optimize Library for Linear and Integer Programming, arXiv:2206.14234v2.

Customer Choice Models vs. Machine Learning

Jacob Feldman, Dennis J. Zhang, Xiaofei Liu, and Nannan Zhang (2021) Customer Choice Models vs. Machine Learning: Finding Optimal Product Displays on Alibaba. Operations Research 70(1):309-328. (Best OM Paper in Operations Research Award: Finalist, pdf, implementation details)

An Exact Solution to Wordle

 Dimitris Bertsimas, Alex Paskov (2024) An Exact Solution to Wordle. Operations Research.

Applications of Operations Research (作業研究) (including Optimization)

為了提高同學們的學習動機，提供以下相關的資訊，以幫助同學們找到方向。也和暑期實習和未來就業中，決策支援系統中的演算法有密切關聯。以下許多的內容屬於碩博士階段的課程，也可以增加同學們就讀研究所的動機：

• Journals: 
• INFORMS Journal on Applied Analytics
• INFORMS is the leading international association for Operations Research & Analytics professionals.
• The mission of INFORMS Journal on Applied Analytics is to publish manuscripts focusing on the practice of operations research and management science and the impact this practice has on organizations throughout the world. 
• Good topics to be explored for the final project
• Ramayya Krishnan and Pascal Van Hentenryck, editors, Advances in Integrating AI & O.R., INFORMS EC2021, Volume 16, April 19, 2021.

Interesting, Important, and Impactful Operations Management

Gerard P. Cachon, Karan Girotra, Serguei Netessine (2020) Interesting, Important, and Impactful Operations Management. Manufacturing & Service Operations Management 22(1):214-222. 

Sparse PCA: A New Scalable Estimator Based On Integer Programming

Kayhan Behdin and Rahul Mazumder, Sparse PCA: A New Scalable Estimator Based On Integer Programming, arXiv:2109.11142v2, 2021. (Julia ahd Gurobi code)

We consider the Sparse Principal Component Analysis (SPCA) problem under the well-known spiked covariance model. Recent work has shown that the SPCA problem can be reformulated as a Mixed Integer Program (MIP) and can be solved to global optimality, leading to estimators that are known to enjoy optimal statistical properties. However, current MIP algorithms for SPCA are unable to scale beyond instances with a thousand features or so. In this paper, we propose a new estimator for SPCA which can be formulated as a MIP. Different from earlier work, we make use of the underlying spiked covariance model and properties of the multivariate Gaussian distribution to arrive at our estimator. We establish statistical guarantees for our proposed estimator in terms of estimation error and support recovery. We propose a custom algorithm to solve the MIP which is significantly more scalable than off-the-shelf solvers; and demonstrate that our approach can be much more computationally attractive compared to earlier exact MIP-based approaches for the SPCA problem. Our numerical experiments on synthetic and real datasets show that our algorithms can address problems with up to 20000 features in minutes; and generally result in favorable statistical properties compared to existing popular approaches for SPCA.

Champion-level drone racing using deep reinforcement learning

Kaufmann, E., Bauersfeld, L., Loquercio, A. et al. Champion-level drone racing using deep reinforcement learning. Nature 620, 982–987 (2023). https://doi.org/10.1038/s41586-023-06419-4

First-person view (FPV) drone racing is a televised sport in which professional competitors pilot high-speed aircraft through a 3D circuit. Each pilot sees the environment from the perspective of their drone by means of video streamed from an onboard camera. Reaching the level of professional pilots with an autonomous drone is challenging because the robot needs to fly at its physical limits while estimating its speed and location in the circuit exclusively from onboard sensors. Here we introduce Swift, an autonomous system that can race physical vehicles at the level of the human world champions. The system combines deep reinforcement learning (RL) in simulation with data collected in the physical world. Swift competed against three human champions, including the world champions of two international leagues, in real-world head-to-head races. Swift won several races against each of the human champions and demonstrated the fastest recorded race time. This work represents a milestone for mobile robotics and machine intelligence, which may inspire the deployment of hybrid learning-based solutions in other physical systems.

Communication Efficient Fair and Robust Federated Learning

Yaodong Yu, Sai Praneeth Karimireddy, Yi Ma, and Michael I. Jordan, Scaff-PD: Communication Efficient Fair and Robust Federated Learning, arXiv:2307.13381.

We present Scaff-PD, a fast and communication-efficient algorithm for distributionally robust federated learning. Our approach improves fairness by optimizing a family of distributionally robust objectives tailored to heterogeneous clients. We leverage the special structure of these objectives, and design an accelerated primal dual (APD) algorithm which uses bias corrected local steps (as in Scaffold) to achieve significant gains in communication efficiency and convergence speed. We evaluate Scaff-PD on several benchmark datasets and demonstrate its effectiveness in improving fairness and robustness while maintaining competitive accuracy. Our results suggest that Scaff-PD is a promising approach for federated learning in resource-constrained and heterogeneous settings.