Robust Stochastic Optimization Made Easy

Chen, Zhi, and Peng Xiong. 2021. RSOME in Python: an open-source package for robust stochastic optimization made easy. Optimization Online.

Chen, Zhi, Melvyn Sim, Peng Xiong. 2020. Robust stochastic optimization made easy with RSOME. Management Science 66(8) 3329–3339.

RSOME in Python (Also solver interfaces)

RSOME (Robust Stochastic Optimization Made Easy) is an open-source Python package for modeling generic optimization problems. Models in RSOME are constructed by variables, constraints, and expressions that are formatted as N-dimensional arrays. These arrays are consistent with the NumPy library in terms of syntax and operations, including broadcasting, indexing, slicing, element-wise operations, and matrix calculation rules, among others. In short, RSOME provides a convenient platform to facilitate developments of optimization models and their applications.

Training One Million Machine Learning Models in Record Time with Ray

Eric Liang and Robert Nishihara, Training One Million Machine Learning Models in Record Time with Ray, Anyscale, December 17, 2022.

Ray and Anyscale are used by companies like Instacart to speed up machine learning training workloads (often demand forecasting) by 10x compared with tools like Celery, AWS Batch, SageMaker, Vertex AI, Dask, and more.

In this blog, we’ll cover:

• Why companies are doing many model training
• How to use Ray to train multiple models
• The properties of Ray that enable efficient many model training 

The Forward-Forward Algorithm

Geoffrey Hinton, The Forward-Forward Algorithm: Some Preliminary Investigations, 2022. (code)

The aim of this paper is to introduce a new learning procedure for neural networks and to demonstrate that it works well enough on a few small problems to be worth serious investigation. The Forward-Forward algorithm replaces the forward and backward passes of backpropagation by two forward passes, one with positive (i.e. real) data and the other with negative data which could be generated by the network itself. Each layer has its own objective function which is simply to have high goodness for positive data and low goodness for negative data. The sum of the squared activities in a layer can be used as the goodness but there are many other possibilities, including minus the sum of the squared activities. If the positive and negative passes can be separated in time, the negative passes can be done offline, which makes the learning much simpler in the positive pass and allows video to be pipelined through the network without ever storing activities or stopping to propagate derivatives.

半導體產業的挑戰

魏哲家，半導體產業的挑戰，玉山科技論壇，2022-12-17 

為什麼聰明人會做蠢事 ?

姚怡平譯，為什麼聰明人會做蠢事? 顛覆高智商等於絕對聰明的常理，助你找出決策的關鍵智慧，商業周刊，2020

David Robson, The Intelligence Trap: Revolutionise Your Thinking and Make Wiser Decisions, Hodder & Stoughton, 2019.

Waki 瓦基，《為什麼聰明人會做蠢事？》讀書心得：掌握變聰明的３種方法，2021-09-01 (也有Podcast)

NeurIPS 2022

 Some information about NeurIPS 2022: 

• Outstanding Paper, Datasets And Benchmarks
• Nice summary by Jim (Linxi) Fan
• Google search "NeurIPS 2022 summary"
• 李宏毅 (Hung-yi Lee)，Chat GPT (可能)是怎麼煉成的 - GPT 社會化的過程

Data Science and Machine Learning

Dirk P. Kroese, Zdravko Botev, Thomas Taimre, and Radislav Vaisman, Data Science and Machine Learning: Mathematical and Statistical Methods, Chapman & Hall, 2019. (Lecture notes and Python files)

The purpose of Data Science and Machine Learning: Mathematical and Statistical Methods is to provide an accessible, yet comprehensive textbook intended for students interested in gaining a better understanding of the mathematics and statistics that underpin the rich variety of ideas and machine learning algorithms in data science.

The Impacts of Technological Invention on Economic Growth

A Reamer, The impacts of technological invention on economic growth–a review of the literature, The George Washington Institute of Public Policy, 2014. 

Interpretable Machine Learning

Cynthia Rudin, Chaofan Chen, Zhi Chen, Haiyang Huang, Lesia Semenova, and Chudi Zhong, Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges. Statistics Surveys, 2022. 

Interpretability in machine learning (ML) is crucial for high stakes decisions and troubleshooting. In this work, we provide fundamental principles for interpretable ML, and dispel common misunderstandings that dilute the importance of this crucial topic. We also identify 10 technical challenge areas in interpretable machine learning and provide history and background on each problem. Some of these problems are classically important, and some are recent problems that have arisen in the last few years. These problems are: (1) Optimizing sparse logical models such as decision trees; (2) Optimization of scoring systems; (3) Placing constraints into generalized additive models to encourage sparsity and better interpretability; (4) Modern case-based reasoning, including neural networks and matching for causal inference; (5) Complete supervised disentanglement of neural networks; (6) Complete or even partial unsupervised disentanglement of neural networks; (7) Dimensionality reduction for data visualization; (8) Machine learning models that can incorporate physics and other generative or causal constraints; (9) Characterization of the “Rashomon set” of good models; and (10) Interpretable reinforcement learning. This survey is suitable as a starting point for statisticians and computer scientists interested in working in interpretable machine learning.

培養思辨能力

• Derek Bok, Our Underachieving Colleges: A Candid Look at How Much Students Learn and Why They Should Be Learning More, Princeton University Press, 2005. (張善楠譯，大學教了沒？：哈佛校長提出的 8 門課，天下文化，2008)
• 以課程而言，書中的部份例子，(143 頁) 具備有量化素養 (quantitative literacy) 以培養思辨能力：算術能力、資料處理、電腦操作、模型化能力、統計概念、機率估算、推理能力

Data-driven research in retail operations

M. Qi, H.Y. Mak, and Z.J.M. Shen, Data‐driven research in retail operations—A review, Naval Research Logistics, 2020, 67 (8), 595-616. (Open access)

We review the operations research/management science literature on data-driven methods in retail operations. This line of work has grown rapidly in recent years, thanks to the availability of high-quality data, improvements in computing hardware, and parallel developments in machine learning methodologies. We survey state-of-the-art studies in three core aspects of retail operations—assortment optimization, order fulfillment, and inventory management. We then conclude the paper by pointing out some interesting future research possibilities for our community.