UDC 330.115
1 International Institute for Applied Systems Analysis, Laxenburg, Austria
 ermol@iiasa.ac.at
|
2 International Institute for Applied Systems Analysis,
Laxenburg, Austria; V.M. Glushkov Institute of Cybernetics, National Academy of Sciences of Ukraine,
Kyiv, Ukraine
ermoliev@iiasa.ac.at
|
|
|
6 International Institute for Applied Systems Analysis, Laxenburg, Austria
kahil@iiasa.ac.at
|
CONNECTIONS BETWEEN ROBUST STATISTICAL ESTIMATION,
ROBUST DECISION=MAKING WITH TWO-STAGE STOCHASTIC
OPTIMIZATION, AND ROBUST MACHINE LEARNING PROBLEMS
Abstract. The paper discusses connections between the problems of two-stage stochastic programming, robust decision-making, robust statistical estimation, and machine learning. In the conditions of uncertainty, possible extreme events and outliers, these problems require quantile-based criteria, constraints, and “goodness-of-fit” indicators. The two-stage STO problems with quantile-based criteria can be effectively solved with the iterative stochastic quasigradient (SQG) solution algorithms. The SQG methods provide a new type of machine learning algorithms that can be effectively used for general-type nonsmooth, possibly discontinuous, and nonconvex problems, including quantile regression and neural network training. In general problems of decision-making, feasible solutions and concepts of optimality and robustness are characterized from the context of decision-making situations. Robust ML approaches can be integrated with disciplinary or interdisciplinary decision-making models, e.g., land use, agricultural, energy, etc., for robust decision-making in the conditions of uncertainty, increasing systemic interdependencies, and “unknown risks.”
Keywords: two-stage STO, robust decision-making and statistical estimation, robust quantile regression, machine learning, general problems of robust decision making, systemic risks, uncertainties.
full text
REFERENCES
- Ermoliev Yu., Wets R.J.-B. (Eds). Numerical Techniques for Stochastic Optimization. Heidelberg: Springer Verlag, 1988. 573 p.
- Ermoliev Y., Hordijk L. Global changes: Facets of robust decisions. In: Coping with Uncertainty: Modeling and Policy Issue. Marti K., Ermoliev Y., Makowski M., Pflug G. (Eds.). Berlin: Springer Verlag, 2003. P. 4–28.
- Ermoliev Y. Methods of Stochastic Programming. Moscow: Nauka, 1976. 340 p. (In Russian).
- Ermoliev Y. Stochastic quasigradient methods. In: Encyclopedia of Optimization. Pardalos P.M. (Ed.). New York: Springer Verlag, 2009. P. 3801–3807.
- Ermoliev Y. Two-stage stochastic programming: Quasigradient method. In: Encyclopedia of Optimization. Pardalos P.M. (Ed.). New York: Springer Verlag, 2009. P. 3955–3959.
- Ermoliev Y. Stochastic quasigradient methods in minimax problems. In: Encyclopedia of Optimization. Pardalos P.M. (Ed.). New York: Springer Verlag, 2009. P. 3813–3818.
- Ermoliev Y., Gaivoronski A. Stochastic quasigradient methods for optimization of discrete event systems. Annals of Operation Research. 1992. Vol. 39, Iss. 1. P. 1–39.
- Ermoliev Y., Zagorodny A.G., Bogdanov V.L., Ermolieva T., Havlik P., Rovenskaya E., Komendantova N., Obersteiner M. Robust food–energy–water–environmental security management: stochastic quasigradient procedure for linkage of distributed optimization models under asymmetric information and uncertainty. Cybernetics and Systems Analysis. 2022. Vol. 58, N 1. P. 45–57. https://doi.org/10.1007/s10559-022-00434-5 .
- Ermoliev Y., von Winterfeldt D. Systemic risk and security management. In: Managing Safety of Heterogeneous Systems. Ermoliev Y., Makowski M., Marti K. (Eds.). Lecture Notes in Economics and Mathematical Systems. 2012. Vol. 658. P. 19–49.
- Ermolieva T., Havlik, P., Fran, S., Kahi, T., Balkovi, J., SkalskН R., Ermoliev Y., Knopov P.S., Borodina O.M., Gorbachuk V.M. A risk-informed decision-making framework for climate change adaptation through robust land use and irrigation planning. Sustainability. 2022. Vol. 14, Iss. 3. P. 1430. https://doi.org/10.3390/su14031430 .
- Ermolieva T., HavlЗk P., Ermoliev Y., Mosnier A., Obersteiner M., Leclere D., Khabarov N., Valin H., Reuter W. Integrated management of land use systems under systemic risks and security targets: A stochastic global biosphere management model. Journal of Agricultural Economics. 2016. Vol. 67, Iss. 3. P. 584–601.
- Borodina O.M., Kyryziuk S.V., Fraier O.V., Ermoliev Y.M., Ermolieva T.Y., Knopov P.S., Horbachuk V.M. Mathematical modeling of agricultural crop diversification in ukraine: scientific approaches and empirical results. Cybernetics and Systems Analysis. 2020. Vol. 56, N 2. P. 213–222. https://doi.org/10.1007/s10559-020-00237-6.
- Gao J., Xu X., Cao G.-Y., Ermoliev Y., Ermolieva T., Rovenskaya E. Strategic decision-support modeling for robust management of the food–energy–water nexus under uncertainty. Journal of Cleaner Production. 2021. Vol. 292. 125995. https://doi.org/10.1016/j.jclepro.2021.125995.
- Ortiz-Partida J.P., Kahil T., Ermolieva T., Ermoliev Y., Lane B., Sandoval-Solis S., Wada Y. A two-stage stochastic optimization for robust operation of multipurpose reservoirs. Water Resources Management. 2019. Vol. 33, Iss. 11. P. 3815–3830. https://doi.org/10.1007/s11269-019-02337-1.
- Ren M., Xu X., Ermolieva T., Cao G.-Y., Yermoliev Y. The optimal technological development path to reduce pollution and restructure iron and steel industry for sustainable transition. International Journal of Science and Engineering Investigations. 2018. Vol. 7, Iss. 73. P. 100–105.
- Huber P. Robust Statistics. New York; Chichester; Brisbane; Toronto; Singapore: John Wiley & Sons, 1981. 317 p.
- Vapnik V.N. The Nature of Statistical Learning Theory. New York: Springer-Verlag, 1995. 334 p.
- Knopov P.S., Kasitskaya E.J. Empirical Estimates in Stochastic Optimization and Identification. Berlin: Springer Verlag, 2002. 250 p.
- Ermolieva T., Ermoliev Y., Obersteiner M., Rovenskaya E. Two-stage nonsmooth stochastic optimization and iterative stochastic quasigradient procedure for robust estimation, machine learning and decision making. In: Resilience in the Digital Age. Ch. 4. Roberts F.S., Sheremet I.A. (Eds.). Cham: Springer, 2021. P. 45–74.
- Nesterov Y. Introductory Lectures on Convex Optimization. New York: Springer New York, 2004. XVIII, 236 p.
- Reddi S.J., Hefny A., Sra S., Poczos B., Smola A.J. On variance reduction in stochastic gradient descent and its asynchronous variants. In: Advances in Neural Information Processing Systems 28. Cortes C., Lawrence N.D., Lee D.D., Sugiyama M., Garnett R. (Eds.). Proc. Annual Conference on Neural Information Processing Systems 2015 (7–12 December 2015, Montreal, Canada). Montreal, 2015. P. 2647–2655.
- Robbins H., Monro S. A stochastic approximation method. The Annals of Mathematical Statistics. 1951. Vol. 22, Iss. 3. P. 400–407.
- Polyak B.T., Juditsk A.B. Acceleration of stochastic approximation by averaging. SIAM Journal on Control and Optimization. 1992. Vol. 30, Iss. 4. P. 838–855.
- Clarke F.H. Optimization and Nonsmooth Analysis. New York: John Wiley & Sons, 1983. XIII, 308 p.
- Rockafeller T. The Theory of Subgradient and its Application to Problems of Optimization: Convex and Nonconvex Functions. Berlin: Heldermann Verlag, 1981. 107 p.
- Ermoliev Y., Shor N. On minimization of nondiferentiable functions. Kibernetika. 1967. Vol. 3, N 1. P. 101–102.
- Ermoliev Y., Norkin V. On nonsmooth and discontinuous problems of stochastic systems optimization. Europ. J. Oper. Res. 1997. Vol. 101, Iss. 2. P. 230–244.
- Gaivoronski A. Convergence properties of backpropagation for neural nets via theory of stochastic quasigradient methods: Part 1. Optimization Methods and Software. 1994. Vol. 4, Iss. 2. P. 117–134.
- Gorbachuk V.M., Ermoliev Y., Ermolieva T., Dunajevskij M.S. Quantile-based regression for the assessment of economic and ecological risks. Proc. 5th International Scientific Conference on Computational Intelligence (15–20 April 2019, Uzgorod, Ukraine). Uzgorod, 2019. P. 188–190.
