Cybernetics And Systems Analysis logo
Editorial Board Announcements Abstracts Authors Archive
KIBERNETYKA TA SYSTEMNYI ANALIZ
International Theoretical Science Journal
Volume 62 >>> № 4 JULY — AUGUST 2026
-->


DOI 10.34229/KCA2522-9664.26.4.1
UDC 519.23:004.855

O.S. Balabanov
Institute of Software Systems, National Academy of Sciences of Ukraine, Kyiv, Ukraine,
 olexastep@gmail.com


REGRESSION ANALYSIS AND CAUSAL MODELS (REVIEW)

Abstract. We present basic guidelines for utilizing a regression technique in analytical and predictive tasks, with emphasis on prediction of causal effects of interventions (manipulations). It is known that regression technique strives to automatically configure a model which provides best expression (under given restrictions) for a target variable in terms of available regressors. This makes a regression technique be destined for “passive prediction” task (“what value of target variable is expected when certain values of predictors are observed?”). But more often a researcher needs an answer the “active prediction” question: what value a target variable would obtain if certain predictors are (externally) set to chosen values (manipulated). To get answer to the question, one should modify respective part of the causal model inferred from observational data. To clarify suggestions and comments given in the text, we delineate several kinds of regression-based tasks. The kinds of tasks are the following: “passive prediction”; retro-recovery of target variable value; causal effect prediction; estimation of causal influence coefficients or structural coefficients; causal diagnostics and discovery; counterfactual analysis. We review and explain several basic approaches and techniques to the above tasks (mainly for linear models), including covariate adjustment. The extended back-door criterion and generalized versions of the instrumental variable methods are outlined. It is explained the distinction between regressor selection strategy for “passive prediction” and that for causal effect prediction.

Keywords: regression, causal effect, prediction, regression coefficient, causal relation, covariate adjustment.


full text

REFERENCES

    1. Kendall M.G., Stuart A. The advanced theory of statistics. Inference and relationship. London: C. Griffin, 1979. Vol. 2. 748 p.
    2. Draper N.R., Smith H. Applied regression analysis. N.Y.: John Wiley, 1998. 736 p. https://doi.org/10.1002/9781118625590.
    3. Cohen J., Cohen P., West S.G., Aiken L.S. Applied multiple regression/correlation analysis for the behavioral sciences. Mahwah, New Jersey: Lawrence Erlbaum Associates, Inc., 2003. 703 p.
    4. Johnson R., Wichern D. Applied multivariate statistical analysis. Pearson Education Ltd., 2014. 770 p.
    5. Harrell F.E. Regression modeling strategies. With applications to linear models, logistic and ordinal regression, and survival analysis. Cham: Springer, 2015. 582 p. https://doi.org/10.1007/978-3-319-19425-7.
    6. Efron B., Hastie T. Computer age statistical inference. N.Y.: Cambridge University Press, 2016. 475 p.
    7. Hastie T., Tibshirani R., Friedman J. The elements of statistical learning. N.Y.: Springer, 2009. 745 p. https://doi.org/10.1007/978-0-387-84858-7.
    8. Greene W.H. Econometric analysis. N.Y.: Pearson Education, Inc., 2018. 1126 p.
    9. Wooldridge J. Introductory econometrics. Modern approach. Mason, OH: South-Western, Cengage Learning, 2012. 910 p.
    10. Kline R.B. Principles and practice of Structural Equation Modeling. N.Y.: The Guilford Press, 2023. 494 p.
    11. Pearl J. Causality: models, reasoning, and inference. Cambridge: Cambridge Univ. Press, 2009. 464 p.
    12. Pearl J., Glymour M., Jewell N.P. Causal inference in statistics. A primer. John Wiley & Sons, 2016. 136 p.
    13. Hernїn M.A., Robins J.M. Causal inference: What if. Boca Raton: Chapman&Hall/CRC Press, 2024. 312 p.
    14. Spirtes P., Glymour C., Scheines R. Causation, prediction and search. N.Y.: MIT Press, 2001. 543 p.
    15. Balabanov O.S. Causal discovery from Markov properties under latent confounders. Cybernetics and Systems Analysis. 2024. Vol. 60, N 3. P. 359–374. https://doi.org/10.1007/s10559-024-00677-4.
    16. VanderWeele T.J., Shpitser I. On the definition of a confounder. The Annals of Statistics. 2013. Vol. 41, N 1. P. 196–220.
    17. Tian J., Pearl J. A general identification condition for causal effects. Proc. of the 18th National Conf. on Artificial Intelligence (AAAI-2002). 2002. Vol. 18. P. 567–573.
    18. Tian J. Identifying direct causal effects in linear models. Proc. of the 20th National Conf. on Artificial Intelligence (AAAI-2005). Proc. of AAAI. 2005. Vol. 20. P. 346–352.
    19. Shpitser I., VanderWeele T., Robins J.M. On the validity of covariate adjustment for estimating causal effects. Proc. of the 26th Conf. on Uncertainty in Artificial Intelligence (UAI-2010). Corvallis, Oregon: AUAI Press, 2010. P. 527–536.
    20. Maathuis M.H., Colombo D. A generalized back-door criterion. The Annals of Statistics. 2015. Vol. 43, N 3. P. 1060–1088.
    21. Van der Zander B., M., Textor J. Separators and adjustment sets in causal graphs: Complete criteria and an algorithmic framework. Artificial Intelligence. 2019. Vol. 270. P. 1–40. https://doi.org/10.1016/j.artint.2018.12.006.
    22. Chaudhuri S., Richardson T. Using the structure of d-connecting paths as a qualitative measure of the strength of dependence. Proc. of the 19th Conf. on Uncertainty in Artificial Intelligence (UAI-2003). 2003. P. 116–123. https://doi.org/10.48550/arXiv.1212.2469.
    23. Bowden R.J., Turkington D.A. Instrumental variables. Cambridge University Press, 1984. https://doi.org/10.1017/ccol0521262410.
    24. Henckel L., Buttenschoen M., Maathuis M.H. Graphical tools for selecting conditional instrumental sets. Biometrika. 2024. Vol. 111, N 3. P. 771–788. https://doi.org/10.1093/biomet/asad066.
    25. Balabanov O.S. Induced dependence, factor interaction, and discriminating between causal structures. Cybernetics and Systems Analysis. 2016. Vol. 52, N 1. P. 8–19. https://doi.org/10.1007/s10559-016-9794-5.
    26. Ding P., Miratrix L.W. To adjust or not to adjust? Sensitivity analysis of M-bias and butterfly-bias. Journal of Causal Inference. 2015. Vol. 3, N 1. P. 41–57. https://doi.org/10.1515/jci-2013-0021.
    27. VanderWeele T.J. Principles of confounder selection. European Journal of Epidemiology. 2019. Vol. 34, N 3, P. 211–219. Corrected publication 2021. https://doi.org/10.1007/s10654-019-00494-6.
    28. Greenland S. Invited commentary: variable selection versus shrinkage in the control of multiple confounders. American Journal of Epidemiology. 2008. Vol. 167, N 5. P. 523–529. https://doi.org/10.1093/aje/kwm355.
    29. Chen B., Pearl J. Graphical Tools for Linear Structural Equation Modeling. Tech. Report 432. CSD, University of California, Los Angeles, CA, 2025. 25 p. URL: https://bayes.cs.ucla.edu/jp_home.html.
    30. Brito C., Pearl J. A new identification condition for recursive models with correlated errors. Journal of Structural Equation Modeling. 2002. Vol. 9, N 4. P. 459–474. https://doi.org/10.1207/S15328007SEM0904_1.
    31. Brito C., Pearl J. A graphical criterion for the identification of causal effects in linear models. Proc. of the 18th National Conf. on Artificial Intelligence (AAAI). Menlo Park, CA: AAAI Press/The MIT Press, 2002. P. 533–538.
    32. Brito C., Pearl J. Graphical condition for identification in recursive SEM. Proc. of the 22nd Conf. on Uncertainty in Artificial Intelligence. Arlington, Virginia: AUAI Press, 2006. P. 47–54.
    33. Tian J. Parameter identification in a class of linear structural equation models. Proc. of the 21st Intern. Joint Conf. on Artificial Intelligence (IJCAI-2009). Palo Alto, CA: AAAI Press, 2009. P. 1970–1975. URL: https://www.ijcai.org/proceedings/2009.
    34. Foygel R., Draisma J., Drton M. Half-trek criterion for generic identifiability of linear structural equation models. The Annals of Statistics. 2012. Vol. 40, N 3. P. 1682–1713. https://doi.org/10.1214/12-AOS1012.
    35. Chen B., Kumor D., Bareinboim E. Identification and model testing in linear structural equation models using auxiliary variables. Proc. of the 34th Intern. Conf. on Machine Learning. 2017. P. 757–766.
    36. Kumor D., Chen B., Bareinboim E. Efficient identification in linear structural causal models with instrumental cutsets. Proc. Advances in Neural Information Processing Systems (NeurIPS). 2019. P. 12477–12486.
    37. Kumor D., Cinelli C., Bareinboim E. Efficient identification in linear structural causal models with auxiliary cutsets. Proc. Intern. Conf. on Machine Learning (ICML-2020). Proc. Machine Learning Research. 2020. Vol. 119. P. 5501–5510.
    38. Deorfler J., van der Zander B., Bleaser M., M. On the complexity of identification in linear structural causal models. arXiv:2407.12528 [cs.AI]. 2024. http://doi.org/10.48550/arXiv.2407.12528.
    39. Glymour C., Zhang K., Spirtes P. Review of causal discovery methods based on graphical models. Frontiers in Genetics. 2019. Vol. 10. Article number 524. https://doi.org/10.3389/fgene.2019.00524.
    40. Zanga A., Ozkirimli E., Stella F. A survey on causal discovery: Theory and practice. Intern. J. of Approximate Reasoning. 2022. Vol. 151. P. 101–129. https://doi.org/10.1016/j.ijar.2022.09.004.
    41. Cinelli C., Feller A., Imbens G., Kennedy E., Magliacane S., Zubizarreta J. Challenges in statistics: A dozen challenges in causality and causal inference. arXiv:2508.17099 [stat.ME]. 2025. http://doi.org/10.48550/arXiv.2508.17099.
    42. Balabanov O.S. Logic of causal inference from data under presence of latent confounders. Cybernetics and Systems Analysis. 2022. Vol. 58, N 2. P. 171–185. http://doi.org/10.1007/s10559-022-00448-z.
    43. Perkovi E., Textor J., Kalisch M., Maathuis M.H. Complete graphical characterization and construction of adjustment sets in Markov equivalence classes of ancestral graphs. Journal of Machine Learning Research. 2018. Vol. 18. P. 1–62.
    44. Rotnitzky A., Smucler E. Efficient adjustment sets for population average causal treatment effect estimation in graphical models. Journal of Machine Learning Research. 2020. Vol. 21. P. 1–86.
    45. Henckel L., Perkovic E., Maathuis M.H. Graphical criteria for efficient total effect estimation via adjustment in causal linear models. Journal of the Royal Statistical Society: Series B. 2022. Vol. 84. P. 579–599. http://doi.org/10.1111/rssb.12451.
    46. Smucler E., Sapienza F., Rotnitzky A. Efficient adjustment sets in causal graphical models with hidden variables. Biometrika. 2022. Vol. 109, N 1. P. 49–65. https://doi.org/10.1093/biomet/asab018.
    47. Jaber A., Ribeiro A., Zhang J., Bareinboim E. Causal identification under Markov equivalence: Calculus, algorithm, and completeness. 36th Conference on Neural Information Processing Systems, 2022. Advances in Neural Information Processing Systems. 2022. Vol. 35. P. 3679–3690.
    48. Handbook of causal analysis for social research. Morgan S.L. (Ed.). Dordrecht: Springer Science+Business Media, 2013. 424 p. https://doi.org/10.1007/978-94-007-6094-3.
    49. Cheng D., Li J., Liu L., Liu J., Le T.D. Data-driven causal effect estimation based on graphical causal modelling: A survey. ACM Computing Surveys. 2024. Vol. 56, N 5. P. 1–37. https://doi.org/10.1145/3636423.
    50. Witte J., Henckel L., Maathuis M.H., Didelez V. On efficient adjustment in causal graphs. Journal of Machine Learning Research. 2020. Vol. 21. P. 1–45. https://doi.org/10.3929/ETHZ-B-000459196.
    51. Zhang W., Liu L., Li J. Treatment effect estimation with disentangled latent factors. Proc. of the 35th AAAI Conf. on Artificial Intelligence (AAAI-2021). 2021. Vol. 35. P. 10923–10930.
    52. Van der Zander B., Textor J., Liskiewicz M. Efficiently finding conditional instruments for causal inference. Proc. of the 24th Intern. Joint Conf. on Artificial Intelligence (IJCAI-2015). AAAI Press, 2015. P. 3243–3249.
    53. Chen B., Pearl J., Bareinboim E. Incorporating knowledge into structural equation models using auxiliary variables. Proc. of the 25th Intern. Joint Conf. on Artificial Intelligence, (IJCAI-2016). 2016. P. 3577–3583.
    54. Van der Laan J.M., Rose S. Targeted learning: Causal inference for observational and experimental data. New York: Springer, 2011. 628 p. https://doi.org/10.1007/978-1-4419-9782-1.
    55. Cui P., Shen Zh., Li Sh. et al. Causal inference meets machine learning. Proc. of the 26th ACM SIGKDD Intern. Conf. on Knowledge Discovery & Data Mining. 2020. P. 3527–3528. https://doi.org/10.1145/3394486.3406460.
    56. The SAGE handbook of regression analysis and causal inference. Best H., Wolf C. (Eds.). London; Los Angeles; New Delhi; Singapore: SAGE, 2015. 414 p.



Editorial Board Announcements Abstracts Authors Archive
about scope office editorial board instructions for authors subscriptions contacts
© 2026 Kibernetika.org. All rights reserved.