DOI
10.34229/KCA2522-9664.26.4.3
UDC 004.8
D. Zharkov
V.M. Glushkov Institute of Cybernetics, National Academy of Sciences of Ukraine,
Kyiv, Ukraine, 
dima.zharkov@gmail.com
V. Tulchinsky
V.M. Glushkov Institute of Cybernetics, National Academy of Sciences of Ukraine,
Kyiv, Ukraine, dep145@gmail.com
А METHOD TO IMPROVE THE QUALITY OF MULTI-CLASS
CLASSIFICATION USING LLM
Abstract. The quality of classification with large language models in automatic mode decreases sharply when the number of alternatives raises. The results of the problem experimental study are presented on the example of automatic assignment of advertisements to a certain category. An approach based on a combination of query engineering methods with pairwise comparisons is proposed.
Keywords: machine learning, artificial intelligence, large language models, multiclass classification, word vector, pairwise comparison method.
full text
REFERENCES
- Miller G.A. The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review. 1956. Vol. 63, Iss. 2. P. 81–97. https://doi.org/10.1037%2Fh0043158.
- Saaty T.L. Relative measurement and its generalization in decision making: Why pairwise comparisons are central in mathematics for the measurement of intangible factors — the analytic hierarchy/network process. Review of the Royal Academy of Exact, Physical and Natural Sciences. Series A: Mathematics (RACSAM). 2008. Vol. 102, Iss. 2. P. 251–318. https://doi.org/10.1007%2Fbf03191825.
- Prompt engineering guide. URL: https://www.promptingguide.ai/ (дата звернення 12.02.2025).
- Lu Z., Tian J., Wei W., et al. Mitigating boundary ambiguity and inherent bias for text classification in the era of large language models. arXiv:2406.07001v1 [cs.CL] 11 Jun 2024. https://doi.org/10.48550/arXiv.2406.07001.
- Al Faraby S., Romadhony A., Adiwijaya. Analysis of LLMs for educational question classification and generation. Computers and Education: Artificial Intelligence. 2024. Vol. 7. Article number 100298. https://doi.org/10.1016/j.caeai.2024.100298.
- Al Faraby S., Adiwijaya, Romadhony A. Educational question classification with pre-trained language models. Proc. 2022 Seventh International Conference on Informatics and Computing (ICIC) (08-09 December 2022, Denpasar, Bali, Indonesia). Denpasar, 2022. P. 1–6. https://doi.org/10.1109/ICIC56845.2022.10006957.
- Kostina A., Dikaiakos M.D., Stefanidis D., Pallis G. Large language models for text classification: Case study and comprehensive review. arXiv:2501.08457v1 [cs.CL] 14 Jan 2025. https://doi.org/10.48550/arXiv.2501.08457.
- Wysocki W., Ochodek M. Leveraging LLM-based data augmentation for automatic classification of recurring tasks in software development projects. Journal of Systems and Software. 2025. Vol. 231. Article number 112641. https://doi.org/10.1016/j.jss.2025.112641.
- Kokkodis M., Demsyn-Jones R., Raghavan V. Beyond the hype: Embeddings vs. prompting for multiclass classification tasks. arXiv:2504.04277v3 [cs.LG] 11 Nov 2025. https://doi.org/10.48550/arXiv.2504.04277.
- Qin Z., Jaderman R., Hui K., et al. Large language models are effective text rankers with pairwise ranking prompting. arXiv:2306.17563v2 [cs.IR] 28 Mar 2024. https://doi.org/10.48550/arXiv.2306.17563.
- Ads-Board dataset description. URL: https://github.com/dimazharkov/datasets/tree/main/ ads-board (дата звернення 12.02.2026).
- SentenceTransformers documentation. URL: https://sbert.net/ (дата звернення 12.02.2026).
- Ukrainian paraphrase multilingual MPNet base. URL: https://huggingface.co/lang-uk/ ukr-paraphrase-multilingual-mpnet-base (дата звернення 12.02.2026).
- LLM-driven text classification experiments. URL: https://github.com/dimazharkov/llm-classification (дата звернення 12.02.2026).
