Cybernetics And Systems Analysis logo
Editorial Board Announcements Abstracts Authors Archive
KIBERNETYKA TA SYSTEMNYI ANALIZ
International Theoretical Science Journal
Volume 62 >>> № 4 JULY — AUGUST 2026
-->


DOI 10.34229/KCA2522-9664.26.4.2
UDC 004.89

R. Pravosud
Taras Shevchenko National University of Kyiv, Kyiv, Ukraine,
 pronod9999@gmail.com

O. Marchenko
Taras Shevchenko National University of Kyiv, Kyiv, Ukraine,
 omarchenko@univ.kiev.ua


DIALOGUE STYLE TRANSFER FOR MULTIPLE CHARACTERS

Abstract. Dialogue style transfer aims to generate responses that preserve semantic coherence while adhering to a specified stylistic persona. Recent work has shown that reinforcement learning combined with parameter-efficient fine-tuning can effectively optimize stylistic objectives, but existing approaches are typically limited to a single character or rely on large, resource-intensive models. In this paper, we present a unified reinforcement learning framework for multi-character dialogue style transfer that is scalable to small, distilled, and quantized language models. Our approach combines Proximal Policy Optimization with Low-Rank Adaptation and a multi-class style reward model, enabling a single model to generate dialogue in multiple philosophical character styles through explicit conditioning. We evaluate the proposed method across multiple model sizes, including GPT2-Large, GPT2, DistilGPT2, and INT8-quantized GPT2, and compare against untuned GPT2 baselines. Experimental results show that reinforcement learning substantially improves stylistic accuracy over untuned models, while smaller models retain most of the performance at significantly reduced computational cost. These findings demonstrate that the proposed framework is effective, scalable, and suitable for deployment in low-resource environments.

Keywords: dialogue generation, NLP, reinforcement learning, style transfer, PEFT, PPO.


full text

REFERENCES

    1. Li J., Monroe W., Shi T., et al. Adversarial learning for neural dialogue generation. Proc. 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP) (September 2017, Copenhagen, Denmark). Copenhagen, 2017. P. 2157–2169. https://doi.org/10.18653/v1/ D17-1230.
    2. See A., Roller S., Kiela D., Weston J. What makes a good conversation? How controllable attributes affect human judgments. Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL) (June 2019, Minneapolis, USA). Minneapolis, 2019. Vol. 1, P. 1702–1723. https://doi.org/10.18653/v1/N19-1170.
    3. Radford A., Wu J., Child R., et al. Language models are unsupervised multitask learners. Technical report. OpenAI Blog. 2019. URL: https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf.
    4. Brown T.B., Mann B., Ryder N., et al. Language models are few-shot learners. NeurIPS. 2020. arXiv:2005.14165v4 [cs.CL] 22 Jul 2020. https://doi.org/10.48550/arXiv.2005.14165.
    5. Jin D., Jin Z., Hu Z., et al. Deep learning for text style transfer: A survey. Computational Linguistics. 2022. Vol. 48, Iss. 1. P. 155–205. https://doi.org/10.1162/v1/coli_a_00426.
    6. Li X.L., Liang P. Prefix-tuning: Optimizing continuous prompts for generation. Proc. 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (August 2021, online). Vol. 1. P. 4582–4597. https://doi.org/10.18653/v1/2021.acl-long.353.
    7. Peng N. Controllable text generation for open-domain creativity and fairness. Proc. Thirty-First International Joint Conference on Artificial Intelligence (IJCAI 2022) (23–29 July 2022, Vienna, Austria). Vienna, 2022. P. 5821–5825. https://doi.org/10.24963/ijcai.2022/818.
    8. Sutton R.S., Barto A.G. Reinforcement learning: An introduction. IEEE Transactions on Neural Networks. 1998. Vol. 9, N 5. P. 1054–1054. https://doi.org/10.1109/TNN.1998.712192.
    9. Hu E.J., Shen Y., Wallis P., et al. LoRA: Low-rank adaptation of large language models. arXiv:2106.09685v2 [cs.CL] 16 Oct 2021. https://doi.org/10.48550/arXiv.2106.09685.
    10. Xu H., Yan Z., Xuan J., et al. Improving proximal policy optimization with alpha divergence. Neurocomputing. 2023. Vol. 534. P. 94–105. https://doi.org/10.1016/j.neucom.2023.02.008.
    11. Sanh V., Debut L., Chaumond J., Wolf T. DistilBERT, a distilled version of BERT. arXiv:1910.01108v4 [cs.CL] 1 Mar 2020. https://doi.org/10.48550/arXiv.1910.01108.
    12. Zhang T., Kishore V., Wu F., et al. BERTScore: Evaluating text generation with BERT. arXiv:1904.09675v3 [cs.CL] 24 Feb 2020. https://doi.org/10.48550/arXiv.1904.09675.
    13. Zhu Y., Lu S., Zheng L., et al. Texygen: A benchmarking platform for text generation models. arXiv:1802.01886v1 [cs.CL] 6 Feb 2018. https://doi.org/10.48550/arXiv.1802.01886.
    14. Tu H., Yang Z., Yang J., et al. PCAE: A framework of plug-in conditional auto-encoder for controllable text generation. Knowledge-Based Systems. 2022. Vol. 256. Article number 109766. https://doi.org/10.1016/j.knosys.2022.109766.
    15. Ouyang L., Wu J., Jiang X., et al. Training language models to follow instructions with human feedback. Proc. 36th Conference on Neural Information Processing Systems (NeurIPS 2022) (28 November – 9 December 2022, New Orlean, USA). New Orlean, 2022. Advances in Neural Information Processing Systems 35. P. 27730–27744. URL: https://proceedings.neurips.cc/paper/2022/hash/b1efde53be364a73914f58805a001731-Abstract-Conference.html.
    16. Dettmers T., Lewis M., Belkada Y., Zettlemoyer L. LLM.int8(): 8-bit matrix multiplication for transformers at scale. arXiv:2208.07339v2 [cs.LG] 10 Nov 2022. https://doi.org/10.48550/arXiv.2208.07339.
    17. Stiennon N., Ouyang L., Wu J., et al. Learning to summarize with human feedback. Proc. 34th Conference on Neural Information Processing Systems (NeurIPS 2020) (6–12 December 2020, Virtual Event). Advances in Neural Information Processing Systems 33. P. 3008–3021. URL: https://proceedings.neurips.cc/paper/2020/hash/1f89885d556929e98d3ef9b86448f951-Abstract.html.
    18. Luo F., Li P., Zhou J., et al. A dual reinforcement learning framework for unsupervised text style transfer. arXiv.1905.10060v1 [cs.CL] 24 May 2019. https://doi.org/10.48550/arXiv.1905.10060.
    19. Pfeiffer J., Kamath A., Rckl A., et al. AdapterFusion: Non-destructive task composition for transfer learning. Proc. 16th Conference of the European Chapter of the European Chapter of the Association for Computational Linguistics (April 2021, online). Main vol. P. 487–503. https://doi.org/10.18653/v1/2021.eacl-main.39.
    20. Pravosud R., Adamov O., Marchenko O. Dialogue style transfer with reinforcement learning and parameter-efficient fine-tuning. In: Flexible Query Answering Systems. De Tr G., et al. (Eds.). FQAS 2025. Lecture Notes in Computer Science. 2025. Vol 16119. P. 232–240. https://doi.org/10.1007/978-3-032-05607-8_22.



Editorial Board Announcements Abstracts Authors Archive
about scope office editorial board instructions for authors subscriptions contacts
© 2026 Kibernetika.org. All rights reserved.