C&SA | Contents

Volume 61 >>> № 2 MARCH — APRIL 2025

-->

DOI 10.34229/KCA2522-9664.25.2.13

UDC 004.855.5

L. Smets¹, D. Rachkovskij², E. Osipov³,
O. Volkov⁴, W. van Leekwijck⁵, S. Latre⁶

¹ University of Antwerp, Department of Computer Science, and imec (IDLab), Antwerp, Belgium

Laura.Smets@uantwerpen.be

² Lulea University of Technology, Luleе, Sweden

Dmitri.Rachkovskij@ltu.se

Institute of Information Technologies and Systems, National Academy of Sciences of Ukraine, Kyiv, Ukraine

dar@infrm.kiev.ua

³ Luleе University of Technology, Lulea, Sweden

Evgeny.Osipov@ltu.se

⁴ Institute of Information Technologies and Systems, National Academy of Sciences of Ukraine, Kyiv, Ukraine

alexvolk@ukr.net

⁵ imec, Leuven, Belgium

Werner.VanLeekwijck@imec.be

⁶ University of Antwerp, Department of Computer Science, and imec (IDLab), Antwerp, Belgium

Steven.Latre@uantwerpen.be

CLASSIFICATION PERFORMANCE OF CONFIDENCE-DRIVEN CENTROIDS

Abstract. Hyperdimensional computing (HDC) is a powerful algorithmic framework at the intersection of symbolic and neural network Artificial Intelligence. In particular, HDC has received significant atten- tion as a suitable candidate for low-resource machine learning tasks, exemplified by wearable Internet of Things. To solve classification tasks, HDC transforms input data to a high-dimensional space and uses simple component-wise vector operations to create, train, and operate the classification model. While the classical centroid model has been often used in HDC, iterative updating of centroids with wrongly classified samples improves the classification performance. In this paper, using a large and variable collection of 121 UCI datasets, we explore how confidence-driven training of centroids formed from HDC representations further improves the classification accuracy.

Keywords: centroid, linear classifier, non-linear data transformation, hyperdimensional computing, vector symbolic architecture.

full text

REFERENCES

1. Kanerva P. Hyperdimensional computing: An introduction to computing in distributed representation with high-dimensional random vectors. Cognitive Computation. 2009. Vol. 1, Iss. 2. P. 139–59. https://doi.org/10.1007/ .

2. Gayler R.W. Vector symbolic architectures answer Jackendoff’s challenges for cognitive neuroscience. arXiv:cs/0412059v1 [cs.NE] 13 Dec 2004. https://doi.org/10.48550/ .

3. Rahimi A., Kanerva P., Benini L., Rabaey J.M. Efficient biosignal processing using hyperdimensional computing: Network templates for combined learning and classification of ExG signals. Proceedings of the IEEE. 2019. Vol. 107, N 1. P. 123–143. https://doi.org/10.1109/ .

4. Widdows D., Cohen T. Reasoning with vectors: A continuous model for fast robust inference. Logic Journal of the IGPL. 2015. Vol. 23, Iss. 2. P. 141–173. https://doi.org/10.1093/ .

5. Vdovychenko R., Tulchinsky V. Increasing the semantic storage density of sparse distributed memory. Cybernetics and Systems Analysis. 2022. Vol. 58, N. 3. P. 331–342. https://doi.org/10.1007/ .

6. Vdovychenko R., Tulchinsky V. Sparse distributed memory for sparse distributed data. In: Intelligent Systems and Applications. IntelliSys 2022. Arai K. (Eds). LNNS. Cham: Springer, 2022. Vol 542. P. 74–81. https://doi.org/10.1007/ .

7. Rachkovskij D.A. Linear classifiers based on binary distributed representations. Intern. J. Inform. Theories & Appl. 2007. Vol. 14, N 3. P. 270–274.

8. Rahimi A., Kanerva P., Rabaey J.M. A robust and energy-efficient classifier using brain-inspired hyperdimensional computing. Proc. 2016 International Symposium on Low Power Electronics and Design (ISLPED’16) (8–10 August 2016, San Francisco Airport CA USA). San Francisco, 2016. P. 64–69. https://doi.org/10.1145/ .

9. Мисуно И.С., Рачковский Д.А., Слипченко С.В., Соколов А.М. Поиск текстовой информации с помощью векторных представлений. Проблеми програмування. 2005. № 4. С. 50–59. http://dspace.nbuv.gov.ua/ .

10. Imani M., Kong D., Rahimi A., Rosing T. Voicehd: Hyperdimensional computing for efficient speech recognition. Proc. 2017 IEEE International Conference on Rebooting Computing (ICRC) (08–09 November 2017, Washington, DC, USA). Washington, DC, 2017. P. 1–8. https://doi.org/10.1109/ .

11. Kim Y., Imani M., Rosing T.S. Efficient human activity recognition using hyperdimensional computing. Proc. 8th International Conference on the Internet of Things (IOT’18) (15–18 October 2018, Santa Barbara, California, USA). Santa Barbara, 2018. P. 1–6. https://doi.org/10.1145/ .

12. Moin A., Zhou A., Rahimi A., et al. A wearable biosensing system with in-sensor adaptive machine learning for hand gesture recognition. Nature Electronics. 2021. Vol. 4. P. 54–63. https://doi.org/10.1038/ .

13. Zhou A., Muller R., Rabaey J. Memory-efficient, limb position-aware hand gesture recognition using hyperdimensional computing. arXiv:2103.05267v1 [cs.LG] 9 Mar 2021. https://doi.org/10.48550/ .

14. Schlegel K., Neubert P., Protzel P. HDC-minirocket: Explicit time encoding in time series classification with hyperdimensional computing. arXiv:2202.08055v1 [cs.LG] 16 Feb 2022. https://doi.org/10.48550/ .

15. Rachkovskij D.A. Shift-equivariant similarity-preserving hypervector representations of sequences. Cognitive Computation. 2024. Vol. 16, Iss. 3. P. 909–923. https://doi.org/10.1007/ .

16. Kleyko D., Khan S., Osipov E., Yong S.P. Modality classification of medical images with distributed representations based on cellular automata reservoir computing. Proc. 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017) (18–21 April 2017, Melbourne, VIC, Australia). Melbourne, 2017. P. 1053–1056. https://doi.org/10.1109/.

17. Watkinson N., Givargis T., Joe V., Nicolau A., Veidenbaum A. Detecting COVID-19 related pneumonia on ct scans using hyperdimensional computing. Proc. 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) (01–05 November 2021, Mexico). Mexico, 2021. P. 3970–3973. https://doi.org/10.1109/ .

18. Lukovich V., Goltsev A., Rachkovskij D. Neural network classifiers for micromechanical equipment diagnostics and micromechanical product quality inspection. Proc. EUFIT’97 (8–11 September 1997, Aachen, Germany). Aachen, 1997. P. 534–536.

19. Kussul E.M., Kasatkina L.M., Rachkovskij D.A., Wunsch D.C. Application of random threshold neural networks for diagnostics of micro machine tool condition. Proc. 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227) (04–09 May 1998, Anchorage, AK, USA). Anchorage, 1998. P. 241–244. https://doi.org/10.1109/ .

20. Goltsev A., Rachkovskij D. Combination of the assembly neural network with a perceptron for recognition of handwritten digits arranged in numeral strings. Pattern Recognition. 2005. Vol. 38, Iss. 3. P. 315–322. https://doi.org/10.1016/ .

21. Manabat A.X., Marcelo C.R., Quinquito A.L., Alvarez A. Performance analysis of hyper- dimensional computing for character recognition. Proc. 2019 International Symposium on Multimedia and Communication Technology (ISMAC) (19–21 August 2019, Quezon City, Philippines). Quezon City, 2019. P. 1–5. https://doi.org/10.1109/ .

22. Rachkovskij D.A. Representation of spatial objects by shift-equivariant similarity-preserving hypervectors. Neural Computing and Applications. 2022. Vol. 34, Iss. 24. P. 22387–22403. https://doi.org/10.1007/.

23. Smets L., Leekwijck W.V., Tsang I.J., LatrБ S. An encoding framework for binarized images using hyperdimensional computing. Frontiers in Big Data. 2024. Vol. 7. https://doi.org/10.3389/ .

24. Neubert P., Schubert S., Protzel P. An introduction to hyperdimensional computing for robotics. KI — Kunstliche Intelligenz. 2019. Vol. 33, Iss. 4. P. 319–330, https://doi.org/10.1007/ .

25. Kleyko D., Rachkovskij D.A., Osipov E., Rahimi A. A survey on hyperdimensional computing aka vector symbolic architectures, part II: Applications, cognitive models, and challenges. ACM Computing Serveys. 2023. Vol. 55, Iss. 9. Article number 175. P. 1–52. https://doi.org/ 10.1145/ .

26. Ge L., Parhi K.K. Classification using hyperdimensional computing: A review. IEEE Circuits and Systems Magazine. 2020. Vol. 20, Iss. 2. P. 30–47. https://doi.org/10.1109/ .

27. Aygun S., Moghadam M.S., Najafi M.H., Imani M. Learning from hypervectors: A survey on hypervector encoding. arXiv:2308.00685v1 [cs.LG] 1 Aug 2023. https://doi.org/10.48550/ .

28. VergБs P., Heddes M., Nunes I., Givargis T., Nicolau A. Classification using hyperdimensional computing: A review with comparative analysis. Research Square, 2023. Preprint. https://doi.org/10.21203/ .

29. Imani M., Morris J., Bosch S., Shu H., Micheli G.D., Rosing T. AdaptHD: Adaptive efficient training for brain-inspired hyperdimensional computing. Proc. 2019 IEEE Biomedical Circuits and Systems Conference (BioCAS) (17–19 October 2019, Nara, Japan). Nara, 2019. P. 1–4. https://doi.org/10.1109/ .

30. Imani M., Bosch S., Datta S., Ramakrishna S., Salamat S., Rabaey J.M., Rosing T. QuantHD: A quantization framework for hyperdimensional computing. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems. 2019. Vol. 39, Iss. 10. P. 2268–2278. https://doi.org/10.1109/ .

31. Imani M., Bosch S., Javaheripi M., Rouhani B., Wu X., Koushanfar F., Rosing T. SemiHD: Semi-supervised learning using hyperdimensional computing. Proc. 2019 IEEE/ACM International Conference on Computer-Aided Design (ICCAD) (04–07 November 2019, Westminster, CO, USA). Westminster, 2019. P. 1–8. https://doi.org/10.1109/ .

32. Imani M., Salamat S., Khaleghi B., Samragh M., Koushanfar F., Rosing T. SparseHD: Algorithm-hardware co-optimization for efficient high-dimensional computing. Proc. 2019 IEEE 27th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM) (28 April – 01 May 2019, San Diego, CA, USA). San Diego, 2019. P. 190–198. https://doi.org/10.1109/ .

33. Morris J., Imani M., Bosch S., Thomas A., Shu H., Rosing T. CompHD: Efficient hyperdimensional computing using model compression. Proc. 2019 IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED) (29–31 July 2019, Lausanne, Switzerland). Lausanne, 2019. P. 1–6. https://doi.org/10.1109/ .

34. Hernїndez-Cano A., Matsumoto N., Ping E., Imani M. OnlineHD: Robust, efficient, and single- pass online learning using hyperdimensional system. Proc. 2021 Design, Automation & Test in Europe Conference & Exhibition (DATE) (01–05 February 2021, Grenoble, France). Grenoble, 2021. P. 56–61. https://doi.org/10.23919/ .

35. Zou Z., Kim Y., Imani F., Alimohamadi H., Cammarota R., Imani M. Scalable edge-based hyperdimensional learning system with brain-like neural adaptation. Proc. International Conference for High Performance Computing, Networking, Storage and Analysis (SC’21) (14–19 November 2021, St. Louis Missouri, USA). St. Louis, 2021. Article number 38. https://doi.org/10.1145/ .

36. Duan S., Liu Y., Ren S., Xu X. LeHDC: Learning-based hyperdimensional computing classifier. arXiv:2203.09680v2 [cs.LG] 1 Apr 2022. https://doi.org/10.48550/ .

37. Pale U., Teijeiro T., Atienza D. Multi-centroid hyperdimensional computing approach for epileptic seizure detection. Frontiers in Neurology. 2022. Vol. 13. https://doi.org/10.3389/ .

38. Wang J., Huang S., Imani M. DistHD: A learner-aware dynamic encoding method for hyper- dimensional classification. Proc. 2023 60th ACM/IEEE Design Automation Conference (DAC) (09–13 July 2023, San Francisco, CA, USA). San Francisco, 2023. P. 1–6. https://doi.org/10.1109/ .

39. Fernїndez-Delgado M., Cernadas E., Barro S., Amorim D., Fernїndez-Delgado A. Do we need hundreds of classifiers to solve real world classification problems? Journal of Machine Learning Research. 2014. Vol. 15, N 1. P. 3133–3181. https://dl.acm.org/doi/ .

40. Plate T.A. Holographic reduced representations. IEEE Transactions on Neural Networks. 1995. Vol. 6, N 3. P. 623–641. https://doi.org/10.1109/ .

41. Plate T.A. Holographic reduced representation: Distributed representation for cognitive structures. Stanford: CSLI Publications, 2003. 300 p.

42. Gayler R.W. Multiplicative binding, representation operators & analogy. Proc. Advances in Analogy Research: Integration of Theory and Data from the Cognitive, Computational, and Neural Sciences (17–20 July 1998, Sofia, Bulgaria). Sofia, 1998.

43. Kanerva P. The spatter code for encoding concepts at many levels. Proc. International Conference on Artificial Neural Networks (ICANN) (26–29 May 1994, Sorrento, Italy) Sorrento, 1994. P. 226–229. https://doi.org/10.1007/ .

44. Kanerva P. Binary spatter-coding of ordered k-tuples. Proc. 6th International Conference on Artificial Neural Networks (ICANN 96) (16–19 July 1996, Bochum, Germany). Bochum, 1996. P. 869–873. https://doi.org/10.1007/ .

45. Laiho M., Poikonen J.H., Kanerva P., Lehtonen E. High-dimensional computing with sparse vectors. Proc. 2015 IEEE Biomedical Circuits and Systems Conference (BioCAS) (22–24 October 2015, Atlanta, GA, USA). Atlanta, 2015. P. 1–4. https://doi.org/10.1109/ .

46. Rachkovskij D.A., Slipchenko S.V., Kussul E.M., Baidyk T.N. Sparse binary distributed encoding of scalars. Journal of Automation and Information Sciences. 2005. Vol. 37, Iss. 6. P. 12–23. https://doi.org/10.1615/.

47. Kleyko D., Rahimi A., Rachkovskij D.A., Osipov E., Rabaey J.M. Classification and recall with binary hyperdimensional computing: Tradeoffs in choice of density and mapping characteristics. IEEE Transactions on Neural Networks and Learning Systems. 2018. Vol. 29, Iss. 12. P. 5880–5898. https://doi.org/10.1109/TNNLS.2018.2814400 .

48. Rachkovskij D.A., Fedoseeva T.V. On audio signals recognition by multilevel neural network. Proc. The International Symposium on Neural Networks and Neural Computing (NEURONET’90) (10–14 September 1990, Prague, Czechoslovakia). Prague, 1990. P. 281–283. https://www.researchgate.net/ .

49. Kussul E.M., Rachkovskij D.A., Baidyk T.N. On image texture recognition by associative-projective neurocomputer. Proc. ANNIE’91 Conference “Intelligent Engineering Systems through Artificial Neural Networks” (10–13 November 1991, St. Louis, MO, USA) St. Louis, 1991. P. 453–458. https://www.researchgate.net/ .

50. Rachkovskij D.A., Slipchenko S.V., Misuno I.S., Kussul E.M., Baidyk T.N. Sparse binary distributed encoding of numeric vectors. Journal of Automation and Information Sciences. 2005. Vol. 37, Iss. 11. P. 47–61. https://doi.org/10.1615/ .

51. Smets L., Leekwijck W.V., Tsang I.J., Latre S. Training a hyperdimensional computing classifier using a threshold on its confidence. Neural Computation. 2023. Vol. 35, Iss. 12. P. 2006–2023. https://doi.org/10.1162/ .

52. Nazemi M., Esmaili A., Fayyazi A., Pedram M. SynergicLearning: Neural network-based feature extraction for highly-accurate hyperdimensional learning. Proc. IEEE/ACM International Conference on Computer-Aided Design (ICCAD’20) (2–5 November 2020, Virtual Event, USA). Virtual Event, 2020. Article Number 89. P. 1–9. https://doi.org/10.1145/ .

53. Imani M., Yin X., Messerly J., Gupta S., Niemier M., Hu X.S., Rosing T. SearcHD: A memory-centric hyperdimensional computing with stochastic training. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems. 2020. Vol. 39, Iss. 10. P. 2422–2433. https://doi.org/10.1109/ .

54. Hsiao Y.-R., Chuang Y.-C., Chang C.-Y., Wu A.-Y. Hyperdimensional computing with learnable projection for user adaptation framework. In: Artificial Intelligence Applications and Innovations. AIAI 2021. Maglogiannis I., Macintyre J., Iliadis L. (Eds.). IFIP Advances in Information and Communication Technology. Cham: Springer, 2021. Vol 627. P. 436–447. https://doi.org/10.1007/.

55. Huang C.-T., Chang C.-Y., Chuang Y.-C., Wu A.-Y. PQ-HDC: Projection-based quantization scheme for flexible and efficient hyperdimensional computing. Proc. 17th IFIP WG 12.5 International Conference (AIAI 2021) (25–27 June 2021, Hersonissos, Crete, Greece). IFIPAICT. 2021. Vol. 627. P. 425–435. https://doi.org/10.1007/ .

56. Heddes M., Nunes I., Givargis T., Nicolau A., Veidenbaum A. Hyperdimensional hashing: A robust and efficient dynamic hash table. Proc. 59th ACM/IEEE Design Automation Conference (DAC’22) (10–14 July 2022, San Francisco, California, USA). San Francisco, 2022. P. 907–912. https://doi.org/10.1145/ .

57. Kussul E., Baidyk T., Kasatkina L., Lukovich V. Rosenblatt perceptrons for handwritten digit recognition. Proc. International Joint Conference on Neural Networks (IJCNN’01) (Cat. No.01CH37222) (15–19 July 2001, Washington, DC, USA). Washington, 2001. Vol. 2. P. 1516–1520. https://doi.org/10.1109/ .

58. Kussul E., Baidyk T. Improved method of handwritten digit recognition tested on MNIST database. Image and Vision Computing. 2004. Vol. 22, Iss. 12. P. 971–981. https://doi.org/10.1016/ .

59. VergБs P., Givargis T., Nicolau A. RefineHD: Accurate and efficient single-pass adaptive learning using hyperdimensional computing. Proc. 2023 IEEE International Conference on Rebooting Computing (ICRC) (05–06 December 2023, San Diego, CA, USA). San Diego, 2023. P. 1–8. https://doi.org/10.1109/ .

60. Kleyko D., Kheffache M., Frady E.P., Wiklund U., Osipov E. Density encoding enables resource-efficient randomly connected neural networks. IEEE Transactions on Neural Networks and Learning Systems. 2021. Vol. 32, Iss. 8. P. 3777–3783. https://doi.org/10.1109/ .

61. Frady E.P., Kleyko D., Sommer F.T. Variable binding for sparse distributed representations: Theory and applications. IEEE Transactions on Neural Networks and Learning Systems. 2023. Vol. 34, Iss. 5. P. 219–2204. https://doi.org/10.1109/ .