1-bit embeddings


It seems as if the mainstream ANN & LLM models and techniques are moving inexorably towards sparser and lower-resolution encodings and representations.  In this sense, they are converging on sparse 1-bit representations and learning mechanisms that have historically been inspired by computational neuroscience and which have at least a 35 year history with certain milestones described by spike timing codes, sparse distributed memories, rank order codes, sparse N-of-M codes and more recently hyperdimensional computing/VSAs and the BitBrain mechanism. We are going to suggest that instead of going via an expensive and circuitous route and entering this very interesting space indirectly through the back door, there is great value in developing and exploring the more direct route and entering through the front door! 

As well as the usually quoted engineering benefits such as latency, energy and a natural match to event-based sensors and neuromorphic compute, we believe that there are others less often discussed such as great robustness in the presence of internal or external noise and errors, opportunities for continuous learning and dealing with 'the binding problem'.











Go to group wiki Go to wiki users Info


Day Time Location
Wed, 01.05.2024 15:00 - 16:00 Sala panorama
Thu, 09.05.2024 15:00 - 16:00 Sala panorama


Jakub Fil
Michael Hopkins


Santiago Díaz Romero
Christian Fernandez Lorden
Jan Finkbeiner
Michael Hopkins
Edward Jones
Melissa Lober
Filippo Moro
Eleni Nisioti
Naresh Ravichandran