EventSession Formset ID 356

Event: ccnw24 2024
Type: Discussion
Title: 1-bit embeddings


It seems as if the mainstream ANN & LLM models and techniques are moving inexorably towards sparser and lower-resolution encodings and representations.  In this sense, they are converging on sparse 1-bit representations and learning mechanisms that have historically been inspired by computational neuroscience and which have at least a 35 year history with certain milestones described by spike timing codes, sparse distributed memories, rank order codes, sparse N-of-M codes and more recently hyperdimensional computing/VSAs and the BitBrain mechanism. We are going to suggest that instead of going via an expensive and circuitous route and entering this very interesting space indirectly through the back door, there is great value in developing and exploring the more direct route and entering through the front door! 

As well as the usually quoted engineering benefits such as latency, energy and a natural match to event-based sensors and neuromorphic compute, we believe that there are others less often discussed such as great robustness in the presence of internal or external noise and errors, opportunities for continuous learning and dealing with 'the binding problem'.




Speaker: Jakub Fil, Michael Hopkins,
Moderators: Jakub Fil, Michael Hopkins,
Schedule ID start time end time location
437 May 01 2024, 15:00 May 01 2024, 16:00 Sala panorama
473 May 09 2024, 15:00 May 09 2024, 16:00 Sala panorama
List all sessions Update Delete