SUSTech Topology

Fall 2024

This semester, our discussion takes a concentrated form of Topological Data Analysis and Deep Learning Discussion Group on subsequent work to the paper Topology-enhanced machine learning for consonant recognition and on topological analysis and deep learning of human speech data. We meet Thursdays biweekly at 3 pm in M5024. Meanwhile, we relegate general seminar talks on applied and computational topology to the Graduate Topology Seminar.

Conference

There will be a Great Bay Area Topology Conference on topological data analysis and deep learning from November 22 to 24, 2024 at SUSTech. All are welcome.

Presentations and Discussions

Oct 31, '24, Yifei Zhu, A recap and what's next?

Nov 14, '24, Yuhe Qin, Gestalt Principle in Auditory Perception

Nov 28, '24, Zeyang Ding, Topological fingerprints for audio identification

Dec 12, '24, Coding session

Dec 26, '24, Siheng Yi, From graph neural networks to simplicial attention networks

Jan 10, '25, Writing session

Spring 2024

This semester, we aim to understand and experiment on two closely related aspects of applying topology to deep learning: (1) topology of artificial neural network (NN) architectures and (2) topological inputs for designing NNs. Among various topics related in a broad context, we place a continued emphasis on audio and speech signal processing and, more generally, on time series analysis. We meet Thursdays at 2 pm in M4009.

Presentations and Discussions

Jan 19, '24, Organizational meeting

Topology of Neural Network architectures: Feedforward NN $\to$ Recurrent NN $\to$ Reservoir Computing ($\to$ Time-Delayed RC) $\to$ engineering implementation (photonic circuits?) ... CLDNN?
Topological inputs for designing NNs: Convolutional NN $\to$ Topological CNN, in the sense of Carlsson et al. (from [Carlsson et al. 2008] to [Love et al. 2023])
How are the above two aspects related?
Report on [Feng et al.] and what's next?

This research drew inspiration from Carlsson and his collaborators' discovery of the Klein-bottle distribution of high-contrast, local patches of natural images [Carlsson et al. 2008], as well as their subsequent recent work on topological CNNs for learning image and even video data [Love et al. 2023]. By analogy, we aim to understand a distribution space for speech data, even a directed graph structure on it modeling the complex network of speech-signal sequences for practical purposes such as speaker diarisation, and how these topological inputs may enable smarter learning.

Other suggestions?
1. Comparative non-topological machine learning experiment, input spectrograms instead of PD/MP: Done on Jan 25 and improved on Feb 22.
2. How do weight vectors distribute in a CNN trained from the above item?
3. Sequence: Attention mechanism
4. librosa: audio and music processing in Python
5. Interpretable design of encoder–decoder bases from Dr. Yu's Dec 26, '23 talk on generative models for speech processing
Derivation of the Klein-filter formulas in [Love et al. 2023]

Feb 29, '24, Pingyao Feng, How to get a (non-topological) CNN up and running

Mar 7, '24, Zhiwang Yu, How to get a topological CNN up and running

Mar 14, '24, No seminar, Happy $\pi$ Day!

Mar 21, '24, Haiyu Zhang, How to capture the topology of a CNN over the course of training

Mar 28, '24, Zeyang Ding, How to integrate persistent homology into graph neural networks

Apr 4, '24, No seminar, Qingming Festival break

Apr 11, '24, Qingrui Qu, Topological, categorical, statistical, and machine learning methods in natural language processing and computational number theory, I

Apr 18, '24, Siheng Yi, An introduction to multiparameter persistence

Apr 25, '24, Zhiwang Yu and Haiyu Zhang, Progress report on the topology of convolutional neural networks learning phonetic data

May 9, '24, Qingrui Qu, Topological, categorical, statistical, and machine learning methods in natural language processing and computational number theory, II

Jun 19, '24 (2 pm in M5024 / Tencent Meeting: 520-800-4008) Pingyao Feng et al., Panel discussion/tutorial: Artificial-intelligence-aided academic writing, and other applications of computer hardwares/softwares in mathematical study and research

References

Gunnar Carlsson, Topological methods for deep learning (Carlsson also gave several online lectures on the topic.)

Ephy R. Love, Benjamin Filippenko, Vasileios Maroulas, and Gunnar Carlsson, Topological convolutional layers for deep learning

Gunnar Carlsson and Rickard Brüel Gabrielsson, Exposition and interpretation of the topology of neural networks

Gunnar Carlsson and Rickard Brüel Gabrielsson, Topological approaches to deep learning

Xing-Yue Duan, Xiong Ying, Si-Yang Leng, Jürgen Kurths, Wei Lin, and Huan-Fei Ma, Embedding theory of reservoir computing and reducing reservoir network using time delays

Yichen Shen, Nicholas C. Harris, Scott Skirlo et al., Deep learning with coherent nanophotonic circuits

T. N. Sainath, O. Vinyals, A. Senior, and H. Sak, Convolutional, long short-term memory, fully connected deep neural networks

Laura Alessandretti, Andrea Baronchelli, and Yang-Hui He, Machine learning meets number theory: The data science of Birch–Swinnerton-Dyer

Tai-Danae Bradley, Juan Luis Gastaldi, and John Terilla, The structure of meaning in language: Parallel narratives in linear algebra and category theory

Chaolong Ying, Xinjian Zhao, and Tianshu Yu, Boosting graph pooling with persistent homology

Gunnar Carlsson and Mikael Vejdemo-Johansson, Topological data analysis with applications

Applied and Computational Topology Seminar

Fall 2024

Conference

Presentations and Discussions

Spring 2024

Presentations and Discussions

References