Ph.D., Electrical Engineering
Indian Institute of Technology Kanpur
adhirajbanerjee35@gmail.comResearch Portfolio
Ph.D., Electrical Engineering, IIT Kanpur
I work on self-supervised learning, neural tokenization, sequence compression, audio retrieval, source separation, and codec-native audio intelligence. My research develops compact neural tokenizers for efficient multimodal foundation models, as well as codec-native downstream audio processing systems that operate directly over neural codec representations.
Research Interests & Directions
A high-level gallery of the research directions I am developing across paired-view predictive learning, autoregressive neural tokenization, sequence compression, codec-native audio processing, universal sound separation, audio retrieval, and multimodal foundation models.
Tap or click the image again to return to the page.
Research Positioning
My work develops neural tokenizers and sequence compressors that transform continuous sensory signals or long foundation-model token streams into compact symbolic representations for prediction, generation, reasoning, adaptation, retrieval, and efficient downstream modeling.
PairAlign formulates tokenization as conditional sequence generation: an encoder conditions an autoregressive decoder to emit ultra-low-bit-rate token streams while preserving information needed for retrieval, prediction, and downstream modeling.
CodecSep studies neural audio codec latents as computational operating spaces, showing that source-compositional structure can be extracted directly through latent masking. This supports compute-efficient codes-in / latent- processing / codes-out systems for downstream edge/server audio processing, and more broadly motivates structured latent spaces as inspectable interfaces for reliable multimodal foundation models.
I independently formulate problems, design objectives, implement training systems, run experiments, analyze failures, write papers, respond to reviews, and maintain open-source releases.
Selected Publications
First-author papers, written with my Ph.D. supervisor Dr. Vipul Arora as the sole co-author, on self-supervised audio representation learning, neural audio tokenization, acoustic word embeddings, semantic audio retrieval, sequence compression, universal source separation, and codec-native audio intelligence. Together, these works develop compact symbolic and latent interfaces that improve the efficiency, scalability, and downstream utility of audio, speech, and multimodal foundation models.
Open Source
Prompt-guided universal source separation on neural audio codec latents.
wav2tok 2.0Scalable neural audio tokenizer with explicit pairwise token alignment.
wav2tokICLR 2023 deep sequence tokenizer for audio retrieval.
Enc-Dec Pairwise Prediction AWEINTERSPEECH 2023 encoder-decoder acoustic word embedding implementation.
Talks and Presentations
YouTube videos can be played inline; playlists and external presentation pages open in a new tab.
Industry Research Experience
Generative Speech AI Research Intern
Contributed to speech and generative audio research focused on source separation, speech enhancement, and audio generation. Implemented, evaluated, and prototyped deep learning audio models using GPU-cluster-based training and inference workflows. Led quantization and deployment of a speech enhancement model under a tight two-day timeline, and delivered technical lectures to researchers and team leads on SSL, source separation, and generative audio modeling.
Research Consultant / Short-Term Work-for-Hire Research Engineer
Contributed to early-stage research and engineering on neural tokenizers, sequence compression, self-supervised learning, and source separation. Helped convert under-specified internal problem statements into modeling directions and modular training/evaluation pipelines. Worked with DDP/FSDP workflows and H200 GPU clusters, and delivered technical lectures to the team and founders with backgrounds including Google Deepmind and Meta.
Funded Applied Projects
Applied research projects translating neural audio retrieval and speech search methods into deployable systems for public-interest Indian-language and broadcast-archive use cases, including work supported through MeitY funding.
Worked on deploying efficient speech retrieval systems for Indian languages as part of a project funded by the Ministry of Electronics and Information Technology (MeitY), India. Received MeitY project funding/support through this work.
Contributed to a content retrieval system for audio archives using speech and text queries. The project was funded by Prasar Bharati, India.
Leadership & Competitions
Competition experience demonstrating technical leadership, team coordination, and applied machine learning system design under real-time, low-power, and benchmark-driven constraints.
Participated in and led a team in a real-time anomaly detection challenge using embedded sensor data from ground and aerial autonomous systems. Developed an unsupervised pipeline to detect behavioral abnormalities in perimeter-monitoring robots.
Led a team that won 2nd prize in an international competition on real-time anomaly detection for autonomous ground vehicles using sensor data. Developed an unsupervised system to detect abnormal behavior in heterogeneous multi-robot environments under real-time and low-power constraints. The challenge was held as part of IEEE ICAS 2021 and sponsored by the IEEE Signal Processing Society.
Participated in and led a team in an NLP competition to detect and rate humor and offensive content in social media posts. Designed a BERT-based system fine-tuned on humor and sentiment datasets.
Academic Journey
Formal academic training across electronics and communication engineering, signal processing, machine learning, and audio representation learning, with degree and seminar documentation linked for reviewer access.
Indian Institute of Technology Kanpur
Ph.D. affiliation in the Department of Electrical Engineering, IIT Kanpur, supervised by Dr. Vipul Arora. My doctoral work develops structured audio tokens and codec-native representations for retrieval, source separation, and prompt-controlled sound manipulation.
Thesis: Structured Audio Tokens: Alignment-Driven Representations for Retrieval and Prompt-Controlled Sound Manipulation.
Open Seminar completed
University of Engineering and Management (UEM), Kolkata
Undergraduate training in electronics and communication engineering, with foundations in signals, communication systems, electronics, and applied engineering design.
Thesis: Enhancing the gain of a HMSIW Based Semicircular Antenna using Antenna-FSS Composite Structure.
Academic Service
Academic service through teaching assistance, undergraduate tutoring, and peer review for conferences and journals across machine learning, signal processing, audio, and neural networks.
ESC201A: Introduction to Electronics — conducted regular problem-solving sessions and graded lab submissions for a first-year undergraduate course.
Awards & Recognition
University of Engineering and Management (UEM), Kolkata.
Won $1,500 in an international competition on unsupervised anomaly detection in multi-robot systems.
View certificateServed as a conference volunteer in Baltimore and received financial aid from ICML.
Sponsored by Google India Research and ICLR.
Awarded by Microsoft India Research.
Awarded by Samsung Research Institute Bangalore Corporate VP and CTO Mohan Rao Goli for extraordinary research and showcasing a PoC for voice cloning technology during RFest 2024.
View certificateContact
For professional inquiries related to my research, publications, experience, or potential alignment with your team, please feel free to contact me by email or LinkedIn.
