3 papers indexed
Subham Singh, Ashutosh Mishra, Subha Raut
Transformers with relative positional encodings often extrapolate to sequences longer than those seen during training, whereas transformers with learned absolute encodings typically do not. This is a robust empirical regularity, and the explanations offered for it so far are chie…
Subham Singh, Ashutosh Mishra, Subha Raut
The cooldown phase of a warmup-stable-decay (WSD) learning-rate schedule, now a default in large-model pretraining, lowers the final training loss in some settings and does nothing in others. We give a provable account of which case obtains, and it turns on two properties togethe…
Bishal Lamichhane, Aniket Kumar Singh, Suman Devkota, Uttam Dhakal, Subham Singh, Chandra Dhakal
This study analyzes a network of musical influence using machine learning and network analysis techniques. A directed network model is used to represent the influence relations between artists as nodes and edges. Network properties and centrality measures are analyzed to identify…