Investigating the Implicit Bias of Activations in Coordinate-MLPs
Link to Full Report (PDF) Contributions: Extensive experiments on 2D video approximation and sparse 3D reconstruction. Analysis of activation function performance under varying data sparsity conditions. Implementation of a geometric initialisation scheme for Gaussian activation. Authors: Ruben Schenk, Bruce Balfour, Alexandra Trofimova Institution: ETH Zurich, Institute for Visual Computing Overview Coordinate-based Multi-Layer Perceptrons (MLPs) have gained significant traction in recent years for their ability to approximate complex signals such as 2D images, 3D shapes, and even 4D spatiotemporal data. However, a key aspect that remains underexplored is the implicit bias introduced by various activation functions within these networks. This project delves into the effects of different activation functions on the representational capacity of coordinate-MLPs, with a particular focus on their performance in 2D video approximation and sparse 3D reconstruction tasks. ...