Approximating dynamical systems from data is an important and challenging problem. Incorporating knowledge about physical laws that govern the dynamical process can help to reduce data requirements and improve prediction accuracy. Here, we discuss how to approximate Hamiltonian functions of energy-conserving dynamical systems by solving an associated linear partial differential equation. We employ neural network activation functions as basis functions for the solution and evaluate the performance of data-agnostic and data-driven weight sampling algorithms to construct this basis.