Methods and systems for computationally processing data with a multi-layer convolutional neural network (CNN) having an input and output layer, and one or more intermediate layers are described. Input data represented in a form of evaluations of continuous functions on a sphere may be received at a computing device and input to the input layer. The input layer may compute outputs as covariant Fourier space activations by transforming the continuous functions into spherical harmonic expansions. The output activations from the input layer may be processed sequentially through each of the intermediate layers. Each, intermediate layer may apply Ciebsch-Gordan transforms to compute respective covariant Fourier space activations as input to an immediately next layer, without computing any intermediate inverse Fourier transforms or forward Fourier transforms. Finally, the respective covariant Fourier space activations of the last intermediate layer may be processed in the output layer of the CNN to compute invariant activations.