Filename Size Access Description License


This dissertation consists of three pieces of work. The first work aims to set up the connection between tropical geometry and feedforward neural networks. We discovered that, mathematically, a feedforward neural network equipped with rectified linear units (ReLU) is a tropical rational function. This connection provides a new approach to understand and analyze deep neural networks. Among other things, we show that the decision boundary derived from an ReLU neural network is contained by a tropical hypersurface of a tropical polynomial in companion with the network. Moreover, we associate functions represented by feedforward neural networks with polytopes and show that a two layer network can be fully characterized by zonotopes which also serve as the building blocks for deeper networks. Also, the number of vertices on the polytopes provides an upper bound on the number of linear regions of the function expressed by the network. We show that this upper bound grows exponentially with the number of layers but only polynomially with respect to number of hidden nodes in each layer.,In the second work, we propose an attention model in continuous vector space for content-based neural memory access. Our model represents knowledge graph entities as low-dimensional vectors while expressing context-dependent attention as a Gaussian scoring function over the vector space. We apply such a model to perform tasks such as knowledge graph completion and complex question answering. The proposed attention model can handle both the propagation of the uncertainty when following a series of relations and also the conjunction of conditions in a natural way. On a dataset of soccer players who participated in the FIFA World Cup 2014, we demonstrate that our model can handle both path queries and conjunctive queries well.,The third work focus on building finite complex frames generated by cyclic vectors under the action of non-commutative groups. We inspect group frames in the space of operators associated with the group's von Neumann algebra. The searching for a proper cyclic vector is then transformed to finding the intersection of a convex set that prescribes the coherence constraints and a subset of Hermitian rank-one operators. An alternating projection algorithm is employed to search for their intersection and an heuristic extrapolation technique is adapted to accelerate the computation. In the experiments, we applied our model to Heisenberg groups and finite affine groups. In the case of Heisenberg group, our method is able to find cyclic vectors that generate equiangular tight frames up to numerical precision.


Additional Details


Download Full History