We implement quantum optimal control algorithms for closed and open quantum systems based on automatic differentiation. Automatic differentiation allows us to specify advanced optimization criteria and incorporate them in the optimization process with ease. We shown that the use of GPUs can speed up calculations by more than an order of magnitude. Our strategy facilitates efficient numerical simulations on affordable desktop computers, and exploration of a host of optimization constraints and system parameters relevant to real-life experiments. We demonstrate optimization of closed quantum evolution based on fine-grained evaluation of performance at each intermediate time step, thus enabling more intricate control on the evolution path, suppression of departures from the truncated model subspace, as well as minimization of the physical time needed to perform high-fidelity state preparation and unitary gates. The optimizer is also used to find a universal set of gates for the protected superconducting qubits of fluxonium and 0-π. The optimization algorithm for open quantum systems utilizes quantum trajectories. Using trajectories allows for optimizing open systems with less computational cost than the regular density-matrix approaches in most realistic optimization problems. We introduce an improved-sampling algorithm which minimizes the number of trajectories needed per optimization iteration. Together with employing stochastic gradient descent techniques, this reduces the complexity of optimizing many realistic open quantum systems to the complexity encountered with closed systems. We utilize the optimizer in a variety of applications to demonstrate how the use of quantum trajectories significantly reduces the memory requirements while achieving a multitude of simultaneous optimization targets. Demonstrated targets include high state-transfer fidelities despite dissipation, faster gate times and maximization of qubit-readout fidelity while maintaining the quantum non-demolition nature of the measurement and allowing for subsequent fast resonator reset.