Cognitive control is essential for implementing goal-directed behavior in daily life. In this study, we used electroencephalography (EEG) to examine two putative neural mechanisms that support cognitive control engagement: frontal midline theta (fmTheta) and pre-stimulus alpha (i.e., pre-cue alpha) in a cued task-switching paradigm. We tested the hypothesis that fmTheta power functions as a neural marker of proactive and reactive control, and that pre-cue alpha serves as a context updating mechanism. Thirty individuals completed the cued task-switching paradigm while EEG was recorded. We observed higher cue-locked fmTheta power in trials where proactive control was possible (i.e., where predictive cues provided the most information about the response from the target stimulus that would be required), although this effect was later shown to be partly driven by aperiodic components of the EEG rather than true oscillatory theta. Contrary to expectations, target-locked fmTheta power was higher when the cues stayed the same between trials compared to when they switched from trial to trial. However, across several conditions, higher target-locked fmTheta power was associated with faster reaction times, suggesting more efficient reactive control engagement. Finally, the results also supported the hypothesis that pre-cue alpha power was associated with faster reaction times, reflecting inhibition of task-irrelevant information. However, there was no association with reduced switch costs that would indicate a context-updating function. Together, these findings reinforce the role of fmTheta as a neural marker of proactive and reactive control, demonstrate individual differences in greater fmTheta power and efficient control engagement, and highlight the role of pre-cue alpha in facilitating task performance.