When an atom strongly couples to a cavity, the two systems can coherently exchange a single quantum excitation through the process of vacuum Rabi oscillation. Controlling this process enables precise synthesis of non-classical light, which plays a central role in quantum information and measurement. Although this control has been realized in microwave-frequency devices, it has been difficult to achieve at optical frequencies, which are essential for quantum communication and metrology. Here, we demonstrate coherent control of vacuum Rabi oscillation in an optical frequency device. We use a photonic molecule composed of two coupled nanocavities to simultaneously achieve strong coupling and a cavity-enhanced a.c. Stark shift. The Stark shift tunes a single quantum dot onto resonance with the photonic molecule on picosecond timescales, creating fast coherent transfer of energy between an atomic and photonic excitation. These results enable ultrafast control of light–matter quantum interactions in a nanophotonic device platform.