Worldwide growth in high-speed rail (HSR) networks has brought a demand for improved structural performance of the bridges that make up a large portion of many of these HSR systems. While some improvement has been made via optimization either of the bridge or the passenger cars, the design criteria of passenger comfort has yet to be addressed in bridge optimization. The transient dynamics of vehicle–bridge interaction between a high-speed train and bridge make such structural optimization of these systems challenging. In this paper, we derive an approach for topology optimization of high-speed rail bridges including vehicle–bridge interaction that enables direct consideration of the passenger comfort in the objective function. Assuming constant contact between the vehicle’s wheels and the bridge, the two systems are combined into a single-state space system. The resulting system matrices are time dependent, as they are a function of the wheel contact locations which change as the vehicles move over the bridge. The equations of motion and the adjoint sensitivities are derived and solved numerically in the time domain. Several numerical examples are provided based on high-speed rail applications that minimize a multi-objective function comprised of bridge and vehicle responses, including passenger comfort. These examples generate topologies that improve passenger comfort at only a small cost to the bridge response and demonstrate the dependence of optimal topology on train speed and length. The proposed method offers the potential for improving high-speed rail passenger comfort through optimization of bridge topology by accounting for the vehicle–bridge interaction effects.