Time-reversal invariance (TRS) and inversion symmetry (IS) are responsible for the topological band structure in Dirac semimetals (DSMs). These symmetries can be broken by applying an external magnetic or electric field, resulting in fundamental changes to the ground state Hamiltonian and a topological phase transition. We probe these changes using universal conductance fluctuations (UCF) in the prototypical DSM, Cd3As2. With increasing magnetic field, the magnitude of the UCF decreases by a factor of Formula Presented, in agreement with numerical calculations of the effect of broken TRS. In contrast, the magnitude of the UCF increases monotonically when the chemical potential is gated away from the charge neutrality point. We attribute this to Fermi surface anisotropy rather than broken IS. The concurrence between experimental data and theory provides unequivocal evidence that UCF are the dominant source of fluctuations and offers a general methodology for probing broken-symmetry effects in topological quantum materials.