When you walk into a dark room and turn the light on, do the electrons travel all the way from the switch to the light?

The energy to light that bulb comes from the electric field E that travels through the wire at the speed of light (in that particular medium, typically copper). That's why the light comes on seemingly immediately when the switch is turned to ON.  

But the electrons do not...not...travel that fast. In fact the proper terminology for the motion of free electrons through a medium is called electron drift. They call it drift because, compared to the E field and the speed of light, these electrons are actually moving, drifting, quite slowly. [See source.] The actual speed depends on the voltage, the cross sectional area, the medium, and so on. In a cathode tube, for example, the drift speed is about 1/10 the speed of light. But that's about as fast as it gets.  

Current, measured in amps, is defined by 1 Coulomb/sec = 1 amp. 1 Coulomb's worth of electrons is about 6.25E18 electrons. So even in a modest 1 amp system there are a billion billion electrons per second drifting in that copper wire, for example. But they are still moving, drifting, at millimeters per second velocity. In DC circuits, those electrons drift away from the cathode end, towards the anode end. In AC circuits, they draft back and forth, following the cycles of the alternating current.  

So, the answer is...no. The electrons do not flow all the way from the switch to the light in an AC circuit. But if you leave the light on long enough, at 1E-3 m/s speeds, the electrons in a DC circuit will eventually get to that bulb