Problem: For a while I had been having an intermittent problem with my Magic Mouse. When I moved the cursor with the magic mouse, the cursor would then move to a random location a second or so after the actual move. The problem seemed to occur when I returned to either working at home or working at my office. The problem often seemed to correct itself after a few hours.
Solution: The second solution presented here by Tom Nelson seems to have fixed the problem. The problem appears to be caused by dust on the surface or on the sensor. Thus, giving the mouse pad surface a wipe and cleaning the optical sensor with a blast of air seems to fix the problem.
I assume that I was noticing the problem after returning to my home or office because dust and grit builds up over time.