A state machine is just a way to roll a bunch of program actions into one change. When you go from state A to state B, you can be sure that x, y, and z will happen. It has the added benefit of making it easier to be sure that you always go through state B when you're moving from state A to state C, for example -- it lets you plan out how this part of your game will run without having to track every change involved.
Using state machines is like using object oriented code -- a lot of people find it helpful, but you may or may not, depending on what you're doing, how you're doing it, and how you're used to doing things. I'd advise every programmer to learn how to implement state machines, but it's hard to tell you if and how you should use them without knowing everything about your project (and your programming style).
