The official limit to be away from your chair is 40 minutes. If you see one unoccupied for longer than that, don't be shy about folding up the towels and placing them and anything else in the chair/clam off to the side.
People have done that to me. I deserved it (I tried to claim a clamshell and sneak off to breakfast), and I just took my things when I got back and went somewhere else. If someone tries to make an issue of it, just say that the chair was empty when you got there, and you have no idea how their stuff got moved.