Regarding the sleeping, our pediatrician recommended the book "Solve Your Child's Sleep Problems" by Richard Ferber, MD. As long as you are coming to get him and sleeping with him in the spare room, I think he will continue to wake. Sleeping with Mommy is much more fun than sleeping alone! If you are OK with this, that's great. If you really want him to sleep in his room, you may have to let him cry it out some (I know this is a very controversial issue). We did this with our girls by going in at intervals (immediately, then five minutes later, then ten minutes later, and then every fifteen minutes) to reassure our daughter that we were there and she is OK but that it is bed time. Just pop your head in the door, but do not pick him up! This worked well for us, and both girls are now great sleepers (and confident, loving children--not scared for life by crying for 15 minutes a few nights!).
We have a monkey for a two year old who will climb anything, so I understand your gate situation. We have a gate at the top of our stairs. Surprisingly, she has never tried to climb this gate. We keep her door open at night, but we do put two pressure gates against her door frame. This way, her door has an open feeling, but she is safely gated in. Good luck!