Is it ok to use 4x4 sheets of plywood sans 4x8 to reinforce the existing subfloor. Since I have a multi-level townhome it would be significantly easier to bring in 4x4 sheets around the tight corners without damaging existing walls and fixtures.

Not a problem ... if you intending to use this as an underlayment over the existing subfloor. Keep each seam offset and staggered leaving an 1/8" spacing between each sheet.... hitting a floor joist does not matter here.

Thanks, that's all I'm planning. Most my rooms are pretty small and when I looked at the room I'm working on now, I noticed out of all the OSB subfloors sheets, only 2 were even a full 4x8 and most the rest weren't even close. Having to offset all the sheets in rooms that are under 150sq ft just didn't seem to require 4x8s.

Be good to yourself .... re-nail the OSB if necessary, take down any proud seams and deal with any flatness issues now, before the underlayment goes down.