Aaron Dunketon has such a neat idea for a birdhouse over at his blog, that I want to put in some more brickwork just to use it. It consists of five custom-shaped bricks that when set in a wall, create a nesting cavities for house sparrows. I'd be curious to see if other designs are possible for some of the other songbirds.
The cavities can be cleaned out, although it requires repointing the inserts. It's not always necessary however as birds do a pretty good job of cleaning out their space each spring. One concern I'd have around here is whether or not the squirrels would be able to widen the whole to commandeer the space.