Cafe Porta Alba has the best Neapolitan-style pizza I've had. Caveat: It recently moved, and I haven't been to the new location yet, but I believe it's under the same management, so I'm operating under the assumption that it's still as good!
Batch Bakehouse is my favorite bakery in Madison. It opened a few months ago, and nothing I've had there has disappointed. They have a variety of artisan breads, as well as pastries.