I have a wireless access point that is sometimes turned on, sometimes off. I use what security I can. I have needed to broadcast SSID a few times to get some devices to connect, but it's usually off. Most of my networking is wired so I don't care too much about throughput on the wireless.
I live in an apartment and I'm sure there are neighbors who could connect to my system.
As far as wardriving goes, I've driven around with my PDA's WiFi active and running wififofum. Between work and home one day (20 miles or so), I picked up about 50 networks as I drove by - sometimes at 40-45 mph. These were from homes and businesses. Only about half had WEP enabled. Many of them had SSIDs like LINKSYS or BELKIN or numbers that looked like sequential installations from a cable company or something.
I would never assume that things like MAC filtering are sufficient - or I have a different definition of "serious hacker." While I don't think people are necessarily going to search for my network and expend a little effort to crack my WEP and MAC filtering, it's not hard for them to do. I think more about the bored teenager who thinks it's cool to break in to someone's network. I'm sure there is stuff you can download from the net that runs for a little while with no interaction and delivers the necessary information to break WEP and spoof a MAC address. With the proliferation of wireless networking offered by the ADSL and cable internet companies, more and more kids (or others) are going to have access to the hardware and see this as a fun thing to do. With enough traffic on your network, it only takes a couple hours to crack this stuff - and they don't need to do anything except sit back.
I'm not too concerned about people piggy-backing onto my network to get to the internet. While I don't want them using the bandwidth that I'm paying for, at least it's not a security risk. But I don't want them accessing the machines on my network (or controlling my apartment, for that matter). This would let them know things about when I'm home, who calls me, etc. This kind of information can be valuable to many people who I wouldn't necessarily want to have it. Also, I have a lot of media files (music, recorded video, etc.) on hard drives. All of this is stuff that I purchased or recorded under "fair use." But, if someone gets into my network and starts copying or playing them, then I may technically be distributing these files and violating copyright laws - something I'd rather not do.