I'm Edinburgh based, and I'd agree that Dunbar is your best bet. Best fishing is probably June to September, although it'll be picking up nicely during May. Slipway cost is about £8 per launch, with a season ticket around £55 or so.
I normally fish on the drift within half a mile of the shore, mainly between West Barns lighthouse and Torness, and you're talking 40-70 feet of water. Species mainly mackerel, codling, ling, pollock, coalfish and wrasse with other odds and sods as well.
Bait-wise I use lug, rag, mussel and mackerel. Rag is good for pollock, and the mussel for wrasse. Mackerel best for ling, whilst cod'll munch any of these. In practice, during the summer, I'm happy going out without bait and simply picking up mackerel for bait.
Anytime from June onwards I'll tend to leave a set of feathers/flasher style lures fishing by themselves in mid-water to get a supply of mackerel. Pollock and cod will often have a go at 4-6" long x quarter inch belly strip of mackerel fished anything up to 20 feet off the bottom.
That said, my basic rig is a two hook muppet rig fished as hard on the bottom as possible. Usually try to fish mackerel strip on the bottom hook for ling/cod (usually use 50-60lb line to combat teeth damage) and maybe another bait on the top hook to try for other species as well as cod. A leadhead with jellyworm will also work well at times, especially for cod/pollock.
Don't expect huge fish - most are under 4lbs, and I've only seen one over 10lbs, but you can get good numbers.
There are plenty of other options, but this should get you started.
Regards,
Doug