Your best bet for mackeral and bass in the summer is to explore some of the rock marks around the open sea coast. There are loads of accessible spots all around the coast of the county. I suggest that you get a map and explore.
There are tackle shops in Milford Haven, Pembroke Dock, Haverfordwest and Tenby.
You don't need a licence for fishing in the sea at the moment.
Regarding Neyland, it's central to the county and provides reasonable access to most places, it's not a picture postcard town though. Within the county there are loads of opportunities for walking and the other interests that you list. You could do worse than have a look at the National Park web site, that should give you some ideas.
There are also a few charter boats that are based in the Marina at Neyland if you fancy getting out on the water.
All the best.
Richard.