No doubt others may disagree, but I think you may be going into the realms of overkill, using a lathe-like set-up for shaping cork handles.
Cork is a very soft material, and you are likely to remove large amounts of material using such an apparatus, even with fine grades of abrasive cloth, rather than tools.
Unless you are hell-bent on trying to produce a perfectly cylinderical handle, I'd seriously suggest that you consider doing this by hand using progressively finer grades of abrasive paper: it takes relatively little time and is a much more tactile process, allowing you to stop removing cork as and when the handle feels 'right' to your hands ... regardless of whether that produces a true cylinder or not.
I shaped my first rod handle like this neck-end of 40 years ago (still have and use the rod, as it was built on a very nice (fibreglass ) blank), and have been doing it ever since, most recently just a couple of months back.