To check whether the servo is the problem, remove the hose to the inlet manifold - a straight pull, it should come out complete with the plastic widget leaving a bare vacuum case. Then you should be able to poke a finger or a dipstick into the vacuum tank. If the source of leakage is the servo it should be very obviously full of hydraulic fluid. If not, or you don't find enough to be convinced, then the likely culprit is the rear caliper seals, which are capable of leaking into the enclosed area that houses the adjuster and handbrake mechanism without it showing as drips on the floor.
Best bet with either is to fit service exchange that have been rebuilt with stainless steel sleeves to the cylinder bores. Obvious first port of calls for thesae are Wynn's or Rover-Classics.
Chris