How to restrict specific source ip address to access virtual ip address

How to restrict specific Client IP address to access Load Balancing Virtual Server...