SAML + LDAP using Nfactor gives error "You are not allowed to login. Please contact your administrator"

After hitting Netscaler for login, you are redirected to SAML and successfully login. Afterwards, you are redirected back to Netscaler and receive the error"You are not allowed to login. Please contact your administrator"...