Let $E/F$ be a quadratic extension of non-archimedean local fields, and let $\ell$ be a prime number different from the residual characteristic of $F$. For a complex cuspidal representation $\pi$ of $GL(n,E)$, the Asai $L$-factor $L^+(X,\pi)$ has a pole at $X=1$ if and only if $\pi$ is $GL(n,F)$-distinguished... In this paper we solve the problem of characterising the occurrence of a pole at $X=1$ of $L^+(X,\pi)$ when $\pi$ is an $\ell$-modular cuspidal representation of $GL(n,E)$: we show that $L^+(X,\pi)$ has a pole at $X=1$ if and only if $\pi$ is a relatively banal distinguished representation; namely $\pi$ is $GL(n,F)$-distinguished but not $\vert\det(~ )|_{F}$-distinguished. This notion turns out to be an exact analogue for the symmetric space $GL(n,E)/GL(n,F)$ of M\' inguez and S\'echerre's notion of banal cuspidal $\overline{\mathbb{F}}_\ell$-representation of $GL(n,F)$. Along the way we compute the Asai $L$-factor of all cuspidal $\ell$-modular representations of $GL(n,E)$ in terms of type theory, and prove new results concerning lifting and reduction modulo $\ell$ of distinguished cuspidal representations. Finally, we determine when the natural $GL(n,F)$-period on the Whittaker model of a distinguished cuspidal representation of $GL(n,E)$ is nonzero. (read more)