We construct a set $H$ of orthogonal polynomial sequences that contains all the families in the Askey scheme and the $q$-Askey scheme. The polynomial sequences in $H$ are solutions of a generalized first-order difference equation which is determined by three linearly recurrent sequences of numbers... Two of these sequences are solutions of a difference equation of order three and the other sequence satisfies a related difference equation of order five. We obtain explicit expressions for the coefficients of the orthogonal polynomials and for the generalized moments with respect to a basis of Newton type of the space of polynomials. We also obtain explicit formulas for the coefficients of the three-term recurrence relation satisfied by the polynomial sequences in $H$. The set $H$ contains all the 15 families in the Askey scheme and all the 29 families in the $q$-Askey scheme. Each of these families is obtained by direct substitution of appropriate values for the parameters in our general formulas. The only cases that require some limits are the Hermite and continuous $q$-Hermite polynomials. We present the values of the parameters for some of the families. (read more)