is infinite. They say, |x> is not
normalizable and mean that it is not in the Hilbert space of
well-behaved pure states.
Caution: Physicists often use different bases which may cause confusing
notation. For example = 0 if x and y are distinct
positions, and

= 0 if p and q are distinct momenta,
the inner product of a momentum bra

(or vice versa) is never zero. (Exercise: Verify this by computing
explicit formulas for

and !) Thus, unlike in mathematics,
the formulas are not invariant under substitution of letters for
the variables!
About the pitfalls when not using the required care, I recommend reading
F. Gieres,
Mathematical surprises and Dirac's formalism in quantum mechanics,
Rep. Prog. Phys. 63 (2000) 1893-1931.
quant-ph/9907069
and
G. Bonneau, J. Faraut, G. Valent,
Self-adjoint extensions of operators and the teaching of quantum
mechanics,
Amer. J. Phys. 69 (2001) 322-331.
quant-ph/0103153