Let $x$ be a real number different from $-1, 0$ and $1$.
$$\left\{ x^{3}, x^{2}, x, -x, -\frac{1}{x} \right\}$$
When the elements of the set are arranged from smallest to largest, which element never occupies the exact middle position?
A) $x^{3}$
B) $x^{2}$
C) $x$
D) $-x$
E) $-\frac{1}{x}$