Using the substitution u = x 2 {\displaystyle u=x^{2}} , we have d u = 2 x d x {\displaystyle du=2x\,dx} , so that
∫ 6 x 3 e − x 2 d x = ∫ 3 x 2 e − x 2 ⋅ 2 x d x = ∫ 3 u e − u d u {\displaystyle \int 6x^{3}e^{-x^{2}}\,dx=\int 3x^{2}e^{-x^{2}}\cdot 2x\,dx=\int 3ue^{-u}\,du}
Then we apply the integration by parts for f ( u ) = u {\displaystyle f(u)=u} and g ( u ) = − e − u {\displaystyle g(u)=-e^{-u}} to get
∫ u e − u d u = − u e − u − ∫ − e − u d u = − u e − u − e − u + C {\displaystyle \int ue^{-u}\,du=-ue^{-u}-\int -e^{-u}\,du=-ue^{-u}-e^{-u}+C}
Plugging u = x 2 {\displaystyle u=x^{2}} back,
∫ 6 x 3 e − x 2 d x = 3 ( − u e − u − e − u + C ) = − 3 x 2 e − x 2 − 3 e − x 2 + C {\displaystyle \int 6x^{3}e^{-x^{2}}\,dx=3(-ue^{-u}-e^{-u}+C)=-3x^{2}e^{-x^{2}}-3e^{-x^{2}}+C}
and so
∫ 0 ∞ 6 x 3 e − x 2 d x = lim t → ∞ ∫ 0 t 6 x 3 e − x 2 d x = lim t → ∞ − 3 x 2 e − x 2 − 3 e − x 2 | x = 0 t = lim t → ∞ ( − 3 t 2 e − t 2 − 3 e − t 2 + 3 ) = 3 {\displaystyle \int _{0}^{\infty }6x^{3}e^{-x^{2}}\,dx=\lim _{t\to \infty }\int _{0}^{t}6x^{3}e^{-x^{2}}\,dx=\lim _{t\to \infty }-3x^{2}e^{-x^{2}}-3e^{-x^{2}}{\bigg |}_{x=0}^{t}=\lim _{t\to \infty }(-3t^{2}e^{-t^{2}}-3e^{-t^{2}}+3)=3}
In the last equality, we use the L'hospital's rule to evaluate
lim t → ∞ t 2 e − t 2 = lim t → ∞ t 2 e t 2 = lim t → ∞ ( t 2 ) ′ ( e t 2 ) ′ = lim t → ∞ 2 t 2 t e t 2 = lim t → ∞ 1 e t 2 = 0. {\displaystyle {\begin{aligned}\lim _{t\to \infty }t^{2}e^{-t^{2}}=\lim _{t\to \infty }{\frac {t^{2}}{e^{t^{2}}}}=\lim _{t\to \infty }{\frac {(t^{2})'}{(e^{t^{2}})'}}=\lim _{t\to \infty }{\frac {2t}{2te^{t^{2}}}}=\lim _{t\to \infty }{\frac {1}{e^{t^{2}}}}=0.\end{aligned}}}
Therefore, we get ∫ 0 ∞ 6 x 3 e − x 2 = 3 {\displaystyle \int _{0}^{\infty }6x^{3}e^{-x^{2}}=\color {blue}3}
, we have 
, so that
and 
to get
back,
