\((D^2 + 1)y = x\sin x + (1 + x^2) e^x\)
Auxiliary equation is m2 + 1 = 0
⇒ m = \(\pm i\)
\(C.F. = C_1\cos x + C_2 \sin x\)
\(P.I. = \frac 1{D^2 + 1} (x\sin x + (1 +x^2)e^x)\)
\(= e^x \frac1{(D + 1)^2 + 1} (1 + x^2) + x \frac 1{D^2 + 1} \sin x + \left\{\frac{-2D}{(D^2 + 1)^2}\right\} \sin x\)
\(= e^x \frac 1{D^2 + 2D + 2} (1 + x^2) + x \times \frac{-x\cos x}{2\times 1} - \frac{2\cos x}{(D^2 + 1)^2}\)
\(= \frac{e^x}2 \left(1 + \frac{D^2 + 2D}2\right)^{-1} (1 + x^2) - \frac {x^2}2 \cos x - 2Re \left(\frac{e^{ix}}{(D^2 + 1)^2}\right)\)
\(= \frac{e^x}2 \left(1-\frac{D^2 + 2D}2 + \left(\frac{D^2 + 2D}2\right)^2 + ....\right) ( 1 + x^2) - \frac{x^2}2 \cos x - 2Re \left(\frac{x^2}{2!}e^{ix}\right)\)
\(= \frac{e^x}2 \left(1 + x^2 - \frac{2 + 4x}2 + 2\right) - \frac{x^2 \cos x}2 - 2\frac{x^2 \cos x}2\)
\(= \frac{e^x}2 (x^2 - 2 x + 2) - \frac 12 x^2 \cos x - x^2 \cos x\)
\(= \frac{e^x}2 (x^2 - 2x + 2) - \frac 32 x^2 \cos x\)
\(\therefore y = C.F. + P.I.\)
\(= C_1\cos x + C_2\sin x + \frac{e^x}2(x^2 - 2x + 2) - \frac 3 2 x^2 \cos x\)
which is the solution of given differential equation.