Correct option is (2) 729
\(A=\left[\begin{array}{cc}\sqrt{2} & 1 \\ -1 & \sqrt{2}\end{array}\right] \Rightarrow \operatorname{det}(A)=3\)
\(B=\left[\begin{array}{ll}1 & 0 \\ 1 & 1\end{array}\right] \Rightarrow \operatorname{det}(B)=1\)
Now
\(\mathrm{C}=\mathrm{ABA}^{\mathrm{T}} \Rightarrow \operatorname{det}(\mathrm{C})=(\operatorname{det}(\mathrm{A}))^{2} \mathrm{x} \operatorname{det}(\mathrm{B})\)
\(|C|=9\)
Now
\(|\mathrm{X}|=\left|\mathrm{A}^{\mathrm{T}} \mathrm{C}^{2} \mathrm{~A}\right|\)
\(=\left|\mathrm{A}^{\mathrm{T}}\right||\mathrm{C}|^{2}|\mathrm{~A}|\)
\(=|\mathrm{A}|^{2}|\mathrm{C}|^{2}\)
\(=9 \times 81\)
\(=729\)