\(\begin{vmatrix}ax+by&ay+bz&az+bx\\ay+bz&az+bx&ax+by\\az+bx&ax+by&ay+bz\end{vmatrix}\)
Applying R1 → R1 + R2 + R3
\(=\begin{vmatrix}a(x+y+z)+b(x+y+z)&a(x+y+z)+b(x+y+z)&a(x+y+z)+b(x+y+z)\\ay+bz&az+bx&ax+by\\az+bx&ax+by&ay+bz\end{vmatrix}\)
\(=\begin{vmatrix}(a+b)(x+y+z))&(a+b)(x+y+z)&(a+b)(x+y+z)\\ay+bz&az+bx&ax+by\\az+bx&ax+by&ay+bz\end{vmatrix}\)
Applying R1 → \(\frac{R_1}{(a+b)(x+y+z)}\)
\(=(a+b)(x+y+z)\begin{vmatrix}1&1&1\\ay+bz&az+bx&ax+by\\az+bx&ax+by&ay+bz\end{vmatrix}\)
Applying C2 → C2 - C3, C3 → C3 - C1
= (a + b) (x + y + z)\(\begin{vmatrix}1 & 0&0\\ay+bz&a(z-y)+b(x-z)&a(x-y)+b(y-z)\\az+bx&a(x-z)+b(y-x)&a(y-z)+b(z-x)\end{vmatrix}\)
Expanding determinant along row R1
= (a + b) (x + y + z)(a2(2yz - y2 - z2) + ab(x-z)(y-z) + ab(z-x)(z-y)+b2 (2xz - x2 - y2) - a2(x2 - xz - xy + yz) - ab(x - y)(y - x) - ab (y - z)(x - z) - b2(y2 - xy - yz + xz))
= (a + b)(x + y + z) (a2 (2yz - y2 - z2 - x2 + xz + xy - yz) + b2(2xz - x2 - y2 - y2 + xy + yz - xz) + ab(xy-yz - xz +z2 - xz - yz + xy - xy + y2 + xy - xy + xz + yz - z2))
= (a + b) (x + y + z) (a2(xy + yz + xz - x2 - y2 - z2) + b2(xy + yz + xz - x2 - y2 - z2) + ab(-2xy - xz - yz + z2 + x2 + y2))
= (a + b) (a2 + b2 - ab) (x + y + z) (xy + yz + xz - x2 - y2 - z2)
= (a3 + b3) (x + y + z) (xy + yz + xz - x2 - y2 - z2)----(1)
Now, \(\begin{vmatrix}x&y&z\\y&z&x\\z&x&y\end{vmatrix}\) = (x + y + z)\(\begin{vmatrix}1&1&1\\y&z&x\\z&x&y\end{vmatrix}\)
(By applying R1 → \(\frac1{x+y+z}\)(R1 + R2 + R3))
= (x + y + z) \(\begin{vmatrix}1&0&0\\y&z-y&x-y\\z&x-z&y-z\end{vmatrix}\)
By applying
(C2 → C2 - C1, C3 → C3 - C1)
= (x + y + z) [(z - y) (y - z) - (x - z) (x - y)]
(On expanding determinant along row R1)
= (x + y + z) (yz - y2 - z2+ yz - x2 + xy + xz - yz)
= (x + y + z) (xy + yz + xz - x2 - y2 - z2)----(2)
Hence, from (1) and (2), we can conclude
\(\begin{vmatrix}ax+by&ay+bz&az+bx\\ay+bz&az+bx&ax+by\\az+bx&ax+by&ay+bz\end{vmatrix}\) = a3 + b3\(\begin{vmatrix}x&y&z\\y&z&x\\z&x&y\end{vmatrix}\)