Given, cos–1α + cos–1β + cos–1γ = 3π … (1)
Principal value of cos-1 x is [0, π]
So, maximum value which cos-1 x can have is π.
So, if (1) is correct then all the three terms i.e,
cos–1α, cos–1β, cos–1γ should be equal to π
So, cos–1α = π
cos–1β = π
cos–1γ = π
So, α = β = γ = -1
So, α(β + γ) + β (γ + α) + γ (α + β)
= (-1)(-1-1) + (-1)(-1-1) + (-1)(-1-1)
= 3(-1)(-2)
= 6
