(b) \(\sqrt{a^2-b^2}\)
Let ∠BMA = ∠BMC = x
Then ∠CMD = π – 2x
Let MD = y. Then AM = (a – y)
In ∆CMD,
tan (π – 2x) = \(\frac{b}{y}\) ...(i)
In ∆BAM, tan x = \(\frac{b}{a-y}\) ...(ii)
From (i), we have tan (π – 2x) = – tan 2x = \(\frac{b}{y}\)
⇒ – 2bya + 2by2 = ba2 + by2 – 2aby – b3
⇒ by2 = b(a2 – b2)
⇒ y = \(\sqrt{a^2-b^2}\)