Concept:
Gauss law states that flux leaving any closed surface is equal to the charge enclosed by that surface:
\({\rm{\Psi }} = \mathop \oint \limits_S \vec D \cdot d\vec S = {Q_{enclosed}} = \mathop \smallint \limits_V \rho V \cdot dV\)
Analysis:
For the flux density to be zero at radius r = 10 m, the total charge enclosed must be zero.
Total charge enclosed on 2m radius sphere will be:
\(4{\rm{\pi }}\left( {{{\rm{r}}^2}} \right)20\frac{{{\rm{nC}}}}{{{{\rm{m}}^2}}} = 320{\rm{\pi \;nC}}\)
Total change on 4 m radius sphere will be:
\(4{\rm{\pi }}\left( {{{\rm{r}}^2}} \right) \times - 4\frac{{{\rm{nC}}}}{{{{\rm{m}}^2}}} = - 256{\rm{\pi \;nC}}\)
The total charge on 8 m radius sphere will be:
\(4{\rm{\pi }}\left( {{{\rm{r}}^2}} \right) \times {{\rm{\rho }}_{\rm{s}}}\frac{{{\rm{nC}}}}{{{{\rm{m}}^2}}} = 256{{\rm{\rho }}_{\rm{s}}}{\rm{\pi \;nC}}\)
The total charge enclosed should be zero. So,
\(320{\rm{\pi }} - 256{\rm{\pi }} + 256{{\rm{\rho }}_{\rm{s}}}{\rm{\pi \;}} = 0\)
Hence,
\( {{\rm{\rho }}_{\rm{s}}} = -0.25{\rm{\;nC}}/{{\rm{m}}^2}\)