During the Holi fire, the temperature of the air just above the fire becomes much greater than that of the air further up. The hot air has lower density (mass per unit volume) and lower refractive index. It becomes an optically rarer medium. The cool air has higher density and higher refractive index. It is an optically denser medium relative to hot air. Hence, in refraction of light, the angle of refraction changes continuously due to a continuous variation in refractive index.
As the physical conditions of air change rapidly, the apparent position of an object fluctuates rapidly. This gives rise to the flickering of an object seen through a turbulent stream of hot air rising above the Holi fire.