To begin with our stock exchanges really lack data. Both NSE & BSE do not have small cap data prior to 2017. For BSE, I could get it only from December 2017.
I am sure if you are tracking the market since 2017/18, you would have noticed the sharp fall in the small cap index post the Feb 1st budget of 2018 when the Long term capital gains(LTCG) was levied at the rate of 10% from an earlier rate of 0% existing for the last 14 years.
But when you look at the US Small cap index movement during the period during the budget period(Fig.1), there was a similar fall but on a lesser extent due to the dampening impact of LTCG introduction in India.
Post that the US small cap index was at the similar levels for a long time. But even if there was little bit of fall in the US market, the Indian market fall heavily.
From my interpretation there could be 2 potential reason for that:
1) FII Sell Off (FII still dictates the movement of our indices)
2) Very poor economic condition in India (In slow down the larger peers do relatively better)
3) Global economic cycle which began in 2013-14 peaked in 2018.