The operation of a Geiger-mode avalanche photo diode (GM-APD) LIDAR is severely disturbed by background noise, which makes it challenging to rapidly and accurately estimate the target depth. Therefore, we propose an adaptive fading Kalman depth estimation technique based on chi-square hypothesis testing for GM-APD LIDARs. First, by analyzing the consistency of the echo photon distribution between adjacent pixels of the GM-APD, the pixels are fused to rapidly obtain the echo data of the target surface, thereby accelerating the depth estimation process. Second, we design a chi-square hypothesis test condition based on the statistical characteristics of the innovation vector, which can help evaluate whether the fading factor is introduced at the current moment, promote the convergence of the algorithm, and reduce the depth estimation time. Third, we propose a fading memory index weighted method to adaptively adjust the weight of the observed values to accurately estimate the innovation matrix covariance and determine the optimal fading factor. We demonstrate the effectiveness of the proposed algorithm through simulations and experiments. The results show that the proposed algorithm can rapidly and accurately estimate the target depth in the presence of strong background noise.