As an important part of fire prevention and control, there are high standards for real-time, anti-interference, and accuracy about flame detection. At present, flame target detection methods lack comprehensive research on the above three indicators. To solve this problem, we propose a lightweight anti-interference flame detection method based on improved YOLOv4-tiny. On the ground of dynamic characteristics of flame changing with time, a double-stream structure of flame detection model is designed. First, deeply separable convolution is applied to YOLOv4-tiny, resulting in a lighter backbone network. Second, in the feature extraction stage, the learning ability of the network to shallow features is improved by further integrating multiscale features, and the efficient channel attention module is introduced into the feature pyramid network to further improve the accuracy. Finally, intersection over union (IOU) postprocessing algorithm is used to shield the interference of fire-like targets effectively. Experimental results show that the parameter quantity of our methods are 3.95 and 4.22 MB, respectively, and accordingly their precisions reach 94.10% and 94.27%. Detection times are 35 and 46 ms, separately. In addition, after using IOU postprocessing algorithm, mAP and F1 composite index both increased to varying degrees, the consumption time is only 0.23 ms.