In scenarios with strong light sources, images captured by surveillance cameras often exhibit lens flare, which can significantly blur or even completely obscure the details of the surveillance footage, thereby reducing the accuracy of the monitoring data. Existing algorithms for removing lens flare often result in blurred edge details in the damaged areas of the image and suffer from low computational efficiency. To address these issues, this paper proposes an algorithm for removing lens flare from images by integrating semantic information. Based on the principles of optical flow, this algorithm constructs a feature map fusion model to learn and enhance the semantic information between adjacent feature maps, thereby capturing the semantic information in the lens flare areas and restoring the lost edge details of the damaged regions. In addition, an improved channel attention mechanism is introduced, which uses a scaling factor to normalize the features of irrelevant background areas and lens flare, reducing the weight of features from irrelevant areas, and specifically extracting semantic features from the lens flare area to improve the computational efficiency of the algorithm. The feature map fusion model and the improved channel attention mechanism are embedded into the skip connections and decoder pathways of U-Net, focusing on fusing high and low-level semantic information and specifically targeting the lens flare areas, thus effectively removing lens flare from images. Experiments on the Flare7K++ dataset from Nanyang Technological University’s S-Lab show that our proposed algorithm achieves a peak signal-to-noise ratio of 27.7643 dB, a structural similarity index measure of 0.9739, and a reduced learned perceptual image patch similarity of 0.0437, outperforming existing methods and demonstrating the effectiveness of our approach in the field of lens flare removal.