Underwater images often suffer from problems such as low contrast, color distortion, and blurred details, which have a negative impact on subsequent image processing tasks. To mitigate such problems, we propose an algorithm that combines an underwater physical imaging model with a convolutional neural network. The physical imaging model has two important types of parameters: background scattering parameters and direct transmission parameters. For the background scattering parameters, we divide them into three levels, which are primary information, secondary information, and advanced information, and design three subnetworks for feature extraction to represent them. For the direct transmission parameters, we decompose them into two levels, which are shallow transmission information and deep transmission information, and also design two subnetworks to indicate them. Experimental results show that, compared with other enhancement algorithms, the proposed algorithm can not only effectively correct the color deviation and enhance the object edges and texture details but also can obtain superior values of objective evaluation metrics in terms of peak signal-to-noise ratio, structural similarity, underwater color image quality evaluation, and underwater image quality measure.