Soccer player and ball detection and tracking have emerged as an area of intense interest among many analysts and researchers. This is because it aids coaches in team performance evaluation and decision-making to achieve optimal results. However, existing methodologies have failed to effectively detect and track the ball when it moves at high velocity and also to track players under occlusion conditions. You only look once (YOLOv3) and simple online real-time (SORT)-based soccer ball and player tracking approach is proposed, for accurately classifying the detected objects in soccer video and track them in various challenging situations. The proposed methodology consists of two parts: (i) YOLOv3 can detect and classify the objects (i.e., player, soccer ball, and background) and eliminate the detected objects outside the playfield as background; (ii) tracking is achieved using SORT algorithm which employs a Kalman filtering and bounding box overlap. The proposed model achieves tracking accuracy of 93.7% on multiple object tracking accuracy metrics with a detection speed of 23.7 frames per second (FPS) and a tracking speed of 11.3 FPS.