Research on data mining and short-term power load forecasting based on Hadoop

Xiannan Huang; Juhua Hong; Shicheng Huang; Linyao Zhang; Lin Liu; Siyu Yang; Weiwei Lin

doi:10.1117/12.2671337

1 June 2023 Research on data mining and short-term power load forecasting based on Hadoop

Xiannan Huang, Juhua Hong, Shicheng Huang, Linyao Zhang, Lin Liu, Siyu Yang, Weiwei Lin

Proceedings Volume 12625, International Conference on Mathematics, Modeling, and Computer Science (MMCS2022); 126252S (2023) https://doi.org/10.1117/12.2671337
Event: International Conference on Mathematics, Modeling and Computer Science (MMCS2022),, 2022, Wuhan, China

Abstract

With the rapid development of smart grid technology, the amount of power data information increases more and more, and the traditional centralized processing method has been unable to meet the requirements of power system operation. In order to better meet the storage and analysis goals of power data, researchers propose to use distributed power big data. On the basis of understanding the research status of data mining and load prediction algorithms, this paper focuses on the data composite characteristics and change rules of power AI competition according to the big data platform, and constructs the user clustering model with Mahout as the core and the multi-algorithm fusion prediction model with Spark as the core. Combined with the final research results, it is shown that Spark and Mahout, two frameworks in the Hadoop ecosystem, fully consider the advantages and disadvantages of different frameworks, and effectively control the time and cost of experimental analysis. The former can be regarded as a short-term composite forecasting platform, while the latter belongs to a data mining framework.

Citation Download Citation

Xiannan Huang, Juhua Hong, Shicheng Huang, Linyao Zhang, Lin Liu, Siyu Yang, and Weiwei Lin "Research on data mining and short-term power load forecasting based on Hadoop", Proc. SPIE 12625, International Conference on Mathematics, Modeling, and Computer Science (MMCS2022), 126252S (1 June 2023); https://doi.org/10.1117/12.2671337

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available