[1]WANG Dewen,SUN Zhiwei.A method for cluster analysis of electric power consumers based on in-memory computing[J].CAAI Transactions on Intelligent Systems,2015,10(4):569-576.[doi:10.3969/j.issn.1673-4785.201411011]
Copy

A method for cluster analysis of electric power consumers based on in-memory computing

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 10 Number of periods: 2015 4 Page number: 569-576 Column: 学术论文—机器学习 Public date: 2015-08-25
Title:
A method for cluster analysis of electric power consumers based on in-memory computing
Author(s):
WANG Dewen;  SUN Zhiwei
School of Control and Computer Engineering, North China Electric Power University, Baoding 071003, China
Keywords:
big data; smart electricity consumption; resilient distributed data set; in-memory computing; cluster analysis
CLC:
TP18
DOI:
10.3969/j.issn.1673-4785.201411011
Abstract:
With the rapid growth of electricity consumption data collected by smart electric meters and data acquisition terminals, the traditional data analysis method cannot meet the demand of smart power consumption behavior analysis in the big data environment. Since K-means algorithm demonstrates high calculation efficiency, easy parallelization and other characteristics, a method for improving and parallelizing K-means with the resilient distributed data set and parallel in-memory computing framework is presented, the running time of job operation and I/O operations is reduced, and the ability of clustering analysis is improved. The experimental data set is built by preprocessed electricity consumption data. Eexperimental results show that the accuracy rate by this cluster analysis method for electric power users is obviously better than the single machine K-means algorithm. The processing speed and ability of this method are superior to the single machine and the clustering method based on MapReduce parallel computing framework, and this method has good adaptability for the growth of data.
References:
[1] 王蓓蓓, 李扬, 高赐威. 智能电网框架下的需求侧管理展望与思考[J]. 电力系统自动化, 2009, 33(20): 17-22. WANG Beibei, LI Yang, GAO Ciwei. Demand side management outlook under smart grid infrastructure[J]. Automation of Electric Power Systems, 2009, 33(20): 17-22.
[2] 何永秀, 王冰, 熊威, 等. 基于模糊综合评价的居民智能用电行为分析与互动机制设计[J]. 电网技术, 2012, 36(10): 247-252. HE Yongxiu, WANG Bing, XIONG Wei, et al. Analysis of residents’ smart electricity consumption behavior based on fuzzy synthetic evaluation and the design of interactive mechanism[J]. Power System Technology, 2012, 36(10): 247-252.
[3] 宋亚奇, 周国亮, 朱永利. 智能电网大数据处理技术现状与挑战[J]. 电网技术, 2013, 37(4): 927-935. SONG Yaqi, ZHOU Guoliang, ZHU Yongli. Present status and challenges of big data processing in smart grid [J]. Power System Technology, 2013, 37(4): 927-935.
[4] 何清. 物联网与数据挖掘云服务[J]. 智能系统学报, 2012, 7(3): 189-194. HE Qing. The Internet of things and the data mining cloud service[J]. CAAI Transactions on Intelligent Systems, 2012, 7(3): 189-194.
[5] 冯晓蒲, 张铁峰. 基于实际负荷曲线的电力用户分类技术研究[J]. 电力科学与工程, 2010, 26(9): 18-22. FENG Xiaopu, ZHANG Tiefeng. Research on electricity users classification technology based on actual load curve[J]. Electric Power Science and Engineering, 2010, 26(9): 18-22.
[6] 李培强, 李欣然, 陈辉华, 等. 基于模糊聚类的电力负荷特性的分类与综合[J]. 中国电机工程学报, 2005, 25(24): 73-78. LI Peiqiang, LI Xinran, CHEN Huihua, et al. The characteristics classification and synthesis of power load based on fuzzy clustering[J]. Proceedings of the CSEE, 2005, 25(24): 73-78.
[7] 段铷, 张彩庆, 刘爱芳. 模糊聚类在电力用户分类中的应用[J]. 电力需求侧管理, 2005, 7(5): 18-20. DUAN Ru, ZHANG Caiqing, LIU Aifang. Application of fuzzy clustering method in classification of electricity customers[J]. Power DSM, 2005, 7(5): 18-20.
[8] 张素香, 刘建明, 赵丙镇, 等. 基于云计算的居民用电行为分析模型研究[J]. 电网技术, 2013, 37(6): 1542-1546. ZHANG Suxiang, LIU Jianming, ZHAO Bingzhen, et al. Cloud computing-based analysis on residential electricity consumption behavior[J]. Power System Technology, 2013, 37(6): 1542-1546.
[9] 毛典辉. 基于MapReduce的Canopy-Kmeans改进算法[J]. 计算机工程与应用, 2012, 48(27): 22-26. MAO Dianhui. Improved Canopy-Kmeans algorithm based on MapReduce[J]. Computer Engineering and Applications, 2012, 48(27): 22-26.
[10] ZAHARIA M, CHOWDHURY M, FRANKLIN M J, et al. Spark: cluster computing with working sets[C] //Proceedings of the 2nd USENIX Conference on Hot Topics in Cloud Computing. Berkeley, CA, USA: USENIX Association, 2010.
[11] 赵薇, 刘杰, 叶丹. 基于组件的大数据分析服务平台[J]. 计算机科学, 2014, 41(9): 75-79. ZHAO Wei, LIU Jie, YE Dan. Module based big data analysis platform[J]. Computer Science, 2014, 41(9): 75-79.
[12] 赵莉, 候兴哲, 胡君, 等. 基于改进 k-means 算法的海量智能用电数据分析[J]. 电网技术, 2014, 38(10): 2715-2720. ZHAO Li, HOU Xingzhe, HU Jun, et al. Improved k-means algorithm based analysis on massive data of intelligent power utilization[J]. Power System Technology, 2014, 38(10): 2715-2720.
[13] 程艳柳. 基于云计算的智能电网数据挖掘的研究[D]. 保定: 华北电力大学, 2013:15-20. CHENG Yanliu. Research on smart grid data mining based on cloud computing[D]. Baoding: North China Electric Power University, 2013:15-20.
[14] ZAHARIA M, CHOWDHURY M, DAS T, et al. Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing[C] //Proceedings of the 9th USENIX Conference on Networked Systems Design and Implementation. Berkeley, USA: USENIX Association, 2012:1-14.
[15] LIN X Q, WANG P, WU B. Log analysis in cloud computing environment with Hadoop and Spark[C] //2013 5th IEEE International Conference on Broadband Network & Multimedia Technology (IC-BNMT). Guilin, China: IEEE, 2013: 273-276.
[16] GU L, LI H. Memory or time: performance evaluation for iterative operation on Hadoop and Spark[C]. 2013 IEEE 10th International Conference on High Performance Computing and Communications & 2013 IEEE International Conference on Embedded and Ubiquitous Computing (HPCC_EUC). Zhangjiajie, China: IEEE, 2013: 721-727.
[17] 海沫, 张书云, 马燕林. 分布式环境中聚类问题算法研究综述[J]. 计算机应用研究, 2013, 30(9): 2561-2564. HAI Mo, ZHANG Shuyun, MA Yanlin. Algorithm review of distributed clustering problem in distributed environments[J]. Application Research of Computers, 2013, 30(9): 2561-2564.
[18] 余晓山, 吴扬扬. 基于MapReduce的文本层次聚类并行化[J]. 计算机应用, 2014, 34(6): 1595-1599, 1680. YU Xiaoshan, WU Yangyang. Parallel text hierarchical clustering based on MapReduce[J]. Journal of Computer Applications, 2014, 34(6): 1595-1599, 1680.
[19] MCCALLUM A, NIGAM K, UNGAR L H. Efficient clustering of high-dimensional data sets with application to reference matching[C]//Proceedings of the Sixth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM, 2000: 169-178.
[20] KANUNGO T, MOUNT D M, NETANYAHU N S, et al. An efficient k-means clustering algorithm: Analysis and implementation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(7): 881-892.
Similar References:

Memo

-

Last Update: 2015-08-28

Copyright © CAAI Transactions on Intelligent Systems