| 引用本文: | 戴志辉,贺欲飞,张洪嘉,等.基于Linear Transformer-CNN模型的继电保护系统缺陷文本深度挖掘技术[J].电力系统保护与控制,2025,53(21):146-155.[点击复制] |
| DAI Zhihui,HE Yufei,ZHANG Hongjia,et al.Deep mining technology for relay protection system defect text based on Linear Transformer-CNN model[J].Power System Protection and Control,2025,53(21):146-155[点击复制] |
|
| 摘要: |
| 针对当前继电保护系统中缺陷文本数据规模庞大，而传统挖掘方法普遍存在特征提取不充分、语义识别精度有限、运行效率不高等问题，提出一种基于Linear Transformer-CNN模型的缺陷文本深度挖掘方法。首先，对海量缺陷文本进行预处理，将处理好的单词输入到MacBERT中生成综合词向量。之后，将线性注意力机制引入Transformer中，以提高整体运行效率。然后，添加多层CNN模块以弥补Linear Transformer对缺陷文本特征提取的不足。最后，将综合词向量输入到多层CNN和Linear Transformer模块中，分别提取缺陷文本局部关键特征和长距离语义特征，经融合后通过SoftMax进行缺陷文本分类。实验结果表明，相比于传统的文本挖掘方法，该方法的训练和测试时间更短，且分类准确率达到94.24%，实现了对缺陷文本的快速精准分类。 |
| 关键词: 继电保护系统 缺陷文本 深度学习 注意力机制 文本分类 |
| DOI：10.19783/j.cnki.pspc.241555 |
| 投稿时间：2024-11-12修订日期：2025-02-17 |
| 基金项目:国家自然科学基金项目资助(51877084) |
|
| Deep mining technology for relay protection system defect text based on Linear Transformer-CNN model |
| DAI Zhihui,HE Yufei,ZHANG Hongjia,ZHANG Fuze,HAN Xiao |
| (Hebei Key Laboratory of Distributed Energy Storage and Microgrid (North China Electric
Power University), Baoding 071003, China) |
| Abstract: |
| In view of the large volume of defective texts in the current relay protection systems and the limitations of traditional mining methods, such as insufficient text feature extraction, inaccurate semantic recognition, and low operation efficiency, a defect text deep mining method based on the Linear Transformer-CNN model is proposed. First, massive defect text data are preprocessed, and the processed words are input into MacBERT to generate comprehensive word embeddings. Next, a linear attention mechanism is introduced into the Transformer to improve overall operation efficiency. Then, a multi-layer CNN module is added to compensate for the Linear Transformer’s limited ability to extract defect text features. Finally, the comprehensive word embeddings are fed into the multi-layer CNN and Linear Transformer modules to extract local key features and long-distance semantic features of defect texts, respectively. The fused features are then classified using a SoftMax layer. Experimental results show that, compared with traditional text mining methods, the training and testing time of the proposed method is shorter, and the classification accuracy reaches 94.24%, enabling fast and accurate classification of defect texts. |
| Key words: relay protection system defect text deep learning attention mechanism text classification |
