Deep Residual Shrinkage Network: Nthowa ya Artificial Intelligence yakusandira Data iyo yili na Noise yinandi

Deep Residual Shrinkage Network ni mtundu uwemi wa Deep Residual Network. Mwakudumura, Deep Residual Shrinkage Network yikusazga pamoza Deep Residual Network, attention mechanisms, na soft thresholding functions.

Tiyeni tipulikiske umo Deep Residual Shrinkage Network yikugwilira ntchito mu nthowa iyi. Chakwamba, network iyi yikugwiritsa ntchito attention mechanisms kumanya features izo ni zambura kukhumbikwa (unimportant features). Pamanyuma, network yikutora soft thresholding functions na kusintha features zambura kukhumbikwa izi kuŵa zero. Mwakupambanako, network yikumanya features zakukhumbikwa (important features) na kuzisunga. Nthowa iyi yikukhozga deep neural network. Nthowa iyi yikovwira network kusanga useful features kufuma ku signals izo zili na noise.

1. Chifukwa chakuchitira Research iyi (Research Motivation)

Pakwamba, noise yikusangika nyengo zose para algorithm yikuchita classify samples. Viyelezgero vya noise iyi ni Gaussian noise, pink noise, na Laplacian noise. Kuyowoya mwakusazgikira, kanandi samples zikuŵa na uthenga uwo ulije ntchito ku classification task yasono. Tingatora uthenga wambura ntchito uwu nga ni noise. Noise iyi yingachepeska mphamvu ya classification. (Soft thresholding ni sitepe yakuzirwa chomene mu signal denoising algorithms zinandi.)

Tiyelezgere kuti ŵanthu ŵakuyowoya mumphepete mwa msewu. Mungapulika kulira kwa magalimoto na mawilo. Panyake tikukhumba kuchita speech recognition pa signals izi. Kulira kwa magalimoto uku kutimbanizgenge results. Kuyana na deep learning, deep neural network yikwenera kuwuskapo features za kulira kwa magalimoto na mawilo. Ichi chikovwira kuti features izi zileke kutimbanizga speech recognition results.

Chachiŵiri, unandi wa noise kanandi ukupambana pakati pa samples. Kusintha uku kukuchitika nanga ni mu dataset yimoza. (Kusintha uku kukuyana waka na attention mechanisms. Tiyeni titore image dataset nga ni chiyelezgero. Malo gha target object ghangapambana mu zithunzi zakupambanapambana. Attention mechanisms zingawona malo gheneko gha target object mu chithunzi chilichose.)

Mwachiyelezgero, tiyeni ti-training cat-and-dog classifier na zithunzi zinkhondi (5) izo zili na label ya “dog.” Chithunzi 1 chingaŵa na ntchebe na mbewa. Chithunzi 2 chingaŵa na ntchebe na atanda (goose). Chithunzi 3 chingaŵa na ntchebe na nkhuku. Chithunzi 4 chingaŵa na ntchebe na mbunda (donkey). Chithunzi 5 chingaŵa na ntchebe na bakha. Panyengo ya training, vinthu vyambura kukhumbikwa ivyo vikutimbanizga classifier. Vinthu ivi ni mbewa, atanda, nkhuku, mbunda, na bakha. Ichi chikupangiska kuti classification accuracy yikhire. Usange tingamanya vinthu vyambura kukhumbikwa ivi. Mbwenu, tingawuskapo features za vinthu ivi. Mu nthowa iyi, tingakweza accuracy ya cat-and-dog classifier.

2. Nthowa ya Soft Thresholding (Soft Thresholding)

Soft thresholding ni sitepe yakuzirwa mu signal denoising algorithms zinandi. Algorithm yikuwuskapo features usange absolute values za features izo ni zichoko kuluska threshold yanyake. Algorithm yiku-shrinks features kuluta ku zero usange absolute values za features ni zikuru kuluska threshold iyi. Ŵaswiri ŵangagwiritsa ntchito formula iyi kuti ŵapange soft thresholding:

Derivative ya soft thresholding output kuyana na input ni:

Formula ya pachanya yikulongora kuti derivative ya soft thresholding ni 1 panji 0. Ichi chikuyana waka na ReLU activation function. Ntheura, soft thresholding yingachepeska suzgo la gradient vanishing na gradient exploding mu deep learning algorithms.

Mu soft thresholding function, kukhazika threshold kukwenera kulondezga malango ghaŵiri. Chakwamba, threshold yikwenera kuŵa nambara ya positive. Chachiŵiri, threshold yileke kuluska maximum value ya input signal. Kwambura nthena, output yose yiŵenge zero.

Kusazgirapo apa, threshold yikwenera kulondezga dango lachitatu. Sample yiliyose yikwenera kuŵa na threshold yakeyake kuyana na unandi wa noise mu sample iyo.

Chifukwa ntchakuti unandi wa noise kanandi ukupambana mu samples. Mwachiyelezgero, Sample A yingaŵa na noise yichoko kweni Sample B yili na noise yinandi mu dataset yimoza. Apa, Sample A yikwenera kugwiritsa ntchito threshold yichoko panyengo ya soft thresholding. Sample B yikwenera kugwiritsa ntchito threshold yikuru. Nangauli features na thresholds izi zilije ng’anamuro lakudunjika mu deep neural networks, kweni logic yake njimoza. Ichi chikung’anamura kuti sample yiliyose yikwenera kuŵa na threshold yakujiyimira payekha. Unandi wa noise ndiyo ukusankha threshold iyi.

3. Nthowa ya Attention Mechanism (Attention Mechanism)

Ŵaswiri ŵangapulikiska attention mechanisms mu nkhani ya computer vision. Maso gha nyama ghangapambaniska vinthu mwakulaŵiska malo ghose mwaluŵiro. Pamanyuma, maso ghakuika attention pa target object. Ichi chikovwira kuti maso ghawone details zinandi. Panyengo yimoza, maso ghakusulako uthenga wambura kukhumbikwa. Kuti mumanye vinandi, ŵerengani mabuku gha attention mechanisms.

Squeeze-and-Excitation Network (SENet) ni nthowa yiphya ya deep learning iyo yikugwiritsa ntchito attention mechanisms. Mu samples zakupambana, feature channels zakupambana zikovwira munthowa zakupambana pa classification task. SENet yikugwiritsa ntchito sub-network yichoko kusanga gulu la weights. Pamanyuma, SENet yikuchita multiplies ma weights agha na features za mu channels izo. Sitepe ili likusintha ukuru wa features mu channel yiliyose. Tingatora sitepe ili nga ni kuika attention yakupambana pa feature channels zakupambana.

Mu nthowa iyi, sample yiliyose yili na gulu la weights lakujiyimira palekha. Ichi chikung’anamura kuti weights za samples ziŵiri zilizose ni zakupambana. Mu SENet, nthowa yeneko yakusangira weights ni “Global Pooling → Fully Connected Layer → ReLU Function → Fully Connected Layer → Sigmoid Function.”

4. Soft Thresholding pamoza na Deep Attention Mechanism

Deep Residual Shrinkage Network yikugwiritsa ntchito kawonekero ka SENet sub-network. Network yikugwiritsa ntchito kawonekero aka kuti yipange soft thresholding pasi pa deep attention mechanism. Sub-network (iyo yili mu bokosi liswesi) yiku- Learn a set of thresholds. Pamanyuma, network yikuchita soft thresholding pa feature channel yiliyose pakugwiritsa ntchito thresholds izi.

Mu sub-network iyi, chakwamba system yikupenda absolute values za features zose mu input feature map. Pamanyuma, system yikuchita global average pooling na averaging kuti yisange feature, iyo tikuyichema A. Mu nthowa yinyake, system yikutora feature map na kuyinjizga mu fully connected network yichoko pamanyuma pa global average pooling. Fully connected network iyi yikugwiritsa ntchito Sigmoid function nga ni layer yaumaliro. Function iyi yikusintha output kuŵa pakati pa 0 na 1. Ichi chikufumiska nambara (coefficient), iyo tikuyichema α. Tingalemba threshold yaumaliro nga ni α × A. Ntheura, threshold ni zotsatira za nambara ziŵiri izo zachita multiply. Nambara yimoza yili pakati pa 0 na 1. Nambara yinyake ni average ya absolute values za feature map. Nthowa iyi yikuoneseska kuti threshold ni positive. Nthowa iyi yikuoneseska so kuti threshold ni yikuru chomene yayi.

Kusazgirapo, samples zakupambana zikupanga thresholds zakupambana. Ntheura, tingapulikiska nthowa iyi nga ni attention mechanism yapadera. Mechanism iyi yikumanya features izo ni zambura kukhumbikwa pa ntchito yasono. Mechanism yikusintha features izi kuŵa nambara za pafupi na zero kupyolera mu convolutional layers ziŵiri. Pamanyuma, mechanism yikusintha features izi kuŵa zero pakugwiritsa ntchito soft thresholding. Mwakupambanako, mechanism yikumanya features zakukhumbikwa pa ntchito yasono. Mechanism yikusintha features izi kuŵa nambara za kutali na zero kupyolera mu convolutional layers ziŵiri. Paumaliro, mechanism yikusunga features izi.

Paumaliro, tikuwunjika (Stack many basic modules) chiŵerengero cha basic modules. Tikuikamo so convolutional layers, batch normalization, activation functions, global average pooling, na fully connected output layers. Ichi chikuzenga Deep Residual Shrinkage Network yose.

5. Mphamvu yakugwira ntchito kulikose (Generalization Capability)

Deep Residual Shrinkage Network ni nthowa ya feature learning iyo yingagwira ntchito kulikose. Chifukwa ntchakuti, mu ntchito zinandi za feature learning, samples kanandi zikuŵa na noise. Samples zikuŵa so na uthenga wambura kukhumbikwa. Noise na uthenga wambura kukhumbikwa vingatimbanizga feature learning. Mwachiyelezgero:

Tiwoneso image classification. Chithunzi chingatoso vinthu vinyake vinandi panyengo yimoza. Tingatora vinthu ivi nga ni “noise.” Deep Residual Shrinkage Network yingagwiritsa ntchito attention mechanism. Network yikuwona “noise” iyi. Pamanyuma, network yikugwiritsa ntchito soft thresholding kusintha features za “noise” iyi kuŵa zero. Ichi chingakweza accuracy ya image classification.

Tiwone speech recognition. Chomenechomene mu malo gha noise yinandi nga ni mumphepete mwa msewu panji mukati mwa fakitale. Deep Residual Shrinkage Network yingakweza accuracy ya speech recognition. Panji titi, network yikupeleka nthowa (methodology). Nthowa iyi yili na nkhongono zakukwezga accuracy ya speech recognition.

Uko kwafuma ukhaliro (Reference)

Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Michael Pecht, Deep residual shrinkage networks for fault diagnosis, IEEE Transactions on Industrial Informatics, 2020, 16(7): 4681-4690.

https://ieeexplore.ieee.org/document/8850096

BibTeX

@article{Zhao2020,
  author    = {Minghang Zhao and Shisheng Zhong and Xuyun Fu and Baoping Tang and Michael Pecht},
  title     = {Deep Residual Shrinkage Networks for Fault Diagnosis},
  journal   = {IEEE Transactions on Industrial Informatics},
  year      = {2020},
  volume    = {16},
  number    = {7},
  pages     = {4681-4690},
  doi       = {10.1109/TII.2019.2943898}
}

Umo ntchito iyi yawovwilira (Academic Impact)

Paper iyi yapokera ma citations ghakujumpha 1,400 pa Google Scholar.

Kuyana na kafukufuku, ŵaswiri ŵagwiritsa ntchito Deep Residual Shrinkage Network (DRSN) mu mabuku/kafukufuku wakujumpha 1,000. Ntchito izi zili mu mbali zinandi. Mbali izi zikusazgapo mechanical engineering, electrical power, vision, healthcare, speech, text, radar, na remote sensing.