Incamake ya Deep Residual Shrinkage Network: Ubuhinga bwa Artificial Intelligence bwo gukoresha ku ma Data arimo Noise nyinshi

Deep Residual Shrinkage Network ni ubwoko bugezweho bwa Deep Residual Network bwagaruwe neza. Mu make, Deep Residual Shrinkage Network ihuriza hamwe Deep Residual Network, ama attention mechanisms, hamwe na soft thresholding functions.

Turashobora gutahura uko Deep Residual Shrinkage Network ikora muri ubu buryo. Ubwa mbere, iyo network ikoresha ama attention mechanisms kugira ngoimenye ama features adakenewe (unimportant features). Hanyuma, iyo network ikoresha soft thresholding functions kugira igire ayo ma features adakenewe zero. Nayo ku rundi ruhande, iyo networkiramenya ama features y’ingirakamaro (important features) hanyuma ikayagumizaho. Uku gukora gutya kuratuma ubushobozi bwa deep neural network bwiyongera cane. Ibi bifasha network gukura ama features afise akamaro mu ma signals arimwo noise.

1. Impamvu zateye ubu bushakashatsi (Research Motivation)

Ubwa mbere, igihe algorithm iriko iratandukanya ama samples, noise ntishobora kubura. Akarorero k’iyo noise harimwo Gaussian noise, pink noise, hamwe na Laplacian noise. Mu buryo bwagutse, ama samples akenshi aba arimwo amakuru adakenewe kuri uwo murimo wo gutandukanya (classification task). Ayo makuru adakenewe dushobora kuyafata nka noise. Iyo noise ishobora gutuma classification itagenda neza. (Soft thresholding ni intambwe nkuru cane muri za algorithms nyinshi zo gukura noise muri signal).

Dufate akarorero k’ikiyago iruhande y’ibarabara. Muri iryo jwi hashobora kuba harimwo amahoni y’imiduga hamwe n’amajwi y’amapine. Dushobora gukora speech recognition kuri ayo ma signals. Amajwi yo inyuma (background sounds) ategerezwa kwivanga mu biva muri iyo recognition. Mu buryo bwa deep learning, iyo deep neural network ikwiye gukura ama features ajanye n’amahoni hamwe n’amapine. Ibi bituma ayo ma features atazana ingaruka mbi kuri speech recognition results.

Ubwa kabiri, ingano ya noise akenshi irahindagurika hagati y’ama samples atandukanye. Ibi biraba naho yoba ari muri dataset imwe. (Iri hindagurika rifitaniye isano n’ama attention mechanisms. Dufate akarorero ka dataset y’amafoto. Ahantu ikintu turondera (target object) giherereye hashobora kuba hatandukanye muri buri foto. Ama attention mechanisms arashobora kwibanda neza neza ahantu iryo shusho riri muri buri foto.)

Dufate akarorero ko kwigisha classifier gutandukanya injangwe n’imbwa, tukaba dufise amafoto atanu yanditseko “dog” (imbwa). Ifoto ya 1 ishobora kuba irimwo imbwa n’imbeba. Ifoto ya 2 ishobora kuba irimwo imbwa n’igisata (goose). Ifoto ya 3 ishobora kuba irimwo imbwa n’inkoko. Ifoto ya 4 ishobora kuba irimwo imbwa n’indogoba. Ifoto ya 5 ishobora kuba irimwo imbwa n’imbata (duck). Mu gihe c’imyitozo (training), ibintu bidakenewe bizobangamira iyo classifier. Ibyo bintu birimwo imbeba, ibisata, inkoko, indogoba, n’imbata. Uku kubangamira gutuma classification accuracyigabanuka. Hamwe twoshobora kumenya ibi bintu bidakenewe, hanyuma tugakura ama features bijanye navyo, dushobora kuzamura accuracy y’iyo cat-and-dog classifier.

2. Uko soft thresholding ikora

Soft thresholding ni intambwe nkuru cane muri za algorithms nyinshi zo gukura noise muri signal (signal denoising). Iyo algorithm irahanagura ama features igihe ama absolute values yayo ari munsi y’urugero runaka (threshold). Iyo algorithm igabanya ama featuresakajya kuri zero igihe ama absolute values yayo ari hejuru y’iyo threshold. Abashakashatsi barashobora gushira mu ngiro soft thresholding bakoresheje iyi formula:

Derivative ya soft thresholding output ugereranije na input ni:

Iyo formula yo hejuru yerekana ko derivative ya soft thresholding ari 1 canke 0. Ibi birasa neza na ReLU activation function. Rero, soft thresholding irashobora kugabanya ingaruka za gradient vanishing hamwe na gradient exploding muri za algorithms za deep learning.

Muri soft thresholding function, uko bategura threshold bategerezwa gukurikiza ibisabwa bibiri. Icya mbere, threshold itegerezwa kuba igiharuro ciza (positive number). Icya kabiri, threshold ntishobora kurenga agaciro kanini (maximum value) ka input signal. Bitagenze uko, output yose izoca iba zero.

Kandi, birashimwa ko threshold yokurikiza n’igisabwa ca gatatu. Buri sample yikwiye kugira threshold yayo yihariye bivanye n’ingano ya noise iri muri iyo sample.

Imvo nuko ingano ya noise akenshi iba itandukanye mu ma samples. Urugero, Sample A ishobora kuba irimwo noise nke mugihe Sample B irimwo noise nyinshi muri dataset imwe. Iyo bimeze gutyo, Sample A ikwiye gukoresha threshold ntoya muri soft thresholding. Sample B ikwiye gukoresha threshold nini. Muri ama deep neural networks, naho ayo ma features n’ama thresholds bita ubusobanuro bwabyo busanzwe bwa fizike, ariko logic niyo igumaho. Ni ukuvuga, buri sample ikwiye kugira threshold yigenga. Ingano nyayo ya noise niyo igena iyo threshold.

3. Attention Mechanism

Abashakashatsi barashobora gutahura attention mechanisms mu gisata ca computer vision bitagoranye. Amaso y’ibikoko arashobora gutandukanya ibintu mu gusuzuma vyihuse akarere kose. Hanyuma, amaso yibanda (focus attention) ku kintu nyamukuru (target object). Ibi bituma bishobora kubona udukuru twinshi (more details). Ico gihe nyene, biba biriko birirengagiza amakuru adakenewe. Kugira mumenye vyinshi, mwosoma ibitabo bijanye na attention mechanisms.

Squeeze-and-Excitation Network (SENet) ni uburyo bushasha bwa deep learning bukoresha attention mechanisms. Mu ma samples atandukanye, ama feature channels atandukanye agira uruhare rutandukanye muri classification task. SENet ikoresha sub-network ntoya kugira ibone urutonde rw’ama weights (Learn a set of weights). Hanyuma, SENetigakuba ayo ma weights n’ama features yo muri ayo ma channels (Apply weighting to each feature channel). Iki gikorwa gihindura ingano y’ama features muri buri channel. Dushobora kubona iki gikorwa nko gushira urugero rutandukanye rwa attention ku ma feature channels atandukanye.

Muri ubu buryo, buri sample ifise urutonde rwayo rwigenga rw’ama weights. Ni ukuvuga, ama weights y’ama samples abiri ayo ariyo yose aratandukanye. Muri SENet, inzira yo kuronka ama weights ni “Global Pooling → Fully Connected Layer → ReLU Function → Fully Connected Layer → Sigmoid Function.”

4. Soft Thresholding hamwe na Deep Attention Mechanism

Deep Residual Shrinkage Network ikoresha imiterere (structure) ya SENet sub-network. Iyi network ikoresha iyi structure kugira ngoikore soft thresholding ifashijwe na deep attention mechanism. Iyo sub-network (yerekanwe mu kazitiro gatukura) yiga urutonde rw’ama thresholds (Learn a set of thresholds). Hanyuma, network igakoresha soft thresholding kuri buri feature channel ikoresheje ayo ma thresholds.

Muri iyo sub-network, sisitemu ibanza guharura ama absolute values y’ama features yose ari muri input feature map. Hanyuma, igakora global average pooling no gufata impuzandengo (averaging) kugira ibone feature imwe, yitwa A. Mu yindi nzira, sisitemu yinjiza feature map muri fully connected network ntoya inyuma ya global average pooling. Iyo fully connected network ikoresha Sigmoid function nka layer ya nyuma. Iyo function ituma outputija hagati ya 0 na 1. Ibi bitanga coefficient, yitwa α. Dushobora kwandika final threshold nka α × A. Rero, iyo threshold ni ibiharuro bibiri bikubye. Igiharuro kimwe kiri hagati ya 0 na 1. Ikindi giharuro ni impuzandengo (average) y’ama absolute values ya feature map. Ubu buryo bura assuringa (ensures) ko threshold iba positive. Ubu buryo kandi bura assuringa ko threshold itaba nini cane.

Mbere, ama samples atandukanye atanga ama thresholds atandukanye. Rero, dushobora gutahura ubu buryo nka attention mechanism yihariye. Iyo mechanism iramenya features zidakenewe (irrelevant features) kuri task turimwo. Iyo mechanism ihindura izo featureszikaba ibiharuro biri hafi ya zero biciye muri convolutional layers zibiri. Hanyuma, iyo mechanism igashira izo features kuri zero ikoresheje soft thresholding. Canke, iyo mechanism ikamenya features zikenewe (relevant features). Iyo mechanism igahindura izo featureszikaba ibiharuro biri kure ya zero biciye muri convolutional layers zibiri. Hanyuma, iyo mechanism ikagumiza izo features (retains these features).

Ubwa nyuma, turarunda (stack) igitigiri runaka c’ama basic modules (Stack many basic modules). Turongeramwo kandi ama convolutional layers, batch normalization, activation functions, global average pooling, hamwe na fully connected output layers. Iki gikorwa nico cyubaka Deep Residual Shrinkage Network yuzuye.

5. Ubushobozi bwo gukora hose (Generalization Capability)

Deep Residual Shrinkage Network ni uburyo rusange (general method) bwo kwiga ama features. Impamvu nuko mu mirimo myinshi yo kwiga ama features, ama samples akenshi aba arimwo noise. Ama samples aba arimwo n’amakuru adakenewe. Iyo noise n’ayo makuru adakenewe birashobora kugabanya ubushobozi bwo kwiga ama features. Akarorero:

Dufate image classification. Ifoto ishobora kuba irimwo ibindi bintu vyinshi. Dushobora gufata ibyo bintu nka “noise”. Deep Residual Shrinkage Network ishobora gukoresha attention mechanism. Iyo network ikabona iyo “noise”. Hanyuma, network igakoresha soft thresholding kugira ngoigire ama features ajanye n’iyo “noise” zero. Iki gikorwa gishobora gutuma image classification accuracyizamuka.

Dufate speech recognition. Cane cane, dufate ahantu hari urusaku nko ku barabara canke mu ruganda. Deep Residual Shrinkage Network ishobora kuzamura speech recognition accuracy. Canke byibura, iyo networkitanga uburyo (methodology). Ubwo buryo bufise ubushobozi bwo kuzamura speech recognition accuracy.

References

Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Michael Pecht, Deep residual shrinkage networks for fault diagnosis, IEEE Transactions on Industrial Informatics, 2020, 16(7): 4681-4690.

https://ieeexplore.ieee.org/document/8850096

BibTeX

@article{Zhao2020,
  author    = {Minghang Zhao and Shisheng Zhong and Xuyun Fu and Baoping Tang and Michael Pecht},
  title     = {Deep Residual Shrinkage Networks for Fault Diagnosis},
  journal   = {IEEE Transactions on Industrial Informatics},
  year      = {2020},
  volume    = {16},
  number    = {7},
  pages     = {4681-4690},
  doi       = {10.1109/TII.2019.2943898}
}

Ingaruka mu bya Académie (Academic Impact)

Iyi paper imaze kuronka ama citations arenga 1,400 kuri Google Scholar.

Dufatiye ku bitigiri bituzuye neza, abashakashatsi bamaze gukoresha Deep Residual Shrinkage Network (DRSN) mu bitabo canke ubushakashatsi (publications/studies) birenga 1,000. Izi applications ziri mu bisata vyinshi bitandukanye. Ibyo bisata birimwo mechanical engineering, electrical power, vision, healthcare, speech, text, radar, hamwe na remote sensing.