Deep Residual Shrinkage Network: Uburyo bwa Artificial Intelligence bwo gukoresha kuri Data zirimo Noise nyinshi

Deep Residual Shrinkage Network ni variant yavuguruwe ya Deep Residual Network. Muri make, Deep Residual Shrinkage Network ihuza Deep Residual Network, attention mechanisms, hamwe na soft thresholding functions.

Dushobora kumva imikorere ya Deep Residual Shrinkage Network muri ubu buryo. Mbere na mbere, network ikoresha attention mechanisms kugira ngo itahure features zidafite akamaro (unimportant features). Hanyuma, network ikoresha soft thresholding functions kugira ngo ishyire izo features kuri zero. Ibinyuranye n’ibyo, network itahura features z’ingenzi (important features) kandi ikabika izo features. Iyi process yongerera imbaraga deep neural network. Iyi process ifasha network gukura features zifite akamaro muri signals zirimo noise.

1. Impamvu y’ubushakashatsi (Research Motivation)

Mbere na mbere, noise ntabwo yakwirindwa igihe algorithm irimo gukora classification ya samples. Ingero z’iyo noise harimo Gaussian noise, pink noise, na Laplacian noise. Mu buryo bwagutse, samples zikunze kuba zirimo amakuru adafite aho ahuriye na classification task iri gukorwa. Dushobora gufata aya makuru adakenewe nka noise. Iyi noise ishobora kugabanya classification performance. (Soft thresholding ni intambwe y’ingenzi (key step) muri algorithms nyinshi za signal denoising.)

Reka dufate urugero rw’ikiganiro kiri kubera iruhande rw’umuhanda. Audio ishobora kuba irimo amajwi y’amahembe y’imodoka (horns) n’amapine (wheels). Dushobora gukora speech recognition kuri izo signals. Amajwi yo inyuma (background sounds) azagira ingaruka ku bisubizo. Iyo turebye muri perspective ya deep learning, deep neural network yagakwiye gukuraho features zijyanye n’amahembe y’imodoka hamwe n’amapine. Uku gukuraho (elimination) birinda ko izo features zagira ingaruka kuri speech recognition results.

Icya kabiri, ingano ya noise ikunze gutandukana hagati ya samples. Iri tandukana ribaho n’iyo waba uri muri dataset imwe. (Iri tandukana rifite aho rihurira na attention mechanisms. Reka dufate urugero rwa image dataset. Ahantu hari target object hashobora kuba hatandukanye muri buri image. Attention mechanisms zishobora kwibanda (focus) ahantu hihariye hari target object muri buri image.)

Urugero, tekereza turimo gutoza (training) cat-and-dog classifier dukoresheje amashusho (images) atanu yiswe “dog”. Image 1 ishobora kuba irimo dog na mouse. Image 2 ishobora kuba irimo dog na goose. Image 3 ishobora kuba irimo dog na chicken. Image 4 ishobora kuba irimo dog na donkey. Image 5 ishobora kuba irimo dog na duck. Mu gihe cya training, ibintu bidafite aho bihuriye na task (irrelevant objects) bizabangamira classifier. Ibyo bintu birimo mice, geese, chickens, donkeys, na ducks. Uku kubangamira gutuma classification accuracyigabanuka. Turamutse dushoboye kumenya ibi bintu bidakenewe. Hanyuma, dushobora gukuraho features zijyanye n’ibyo bintu. Muri ubu buryo, dushobora kuzamura accuracy ya cat-and-dog classifier.

2. Soft Thresholding

Soft thresholding ni intambwe y’ingenzi (core step) muri algorithms nyinshi za signal denoising. Algorithm ikuraho features (eliminates features) iyo absolute values z’izo features ziri munsi ya threshold runaka. Algorithmikora shrinking ya features zerekeza kuri zero iyo absolute values z’izo features ziri hejuru y’iyo threshold. Abashakashatsi bashobora gukora soft thresholding bakoresheje formula ikurikira:

Derivative ya soft thresholding output ugereranyije na input ni:

Iyo formula iri hejuru yerekana ko derivative ya soft thresholding ari 1 cyangwa 0. Iyi property ni kimwe n’iya ReLU activation function. Kubera iyo mpamvu, soft thresholding ishobora kugabanya ibyago bya gradient vanishing na gradient exploding muri deep learning algorithms.

Muri soft thresholding function, gushyiraho threshold bigomba kuzuza ibisabwa bibiri. Icya mbere, threshold igomba kuba umubare mwiza (positive number). Icya kabiri, threshold ntabwo igomba kurenga agaciro kanini (maximum value) ka input signal. Bitabaye ibyo, output yose izaba zero.

Byongeye kandi, byaba byiza threshold yujuje ikintu cya gatatu. Buri sample yagakwiye kugira threshold yayo yigenga (independent threshold) bitewe na noise content iri muri iyo sample.

Impamvu ni uko noise content ikunze gutandukana hagati ya samples. Urugero, Sample A ishobora kuba irimo noise nkeya naho Sample B ikaba irimo noise nyinshi muri dataset imwe. Muri iki gihe, Sample A yagakwiye gukoresha threshold ntoya igihe cya soft thresholding. Sample B yagakwiye gukoresha threshold nini. Muri deep neural networks, nubwo izi features na thresholds bitakaza ubusobanuro bwabyo busanzwe bwa physics (explicit physical definitions). Ariko, logic y’ibanze igumaho. Mu yandi magambo, buri sample yagakwiye kugira threshold yigenga. Noise content yihariye niyo igena iyo threshold.

3. Attention Mechanism

Abashakashatsi bashobora kumva neza attention mechanisms mu gice cya computer vision. Visual systems z’inyamaswa zishobora gutandukanya targets zikoresheje gusikana (scanning) vuba agace kose. Nyuma yaho, visual systems zishyira attention yazo (focus attention) kuri target object. Iki gikorwa gituma systems zibasha kubona details nyinshi. Icyarimwe, systems zikora suppressing y’amakuru adakenewe (irrelevant information). Kugira ngo umenye details, wasoma literature zijyanye na attention mechanisms.

Squeeze-and-Excitation Network (SENet) ihagarariye uburyo bushya bwa deep learning bukoresha attention mechanisms. Muri samples zitandukanye, feature channels zitandukanye zigira uruhare rutandukanye kuri classification task. SENet ikoresha sub-network ntoya kugira ngo ibone seti y’ama-weights (a set of weights). Hanyuma, SENet ikuba ayo weights na features za channels zijyanye nayo. Iki gikorwa gihindura ingano (magnitude) ya features muri buri channel. Dushobora gufata iyi process nko gushyira levels zitandukanye za attention kuri feature channels zitandukanye.

Muri ubu buryo, buri sample iba ifite seti y’ama-weights yigenga (independent set of weights). Mu yandi magambo, weights za samples ebyiri izo ari zo zose ziba zitandukanye. Muri SENet, inzira yihariye yo kubona weights ni “Global Pooling → Fully Connected Layer → ReLU Function → Fully Connected Layer → Sigmoid Function.”

4. Soft Thresholding hamwe na Deep Attention Mechanism

Deep Residual Shrinkage Network ikoresha structure ya SENet sub-network. Network ikoresha iyi structure kugira ngo ikore soft thresholding igendeye kuri deep attention mechanism. Iyo sub-network (yerekanwe mu gasanduku gatukura) yiga seti y’ama-thresholds (Learn a set of thresholds). Hanyuma, network ikora soft thresholding kuri buri feature channel ikoresheje izo thresholds.

Muri iyi sub-network, system ibanza kubara absolute values za features zose muri input feature map. Hanyuma, system ikora global average pooling na averaging kugira ngo ibone feature, yitwa A. Mu yindi nzira (path), system yinjiza feature map muri fully connected network ntoya nyuma ya global average pooling. Iyi fully connected network ikoresha Sigmoid function nka layer ya nyuma. Iyi function ikora normalizing ya output ikaza hagati ya 0 na 1. Iyi process itanga coefficient, yitwa α. Dushobora kwandika threshold ya nyuma nka α × A. Bityo, threshold ni igikubo cy’imibare ibiri. Umubare umwe uba hagati ya 0 na 1. Undi mubare ni average ya absolute values ya feature map. Ubu buryo bwizera ko threshold iba positive. Ubu buryo kandi bwizera ko threshold itaba nini cyane (excessively large).

Byongeye, samples zitandukanye zitanga thresholds zitandukanye. Ku bw’iyo mpamvu, dushobora gufata ubu buryo nka attention mechanism yihariye (specialized). Iyi mechanism itahura features zidafite aho zihuriye na task iri gukorwa. Iyi mechanism ihindura izo features ikazishira hafi ya 0 ikoresheje convolutional layers ebyiri. Hanyuma, iyi mechanism ishyira izo features kuri zero ikoresheje soft thresholding. Cyangwa se, iyi mechanism itahura features zifite aho zihuriye na task iri gukorwa. Iyi mechanism ihindura izo features ikazishira kure ya 0 ikoresheje convolutional layers ebyiri. Hanyuma, iyi mechanism ibika (preserves) izo features.

Mu gusoza, dukora stacking y’umubare runaka wa basic modules (Stack many basic modules). Dushyiramo kandi convolutional layers, batch normalization, activation functions, global average pooling, na fully connected output layers. Iyi process yubaka Deep Residual Shrinkage Network yuzuye.

5. Ubushobozi bwo gukora muri rusange (Generalization Capability)

Deep Residual Shrinkage Network ni uburyo rusange (general method) bwa feature learning. Impamvu ni uko muri tasks nyinshi za feature learning, samples zikunze kuba zirimo noise. Samples ziba zirimo n’amakuru adakenewe (irrelevant information). Iyi noise n’amakuru adakenewe bishobora kugira ingaruka kuri performance ya feature learning. Urugero:

Reka turebe image classification. Image ishobora kuba irimo ibindi bintu byinshi icyarimwe. Dushobora gufata ibi bintu nka “noise”. Deep Residual Shrinkage Network ishobora kuba yabasha gukoresha attention mechanism. Network ibona iyi “noise”. Hanyuma, network ikoresha soft thresholding kugira ngo ishyire features zihura n’iyo “noise” kuri zero. Iki gikorwa gifite ubushobozi bwo kuzamura image classification accuracy.

Reka turebe speech recognition. By’umwihariko, ahantu harimo urusaku (noisy environments) nko mu kiganiro iruhande rw’umuhanda cyangwa imbere mu ruganda (factory workshop). Deep Residual Shrinkage Network ishobora kuzamura speech recognition accuracy. Cyangwa byibura, network itanga methodology. Iyi methodology ifite ubushobozi bwo kuzamura speech recognition accuracy.

Reference

Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Michael Pecht, Deep residual shrinkage networks for fault diagnosis, IEEE Transactions on Industrial Informatics, 2020, 16(7): 4681-4690.

https://ieeexplore.ieee.org/document/8850096

BibTeX

@article{Zhao2020,
  author    = {Minghang Zhao and Shisheng Zhong and Xuyun Fu and Baoping Tang and Michael Pecht},
  title     = {Deep Residual Shrinkage Networks for Fault Diagnosis},
  journal   = {IEEE Transactions on Industrial Informatics},
  year      = {2020},
  volume    = {16},
  number    = {7},
  pages     = {4681-4690},
  doi       = {10.1109/TII.2019.2943898}
}

Ingaruka mu by’ubumenyi (Academic Impact)

Iyi paper imaze kubona citations zirenga 1,400 kuri Google Scholar.

Gendeye kuri statistics zitaruzura neza, abashakashatsi bamaze gukoresha Deep Residual Shrinkage Network (DRSN) mu bitabo/inyandiko (publications/studies) zirenga 1,000. Izi applications ziri mu bice bigari bitandukanye. Ibi bice birimo mechanical engineering, electrical power, vision, healthcare, speech, text, radar, na remote sensing.