Na Deep Residual Shrinkage Network: E Dua na Artificial Intelligence Method me Baleta na Highly Noisy Data

Na Deep Residual Shrinkage Network e dua na kena iwalewale e sa improved (vakavinakataki) mai na Deep Residual Network. Ena kena dina, na Deep Residual Shrinkage Network e integrate (vakaduavata) tiko kina na Deep Residual Network, attention mechanisms, kei na soft thresholding functions.

Eda na rawa ni kila na working principle (kena iwalewale ni cakacaka) ni Deep Residual Shrinkage Network ena sala oqo. Matai, na network e vakayagataka na attention mechanisms me identify (kunea) na unimportant features (feature e sega ni bibi). Qai oti, na network e vakayagataka na soft thresholding functions me set (biuta) kina na unimportant features oqori ki na zero. Ia e veicalati sara, na network e identify (kunea) na important features (feature bibi) ka retain (maroroya) talega na important features oqo. Na iwalewale oqo e enhance (vakaukauwataka) na kena rawa ni cakacaka na deep neural network. E vukea na iwalewale oqo na network me extract (tovolea) na useful features (feature yaga) mai na signals e tiko kina na noise.

1. Na vuna ni vakadidike

Matai, na noise (voqa) e sega ni rawa ni drotaki ni sa classify (vakatulewataka) na samples (iyaya) na algorithm. Me kena ivakaraitaki ni noise oqo e wili kina na Gaussian noise, pink noise, kei na Laplacian noise. Ni da vakaraica sara vakalevu, na samples e tiko vakalevu kina na information e sega ni veisemati kei na current classification task (cakacaka ni vakatulewataka e cakacaki tiko). Eda na rawa ni kila na irrelevant information (information e sega ni veisemati) oqo me vaka ga na noise. Na noise oqo e rawa ni vakalailaitaka na classification performance (kena cakacaka ni vakatulewataka). (Soft thresholding e dua na key step (iwalewale bibi) ena vuqa na signal denoising algorithms.)

Me kena ivakaraitaki, ni da vakaivotavota ena dua na veivosaki ena roadside (bati ni gaunisala). Na audio e rawa ni tiko kina na domo ni car horns (davui ni motoka) kei na wheels (yavulovulo). Eda na rawa ni perform (cakava) na speech recognition (vakatulewataka ni vosa) ena signals oqo. Na background sounds (domo e muri) ena inevitably (vakadeitaki) ni na affect (vakaleqa) na results. Mai na deep learning perspective, na deep neural network e dodonu me kauta tani na features e veisemati kei na horns kei na wheels. Na kena kauti tani oqo e prevent (tarova) kina na features me affect (vakaleqa) na speech recognition results.

E karua, na levu ni noise e dau vary (duidui) ena kedra maliwa na samples yadua. Na variation (veidutaitaki) oqo e yaco tikoga, dina mada ga ni samples kece e tu ena loma ni same dataset. (Na variation oqo e tu kina na similarities (veivolekati) kei na attention mechanisms. Me kena ivakaraitaki, ni da vakayagataka na image dataset. Na vanua e tu kina na target object (iyaya bibi) e rawa ni duidui ena veivanua ni images kecega. Na attention mechanisms e rawa ni focus (vakabibitaka) na attention ki na specific location ni target object ena images yadua.)

Me kena ivakaraitaki, ni da train (vakavulica) tiko e dua na cat-and-dog classifier (dau vakatulewataka na pusi kei na koli), sa tiko vei keda e lima na images e tukuna tiko ni oya na “dog”. Na Image 1 e rawa ni tiko kina na dog kei na mouse (kalavo). Na Image 2 e rawa ni tiko kina na dog kei na goose (toa vusivusi). Na Image 3 e rawa ni tiko kina na dog kei na chicken (toa). Na Image 4 e rawa ni tiko kina na dog kei na donkey (asa). Na Image 5 e rawa ni tiko kina na dog kei na duck (ga). Ena gauna ni training, na irrelevant objects (iyaya e sega ni veisemati) ena interfere (veisaqasaqa) kei na classifier. Na objects oqo e wili kina na mice, geese, chickens, donkeys, kei na ducks. Na interference oqo e result (vakavuna) kina na kena decrease (lailai sobu) na classification accuracy. Nanuma mada, ni da rawa ni identify (kunea) na irrelevant objects oqo. O koya gona, eda na rawa ni eliminate (kauta tani) na features e veisemati kei na objects oqo. Ena sala oqo, eda na rawa ni improve (vakavinakataka) na accuracy ni cat-and-dog classifier.

2. Na Soft Thresholding

Na Soft thresholding e dua na core step (iwalewale bibi) ena vuqa na signal denoising algorithms. Na algorithm e eliminate (kauta tani) na features kevaka na absolute values (dina ni levu) ni features e lailai sobu mai na dua na certain threshold (kena iyalayala). Na algorithm e shrink (vakalailaitaka) na features ki na zero kevaka na absolute values ni features e levu cake mai na threshold oqo. E rawa ni ra implement (cakava) na soft thresholding na researchers ena vakayagataki ni formula oqo:

Na derivative (kena iwali) ni soft thresholding output e veisemati kei na input oya:

Na formula e cake e vakaraitaka tiko ni na derivative ni soft thresholding e na either (se) 1 se 0. Na property (kena iwali) oqo e tautauvata kei na property ni ReLU activation function. O koya gona, na soft thresholding e rawa ni reduce (vakalailaitaka) kina na risk (rerevaki) ni gradient vanishing kei na gradient exploding ena deep learning algorithms.

Ena soft thresholding function, na kena setting (vakarautaki) na threshold e dodonu me satisfy (kila) e rua na conditions (ivakarau). Matai, na threshold e dodonu me dua na positive number (naba dina). E karua, na threshold e sega ni rawa ni exceed (sivia) na maximum value (kena levu) ni input signal (signeli ni curu). Ke sega, na output (kena isau) ena entirely zero (sega vakadua).

Talei, e vinaka kevaka na threshold e rawa ni satisfy (kila) e dua na third condition (ivakarau e tolu). Na sample yadua e dodonu me tiko kina nona own independent threshold (threshold dina ka duidui) e vakatau ena noise content (kena levu ni voqa) ni sample oya.

Na vuna oya ni na noise content e dau vary (duidui) tiko ena kedra maliwa na samples. Me kena ivakaraitaki, ena loma ni same dataset, na Sample A e rawa ni lailai sobu na kena noise, ka rawa ni levu cake na noise ni Sample B. Ena gauna e vaka oqo, na Sample A e dodonu me vakayagataka e dua na smaller threshold ni sa soft thresholding tiko. Na Sample B e dodonu me vakayagataka e dua na larger threshold. Ena loma ni deep neural networks, dina mada ga ni na features kei na thresholds oqo e sa sega tu ni tiko kina na explicit physical definitions (vakamacala vakayago e matata), na basic underlying logic (vuna dina) e se tu ga. Ena dua tale na vosa, na sample yadua e dodonu me tiko kina e dua na independent threshold. Na specific noise content (kena levu dina ni voqa) e determine (vakatulewataka) na threshold oqo.

3. Na Attention Mechanism

E rawa ni ra kila vakavinaka na researchers (dau vakadikeva) na attention mechanisms ena field (vanua ni cakacaka) ni computer vision. Na visual systems (kena iwalewale ni raica) ni manumanu e rawa ni distinguish (vakaduiduitaka) na targets ena nodra rapidly scanning (vakasaqa vakatotolo) na entire area (vanua taucoko). Qai oti, na visual systems e focus (vakabibitaka) na attention ki na target object (iyaya bibi). Na cakacaka oqo e allow (vakatara) na systems me extract (tovolea) na more details (ka ni lewe). Ena gauna vata ga, na systems e suppress (vakalailaitaka) na irrelevant information (information e sega ni veisemati). Me baleta na kena dina, yalovinaka ni vakaraica na literature (ivola) e veisemati kei na attention mechanisms.

Na Squeeze-and-Excitation Network (SENet) e vakatakarakarataka e dua na relatively new (vou vakalailai) deep learning method e vakayagataka na attention mechanisms. Ena kedra maliwa na samples duidui, na different feature channels (sala ni feature duidui) e contribute (solia) vakaduadua ki na classification task. Na SENet e vakayagataka e dua na small sub-network me rawata kina e dua na set of weights (iwasewase ni weights). Qai oti, na SENet e multiply (vakalewalewa) na weights oqo kei na features ni respective channels. Na cakacaka oqo e adjust (vakavinakataka) na levu ni features ena channel yadua. Eda na rawa ni raica na iwalewale oqo me vaka ga e dua na kena vakamuri na varying levels of attention (duidui ni levu ni attention) ki na different feature channels.

Ena iwalewale oqo, na sample yadua e tiko kina e dua na independent set of weights. Ena dua tale na vosa, na weights ni rua ga na arbitrary samples e duidui. Ena SENet, na specific path (sala dina) ni kena rawati na weights oya na: “Global Pooling → Fully Connected Layer → ReLU Function → Fully Connected Layer → Sigmoid Function.”

4. Na Soft Thresholding kei na Deep Attention Mechanism

Na Deep Residual Shrinkage Network e vakayagataka na structure (kena iyaloyalo) ni SENet sub-network. Na network e vakayagataka na structure oqo me implement (cakava) kina na soft thresholding ena ruku ni deep attention mechanism. Na sub-network (e vakaraitaki tiko ena red box) e learn (vulica) e dua na set of thresholds. Qai oti, na network e vakayagataka na thresholds oqo me cakava kina na soft thresholding ki na feature channel yadua.

Ena sub-network oqo, na system e first (matai) ni calculate (wilika) na absolute values ni features kece ena input feature map. Qai oti, na system e perform (cakava) na global average pooling kei na averaging (wilika na average) me rawata kina e dua na feature, ka volai tiko me A. Ena dua tale na path (sala), na system e vakacuruma na feature map ki na dua na small fully connected network ni oti na global average pooling. Na fully connected network oqo e vakayagataka na Sigmoid function me vaka na final layer. Na function oqo e normalize (vakadodonutaka) na output ena kedrau maliwa na 0 kei na 1. Na iwalewale oqo e yield (solia) kina e dua na coefficient, ka volai tiko me α. Eda na rawa ni vakaraitaka na final threshold me α × A. O koya gona, na threshold e dua na kena product (saumi) ni rua na naba. E dua na naba e tiko ena kedrau maliwa na 0 kei na 1. Ka dua tale na naba oya na average (wili taucoko) ni absolute values ni feature map. Na method oqo e guarantee (vakadeitaka) ni na threshold e positive. Na method oqo e guarantee talega ni na threshold e sega ni na levu sara.

Talei, na samples duidui e result (vakavuna) na thresholds duidui. O koya gona, eda na rawa ni kila na method oqo me vaka ga e dua na specialized attention mechanism. Na mechanism oqo e identify (kunea) na features e sega ni veisemati kei na current task. Na mechanism oqo e transform (veikauyaka) na features oqo me volekata na dina ni zero ena vuku ni rua na convolutional layers. Qai oti, na mechanism e biuta na features oqo ki na zero ena vakayagataki ni soft thresholding. Se, na mechanism e identify (kunea) na features e veisemati kei na current task. Na mechanism oqo e transform (veikauyaka) na features oqo me yawa mai na dina ni zero ena vuku ni rua na convolutional layers. Ena kena iotioti, na mechanism e preserve (maroroya) na features oqo.

Ena kena iotioti, eda na stack (vakalewalewa) e dua na certain number (wili dina) ni basic modules. Eda sa include (biuta) talega kina na convolutional layers, batch normalization, activation functions, global average pooling, kei na fully connected output layers. Na iwalewale oqo e construct (tara) kina na complete Deep Residual Shrinkage Network (Deep Residual Shrinkage Network taucoko).

5. Na Generalization Capability (Kena Rawa ni Vakayagataki Vakalevu)

Na Deep Residual Shrinkage Network e dua na general method (sala taucoko) me baleta na feature learning. Na vuna oya ni na vuqa na feature learning tasks, na samples e dau tiko kina na noise. Na samples e tiko talega kina na irrelevant information. Na noise kei na irrelevant information oqo e rawa ni affect (vakaleqa) na performance ni feature learning. Me kena ivakaraitaki:

Nanuma mada na image classification. Na image e rawa ni tiko vata kina na vuqa tale na objects (iyaya). Eda na rawa ni kila na objects oqo me vaka ga na “noise”. Na Deep Residual Shrinkage Network e rawa ni vakayagataka na attention mechanism. Na network e raica na “noise” oqo. Qai oti, na network e vakayagataka na soft thresholding, me biuta kina na features e veisemati kei na “noise” oqo ki na zero. Na cakacaka oqo e rawa ni improve (vakavinakataka) na image classification accuracy.

Nanuma mada na speech recognition. Na kena dina, nanuma mada na relatively noisy environments (vanua e voqa vakalevu) me vaka na conversational settings (veivosaki) ena bati ni gaunisala se ena loma ni factory workshop (rumu ni cakacaka). Na Deep Residual Shrinkage Network e rawa ni improve (vakavinakataka) na speech recognition accuracy. Se, me lailai sobu, na network e solia e dua na methodology. Na methodology oqo e rawa ni improve (vakavinakataka) na speech recognition accuracy.

6. Academic Impact (Kena Bibi Vuli)

Na paper oqo e sa rawata oti e over (sivia) na 1,400 citations ena Google Scholar.

Vakatau ena incomplete statistics, era sa apply (vakayagataka) na Deep Residual Shrinkage Network (DRSN) na researchers ena over na 1,000 publications/studies. Na applications oqo e oka kina na wide range of fields. Na fields oqo e wili kina na mechanical engineering, electrical power, vision, healthcare, speech, text, radar, kei na remote sensing.

References (Vola Vakamacala)

Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Michael Pecht, Deep residual shrinkage networks for fault diagnosis, IEEE Transactions on Industrial Informatics, 2020, 16(7): 4681-4690.

https://ieeexplore.ieee.org/document/8850096

BibTeX

@article{Zhao2020,
  author    = {Minghang Zhao and Shisheng Zhong and Xuyun Fu and Baoping Tang and Michael Pecht},
  title     = {Deep Residual Shrinkage Networks for Fault Diagnosis},
  journal   = {IEEE Transactions on Industrial Informatics},
  year      = {2020},
  volume    = {16},
  number    = {7},
  pages     = {4681-4690},
  doi       = {10.1109/TII.2019.2943898}
}