U kaven kwagh u Deep Residual Shrinkage Network: Ka gbenda u Artificial Intelligence sha ci u Data u a lu a ihyu kpishi

Deep Residual Shrinkage Network ne, ka kwagh u i sôr i seer sha Deep Residual Network yô. Jighilii yô, Deep Residual Shrinkage Network kohol kwagh u Deep Residual Network, attention mechanisms, man soft thresholding functions i zua imôngo.

Se fatyô u kaven er Deep Residual Shrinkage Network ne ka i er tom yô nahan. Hiihii yô, network ne ka a er tom a attention mechanisms u nengen a features (akaa) a a lu a inja ga la. Mba dondon yô, network la ka a er tom a soft thresholding functions u geman features a a lu a inja ga ne a hingir zero. Kpa, network ne ka a nenge features a a lu a inja yô, a kura a. Gbenda ne ka u wase deep neural network la u seer taver. Kwagh ne ka a wase network la u kuren features a a lu a inja ken signals (akaa a i lamen a mi) a a lu a noise (ihyu) yô.

1. Akaa a a ne ve i er Topsyase ne (Research Motivation)

Hiihii yô, zum u algorithm a lu va nan igbenda i classifying samples yô, noise ka kwagh u se fatyô u palegh ga yô. Ikyav i noise ne ka Gaussian noise, pink noise, man Laplacian noise. U seer kaven yô, samples ka i lu a “information” (kwaghôron) u a lu a inja sha ci u tom u classification la ga yô. Se fatyô u kaven “information” u a lu a inja ga ne er ka noise nahan. Noise ne una fatyô u nan classification la una er tom tsembelee ga. (Soft thresholding ka kwagh u vesen u i eren ken signal denoising algorithms kpishi yô.)

Ikyav i tesen yô, hen ase sha kwagh u ior ve lu lamen kpeghee sha akihir a gbenda yô. Audio (kwagh u ior ve lu lamen la) una fatyô u lun a amar a amato man ajiir a amato. Alaghga se soo u eren speech recognition sha signals mban. Akaa a a lu owon ken ijime la aa bunde iwasen i se zua a mi la. Sha nengen u deep learning yô, gba u deep neural network una ese features a a lu a amar a amato man ajiir a amato la kera. M-ese u esen akaa ne kera la una yange features ne u bundu speech recognition results la.

Sha uhar yô, iyenge i noise la ka i kaha ken samples kposo kposo. Ukwaghakahan mban ka ve lu ken dataset mom je kpaa. (Ukwaghakahan mban ka ve lu kwagh môm a attention mechanisms. Tôô ase image dataset er ka ikyav nahan. Ijiir i target object a lu ken images la ia fatyô u kahan. Attention mechanisms aa fatyô u veren ishima sha ijiir i target object la jighilii ken hanma image.)

Ikyav i tesen yô, tôô ase er se lu trainin cat-and-dog classifier a images utaan a i yer ve er “dog” (u) yô. Image 1 alaghga una lu a u man ibeenegh. Image 2 alaghga una lu a u man go. Image 3 alaghga una lu a u man ikegh. Image 4 alaghga una lu a u man jakaki. Image 5 alaghga una lu a u man adua. Zum u i lu trainin yô, akaa a a lu a inja ga la aa na classifier la ican. Akaa ne ka ibeenegh, go, ikegh, jakaki, man adua. Ican i akaa ne a ne yô, accuracy u classification la una yina. Aluer se fatyô u nengen a akaa a a lu a inja ga ne yô. Nahan yô, se fatyô u esen features a a lu a akaa ne kera. Sha gbenda ne yô, se fatyô u seer accuracy u cat-and-dog classifier la.

2. Soft Thresholding (Soft Thresholding)

Soft thresholding ne ka kwagh u vesen u i eren ken signal denoising algorithms kpishi yô. Aluer absolute values a features la a yina a threshold (kwan u i ver) yô, algorithm la ka a ese features shon kera. Aluer absolute values a features la a hemba threshold la yô, algorithm la ka a sôr features la ve yem ica a zero. Topsyase a fatyô u eren soft thresholding sha u dondon formula ne:

Derivative u soft thresholding output sha ci u input yô ka:

Formula u a lu sha ne tese er derivative u soft thresholding ka 1 shin 0. Kwagh ne ngu kwagh môm a ReLU activation function. Nahan yô, soft thresholding una fatyô u panden kwagh u bo u gradient vanishing man gradient exploding ken deep learning algorithms.

Ken soft thresholding function ne, gba u setting u threshold la una kure akaa ahar. Hiihii yô, threshold la a lu positive number (namba u a hemba zero). Sha uhar yô, threshold la a de hemban maximum value u input signal la ga. Aluer a hemba yô, output la cii una hingir zero.

Heela tseegh ga, doo u threshold la una kure kwagh u sha utar kpaa. Hanma sample yô, i lu a threshold u nan, a har sha iyenge i noise u a lu ken sample shon yô.

Itaki yô, iyenge i noise la ka i kaha ken samples kposo kposo. Ikyav i tesen yô, Sample A alaghga una lu a noise kpeghee, kpa Sample B una lu a noise kpishi ken dataset mom. Nahan yô, zum u i lu eren soft thresholding yô, gba u Sample A una er tom a threshold u kiriki. Sample B di yô, a er tom a threshold u vesen. Ken deep neural networks yô, features man thresholds mban ka ve kera lu a physical definitions (inja i sha aondo) jighilii ga. Kpa, logic (kwaghfan) u a lu ken ijime la ngu kwagh môm. Inja na yô, hanma sample yô, i lu a threshold u nan kposo. Ka iyenge i noise la jighilii ia tese threshold ne ye.

3. Attention Mechanism (Attention Mechanism)

Topsyase a fatyô u kaven attention mechanisms ken computer vision (gbenda u kômputa a nengen akaa) la zange. Akaa a uma a lun a ashe aa fatyô u paven targets sha u nengen ajiir la cii fele. Mba dondon yô, ashe la aa ver attention sha target object la. Kwagh ne ka a na systems la ian i kaven details (akaa a kiriki) a seer. Hen shighe mom la, systems la ka i kibe information u a lu a inja ga la. Sha ci u akaa a seer yô, ôr akaa a i nger sha kwagh u attention mechanisms la.

Squeeze-and-Excitation Network (SENet) ka deep learning method u he u a eren tom a attention mechanisms yô. Ken samples kposo kposo, feature channels kposo kposo ka ve wase classification task la sha igbenda kposo kposo. SENet ka a er tom a sub-network u kiriki u zuan a weights (ikyege). Nahan yô, SENet ka a multiplier weights mban a features a channels shon. Kwagh ne ka a gema vesen u features ken hanma channel. Se fatyô u nengen er process ne ka Apply weighting to each feature channel (u nan ikyege sha hanma feature channel) sha igbenda kposo kposo.

Sha gbenda ne yô, hanma sample ngi a set of weights u nan kposo. Inja na yô, weights a samples ahar cii ka a kaha. Ken SENet yô, gbenda u zuan a weights jighilii yô ka “Global Pooling → Fully Connected Layer → ReLU Function → Fully Connected Layer → Sigmoid Function.”

4. Soft Thresholding a Deep Attention Mechanism (Soft Thresholding with Deep Attention Mechanism)

Deep Residual Shrinkage Network ka a er tom a structure u SENet sub-network la. Network la ka a er tom a structure ne u eren soft thresholding sha deep attention mechanism. Sub-network ne (u i tese ken box u il ne) ka a Learn a set of thresholds. Nahan yô, network la ka a er tom a soft thresholding sha hanma feature channel sha u eren tom a thresholds mban.

Ken sub-network ne, hiihii yô, system la ka a calculate absolute values a features cii ken input feature map. Mba dondon yô, system la ka a er global average pooling man averaging u zuan a feature, i yila er A. Ken gbenda u gen la, system la ka a input feature map la ken fully connected network u kiriki sha gima u global average pooling. Fully connected network ne ka a er tom a Sigmoid function er layer u masetyô. Function ne ka a normalize output la hen atô u 0 man 1. Process ne ka a na coefficient, i yila er α. Se fatyô u tesen threshold u masetyô la er α × A. Nahan yô, threshold la ka product u namba ahar. Namba môm ngu hen atô u 0 man 1. Namba u gen la ka average u absolute values a feature map la. Gbenda ne ka u na threshold la a lu positive. Gbenda ne kpaa ka u na threshold la a ngee gande ga.

Heela tseegh ga, samples kposo kposo ka a na thresholds kposo kposo. Nahan yô, se fatyô u kaven method ne er ka attention mechanism u special yô. Mechanism ne ka a nenge a features a a lu a inja sha ci u task ne ga yô. Mechanism ne ka a gema features mban ve hingir values a a kpeghel 0 yô sha convolutional layers ahar. Nahan yô, mechanism la ka a ver features mban sha zero sha u eren tom a soft thresholding. Shin se fatyô u kaan ser, mechanism ne ka a nenge a features a a lu a inja sha ci u task ne yô. Mechanism ne ka a gema features mban ve hingir values a a lu ica a 0 yô sha convolutional layers ahar. Masetyô yô, mechanism la ka a kura features mban.

Ken m-kure yô, se Stack many basic modules (kohol modules a basic kpishi). Se kohol convolutional layers, batch normalization, activation functions, global average pooling, man fully connected output layers kpaa. Process ne ka a maa Deep Residual Shrinkage Network la cii.

5. Tahav mbu eren tom sha akaa kposo kposo (Generalization Capability)

Deep Residual Shrinkage Network ka general method sha ci u feature learning. Itaki yô, samples ka i lu a noise kpishi ken feature learning tasks kpishi. Samples kpaa ka i lu a information u a lu a inja ga yô. Noise man information u a lu a inja ga ne aa fatyô u yangen feature learning la u eren tom tsembelee. Ikyav i tesen:

Tôô ase kwagh u image classification. Image (foto) mom alaghga una lu a akaa agen kpishi ker. Se fatyô u kaven akaa ne er ka “noise” nahan. Deep Residual Shrinkage Network alaghga una fatyô u eren tom a attention mechanism. Network ne ka a nenge a “noise” ne. Nahan yô, network la ka a er tom a soft thresholding u veren features a a lu a “noise” ne sha zero. Kwagh ne una fatyô u seer image classification accuracy.

Tôô ase kwagh u speech recognition. Jighilii yô, tôô ase ajiir a noise a lu kpishi er ajiir a i lamen kpeghee sha akihir a gbenda shin ken factory workshop nahan. Deep Residual Shrinkage Network alaghga una seer speech recognition accuracy. Shin sha gima yô, network ne ka a na methodology (gbenda). Methodology ne ngu a tahav mbu seer speech recognition accuracy.

Reference

Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Michael Pecht, Deep residual shrinkage networks for fault diagnosis, IEEE Transactions on Industrial Informatics, 2020, 16(7): 4681-4690.

https://ieeexplore.ieee.org/document/8850096

BibTeX

@article{Zhao2020,
  author    = {Minghang Zhao and Shisheng Zhong and Xuyun Fu and Baoping Tang and Michael Pecht},
  title     = {Deep Residual Shrinkage Networks for Fault Diagnosis},
  journal   = {IEEE Transactions on Industrial Informatics},
  year      = {2020},
  volume    = {16},
  number    = {7},
  pages     = {4681-4690},
  doi       = {10.1109/TII.2019.2943898}
}

Kwagh u i fe method ne a mi (Academic Impact)

Paper ne zua a citations (i-kav) a hemba 1,400 sha Google Scholar.

A har sha statistics a i lu a mi yô, topsyase a er tom a Deep Residual Shrinkage Network (DRSN) ken publications/studies a hemba 1,000. Applications mban wa fields (akaa) kpishi ker. Fields mban wa mechanical engineering, electrical power, vision, healthcare, speech, text, radar, man remote sensing ker.