Deep Residual Shrinkage Network: Highly Noisy Data lae hik Artificial Intelligence Method

Deep Residual Shrinkage Network asal mein Deep Residual Network jo hik behtar variant ahey. Bunyadi tor te, he Deep Residual Network, attention mechanisms, ain soft thresholding functions jo integration ahey.

Hek hadd tain, Deep Residual Shrinkage Network je working principle khey heth dinal tareeqay saan samjhi saghje tho: he attention mechanisms khey use kandey unimportant features ji sunhann (identify) kare tho ain soft thresholding functions zariye unhan khey zero set kare tho; bi taraf, he important features khey identify kare tho ain unhan khey retain kare tho. He process deep neural network ji noise wari signals maan useful features extract karann ji salahiyat khey wadhaye tho.

1. Research Motivation

Pehryon, jadahn samples khey classify kyo wyo ahey, ta noise—jihan ta Gaussian noise, pink noise, ain Laplacian noise—jo hojjann lazmi ahey. Wadheek wasee paimany te, samples mein aksar ehri maloomat hundi ahey jehri current classification task lay irrelevant hundi ahey, un khey bi noise samjhi saghje tho. He noise classification performance te manfi asar (negative affect) vijhi saghey thi. (Soft thresholding ghanay signal denoising algorithms mein hik aham marhalo ahey.)

Misal tor te, sadak je kinary guftago doran, audio mein gaddiyun je horns ain pahiyn ja awaz mix thi saghan tha. Jadahn inhan signals te speech recognition kyo wyo ahey, ta nateejay lazmi tor te inhan background sounds khan mutasir thinda. Deep learning je nuqta-e-nazar (perspective) khan, horns ain wheels saan wabasta features khey deep neural network andar eliminate karann gurjey ta jian uhey speech recognition je nateejon khey mutasir na kan.

Byo he ta, sagye dataset mein bi, noise jo miqdar aksar sample dar sample mukhtalif hundo ahey. (He attention mechanisms saan mumasilat rakhy tho; image dataset jo misal wathann, ta target object ji location mukhtalif images mein mukhtalif thi saghey thi, ain attention mechanisms har image mein target object ji makhsoos location te focus kare saghan tha.)

Misal tor te, jadahn hik cat-and-dog classifier khey train kyo wyo ahey, panj images khey “dog” label kayo. Pehri image mein kutta ain kuan (mouse) huje, byi mein kutta ain hans (goose), teen mein kutta ain kukar (chicken), chothi mein kutta ain gadah (donkey), ain panjien mein kutta ain badak (duck) huje. Training doran, classifier lazmi tor te irrelevant objects jihan ta mice, geese, chickens, donkeys, ain ducks khan interference qabool kando, jenhin je nateejay mein classification accuracy ghatjee wehndi. Agar asan inhan irrelevant objects—mice, geese, chickens, donkeys, ain ducks—khey identify kare saghon ain unhan je corresponding features khey eliminate kayon, ta cat-and-dog classifier ji accuracy behtar karann mumkin ahey.

2. Soft Thresholding

Soft thresholding, ghanay signal denoising algorithms jo hik core step ahey. He unhan features khey eliminate kare tho jin ja absolute values hik makhsoos threshold khan ghat hunda ahin ain unhan features khey shrink kare tho jin ja absolute values hen threshold khan vadheek hunda ahin zero je taraf. In khey heth dinal formula zariye implement kare saghje tho:

Soft thresholding output jo input je lihaz khan derivative he ahey:

Jiyan ta mathay dekharyo wyo ahey, soft thresholding jo derivative ya 1 ahey ya 0. He property ReLU activation function wari ahey. Tanhnkare, soft thresholding deep learning algorithms mein gradient vanishing ain gradient exploding je khatray khey ghataye tho.

Soft thresholding function mein, threshold ji setting khey ba sharton (conditions) pooro karann gurjey: pehryon, threshold hik positive number huje; byo, threshold input signal je maximum value khan wadheek na huje, na ta output pooro zero thi wendo.

Wadheek he ta, threshold khey teeji shart poori karann behtar ahey: har sample jo hen je noise content je mutabiq penhjo azadana (independent) threshold hujann gurjey.

Chakaanta noise content aksar samples darmiyan mukhtalif hundo ahey. Misal tor te, sagye dataset mein aam ahey ta Sample A mein ghat noise huje jabkay Sample B mein wadheek noise huje. Hen soorat mein, jadahn denoising algorithm mein soft thresholding kyo wyo ahey, Sample A khey nandho threshold use karann gurjey, jabkay Sample B khey waddo threshold use karann gurjey. Agarcha he features ain thresholds deep neural networks mein penhiji wazeh physical definitions winyaye wehnda ahin, par bunyadi logic sagyi rahandi ahey. Lafzan mein, har sample jo penhjo independent threshold hujann gurjey jehro un je makhsoos noise content tay kyo wyo ahey.

3. Attention Mechanism

Attention mechanisms computer vision je maidan mein samajhann lay nisbatan asan ahin. Janwaran jo visual system poori area khey tezi saan scan kare targets khey distinguish kare saghey tho, bad mein target object te attention focus kare wadheek details extract kare tho ain irrelevant information khey suppress kare tho. Tafseelat lay, baraye meherbani attention mechanisms baray literature d’iso.

Squeeze-and-Excitation Network (SENet) hik nisbatan naya deep learning method ahey jehro attention mechanisms use kare tho. Mukhtalif samples darmiyan, classification task mein mukhtalif feature channels jo hisso (contribution) aksar mukhtalif hundo ahey. SENet hik nandho sub-network use kando ahey ta jian obtain a set of weights (weights jo set hasil karann) ain poe inhan weights khey respective channels je features saan multiply kare tho ta jian har channel mein features je magnitude khey adjust kare saghje. Hen process khey mukhtalif feature channels te mukhtalif levels ji attention apply karann tor d’isi saghje tho.

Hen tareeqay mein, har sample wat weights jo penhjo azadana set hundo ahey. Lafzan mein, kahin bi ba arbitrary samples ja weights mukhtalif hunda ahin. SENet mein, weights hasil karann jo makhsoos rasto he ahey “Global Pooling → Fully Connected Layer → ReLU Function → Fully Connected Layer → Sigmoid Function.”

4. Soft Thresholding with Deep Attention Mechanism

Deep Residual Shrinkage Network mathay dinal SENet sub-network structure khan inspiration wathi deep attention mechanism heth soft thresholding implement kare tho. Sub-network (jehko red box mein dekharyo wyo ahey) zariye, thresholds jo hik set learn kare saghje tho (Learn a set of thresholds) ta jian har feature channel te soft thresholding apply kare saghje.

Hen sub-network mein, input feature map je sabhni features ja absolute values pehryon calculate kya wenda ahin. Poe, global average pooling ain averaging zariye, hik feature hasil kyo wyo ahey, jenhin khey A zahir kyo wyo ahey. Bye rastey mein, global average pooling bad feature map khey hik nandhy fully connected network mein input kyo wyo ahey. He fully connected network penhiji final layer tor Sigmoid function use kare tho ta output khey 0 ain 1 darmiyan normalize kare saghje, jenhin saan hik coefficient miley tho jenhin khey α zahir kyo wyo ahey. Final threshold khey α × A tor express kare saghje tho. Tanhnkare, threshold hik 0 ain 1 darmiyan wary number ain feature map je absolute values je average jo product ahey. He tareeqo yaqeeni banaye tho ta threshold na sirf positive ahey par hadd khan wadheek waddo bi na ahey.

Wadheek he ta, mukhtalif samples ja nateejay mein mukhtalif thresholds milan tha. Nateejan, hek hadd tain, in khey hik specialized attention mechanism tor samjhi saghje tho: he current task lay irrelevant features khey identify kare tho, unhan khey ba convolutional layers zariye zero je qareeb values mein transform kare tho, ain soft thresholding use kandey unhan khey zero set kare tho; ya, he current task lay relevant features khey identify kare tho, unhan khey ba convolutional layers zariye zero khan passy (far) values mein transform kare tho, ain unhan khey preserve kare tho.

Aakhir mein, makhsoos tadaad mein basic modules (Stack many basic modules) khey convolutional layers, batch normalization, activation functions, global average pooling, ain fully connected output layers samait stack karann saan, mukammal Deep Residual Shrinkage Network tameer (construct) thi wye tho.

5. Generalization Capability

Deep Residual Shrinkage Network darhaqeeqat hik aam feature learning method ahey. Chakaanta, ghanay feature learning tasks mein, samples mein ghat ya wadh koch noise ain irrelevant information hundi ahey. He noise ain irrelevant information feature learning performance khey mutasir kare saghey thi. Misal tor te:

Image classification mein, agar hik image mein hik waqt ghanay bya objects hujen, ta inhan objects khey “noise” samjhi saghje tho. Deep Residual Shrinkage Network shayad attention mechanism use kare hen “noise” khey notice kare saghey, ain poe soft thresholding use kare hen “noise” san wabasta features khey zero set kare, jenhin saan image classification accuracy behtar thi saghey thi.

Speech recognition mein, khaas tor te nisbatan noisy mahol mein jihan ta sadak kinary guftago ya factory workshop andar, Deep Residual Shrinkage Network shayad speech recognition accuracy improve kare saghey, ya kam az kam, hik ehro methodology pesh kare tho jehko speech recognition accuracy improve karann ji salahiyat rakhey tho.

Academic Impact

@article{Zhao2020,
  author    = {Minghang Zhao and Shisheng Zhong and Xuyun Fu and Baoping Tang and Michael Pecht},
  title     = {Deep Residual Shrinkage Networks for Fault Diagnosis},
  journal   = {IEEE Transactions on Industrial Informatics},
  year      = {2020},
  volume    = {16},
  number    = {7},
  pages     = {4681-4690},
  doi       = {10.1109/TII.2019.2943898}
}