Deep Residual Shrinkage Network hem Deep Residual Network chem ek sudharit (improved) variant aasa. Mhanjech, Deep Residual Shrinkage Network madhe Deep Residual Network, attention mechanisms, ani soft thresholding functions hancho askar (integration) asta.
Aami Deep Residual Shrinkage Network koxem kaam karta, hem ashem samjun gheunk shakta. Saglyant poilim, hem network attention mechanisms vaprun binn-mahatvache (unimportant) features identify karta. Tea uprant, network soft thresholding functions vaprun hea binn-mahatvachea features-ank zero set karta. Tech bhaxen, hem network mahatvache (important) features identify karta ani tankam samballun dohorta. Haka lagun deep neural network chi ability vadhta. Hem process network-ak noise aslelea signals madhlyan useful features extract karpak modot karta.
1. Research Motivation (Research Motivation)
Poilim mholyar, jednam algorithm samples classify karta, tednam noise asop hem samkem sahajik aasa. Hea noise chya udharonam madhe Gaussian noise, pink noise, ani Laplacian noise hancho samavesh zata. Chodd vistaraan sangpachem zalear, samples madhe zaite pavtt current classification task khatir garjechi nasleli mahiti (irrelevant information) asta. Aami hea irrelevant information-ak noise mhonn samjun gheunk shakta. Ho noise classification performance kumi karunk shakta. (Soft thresholding hem zaitea signal denoising algorithms madhe ek mukhya step asta.)
Udharona khatir, rastyachea kadeak chol’leli gazaali (conversation) chintat. Audio madhe gaddiyanchea horns ani chakanchim (wheels) aavaz asunk shaktat. Aami hea signals-ancher speech recognition karunk shakta. Punn background sounds nishchitponnan result-ancher parinnam kartat. Deep learning chya nodrentlean (perspective), deep neural network-an horns ani wheels khaitr aslele features eliminate karpak zai. Hem elimination keleam mholyar he features speech recognition chya result-ancher parinnam karchena.
Dusrem mholyar, noise chem pramaan (amount) dar ek sample madhe veg-vegem asta. Hem variation ekach dataset madhe aslelea samples madhe legun disun yeta. (Hem variation attention mechanisms sarkem aasa. Ek image dataset chem udharon gheuya. Veg-veglea images madhe target object chem location veg-vegem asunk shakta. Attention mechanisms dar ek image madhe target object chya specific location-acher focus karunk shakta.)
Udharona khatir, aami ek cat-and-dog classifier train kartat ani aamchea kodden “dog” mhonn label keleleo paanch images aasat oxem somzuya. Image 1 madhe dog ani mouse asunk shakta. Image 2 madhe dog ani goose asunk shakta. Image 3 madhe dog ani chicken asunk shakta. Image 4 madhe dog ani donkey asunk shakta. Image 5 madhe dog ani duck asunk shakta. Training chya vellar, irrelevant objects classifier-ak disturb kartat. Hea objects madhe mice, geese, chickens, donkeys, ani ducks hancho samavesh zata. Hea interference-ak lagun classification accuracy kumi zata. Zari aami he irrelevant objects identify karunk shakle, tor aami hea objects-ank correspond korpi features eliminate karunk shaktat. Haka lagun, aami cat-and-dog classifier chi accuracy vadhoung shakta.
2. Soft Thresholding (Soft Thresholding)
Zaitea signal denoising algorithms madhe Soft thresholding hem ek core step aasa. Zari features che absolute values ek tharavik threshold poros kumi asat, tor algorithm te features eliminate karta. Ani zari features che absolute values tea threshold poros chodd asat, tor algorithm tea features-ank zero chya dishen shrink karta. Researchers sokoil dillea formula cho vapor korun soft thresholding implement karunk shaktat:
\[y = \begin{cases} x - \tau & x > \tau \\ 0 & -\tau \le x \le \tau \\ x + \tau & x < -\tau \end{cases}\]Input chya sandarbhan soft thresholding output chem derivative ashem aasa:
\[\frac{\partial y}{\partial x} = \begin{cases} 1 & x > \tau \\ 0 & -\tau \le x \le \tau \\ 1 & x < -\tau \end{cases}\]Voir dillo formula dakhoita ki soft thresholding chem derivative ek tor 1 aasa vo 0 aasa. Hi property ReLU activation function chya property sarki same aasa. Dekhun, soft thresholding deep learning algorithms madhe gradient vanishing ani gradient exploding cho risk kumi karunk shakta.
Soft thresholding function madhe, threshold set kartana don conditions satisfy korcheo poddtta. Poilim, threshold ek positive number aspak zai. Dusrem, threshold input signal chya maximum value poros vhodd asunk favona. Na zalear, output purai zero astolo.
Te bhair, threshold-an tisri condition satisfy keli zalear chodd borem. Dar ek sample khatir, tea sample chya noise content pramann, ek swatacho independent threshold aspak zai.
Hachem karann mholyar noise content dar ek sample madhe veg-vegem asta. Udharona khatir, ekach dataset madhe Sample A madhe kumi noise asunk shakta, punn Sample B madhe chodd noise asunk shakta. Haka lagun, soft thresholding kartana Sample A khatir lhan threshold vapropak zai. Ani Sample B khatir vhodd threshold vapropak zai. Deep neural networks madhe, jari hea features ani thresholds hancho explicit physical definitions nastyat, tori punn taanchem basic underlying logic tech urta. Sadhea utramni sangpachem zalear, dar ek sample khatir ek independent threshold aspak zai. Specific noise content ho threshold tharaita.
3. Attention Mechanism (Attention Mechanism)
Computer vision chya shetran attention mechanisms samjop researchers khatir sopem aasa. Prannianchi (Animals) visual systems purai area fast scan korun targets distinguish karunk shaktat. Tea uprant, visual systems target object-acher attention focus kartat. Hi action systems-ank chodd details extract karpak modot karta. Tech barobar, systems irrelevant information suppress kartat. Chodd mahiti khatir, tumi attention mechanisms-avishim aslelem literature vachunk shakta.
Squeeze-and-Excitation Network (SENet) hi attention mechanisms vaporpi ek navi deep learning method aasa. Veg-veglea samples madhe, veg-vegle feature channels classification task khatir veg-veglo contribution ditat. SENet weights cho ek set mellovpak ek lhan sub-network vapor’ta. Magir, SENet he weights tea tea channels chya features kodden multiply karta. Hem operation dar ek channel madlea features chem magnitude adjust karta. Aami hea process-ak veg-veglea feature channels-ancher veg-veglea level-achem attention divop, ashem manunk shakta.
Hea approach madhe, dar ek sample kodden weights cho ek independent set asta. Mhanjech, konnuy don veg-veglea samples che weights veg-vegle astat. SENet madhe, weights mellovpacho specific path aasa: “Global Pooling → Fully Connected Layer → ReLU Function → Fully Connected Layer → Sigmoid Function.”
4. Deep Attention Mechanism-a sohot Soft Thresholding (Soft Thresholding with Deep Attention Mechanism)
Deep Residual Shrinkage Network SENet sub-network chem structure vapor’ta. Hem network deep attention mechanism chya sokoil soft thresholding implement karpak hea structure cho vapor karta. Hem sub-network (jem red box madhe dakhoilam) Learn a set of thresholds karta (mhanjech thresholds cho set xikta). Uprant, network he thresholds vaprun dar ek feature channel-acher soft thresholding apply karta.
Hea sub-network madhe, system poilim input feature map madlea sogllea features che absolute values calculate karta. Uprant, system global average pooling ani averaging korun ek feature melloita, jeka $A$ mhonn denote kelam. Dusrea path-an, system global average pooling zalea uprant feature map ek lhan fully connected network madhe input karta. Hem fully connected network Sigmoid function-ak last layer koso vapor’ta. Hem function output-ak 0 ani 1 chya modem normalize karta. Hea process vorvim ek coefficient mellta, jeka $\alpha$ mhonn denote kelam. Aami final threshold $\alpha \times A$ asho express karunk shakta. Mhanjech, threshold ho don numbers cho product (gunnakaar) aasa. Ek number 0 ani 1 chya modem aasa. Ani dusro number feature map chya absolute values chem average aasa. Hi method ensure karta ki threshold nemham positive aasa. Tech barobar hi method threshold chodd vhodd aschena mhonn ensure karta.
Anik mholyar, veg-vegle samples veg-vegle thresholds toyar kartat. Dekhun, aami hea method-ak ek specialized attention mechanism mhonn samjun gheunk shakta. Hem mechanism current task-ak irrelevant aslele features identify karta. Hem mechanism hea features-ank don convolutional layers vaprun zero chya lagim aslelea values madhe transform karta. Uprant, hem mechanism soft thresholding vaprun hea features-ank zero set karta. Dusrea bhaxen sangpachem zalear, hem mechanism current task-ak relevant aslele features identify karta. Mechanism hea features-ank don convolutional layers vaprun zero pasun pois aslelea values madhe transform karta. Ani shevtteam, hem mechanism he features preserve karta.
Akherk, aami Stack many basic modules kartat (mhanjech zaite basic modules ekameka-ancher stack kartat). Aami convolutional layers, batch normalization, activation functions, global average pooling, ani fully connected output layers hancho-ui samavesh kartat. Hem process purai Deep Residual Shrinkage Network bandhta.
5. Generalization Capability (Generalization Capability)
Deep Residual Shrinkage Network hi feature learning khatir ek general method aasa. Kityak mholyar, zaitea feature learning tasks madhe samples-ant noise asta. Samples madhe irrelevant information legun asta. Ho noise ani irrelevant information feature learning chya performance-acher parinnam karunk shakta. Udharona khatir:
Image classification chintat. Ek image madhe ekach vellar zaiteo her vostu (objects) asunk shaktat. Aami hea objects-ank “noise” mhonn samjun gheunk shakta. Deep Residual Shrinkage Network attention mechanism cho vapor karpak shakto. Network hea “noise”-ak notice karta. Magir, network soft thresholding vaprun hea “noise”-ak correspond korpi features zero set karta. Hi action image classification chi accuracy vadhoung shakta.
Speech recognition chintat. Khas korun, rastyachea kadeak vo factory workshop madhe gazaali kartana, jhoim chodd noise asta. Deep Residual Shrinkage Network speech recognition chi accuracy vadhoung shakta. Vo kumi-t-kumi, hem network ek methodology offer karta. Hi methodology speech recognition accuracy vadhoupak saksham aasa.
References
Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Michael Pecht, Deep residual shrinkage networks for fault diagnosis, IEEE Transactions on Industrial Informatics, 2020, 16(7): 4681-4690.
https://ieeexplore.ieee.org/document/8850096
BibTeX
@article{Zhao2020,
author = {Minghang Zhao and Shisheng Zhong and Xuyun Fu and Baoping Tang and Michael Pecht},
title = {Deep Residual Shrinkage Networks for Fault Diagnosis},
journal = {IEEE Transactions on Industrial Informatics},
year = {2020},
volume = {16},
number = {7},
pages = {4681-4690},
doi = {10.1109/TII.2019.2943898}
}
Academic Impact (Academic Impact)
Google Scholar-acher hea paper-ak 1,400 poros chodd citations mell’leat.
Apurnn statistics pormannem, researchers-amni Deep Residual Shrinkage Network (DRSN) cho vapor 1,000 poros chodd publications/studies madhe kela. Hea applications madhe mechanical engineering, electrical power, vision, healthcare, speech, text, radar, ani remote sensing sarkea veg-veglea kshetrancho samavesh zata.