Deep Residual Shrinkage Network paneka varian se ampon e-improve dhari Deep Residual Network. Satemmene, Deep Residual Shrinkage Network a-integrasi aghi Deep Residual Network, attention mechanisms, ban soft thresholding functions.
Kita bisa mahami prinsip kerjana Deep Residual Shrinkage Network kalaban cara e baba paneka. Kaping settong, network ngangghuy attention mechanisms kaangguy a-identifikasi features se ta’ penting. Terros, network ngangghuy soft thresholding functions kaangguy madaddi nol features se ta’ penting gapaneka. Se sabaliggha, network nemmo features se penting ban nyimpen features gapaneka. Proses paneka makuwat kamampuanna deep neural network. Proses paneka abanto network kaangguy ngala’ features se aghuna dhari sinyal se ngandung noise.
1. Motivasi Riset
Se kaping settong, noise paneka ta’ bisa e-hindari nalika algoritma akalako klasifikasi samples. Conto dhari noise paneka termasok Gaussian noise, pink noise, ban Laplacian noise. Lebbi luas pole, samples serreng ngandung informasi se ta’ relevan kalaban tugas klasifikasi sataya. Kita bisa ngarte’e informasi se ta’ relevan paneka minangka noise. Noise paneka bisa matoron performa klasifikasi. (Soft thresholding paneka langkah kunci e dalem bannya’ algoritma signal denoising.)
Contona, coba bayangaghi obrolan e pinggir jalan. Audio paneka bisa ngandung sowara klakson motor ban roda. Kita mungkin alako speech recognition e attas sinyal paneka. Sowara latar blakang (background) paneka pasti a-efek ka hasilla. Dhari sudut pandang deep learning, deep neural network kodhu ma-ilang features se ahubungan kalaban klakson ban roda gapaneka. Panyingkiran paneka anyeggha features gapaneka a-pengaruh ka hasil speech recognition.
Se kaping duwa’, jumlah noise biasana bida antarana samples. Parbidaan paneka kadaddian maske e dalem dataset se padha. (Parbidaan paneka andhi’ kamiripan kalaban attention mechanisms. Ngala’ conto dataset gambar. Lokasi objek target bisa bida e antarana gambar. Attention mechanisms bisa fokus ka lokasi spesifik dhari objek target e tiap gambar.)
Contona, coba bayangaghi alatih classifier koceng-ban-pate’ kalaban lema’ gambar se a-label “pate’”. Gambar 1 mungkin ngandung pate’ ban tikus. Gambar 2 mungkin ngandung pate’ ban angsa. Gambar 3 mungkin ngandung pate’ ban ajam. Gambar 4 mungkin ngandung pate’ ban keledai. Gambar 5 mungkin ngandung pate’ ban bebek. E dalem proses training, objek se ta’ relevan bakal aganggu classifier. Objek paneka termasok tikus, angsa, ajam, keledai, ban bebek. Gangguan paneka anyabab aghi akurasi klasifikasi toron. Manabi kita bisa a-identifikasi objek se ta’ relevan gapaneka. Terros, kita bisa ma-ilang features se sesuai kalaban objek gapaneka. Kalaban cara paneka, kita bisa ningkat aghi akurasi dhari classifier koceng-ban-pate’.
2. Soft Thresholding
Soft thresholding paneka langkah inti e dalem bannya’ algoritma signal denoising. Algoritma paneka ma-ilang features manabi nèlay absolut dhari features gapaneka korang dhari threshold tartanto. Algoritma paneka “a-shrink” (manyosot) features ka ara nol manabi nèlay absolut dhari features lebbi rajha dhari threshold paneka. Para peneliti bisa a-implementasi aghi soft thresholding ngangghuy rumus e baba paneka:
\[y = \begin{cases} x - \tau & x > \tau \\ 0 & -\tau \le x \le \tau \\ x + \tau & x < -\tau \end{cases}\]Turunan dhari output soft thresholding terhadap input paneka:
\[\frac{\partial y}{\partial x} = \begin{cases} 1 & x > \tau \\ 0 & -\tau \le x \le \tau \\ 1 & x < -\tau \end{cases}\]Rumus e attas nunjuk aghi ja’ turunan dhari soft thresholding paneka 1 otaba 0. Sifat paneka padha kalaban sifat dhari ReLU activation function. Mangkana, soft thresholding bisa ngorangi risiko gradient vanishing ban gradient exploding e dalem algoritma deep learning.
E dalem soft thresholding function, settigan threshold kodhu amenuwi duwa’ sarat. Kaping settong, threshold kodhu angka positif. Kaping duwa’, threshold ta’ olle lebbi rajha dhari nèlay maksimum sinyal input. Manabi ta’, output-na bakal daddi nol sadaja.
Tambaan pole, threshold saena amenuwi sarat kaping tello’. Tiap sample kodhu andhi’ threshold mandiri adhasar aghi kandungan noise e dalem sample gapaneka.
Alasanna, polana kandungan noise serreng bida e antarana samples. Contona, e dalem dataset se padha, Sample A mungkin ngandung noise se sakone’, manabi Sample B ngandung noise se bannya’. E dalem kadaddian paneka, nalika alako soft thresholding, Sample A saena ngangghuy threshold se lebbi kene’. Sample B saena ngangghuy threshold se lebbi rajha. E dalem deep neural networks, maske features ban thresholds paneka ka-ilangan definisi fisik se teggas. Namong, logika dhasarra tetep padha. Artina, tiap sample kodhu andhi’ threshold se independen. Kandungan noise se spesifik nanto aghi threshold gapaneka.
3. Attention Mechanism
Para peneliti bisa kalaban ghampang mahami attention mechanisms e bidang computer vision. Sistem pangliatan hewan bisa abida aghi target kalaban akilap nyacan (scan) sadaja area. Saterrossa, sistem pangliatan fokus attention (perhatian) ka objek target. Aksi paneka a-izin aghi sistem kaangguy ngala’ detail se lebbi bannya’. E bakto se padha, sistem a-tekkenn informasi se ta’ relevan. Kaangguy detail spesifik, manabi bule, pareksane literatur mangenayi attention mechanisms.
Squeeze-and-Excitation Network (SENet) arupaghi metode deep learning se anyar se ngangghuy attention mechanisms. E antarana samples se bida, feature channels se bida aberri’ kontribusi se bida ka tugas klasifikasi. SENet ngangghuy sub-network kene’ kaangguy olle sakampol weights (Learn a set of weights). Terros, SENet akali aghi weights paneka kalaban features dhari channels se bersangkutan. Operasi paneka a-nyesuwai aghi rajhana features e tiap channel. Kita bisa ngangghuy pandangan ja’ proses paneka nerap aghi tingkatan attention se bida ka feature channels se bida (Apply weighting to each feature channel).
E dalem cara paneka, tiap sample andhi’ sakampol weights se independen. Kalaban kata laen, weights kaangguy duwa’ samples se manabi saos paneka bida. E dalem SENet, jalur spesifik kaangguy olle weights paneka “Global Pooling → Fully Connected Layer → ReLU Function → Fully Connected Layer → Sigmoid Function.”
4. Soft Thresholding kalaban Deep Attention Mechanism
Deep Residual Shrinkage Network ngangghuy struktur SENet sub-network. Network ngangghuy struktur paneka kaangguy alako soft thresholding e baba deep attention mechanism. Sub-network (se e-tunjuk aghi e dalem kotak mera) a-learn sakampol thresholds (Learn a set of thresholds). Terros, network nerap aghi soft thresholding ka tiap feature channel ngangghuy thresholds gapaneka.
E dalem sub-network paneka, sistem kaping settong a-itong nèlay absolut dhari sadaja features e dalem input feature map. Terros, sistem alako global average pooling ban averaging kaangguy olle settong feature, se e-tanda’i minangka A. E jalur laen, sistem amaso’ aghi feature map ka dalem fully connected network kene’ saamponna global average pooling. Fully connected network paneka ngangghuy Sigmoid function minangka lapisan (layer) akhir. Fungsi paneka a-normalize output antarana 0 ban 1. Proses paneka aghasil aghi koefisien, se e-tanda’i minangka α. Kita bisa anyata aghi threshold akhir minangka α × A. Mangkana, threshold paneka hasèl kali dhari duwa’ angka. Settong angka antarana 0 ban 1. Angka laenna paneka rata-rata dhari nèlay absolut feature map. Metode paneka ajamin ja’ threshold paneka positif. Metode paneka jugan ajamin threshold ta’ pati rajha.
Tambaan pole, samples se bida aghasil aghi thresholds se bida. Akibatta, kita bisa mahami metode paneka minangka attention mechanism se khusus. Mekanisme paneka a-identifikasi features se ta’ relevan kalaban tugas sataya. Mekanisme paneka a-transformasi features gapaneka daddi nèlay se semma’ ka nol liwat duwa’ convolutional layers. Terros, mekanisme paneka masetta features gapaneka daddi nol ngangghuy soft thresholding. Otaba, mekanisme paneka a-identifikasi features se relevan kalaban tugas sataya. Mekanisme paneka a-transformasi features gapaneka daddi nèlay se jau dhari nol liwat duwa’ convolutional layers. Akhirra, mekanisme paneka a-preserve (nyimpen) features gapaneka.
Akhirra, kita a-tompuk (stack) sajumlah basic modules (Stack many basic modules). Kita jugan amaso’ aghi convolutional layers, batch normalization, activation functions, global average pooling, ban fully connected output layers. Proses paneka abangun Deep Residual Shrinkage Network se lengkap.
5. Kamampuan Generalisasi
Deep Residual Shrinkage Network paneka metode umum kaangguy feature learning. Alasanna, polana samples serreng ngandung noise e dalem bannya’ tugas feature learning. Samples jugan ngandung informasi se ta’ relevan. Noise ban informasi se ta’ relevan paneka bisa a-pengaruh ka performa feature learning. Contona:
Coba pertimbang aghi image classification. Settong gambar bisa ngandung bannya’ objek laen e bakto se padha. Kita bisa mahami objek-objek paneka minangka “noise”. Deep Residual Shrinkage Network mungkin bisa manfa’at aghi attention mechanism. Network nyadari “noise” paneka. Terros, network ngangghuy soft thresholding kaangguy masetta features se sesuai kalaban “noise” paneka daddi nol. Aksi paneka andhi’ potensi ningkat aghi akurasi image classification.
Coba pertimbang aghi speech recognition. Khusussa, pertimbang aghi lingkungan se noisy, akadi seting obrolan e pinggir jalan otaba e dalem bengkel pabrik. Deep Residual Shrinkage Network mungkin bisa ningkat aghi akurasi speech recognition. Otaba satidha’na, network anawar aghi settong metodologi. Metodologi paneka mampu ningkat aghi akurasi speech recognition.
Referensi
Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Michael Pecht, Deep residual shrinkage networks for fault diagnosis, IEEE Transactions on Industrial Informatics, 2020, 16(7): 4681-4690.
https://ieeexplore.ieee.org/document/8850096
BibTeX
@article{Zhao2020,
author = {Minghang Zhao and Shisheng Zhong and Xuyun Fu and Baoping Tang and Michael Pecht},
title = {Deep Residual Shrinkage Networks for Fault Diagnosis},
journal = {IEEE Transactions on Industrial Informatics},
year = {2020},
volume = {16},
number = {7},
pages = {4681-4690},
doi = {10.1109/TII.2019.2943898}
}
Dampak Akademik
Kertas paneka ampon narema lebbi dhari 1,400 kutipan e Google Scholar.
Adhasar aghi statistik se ta’ lengkap, para peneliti ampon nerap aghi Deep Residual Shrinkage Network (DRSN) e dalem lebbi dhari 1,000 publikasi/studi. Aplikasi paneka ngalingkupi macem-macem bidang. Bidang paneka termasok teknik mesin, tenagha listrik, visi, kesehatan, sowara (speech), teks, radar, ban penginderaan jauh (remote sensing).