Deep Residual Shrinkage Network punika wantah varian utawi soroh sane sampun katincapang saking Deep Residual Network. Sejatine, Deep Residual Shrinkage Network ngintegrasiang Deep Residual Network, attention mechanisms, miwah soft thresholding functions.
Iraga presida ngresepang prinsip kerja saking Deep Residual Shrinkage Network antuk cara puniki. Kapertama, network punika nganggen attention mechanisms anggen narka fitur-fitur sane nenten mabuat (unimportant features). Raris, network punika nganggen soft thresholding functions anggen ngubah fitur-fitur sane nenten mabuat punika dados nol. Tungkalikanne, network punika uning encen fitur sane mabuat tur nglestariang fitur-fitur punika. Proses puniki nincapang kemampuan saking deep neural network. Proses puniki ngwantu network ngerereh fitur sane maguna (useful features) saking sinyal sane madaging noise.
1. Motivasi Panelitian (Research Motivation)
Kapertama, noise wantah wantah parindikan sane nenten mrasidayang kakelidin rikala algoritma ngelompokang utawi klasifikasi sampel. Conto saking noise puniki minakadi Gaussian noise, pink noise, miwah Laplacian noise. Sane luwih luas, sampel sering madaging informasi sane nenten wenten hubunganne ring tugas klasifikasi sane sedek mamargi. Iraga presida ngresepang informasi sane nenten mapaiketan puniki pinaka noise. Noise puniki presida ngawinang asil klasifikasi dados kirang becik. (Soft thresholding wantah langkah utama ring akeh algoritma signal denoising).
Conto-ne, bayangang wenten pabligbagan utawi conversation ring pinggir margi. Ring suara punika, pastika wenten suara klakson mobil miwah suara ban montor. Iraga minab jagi ngamargiang speech recognition ring sinyal punika. Suara latar utawi background sounds punika pastika pacang ngulgul asilne. Saking kacamata deep learning, deep neural network patut ngilangang fitur-fitur sane mawit saking klakson miwah ban punika. Pangilangan fitur puniki matetujon mangda nenten ngusak asil speech recognition.
Sane kaping kalih, akeh noise sering matiosan ring soang-soang sampel. Parindikan puniki mamargi yadiastun ring dataset sane pateh. (Variasi puniki madue pakaikan sareng attention mechanisms. Ambil conto dataset gambar. Genah objek target minab matiosan ring soang-soang gambar. Attention mechanisms presida fokus ring genah objek target sane spesifik ring soang-soang gambar).
Pinaka conto, bayangang iraga ngelatih classifier meong-kuluk (cat-and-dog classifier) antuk limang gambar sane malabel “kuluk” (dog). Gambar 1 minab madaging kuluk miwah bikul (mouse). Gambar 2 minab madaging kuluk miwah angsa (goose). Gambar 3 minab madaging kuluk miwah siap (chicken). Gambar 4 minab madaging kuluk miwah keledai (donkey). Gambar 5 minab madaging kuluk miwah bebek (duck). Rikala proses training, objek-objek sane nenten mapaiketan puniki pacang ngulgul classifier punika. Objek-objek puniki minakadi bikul, angsa, siap, keledai, miwah bebek. Gangguan puniki ngawinang akurasi klasifikasi dados tuun. Yening iraga presida narka objek-objek sane nenten mabuat puniki, raris iraga presida ngilangang fitur-fitur sane mapaiketan ring objek punika. Antuk cara puniki, iraga presida nincapang akurasi saking cat-and-dog classifier.
2. Indik Soft Thresholding (Soft Thresholding)
Soft thresholding wantah langkah inti ring akeh algoritma signal denoising. Algoritma punika ngilangang fitur yening absolute values saking fitur punika alitan ring threshold (wates) tinentu. Algoritma punika “nyutiang” (shrinks) fitur nuju nol yening absolute values saking fitur punika agengan ring threshold punika. Para peneliti presida nerapang soft thresholding nganggen rumus ring sor puniki:
\[y = \begin{cases} x - \tau & x > \tau \\ 0 & -\tau \le x \le \tau \\ x + \tau & x < -\tau \end{cases}\]Turunan (derivative) saking output soft thresholding marep ring input-ne wantah:
\[\frac{\partial y}{\partial x} = \begin{cases} 1 & x > \tau \\ 0 & -\tau \le x \le \tau \\ 1 & x < -\tau \end{cases}\]Rumus ring ajeng nyihnayang indik turunan saking soft thresholding wantah 1 utawi 0. Sifat puniki pateh sareng sifat saking ReLU activation function. Punika mawinan, soft thresholding presida ngirangin risiko gradient vanishing miwah gradient exploding ring algoritma deep learning.
Ring fungsi soft thresholding, panentuan threshold patut nagingin kalih syarat. Kapertama, threshold patut marupa angka positif. Kaping kalih, threshold nenten dados agengan ring nilai maksimum saking sinyal input. Yening nenten, output-ne pacang dados nol makasami.
Sane lianan, threshold punika becikne taler nagingin syarat kaping tiga. Soang-soang sampel patut madue threshold sane mandiri (independent) manut ring akeh noise ring sampel punika.
Alasanne wantah krana akeh noise sering matiosan ring pantaraning sampel. Conto-ne, Sampel A minab madaging noise kidikan, nanging Sampel B madaging noise akehan ring dataset sane pateh. Ring kasus puniki, Sampel A patut nganggen threshold sane alitan rikala soft thresholding. Sampel B patut nganggen threshold sane agengan. Ring deep neural networks, fitur miwah threshold puniki mula kelangan definisi fisik sane pasti. Sakewanten, logika dasar ring ungkurne tetep pateh. Tegesne, soang-soang sampel patut madue threshold sane independen. Akeh noise sane spesifik punika sane nentuang threshold niki.
3. Indik Attention Mechanism (Attention Mechanism)
Para peneliti presida aluh ngresepang attention mechanisms ring bidang computer vision. Sistem panyingakan buron (visual systems of animals) presida mabinayang target antuk ny-scan makasami area antuk gelis. Raris, sistem panyingakan punika fokus ring objek target. Aksi puniki ngawinang sistem presida ngambil detail sane akehan. Ring galah sane pateh, sistem punika neken informasi sane nenten mabuat. Indik detailne, durus cingak literatur ngeninin indik attention mechanisms.
Squeeze-and-Excitation Network (SENet) wantah silih tunggil metode deep learning sane anyar sane nganggen attention mechanisms. Ring sampel sane matiosan, feature channels sane matiosan madue kontribusi sane mabinayan ring tugas klasifikasi. SENet nganggen sub-network alit anggen ngamolihang a set of weights (apuak bobot). Raris, SENet ngalikan bobot puniki sareng fitur saking channel sane mapaiketan. Operasi puniki ngubah ageng-alit fitur ring soang-soang channel. Iraga presida nganggep proses puniki sakadi nerapang tingkat perhatian (attention) sane mabinayan ring feature channels sane matiosan.
Ring pendekatan puniki, soang-soang sampel madue a set of weights sane independen. Tegesne, bobot anggen kalih sampel sane encen ja, pastika mabinayan. Ring SENet, jalur spesifik anggen ngamolihang bobot punika wantah “Global Pooling → Fully Connected Layer → ReLU Function → Fully Connected Layer → Sigmoid Function.”
4. Soft Thresholding antuk Deep Attention Mechanism
Deep Residual Shrinkage Network nganggen struktur saking sub-network SENet. Network punika nganggen struktur niki anggen nerapang soft thresholding ring sor deep attention mechanism. Sub-network punika (sane kacihnayang antuk kotak barak) malajahin a set of thresholds. Raris, network punika nerapang soft thresholding ring soang-soang feature channel nganggen thresholds puniki.
Ring sub-network puniki, sistem kapertama ngitung absolute values saking makasami fitur ring input feature map. Raris, sistem ngamargiang global average pooling miwah averaging anggen ngamolihang asiki fitur, sane kasimbolang pinaka A. Ring jalur sane lianan, sistem nglebengang feature map nuju fully connected network sane alit sasampun global average pooling. Fully connected network puniki nganggen Sigmoid function pinaka layer pinih ungkur. Fungsi puniki ng-normalisasi output ring pantaraning 0 miwah 1. Proses puniki ngasilang koefisien, sane kasimbolang pinaka α. Iraga presida nyurat threshold pinih ungkur pinaka α × A. Punika mawinan, threshold wantah asil kali saking kalih angka. Angka siki ring pantaraning 0 miwah 1. Angka sane lianan wantah rata-rata saking absolute values saking feature map. Metode puniki nyamin mangda threshold punika positif. Metode puniki taler nyamin mangda threshold punika nenten bes ageng.
Malih lianan, sampel sane matiosan ngasilang thresholds sane matiosan. Punika mawinan, iraga presida ngartosang metode puniki pinaka attention mechanism sane khusus. Mekanisme puniki narka fitur sane nenten mapaiketan ring tugas sane sedek mamargi. Mekanisme puniki ngubah fitur-fitur punika dados nilai sane nampek ring nol melarapan antuk kalih convolutional layers. Raris, mekanisme puniki netapang fitur-fitur niki dados nol nganggen soft thresholding. Utawi, mekanisme puniki narka fitur sane mapaiketan ring tugas sane sedek mamargi. Mekanisme puniki ngubah fitur-fitur niki dados nilai sane joh saking nol melarapan antuk kalih convolutional layers. Pinih ungkur, mekanisme puniki nglestariang fitur-fitur punika.
Pinih untat, iraga numpuk (stack) makudang-kudang basic modules. Iraga taler nagingin convolutional layers, batch normalization, activation functions, global average pooling, miwah fully connected output layers. Proses puniki ngawangun Deep Residual Shrinkage Network sane jangkep.
5. Kaprasidaan Generalisasi (Generalization Capability)
Deep Residual Shrinkage Network wantah metode umum anggen feature learning. Alasanne wantah, ring akeh tugas feature learning, sampel-sampel sering madaging noise. Sampel taler madaging informasi sane nenten relevan. Noise miwah informasi nenten relevan puniki presida ngulgul performa feature learning. Conto-ne:
Ring klasifikasi gambar (image classification). Asiki gambar minab madaging akeh objek lianan ring galah sane pateh. Iraga presida ngresepang objek-objek puniki pinaka “noise”. Deep Residual Shrinkage Network minab mrasidayang ngawigunayang attention mechanism. Network punika “eling” ring “noise” niki. Raris, network punika nganggen soft thresholding anggen netapang fitur-fitur sane mapaiketan ring “noise” niki dados nol. Aksi puniki madue potensi nincapang akurasi klasifikasi gambar.
Ring speech recognition. Khususne ring lingkungan sane ma-noise sakadi pabligbagan ring pinggir margi utawi ring tengahing bengkel pabrik. Deep Residual Shrinkage Network minab presida nincapang akurasi speech recognition. Utawi saking kidikne, network punika nyediayang metodologi. Metodologi puniki mrasidayang nincapang akurasi speech recognition.
Pustaka (Reference)
Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Michael Pecht, Deep residual shrinkage networks for fault diagnosis, IEEE Transactions on Industrial Informatics, 2020, 16(7): 4681-4690.
https://ieeexplore.ieee.org/document/8850096
BibTeX
@article{Zhao2020,
author = {Minghang Zhao and Shisheng Zhong and Xuyun Fu and Baoping Tang and Michael Pecht},
title = {Deep Residual Shrinkage Networks for Fault Diagnosis},
journal = {IEEE Transactions on Industrial Informatics},
year = {2020},
volume = {16},
number = {7},
pages = {4681-4690},
doi = {10.1109/TII.2019.2943898}
}
Dampak Akademis (Academic Impact)
Makalah puniki sampun polih langkungan ring 1,400 sitasi ring Google Scholar.
Manut statistik sane nenten jangkep, para peneliti sampun nerapang Deep Residual Shrinkage Network (DRSN) ring langkungan ring 1,000 publikasi/studi. Penerapan puniki ngjangkepang bidang sane luas pisan. Bidang-bidang puniki minakadi teknik mesin (mechanical engineering), kelistrikan (electrical power), vision, kesehatan (healthcare), speech, teks, radar, miwah remote sensing.