Tikugashirei kune inosimba nyika yekusimbisa kudzidza (RL), simba rinoshandura rinogadzirisa hungwaru hwekugadzira. RL inosiya nzira dzechinyakare dzekudzidza, ichipa nzira nyowani apo michina isingaite mabasa chete asi kudzidza kubva mukudyidzana kwega kwega. Rwendo urwu rwekusimbisa kudzidza rucharatidza magadzirirwo airi kuita mabenchmarks matsva mukugona kweAI kugadzirisa matambudziko akaomarara uye kugadzirisa matambudziko matsva, sevanhu.
Ungave uri mudzidzi, anofarira, kana nyanzvi, batana nesu parwendo urwu runonakidza kuburikidza nenyika yekusimudzira kudzidza, uko dambudziko rega rega riri mukana wekukura uye mikana yekuvandudza haina magumo.
Tsanangudzo yekusimbisa kudzidza
Reinforcement learning (RL) ibazi rine simba uye rine simba re machine learning iyo inodzidzisa michina kuita sarudzo kuburikidza nekudyidzana kwakananga nenzvimbo dzayo. Kusiyana nemaitiro echinyakare anovimba nemaseti makuru kana yakagadziriswa hurongwa, RL inoshanda pakuedza-uye-kukanganisa nzira yekudzidza. Iyi nzira inobvumira michina kudzidza kubva kune zvakabuda muzviito zvavo, ichipesvedzera zvakananga sarudzo dzinotevera uye kuratidzira maitiro echisikirwo ekudzidza akafanana neruzivo rwevanhu.
RL inozivikanwa kune akati wandei akakosha maficha anotsigira huwandu hwayo hwekushandisa:
- Autonomous kudzidza. Kusimbisa kudzidza vamiririri vanozvimirira nekufamba kwenguva nekuita sarudzo, kucherechedza mhedzisiro, uye kugadzirisa zvichienderana nekubudirira kana kutadza kwezviito zvavo. Kudzidzira uku kwakakosha pakuvandudza maitiro ehungwaru uye kunobvumira masisitimu eRL kubata mabasa anoda kuchinjika kwakakosha.
- Kushandiswa zvakasiyana-siyana. Kuchinjika kweRL kunoratidzwa mukati meakasiyana akaomesesa uye ane simba masisitimu, kubva kumotokari dzinozvimiririra dzinofambisa traffic kusvika kune yepamusoro-yekutamba-algorithms uye zvirongwa zvekurapa zvemunhu. Izvi zvakasiyana-siyana zvinosimbisa kushanda kweRL kwakafara muzvikamu zvakasiyana.
- Iterative kudzidza uye optimization. Pakati pepakati peRL ndeyekutenderera kutenderera kwekuyedza, kukanganisa, uye kunatsiridza. Iyi iterative process yakakosha kumaapplication ayo mamiriro anoramba achishanduka, sekufamba nekuchinja kwetraffic mapatani kana misika yemari.
- Kubatanidzwa nemhinduro dzevanhu (RLHF). Kuvandudza nzira dzechinyakare dzekusimbisana dzekudzidza, kubatanidzwa kwemhinduro dzevanhu — kunonzi RLHF — kunowedzera nzira yekudzidza nekuwedzera maonero evanhu. Izvi zvinoita kuti masisitimu apindure uye aenderane zviri nani nezvido zvevanhu, izvo zvinonyanya kukosha munzvimbo dzakaoma sekugadzirisa mutauro wechisikigo.
Iyi sumo inogadzirisa nhanho yekuongorora kwakadzama kwezvinhu zveRL uye maitiro, ayo achatsanangurwa muzvikamu zvinotevera. Inokupa iyo yakakosha kumashure inodiwa kuti unzwisise iyo yakakura-siyana pesvedzero uye kukosha kweRL mumaindasitiri akasiyana uye maapplication.
Zvinhu zvekusimbisa kudzidza
Kuvaka pamusoro pekunzwisisa kwedu kwekutanga, ngationgororei zvinhu zvakakosha zvinotsanangura kuti kudzidza kwekusimudzira kunoshanda sei munzvimbo dzakasiyana siyana. Kunzwisisa izvi zvikamu zvakakosha pakubata kuchinjika uye kuoma kweRL masisitimu:
- mhepo mvura nenzvimbo. Kurongeka uko mumiririri weRL anoshanda kunotangira kubva kudhijitari yekunyepedzera kutengesa masheya kusvika kumamiriro ezvinhu akafanana nekufambisa madrones.
- muiti. Muiti wesarudzo muchirongwa cheRL anodyidzana nenharaunda uye anoita sarudzo zvichibva pane yakaunganidzwa data nemhedzisiro.
- Action. Sarudzo chaidzo kana mafambiro anoitwa nemumiririri, ayo anokanganisa zvakananga mhedzisiro yekudzidza.
- mamiriro. Inomiririra mamiriro azvino kana mamiriro sekuonekwa kunoitwa nemumiririri. Inoshanduka zvine simba sezvo mumiriri anoita, achipa mamiriro ekutevera sarudzo.
- Reward. Mhinduro inopiwa mushure mekuita kwega kwega, ine mibairo yakanaka inokurudzira uye zvirango zvinoodza mwoyo mamwe maitiro.
- urongwa. Nzira kana seti yemitemo inotungamira sarudzo dzemumiririri zvichienderana nemamiriro azvino, inonatswa kuburikidza nekudzidza kunoenderera.
- ukoshi. Kufanotaura kwemubairo weramangwana kubva kudunhu rega rega, batsira mumiriri kuisa pamberi nyika kuti zvibatsirwe zvakanyanya.
Zvinhu zvezvakatipoteredza, mumiririri, chiito, nyika, mubairo, mutemo, uye kukosha hazvisi zvikamu zvehurongwa; vanogadzira chimiro chakabatana chinobvumira vamiririri veRL kudzidza uye kugadzirisa zvine simba. Uku kugona kuenderera mberi nekudzidza kubva mukudyidzana mukati menharaunda kunoisa fundo yekusimbisa kubva kune dzimwe nzira dzekudzidza muchina uye inoratidza kugona kwayo kukuru mukushandisa kwakasiyana. Kunzwisisa zvinhu izvi zvega kwakakosha, asi basa ravo rekubatana mukati meRL system rinoratidza simba rechokwadi uye kushanduka kweiyi tekinoroji.
Kuti uone zvinhu izvi mukuita, ngationgororei muenzaniso unoshanda mune maindasitiri marobhoti:
| • mhepo mvura nenzvimbo. Mutsetse wegungano unoshanda robhoti ruoko. • muiti. Ruoko rwerobhoti rwakagadzirirwa kuita mabasa chaiwo. • Action. Mafambiro akadai sekunhonga, kuisa, nekuunganidza zvikamu. • mamiriro. Nzvimbo iripo yeruoko uye mamiriro emutsetse wegungano. • Reward. Mhinduro pamusoro pekuita basa regungano nemazvo uye nemazvo. • urongwa. Nhungamiro inotungamira sarudzo dzemarobhoti kukwenenzvera kutevedzana kwegungano. • ukoshi. Ongororo yekuti ndeapi mafambiro anoburitsa zvinonyanya kushanda pagungano nekufamba kwenguva. |
Uyu muenzaniso unoratidza mashandisirwo anoitwa zvinhu zvekutanga zvekusimbisa kudzidza mumamiriro ezvinhu epasirese, zvichiratidza kugona kwerobhoti kudzidza uye kugadzirisa kuburikidza nekuenderera mberi kwekudyidzana nenzvimbo yaro. Zvishandiso zvakadaro zvinosimbisa kugona kwepamusoro kweRL masisitimu uye zvinopa maonero anoshanda pane dzidziso inokurukurwa. Sezvatinoenderera mberi, tichaongorora mamwe maapplication uye nekuzama zvakadzama mukati mekuoma uye kushandura kugona kwekusimbisa kudzidza, tichiratidza maitiro avo anoshanda uye shanduko yeRL mumamiriro epasirese chaiwo.
Kuongorora kushanda kwekusimbisa kudzidza
Kutenda zvizere kushanda kwekusimbisa kudzidza (RL) munzvimbo dzakasiyana siyana, zvakakosha kuti unzwisise mashandiro ayo mechanics. Pakati payo, RL inotenderera pakudzidza maitiro akanaka kuburikidza nekudyidzana kune simba kwezviito, mibairo, uye zvirango-kuumba izvo zvinozivikanwa seyekusimbisa kudzidza mhinduro loop.
Iyi nzira inosanganisira kutenderera kwezviito, mhinduro, uye kugadzirisa, zvichiita kuti ive nzira ine simba yekudzidzisa michina kuita mabasa zvakanyatsonaka. Heino nhanho-ne-nhanho kupatsanurwa kwemaitiro ekusimbisa kudzidza kunowanzo shanda:
- Tsanangura dambudziko. Ziva zvakajeka iro basa chairo kana kupikisa mumiririri weRL akagadzirirwa kugadzirisa.
- Gadzirisa zvakatipoteredza. Sarudza mamiriro achashanda nemumiririri, angave ari madhijitari akateedzerwa kana mamiriro epasirese chaiwo.
- Gadzira mumiririri. Gadzira mumiririri weRL ane masensa kuti anzwisise zvakaitenderedza uye kuita zviito.
- Tanga kudzidza. Bvumira mumiririri kuti adyidzane nenharaunda yayo, ichiita sarudzo dzinopesvedzerwa nehurongwa hwayo hwekutanga.
- Gamuchira mhinduro. Mushure mechiitiko chega chega, mumiririri anogamuchira mhinduro nenzira yemubairo kana zvirango, izvo zvaanoshandisa kudzidza uye kugadzirisa maitiro ayo.
- Gadzirisa mutemo. Ongorora mhinduro yekunatsa marongero emumiririri, nokudaro achivandudza kugona kwake kuita sarudzo.
- Tsanangurira. Ramba uchinatsiridza mashandiro emumiririri kuburikidza nekudzokorora kudzidza uye mhinduro.
- Shandisa. Kutevera kudzidziswa kwakakwana, shandisa mumiririri kuti abate mabasa epasirese chaiwo kana kushanda mukati memienzaniso yakaoma kunzwisisa.
Kuratidza mashandisirwo emaitiro aya matanho mukuita, funga nezvemuenzaniso weRL mumiririri akagadzirirwa kutonga traffic yemudhorobha:
| • Tsanangura dambudziko. Chinangwa ndechekukwirisa kuyerera kwetraffic pamharadzano yeguta kuti kuderedze nguva yekumirira uye kuwanda. • Gadzirisa zvakatipoteredza. Iyo RL system inoshanda mukati meiyo traffic control network yemharadzano, uchishandisa chaiyo-nguva data kubva kune traffic sensors. • Gadzira mumiririri. Iyo traffic control system pachayo, yakashongedzerwa nema sensors uye masigner controller, inoshanda semumiririri. • Tanga kudzidza. Mumiririri anotanga kugadzirisa nguva dzemwenje dzemotokari zvichibva pane chaiyo-nguva traffic mamiriro. • Gamuchira mhinduro. Mhinduro yakanaka inogamuchirwa yekudzikisa nguva yekumirira uye kuwanda, nepo mhinduro isina kunaka inoitika kana kunonoka kana kuvharika kwemigwagwa kuchiwedzera. • Gadzirisa mutemo. Mumiririri anoshandisa iyi mhinduro kunatsiridza maalgorithms ayo, achisarudza iyo inonyanya kushanda yechiratidzo nguva. • Tsanangurira. Iyo sisitimu inogara ichigadzirisa uye kudzidza kubva kune inoenderera data kuti ivandudze kushanda kwayo. • Shandisa. Kana yangoratidzwa inoshanda, sisitimu inoiswa zvachose kubata traffic pamharadzano. |
Zvakananga zvinhu zveRL system mune ino mamiriro:
| • mhepo mvura nenzvimbo. The traffic system yemharadzano yeguta. • muiti. Iyo traffic control system ine masensa uye masigner controller. • Action. Shanduko kunguva dzemwenje wemotokari uye masaini evanofamba netsoka. • mamiriro. Mamiriro azvino ekuyerera kwetraffic, kusanganisira kuverenga kwemotokari, kuwanda kwetraffic, uye nguva dzechiratidzo. • Reward. Mhinduro inobva pakushanda kwehurongwa mukuderedza nguva dzekumirira. • Pfungwa. Algorithms inokwirisa chiratidzo chenguva kuti iwedzere kuyerera kwetraffic. • ukoshi. Kufanotaura nezvemhedzisiro yeakasiyana-siyana ekugadzirisa nguva pane ramangwana traffic mamiriro. |
Iyi RL system inoramba ichigadzirisa marambi emigwagwa munguva chaiyo kuti iwedzere kuyerera uye kuderedza kutsvikinyidzana kunoenderana nemhinduro yenguva dzose kubva kunharaunda yayo. Zvishandiso zvakadaro hazvingoratidzi kushanda kweRL asi zvakare kuratidza kugona kwayo kuchinjika kune yakaoma uye inochinja mamiriro.
Kunzwisisa RL mukati meyakafara mamiriro ekudzidza muchina
Sezvo isu tichiongorora kuomarara kwekusimbisa kudzidza, zvinova zvakakosha kuti tisiyanise kubva kune dzimwe nzira dzemuchina wekudzidza kuti unyatso tenda zvakasarudzika mashandisiro uye matambudziko. Pazasi pane ongororo yekuenzanisa yeRL maererano nedzidzo inotariswa uye isina anotariswa. Kuenzanisa uku kunovandudzwa nemuenzaniso mutsva weRL's application mune smart grid manejimendi, iyo inosimbisa kugona kweRL uye inoburitsa matambudziko chaiwo ane chekuita neiyi nzira yekudzidza.
Kuenzanisa kuongororwa kwemichina yekudzidza nzira
| Kuonekwa | Inotarisirwa kudzidza | Kudzidza kusingatarisirwe | Yekusimbisa kudzidza |
| Rudzi rweData | Yakanyorwa data | Mashoko asina kunyorwa | Hapana dataset yakagadziriswa |
| Feedback | Yakananga uye pakarepo | hapana | Zvisina kunanga (mibayiro/zvirango) |
| Shandisa maitiro | Classification, regression | Kutsvaga data, kubatanidza | Dynamic kuita sarudzo nharaunda |
| unhu | Inodzidza kubva kune dataset ine mhinduro dzinozivikanwa, yakanakira mhedzisiro yakajeka uye yakananga dzidziso mamiriro. | Inotsvaga mapatani kana zvimiro zvakavanzwa pasina zvakafanotsanangurwa, zvakanakira kuongorora kuongorora kana kutsvaga mapoka edata. | Inodzidza kuburikidza nekuedza uye kukanganisa uchishandisa mhinduro kubva kuzviito, inokodzera nharaunda umo sarudzo dzinotungamira kune zvakasiyana mhedzisiro. |
| mienzaniso | Kuzivikanwa kwemufananidzo, kuona spam | Kupatsanurwa kwemusika, kuona kusinganzwisisike | Mutambo AI, mota dzinozvimiririra |
| matambudziko | Inoda makuru akanyorwa dataset; inogona kusagadzirisa zvakanaka kune isingaonekwe data. | Zvakaoma kuongorora maitiro emodhi pasina zvakanyorwa data. | Kugadzira chirongwa chemubairo chinoshanda chinonetsa; high computational kudiwa. |
Mufananidzo wekusimbisa kudzidza: Smart grid manejimendi
Kuratidza RL's application kupfuura iyo inowanzo kurukurwa yetraffic manejimendi masisitimu uye kuve nechokwadi akasiyana emienzaniso, funga yakangwara grid manejimendi system yakagadzirirwa kukwirisa kugovera simba uye kuderedza marara:
| • Dambudziko tsanangudzo. Vavarira kuwedzera simba remagetsi mukati meguta remagetsi uku uchideredza kudzimwa uye kuderedza kupera kwemagetsi. • Mamiriro ekugadzirisa. Iyo RL sisitimu inosanganiswa mune network ye smart metres uye simba ma routers, ayo anoramba achitarisa chaiyo-nguva mashandisiro esimba uye kugovera metrics. • Kugadzira agent. A smart grid controller, akadzidziswa nekugona mukufungidzira analytics uye akashongedzerwa kuita RL algorithms senge Q-kudzidza kana Monte Carlo nzira, inoshanda semumiririri. • Maitiro ekudzidza. Mumiririri ane simba anogadzirisa nzira dzekugovera simba zvichienderana nekufungidzira modhi yekuda uye kugovera. Semuyenzaniso, Q-kudzidza inogona kushandiswa kunatsa zvishoma nezvishoma nzira idzi kuburikidza nemubairo sisitimu inoongorora kugona kwekugovera kwemagetsi uye kugadzikana kwegridi. • Feedback reception. Mhinduro yakanaka inopiwa kune zviito zvinovandudza kugadzikana nekushanda kwegridi, nepo mhinduro isina kunaka ichigadzirisa kusashanda kana kutadza kwehurongwa, ichitungamira marongero emangwana emumiririri. • Kugadziridzwa kwesarudzo. Mumiririri anogadziridza marongero ayo zvichienderana nekubudirira kwezviito zvekare, kudzidza kutarisira kukanganisa kunogona kuitika uye kugadzirisa kugovera. • Kunatsiridza. Inoenderera mberi data ichipinda uye iterative mhinduro loops inogonesa sisitimu kuvandudza maitiro ayo ekushanda uye kufanotaura chokwadi. • Deployment. Mushure mekugadzirisa, sisitimu inoiswa kuti igadzirise kugovera simba kune akawanda magridi. |
Uyu muenzaniso unoratidza kuti kudzidza kwekusimbisa kunogona kushandiswa sei kune yakaoma masisitimu uko kwakakosha nguva-yekuita sarudzo uye kuchinjika. Inosimbisawo matambudziko akajairika mukusimbisa kudzidza, sekuomerwa kwekumisikidza mibairo inomiririra zvechokwadi zvinangwa zvenguva refu uye kubata yakakwirira computational zvinodiwa zvekuchinja nharaunda.
Hurukuro ye smart grid manejimendi inotitungamira mukutsvagisa nzira dzepamusoro dzekusimbisa kudzidza uye mashandisiro muzvikamu zvakasiyana sehutano, mari, uye kuzvitonga masisitimu. Nhaurirano idzi dzicharatidza zvakare kuti akagadziridzwa sei mazano eRL anogadzirisa matambudziko emumaindasitiri uye nyaya dzetsika dzavanosanganisa.
Kufambira mberi kwezvino mukusimbisa kudzidza
Sezvo kudzidza kwekusimbisa kuri kuramba kuchishanduka, kunosundidzira miganhu yehungwaru hwekugadzira nebudiriro huru yedzidziso uye inoshanda. Ichi chikamu chinoratidza izvi zvitsva zvitsva, zvichitarisa kune akasiyana maapplication anoratidza basa reRL riri kukura munzvimbo dzakasiyana siyana.
Kubatanidzwa nekudzidza kwakadzama
Kudzidza kwakadzama kwekusimbisa kunonatsiridza kugona kweRL kwekuita sarudzo kuburikidza nekuzivikanwa kwepateni yepamberi kubva pakudzidza kwakadzama. Kubatanidzwa uku kwakakosha kune zvikumbiro zvinoda kukurumidza uye kwakaomarara kuita sarudzo. Zvinotaridza kukosha zvakanyanya munzvimbo dzakaita senge yakazvimirira kufamba kwemotokari uye kuongororwa kwekurapa, uko chaiyo-nguva yekugadzirisa data uye kuita sarudzo kwakakosha kuchengetedzeka uye kushanda.
Kubudirira uye maapplication
Kubatana pakati pekudzidza kwekusimbisa uye kudzidza kwakadzama kwakonzera kubudirira kunoshamisa munzvimbo dzakasiyana siyana, kuratidza kugona kweRL kugadzirisa uye kudzidza kubva kune yakaoma data. Heano dzimwe nzvimbo dzakakosha idzo nzira iyi yakabatanidzwa yakaita zvakakosha, ichiratidza kugona kwayo uye kugona kushandura:
- Strategic mutambo uchitamba. DeepMind's AlphaGo muenzaniso wepamusoro wekuti kudzidza kwakadzama kwekusimbisa kunogona sei kukurira matambudziko akaomarara. Nekuongorora ruzivo rwakakura rwemutambo wemutambo, AlphaGo yakagadzira nzira itsva dzakazopfuura dzevamhanyi venyika dzevanhu, ichiratidza simba rekubatanidza RL nekudzidza kwakadzama mukufunga zvine hungwaru.
- Autonomous mota. Muindasitiri yemotokari, kudzidza kwakadzika kwekusimbisa kwakakosha pakuvandudza kuita sarudzo-chaiyo nguva. Mota dzakagadzirirwa neiyi tekinoroji dzinogona kufamba zvakachengeteka uye zvine mutsindo nekuchimbidza ipapo kuchinja mamiriro emigwagwa uye data yezvakatipoteredza. Kushandiswa kwekufungidzira analytics, kunopihwa simba nekudzidza kwakadzama, kunoratidza kufambira mberi kwakakosha mutekinoroji yemotokari, zvinotungamira kune yakachengeteka uye yakavimbika yakazvimirira kutyaira masisitimu.
- Robotics. Marobhoti ari kuwedzera kukwanisa kubata matambudziko matsva nekuda kwekubatanidzwa kwekusimbisa kudzidza nekudzidza kwakadzama. Kubatanidzwa uku kwakakosha muzvikamu zvakaita sekugadzira, uko kurongeka uye kuchinjika kwakakosha. Sezvo marobhoti achishanda munzvimbo dzakasimba dzemaindasitiri, anodzidza kukwenenzvera maitiro ekugadzira uye kuwedzera mashandiro ekushanda kuburikidza nekuenderera kuchinjika.
- Nezveutano. Iko kusanganiswa kweRL uye kudzidza kwakadzama kunoshandura kutarisirwa kwevarwere nekuita zvemunhu marapirwo ekurapa. Algorithms inoshandura zvirongwa zvekurapa zvichibva pakuenderera mberi kwekutarisa, kuwedzera huchokwadi uye kushanda kwekupindira kwekurapa. Iyi nzira yekuchinja inonyanya kukosha kune mamiriro anoda kuramba achigadziriswa kune marapirwo uye kufungidzira kwehutano hwehutano.
Zvinoreva uye tarisiro yeramangwana
Nekubatanidza kusimbaradza kudzidza nekudzidza kwakadzama, hungwaru, masisitimu anochinja anoshanduka achizvimirira, achinatsiridza kudyidzana kwemichina nenyika. Aya masisitimu ari kuwedzera kuita zvinoenderana nezvido zvevanhu uye shanduko yezvakatipoteredza, achiisa mitsva mitsva yekudyidzana tekinoroji.
Case zvidzidzo zvekusimbisa kudzidza muindasitiri
Kutevera kuongorora kwedu kufambira mberi kwakakosha mukusimbisa kudzidza, ngationgororei shanduko yayo muzvikamu zvakasiyana. Izvi zvidzidzo zvenyaya hazvingoratidzi kuchinjika kweRL chete asi zvakare inosimbisa basa rayo mukuvandudza kugona uye kugadzirisa matambudziko akaomarara:
- Mune zvemari, smart algorithms inoshandura mashandiro emusika nekuchinjira kune shanduko, nekudaro inosimudzira manejimendi enjodzi uye pundutso. Kutengeserana kwealgorithmic kwave kushandiswa kwakakosha, kushandisa kusimbisa kudzidza kuita kutengeserana panguva dzakanangana, kuwedzera kugona, uye kuderedza kukanganisa kwevanhu.
- Hutano hunobatsira zvakanyanya kubva kuRL, iyo inovandudza kuchengetwa kwemunhu nekugadzirisa zvine simba marapirwo anoenderana nemhinduro dzemurwere chaidzo. Iyi tekinoroji yakakosha mukugadzirisa mamiriro akaita seshuga uye mukufungidzira hutano hwehutano, uko inobatsira kutarisira nekudzivirira zvingangoitika zvehutano.
- Muindasitiri yemotokari, Kusimbisa kudzidza kunovandudza mashandiro anoita mota dzinozvityaira. Makambani akaita saTesla naWaymo anoshandisa tekinoroji iyi kuongorora data kubva kumasensa emotokari nekukurumidza, zvichibatsira mota kuita sarudzo dziri nani nezve kwekuenda uye nguva yekuita kugadzirisa. Izvi hazvingoite kuti mota dzichengeteke chete asiwo zvinobatsira kuti dzifambe zvakanaka.
- Muchikamu chevaraidzo, RL iri kugadziridza mutambo nekugadzira akangwara asiri-mutambi mavara (NPCs) anochinjika kukudyidzana kwevatambi. Pamusoro pezvo, inovandudza midhiya yekutepfenyura masevhisi nekugadzirisa zvemukati zvinorudziro, izvo zvinosimudzira kubatanidzwa kwevashandisi nekuenderana nezvido zvevaoni.
- Mukugadzira, Kusimbisa kudzidza kunokwidziridza mitsara yekugadzira uye mashandiro echeni yekugovera nekufanotaura kungangotadza kutadza kwemuchina uye kuronga kugadzirisa zvine hungwaru. Ichi chishandiso chinodzikisira nguva yekudzikira uye chinowedzera chigadzirwa, chichiratidza RL maitiro ekuita kwemaindasitiri.
- Energy maneja zvakare inoona kufambira mberi kuburikidza neRL, iyo inogonesa chaiyo-nguva simba rekushandisa mukati me smart grids. Nekufanotaura nekudzidza maitiro ekushandiswa, kusimbaradza kudzidza kunodzikamisa kudiwa uye kupa, kuvandudza kugona uye kusimba kwesimba masisitimu.
Iyi mienzaniso mumaindasitiri akasiyana-siyana inosimbisa kushanda kweRL kwakafara uye kugona kwayo kufambisa hunyanzvi hwetekinoroji, ichivimbisa kumwe kufambira mberi uye kutorwa kweindasitiri yakafara.
Kubatanidzwa kwekusimbisa kudzidza nemamwe matekinoroji
Kusimbisa kudzidza hakusi kungoshandura zvikamu zvechinyakare; iri kupayona kubatanidzwa neiyo-ye-iyo-tekinoroji matekinoroji, kutyaira isina kuongororwa mhinduro nekuvandudza mashandiro:
- Internet Zvinhu (IoT). RL iri kushandura IoT nekuita kuti zvishandiso zvive zvakangwara munguva chaiyo. Semuyenzaniso, smart home systems dzinoshandisa RL kudzidza kubva pamabatiro atinoita navo uye nemamiriro akavatenderedza, otomatiki mabasa akaita sekugadzirisa mwenje uye tembiricha kana kuvandudza kuchengetedzwa. Izvi hazvingochengetedze simba chete asiwo zvinoita kuti hupenyu huve nyore uye huve nyore, zvichiratidza kuti RL inogona sei kuita zvine hungwaru maitiro edu ezuva nezuva.
- Bhizinesi reBlockchain. Munyika yeblockchain, kudzidza kwekusimbisa kunobatsira kugadzira masisitimu akasimba uye anoshanda zvakanyanya. Izvo zvakakosha mukugadzira mitemo inoshanduka inochinjika kune shanduko yezvinodiwa netiweki. Kugona uku kunogona kukurumidzira kutengeserana uye kuderedza mitengo, kuratidza basa reRL mukugadzirisa mamwe matambudziko makuru mu blockchain tekinoroji.
- Augmented reality (AR). RL iri kusimudzira zvakare AR nekuita kuti kudyidzana kwevashandisi kuve kwakasarudzika uye kukwidziridzwa. Iyo inogadzirisa chaiwo zvemukati-chaiyo-nguva zvichibva pane maitiro evashandisi uye nharaunda yavari mairi, zvichiita kuti zviitiko zveAR zviwedzere kuita uye ndezvechokwadi. Izvi zvinonyanya kukosha muzvirongwa zvekudzidzisa uye zvekudzidzisa, uko RL-yakagadzirwa inochinjika nharaunda yekudzidza inotungamira kune zvirinani kudzidza nekubatanidzwa.
Nekubatanidza RL nematekinoroji akaita seIoT, blockchain, uye AR, vagadziri havasi kungovandudza mashandiro anoita masisitimu asiwo kusundira miganho yezvingawanikwe muzvirongwa zvakangwara uye masisitimu akaiswa. Musanganiswa uyu urikumisikidza nhanho yezvakawanda zvakazvimiririra, zvinoshanda, uye zvakagadzirirwa tekinoroji zvikumbiro, zvichivimbisa kufambira mberi kunonakidza kwemaindasitiri uye kushandiswa kwemazuva ese tekinoroji.
Zvishandiso uye masisitimu ekusimbisa kudzidza
Sezvo takaongorora mashandisirwo akasiyana-siyana uye kubatanidza tekinoroji yekusimbisa kudzidza, kukosha kwezvishandiso zvepamberi kugadzira, kuyedza, uye kukwenenzvera masisitimu aya zvinova pachena. Ichi chikamu chinoburitsa makiyi masisitimu uye maturusi akakosha pakugadzira anoshanda maRL mhinduro. Zvishandiso izvi zvakarongedzerwa kuti zvisangane nezvido zvenzvimbo dzakasimba uye matambudziko akaomarara anotarisana neRL, achivandudza zvese kushanda uye kukanganisa kweRL application. Ngatitarisei zvakanyanya mamwe maturusi akakosha ari kusimudzira munda weRL:
- TensorFlow Agents (TF-Agents). Chishandiso chine simba mukati meTensorFlow ecosystem, TF-Agents inotsigira huwandu hwakawanda hwealgorithms uye inonyanya kukodzera kubatanidza mamodheru epamberi nekudzidza kwakadzama, ichizadzisa kufambira mberi kwakambokurukurwa mukubatanidza kudzidza kwakadzama.
- Vhura AI Gym. Inozivikanwa nenzvimbo dzayo dzakasiyana-siyana dzekutevedzera-kubva kumitambo yeAtari yekare kusvika kumitambo yakaoma yemuviri-OpenAI Gym ipuratifomu inobvumidza vanogadzira kuyedza RL algorithms mune akasiyana marongero. Izvo zvakakosha kuti uongorore kuchinjika kweRL mumaseti akafanana kune ayo anoshandiswa mutraffic manejimendi uye smart grids.
- RLlib. Ichishanda paRay framework, RLlib yakagadziridzwa scalable uye yakagovaniswa RL, inobata mamiriro akaomarara anosanganisira akawanda maajenti, senge mukugadzira uye kuzvitonga kurongeka kwemotokari.
- PyTorch yekusimbisa kudzidza (PyTorch-RL). Uchishandisa PyTorch ine simba komputa maficha, iyi seti yeRL algorithms inopa shanduko inodiwa kune masisitimu anogadziridza kune ruzivo rutsva, izvo zvakakosha kumapurojekiti anoda kugara achigadziridzwa zvichienderana nemhinduro.
- Yakagadzikana Baselines. Iyo yakavandudzwa vhezheni yeOpenAI Baselines, Yakagadzikana Baselines inopa yakanyatso nyorwa uye mushandisi-inoshamwaridzika RL algorithms anobatsira vanogadzira kunatsa nekuvandudza nzira dziripo dzeRL, dzakakosha kuzvikamu zvakaita sehutano uye mari.
Maturusi aya haangogadzirise kuvandudzwa kweRL application chete asi zvakare anoita basa rakakosha mukuyedza, kunatsa, uye kuendesa modhi munzvimbo dzakasiyana siyana. Vakashongedzwa nekunzwisisa kwakajeka kwemabasa avo uye mashandisiro, vanogadzira uye vaongorori vanogona kushandisa zvishandiso izvi kuwedzera mikana mukusimbisa kudzidza.
Kushandisa inodyidzana simulations kudzidzisa RL modhi
Mushure mekutsanangudza maturusi ezvishandiso akakosha uye masisitimu anotsigira kukudziridzwa nekukwenenzverwa kwemamodheru ekusimudzira ekudzidza, zvakakosha kuti titarise pane iyo modhi iyi inoedzwa nekunatswa. Interactive kudzidza uye simulation nharaunda kwakakosha kufambisira mberi RL application, ichipa yakachengeteka uye inodzorwa marongero anoderedza njodzi dzepasirese.
Simulation mapuratifomu: Yechokwadi nzvimbo dzekudzidzira
Mapuratifomu akadai seUnity ML-Agents uye Microsoft AirSim anoshanda kwete sezvishandiso chete, asi semasuwo echokwadi, anodyidzana nyika uko RL algorithms inodzidziswa zvakasimba. Aya mapuratifomu akakosha kune madomasi senge akazvimirira kutyaira uye aerial marobhoti, uko chaiyo-yepasirese kuyedzwa kunodhura uye kune njodzi. Kuburikidza neakadzama ekufungidzira, vagadziri vanogona kupikisa uye kukwenenzvera maRL modhi pasi peakasiyana uye akaoma mamiriro ezvinhu, akada kufanana nekusafungira kwenyika chaiko.
Kudyidzana kweDynamic mukudzidza
Iko kusimba kwenzvimbo yekudyidzana yekudzidza inobvumira mhando dzeRL kudzidzira mabasa uye kujairana nezvinetso zvitsva munguva-chaiyo. Uku kuchinjika kwakakosha kune masisitimu eRL akagadzirirwa mashandisirwo epasirese epasirese, sekutonga mapotifolio emari kana kukwenenzvera masisitimu emudhorobha.
Basa mukuenderera mberi nekusimbiswa
Kupfuura kudzidziswa kwekutanga, nharaunda idzi dzakakosha pakuenderera mberi kwekuvandudza uye kusimbiswa kwemhando dzekudzidzira dzekusimbisa. Ivo vanopa chikuva chevagadziri kuti vaedze marongero matsva uye mamiriro, vachiongorora kusimba uye kuchinjika kwealgorithms. Izvi zvakakosha pakuvaka mamodheru ane simba anokwanisa kugadzirisa chaiwo-nyika yakaoma.
Kuwedzera tsvagiridzo uye indasitiri maitiro
Kune vaongorori, nharaunda idzi dzinopfupisa iyo mhinduro mukuvandudza modhi, zvichifambisa kukurumidza kudzokororwa nekuvandudza. Mumashandisirwo ekutengeserana, vanova nechokwadi chekuti RL masisitimu anotariswa zvakanyanya uye akagadziridzwa asati aiswa munzvimbo dzakakosha dzakadai sehutano uye mari, uko kurongeka uye kuvimbika kwakakosha.
Nekushandisa inodyidzana yekudzidza uye yekunyepedzera nharaunda mune yeRL yekuvandudza maitiro, iyo inoshanda yekushandisa uye mashandiro ekushanda kweaya akaomarara algorithms anovandudzwa. Aya mapuratifomu anoshandura ruzivo rwe theoretical kuva mashandisirwo epasirese uye anovandudza kurongeka uye kugona kweRL masisitimu, kugadzirira nzira yekusikwa kwehungwaru, inogadzirisa tekinoroji.
Zvakanakira uye matambudziko ekusimbisa kudzidza
Mushure mekuongorora maturusi akasiyana siyana, kuona mashandisirwo aanoitwa munzvimbo dzakasiyana senge mota dzehutano uye dzinozvityaira, uye kudzidza nezve pfungwa dzakaoma senge yekusimbisa kudzidza yemhinduro loop uye kuti inoshanda sei nekudzidza kwakadzama, isu tava kuenda tarisa mabhenefiti makuru nematambudziko ekusimbisa kudzidza. Ichi chikamu chenhaurirano yedu chinotarisa kuti RL inogadzirisa sei matambudziko akaomarara uye inogadzirisa nyaya dzepasirese, tichishandisa zvatakadzidza kubva muongororo yedu yakadzama.
Advantages
- Complex kugadzirisa dambudziko. Reinforcement learning (RL) inokunda munzvimbo dzisingatarisike uye dzakaoma kunzwisisa, dzinowanzoita zvirinani kupfuura nyanzvi dzevanhu. Muenzaniso wakanaka ndeweAlphaGo, iyo RL system yakahwina mutambo wayo nemakwikwi epasi rose mumutambo weGo. Kupfuura mitambo, RL yanga ichishanda zvinoshamisa mune dzimwe nzvimbo zvakare. Semuyenzaniso, mukutonga kwesimba, masisitimu eRL akavandudza kushanda kwemagidhi emagetsi kupfuura zvaifungwa nenyanzvi. Mhedzisiro iyi inoratidza kuti RL inogona sei kuwana mhinduro nyowani pachayo, ichipa mikana inonakidza yemaindasitiri akasiyana.
- High kuchinjika. Kugona kweRL kukurumidza kugadzirisa mamiriro matsva kunobatsira zvakanyanya munzvimbo dzakaita sedzimotokari dzinozvityaira uye kutengesa masheya. Muminda iyi, masisitimu eRL anogona kushandura maitiro avo nekukasira kuti aenderane nemamiriro matsva, achiratidza kuti ari kuchinjika sei. Semuenzaniso, kushandisa RL kugadzirisa nzira dzekutengesa apo misika yemusika yakaratidza kuti inoshanda zvakanyanya kupfuura nzira dzekare, kunyanya panguva dzisingatarisirwi dzemusika.
- Kuzvisarudzira kuita sarudzo. Sistimu dzekudzidza dzekusimbisa dzinoshanda dzakazvimiririra nekudzidza kubva mukudyidzana kwakananga nenzvimbo dzavari. Kuzvitonga uku kwakakosha munzvimbo dzinoda kukurumidza, kufambiswa nedatha kuita sarudzo, senge marobhoti navigation uye yakasarudzika hutano hwehutano, uko RL inogadzira sarudzo zvichibva pane inoenderera mberi data yevarwere.
- Kubudirira. RL algorithms akavakwa kuti agadzirise kukura kuomarara uye kushanda nemazvo mumashandisirwo mazhinji akasiyana. Uku kugona kuyera kunobatsira mabhizinesi kukura uye kugadzirisa munzvimbo dzakaita sekutenga online uye makore komputa, uko zvinhu zvinogara zvichichinja.
- Kuenderera mberi nekudzidza. Kusiyana nemamwe maAI mamodheru angangoda kudzokororwa nguva nenguva, masisitimu eRL anogara achidzidza nekuvandudza kubva mukudyidzana kutsva, achiita kuti ashande zvakanyanya muzvikamu zvakaita sekufembera kugadzirisa, kwavanogadzirisa zvirongwa zvichibva pane chaiyo-nguva data.
matambudziko
- Data kusimba. RL inoda data yakawanda uye kushamwaridzana nguva dzose, izvo zvakaoma kuwana mukuedza kwekutanga kwemotokari dzinozvifambisa. Kunyangwe kuvandudzwa kwekufananidza uye kugadzira data rekugadzira kunotipa zvirinani kudzidzisa dhatabheti, kuwana emhando yepamusoro-chaiyo-yepasi data ichiri dambudziko hombe.
- Real-nyika yakaoma. Kusafungira uye kunonoka mhinduro muzvirongwa chaizvo zvinoita kuti kudzidzisa maRL modhi kunetse. Nyowani maalgorithms ari kusimudzira mabatiro anoita mamodheru aya kunonoka, asi kugara achichinjika kukusafungidzirika kwemamiriro epasirese chaiwo kuchiri kunetsa.
- Mubairo dhizaini yakaoma. Zvakaoma kugadzira mibairo masisitimu anoenzanisa zviito zvekukurumidza nezvinangwa zvenguva refu. Nhamburiko dzakaita sekugadzira nzira dzekusimbisa dzekusimbisa dzakakosha, asi hadzisati dzanyatsogadzirisa kuomarara mumashandisirwo epasirese.
- High computational zvinoda. RL algorithms inoda simba rakawanda rekombuta, kunyanya kana richishandiswa muhukuru-hukuru kana hwakaoma mamiriro. Kunyangwe paine kuyedza kuita kuti maalgorithms aya anyatsoshanda uye kushandisa zvine simba komputa Hardware senge Graphics Processing Units (GPUs) uye Tensor Processing Units (TPUs), mutengo uye huwandu hwezviwanikwa hunodiwa hunogona kunge hwakanyanya kukwira kumasangano mazhinji.
- Sample kunyatsoshanda. Kudzidza kwekusimbisa kazhinji kunoda data rakawanda kuti rishande zvakanaka, rinova dambudziko guru munzvimbo dzakaita semarobhoti kana hutano hwehutano uko kuunganidza data kunogona kudhura kana kune njodzi. Nekudaro, matekiniki matsva mukudzidzira-isina-policy kudzidza uye batch yekusimbisa kudzidza ari kuita kuti zvikwanise kudzidza zvakawanda kubva kune shoma data. Zvisinei nekuvandudzwa uku, zvichiri kunetsa kuwana mibairo yakanaka kwazvo ine mashoma data mapoinzi.
Nhungamiro yeramangwana uye mamwe matambudziko
Sezvatinotarisa kune ramangwana, kudzidza kwekusimbisa kwakagadzirira kugadzirisa matambudziko aripo uye kuwedzera mashandisirwo ayo. Heano mamwe mafambiro chaiwo uye kuti anotarisirwa kugadzirisa sei matambudziko aya:
- Scalability nyaya. Nepo RL ichingo scalable, ichiri kuda kubata nharaunda dzakakura uye dzakaoma zvakanyanya. Zvitsva mumasisitimu akawanda-agent anotarisirwa kuvandudza kugoverwa kwemabasa emakomputa, ayo anogona kuderedza zvakanyanya mutengo uye kuwedzera mashandiro panguva dzepamusoro-soro, senge mune chaiyo-nguva yeguta-yakafara manejimendi traffic kana yakakwirira-mutoro nguva mumakomputa emakore.
- Kuoma kwezvishandiso zvepasirese. Kuvhara mukaha uripo pakati penzvimbo dzakadzorwa uye kusatarisika kwehupenyu chaihwo kunoramba kuri chinhu chakakosha. Tsvagiridzo yakatarisana nekugadzira ane simba algorithms anokwanisa kushanda pasi pemamiriro akasiyana. Semuyenzaniso, maitiro ekudzidza anochinjika, akaedzwa mumapurojekiti ekufambisa akazvimirira mumamiriro ekunze akasiyana-siyana, ari kugadzirira RL kuti ibate zvakada kufanana zvepasirese zvakaomarara zvakanyanya.
- Reward system dhizaini. Kugadzira masisitimu emubairo anoyananisa zviito zvenguva pfupi nezvinangwa zvenguva refu zvinoramba zvichinetsa. Kuedza kujekesa uye kurerutsa maalgorithms kuchabatsira kugadzira mamodheru ari nyore kududzira uye kuenderana nezvinangwa zvesangano, kunyanya mune zvemari uye hutano, uko mhedzisiro yakakosha.
- Kubatanidzwa kweramangwana uye kubudirira. Kubatanidzwa kweRL nehunyanzvi hweAI tekinoroji senge generative adversarial network (GANs) uye mitauro yechisikigo kugadzirisa (NLP) inotarisirwa kuwedzera zvakanyanya kugona kweRL. Iyi synergy ine chinangwa chekushandisa masimba etekinoroji yega yega kuwedzera kuchinjika kweRL uye kushanda nesimba, kunyanya mumamiriro ezvinhu akaoma. Zviitiko izvi zvakagadzirirwa kuunza mamwe masimba ane simba uye epasi rose zvikumbiro muzvikamu zvakasiyana.
Kuburikidza nekuongorora kwedu kwakadzama, zviri pachena kuti nepo RL ichipa mukana wakakura wekushandura zvikamu zvakasiyana, kubudirira kwayo kunoenderana nekukunda matambudziko makuru. Nekunyatsonzwisisa kusimba uye kusasimba kweRL, vagadziri, uye vaongorori vanogona zvakanyanya kushandisa tekinoroji iyi kutyaira hunyanzvi uye kugadzirisa matambudziko akaomarara munyika chaiyo.
Ethical kufunga mukusimbisa kudzidza
Sezvatinopedzisa kuongorora kwedu kwakadzama kwekudzidza kwekusimbisa, zvakakosha kuti tigadzirise zvazvinoreva - iyo yekupedzisira asi yakakosha chikamu chekuisa masisitimu eRL mumamiriro epasirese chaiwo. Ngatikurukurei mabasa akakosha uye matambudziko anomuka nekubatanidzwa kweRL mune tekinoroji yemazuva ese, tichiratidza kukosha kwekunyatso funga nezvekushandisa kwayo:
- Kuzvimiririra kuita sarudzo. Kudzidza kwekusimbisa kunoita kuti masisitimu aite sarudzo dzakazvimirira, izvo zvinogona kukanganisa zvakanyanya kuchengetedzeka kwevanhu uye kugara kwavo zvakanaka. Semuyenzaniso, mumotokari dzinozvimiririra, sarudzo dzakaitwa neRL algorithms dzinobata zvakananga kuchengetedzeka kwevatyairi nevanofamba netsoka. Izvo zvakakosha kuona kuti sarudzo idzi hadzikuvadze vanhu uye kuti nzira dzakasimba dziripo dzekutadza kwehurongwa.
- Zvekuvanzika zvine hanya. RL masisitimu anowanzo gadzirisa huwandu hwakawanda hwe data, kusanganisira ruzivo rwemunhu. Kudzivirirwa kwakasimba kwekuvanzika kunofanirwa kuitwa kuti ive nechokwadi chekuti mabatirwo edatha anotevera zviyero zvepamutemo uye zvetsika, kunyanya kana masisitimu anoshanda munzvimbo dzemunhu sedzimba kana pamidziyo yemunhu.
- Kusarura uye kururamisira. Kudzivisa rusarura idambudziko rakakura mukutumirwa kweRL. Sezvo masisitimu aya achidzidza kubva kunharaunda dzawo, kusarura mu data kunogona kutungamirira kusarudzo dzisina kunaka. Nyaya iyi inonyanya kukosha mumashandisirwo akaita sekufungidzira mapurisa kana kuhaya, uko maalgorithms akarerekera anogona kusimbisa kusarongeka kuripo. Vagadziri vanofanirwa kushandisa maitiro e-de-biasing uye vachiramba vachiongorora kusarongeka kwavo.
- Kuzvidavirira uye pachena. Kudzikamisa njodzi idzi, panofanirwa kuve negwara rakajeka uye zvirevo zvemaitiro ekusimbisa ekudzidza maitiro. Vagadziri uye masangano anofanirwa kuve pachena nezve maitiro avo eRL masisitimu anoita sarudzo, data ravanoshandisa, uye matanho anotorwa kugadzirisa zvine chekuita nehunhu. Pamusoro pezvo, panofanira kunge paine nzira dzekuzvidavirira uye sarudzo dzekutsvaga kana RL system ikakuvadza.
- Kuvandudza tsika nekudzidziswa: Munguva yekuvandudza uye nhanho dzekudzidzisa, zvakakosha kuti titarise etsika yekutsvagisa data uye kusanganisira akasiyana siyana emaonero. Iyi nzira inobatsira kufanogadzirisa kusarura kungangove uye kuve nechokwadi chekuti masisitimu eRL akasimba uye akanaka pane dzakasiyana siyana dzekushandisa.
- Impact pabasa. Sezvo masisitimu eRL achishandiswa zvakanyanya mumaindasitiri akasiyana, zvakakosha kutarisa mabatiro aanoita mabasa. Vanhu vanotungamira vanofanirwa kufunga uye kuderedza chero mhedzisiro yakaipa pamabasa, senge vanhu vanorasikirwa nemabasa kana mabasa ekuchinja. Vanofanirwa kuve nechokwadi chekuti sezvo mamwe mabasa achiva otomatiki, pane zvirongwa zvekudzidzisa hunyanzvi hutsva nekugadzira mabasa muminda mitsva.
Kuburikidza nekuongorora kwedu kwakadzama, zviri pachena kuti nepo RL ichipa zvinoshamisa mukana wekushandura zvikamu zvakasiyana, kunyatsotarisisa kweaya maitiro ehutsika kwakakosha. Nekuziva nekugadzirisa pfungwa idzi, vagadziri uye vaongorori vanogona kuona kuti tekinoroji yeRL inofambira mberi nenzira inoenderana nemagariro netsika.
mhedziso
| Kunyura kwedu kwakadzama mukusimbisa kudzidza (RL) kwatiratidza kugona kwayo kwakasimba kushandura zvikamu zvakawanda nemichina yekudzidzisa kudzidza nekuita sarudzo kuburikidza nemaitiro ekuedza nekukanganisa. RL's kuchinjika uye kugona kuramba uchivandudza inoita kuti ive sarudzo yakasarudzika yekuvandudza zvese kubva kumotokari dzinozvityaira kuenda kune hutano masisitimu. Nekudaro, sezvo RL inova chikamu chikuru chehupenyu hwedu hwemazuva ese, isu tinofanirwa kufunga zvakadzama maitiro ayo ehunhu. Izvo zvakakosha kuti titarise pane kurongeka, kuvanzika, uye kubuda pachena apo tinoongorora mabhenefiti nematambudziko ehunyanzvi uhu. Zvakare, sezvo RL ichichinja musika webasa, zvakakosha kutsigira shanduko dzinobatsira vanhu kukudziridza hunyanzvi hutsva nekugadzira mabasa matsva. Tichitarisa kumberi, hatifanire kungovavarira kuvandudza tekinoroji yeRL asi zvakare kuona kuti tasangana netsika dzepamusoro dzinobatsira nzanga. Nekubatanidza hutsva nebasa, tinogona kushandisa RL kwete chete kufambira mberi kwehunyanzvi asiwo kukurudzira shanduko dzakanaka munharaunda. Izvi zvinopedzisa ongororo yedu yakadzama, asi ingori kutanga kwekushandisa RL zvine hungwaru kuvaka ramangwana rakajeka uye rakanaka. |
