682:-like learning rule", only, unlike the neural gas, it has no parameters that change over time and it is capable of continuous learning, i.e. learning on data streams. GNG has been widely used in several domains, demonstrating its capabilities for clustering data incrementally. The GNG is initialized with two randomly positioned nodes which are initially connected with a zero age edge and whose errors are set to 0. Since the in the GNG input data is presented sequentially one by one, the following steps are followed at each iteration:
669:
A number of variants of the neural gas algorithm exists in the literature so as to mitigate some of its shortcomings. More notable is perhaps Bernd
Fritzke's growing neural gas, but also one should mention further elaborations such as the Growing When Required network and also the incremental growing
730:
Having a network with a growing set of nodes, like the one implemented by the GNG algorithm was seen as a great advantage, however some limitation on the learning was seen by the introduction of the parameter λ, in which the network would only be able to grow when iterations were a multiple of this
657:
Compared to self-organized map, the neural gas model does not assume that some vectors are neighbors. If two vectors happen to be close together, they would tend to move together, and if two vectors happen to be apart, they would tend to not move together. In contrast, in an SOM, if two vectors are
704:
If the current iteration is an integer multiple of a predefined frequency-creation threshold, a new node is inserted between the node with the largest error (among all) and its topological neighbor presenting the highest error. The link between the former and the latter nodes is eliminated (their
731:
parameter. The proposal to mitigate this problem was a new algorithm, the
Growing When Required network (GWR), which would have the network grow more quickly, by adding nodes as quickly as possible whenever the network identified that the existing nodes would not describe the input well enough.
739:
The ability to only grow a network may quickly introduce overfitting; on the other hand, removing nodes on the basis of age only, as in the GNG model, does not ensure that the removed nodes are actually useless, because removal depends on a model parameter that should be carefully tuned to the
721:
Another neural gas variant inspired in the GNG algorithm is the incremental growing neural gas (IGNG). The authors propose the main advantage of this algorithm to be "learning new data (plasticity) without degrading the previously trained network and forgetting the old input data (stability)."
828:
of the feature vectors, the neural gas algorithm involves sorting, which is a procedure that does not lend itself easily to parallelization or implementation in analog hardware. However, implementations in both parallel software and analog hardware were actually designed.
527:
743:
The "Plastic Neural Gas" model solves this problem by making decisions to add or remove nodes using an unsupervised version of cross-validation, which controls an equivalent notion of "generalization ability" for the unsupervised setting.
711:
If the stopping criterion is not met, the algorithm takes a following input. The criterion might be a given number of epochs, i.e., a pre-set number of times where all data is presented, or the reach of a maximum number of
365:
705:
errors are decreased by a given factor) and the new node is connected to both of them. The error of the new node is initialized as the updated error of the node which had the largest error (among all).
49:. The algorithm was coined "neural gas" because of the dynamics of the feature vectors during the adaptation process, which distribute themselves like a gas within the data space. It is applied where
942:
Progress in pattern recognition, image analysis and applications: 12th
Iberoamerican Congress on Pattern Recognition, CIARP 2007, Viña del Mar-Valparaiso, Chile, November 13–16, 2007; proceedings
1258:
T. Martinetz, S. Berkovich, and K. Schulten. "Neural-gas" Network for Vector
Quantization and its Application to Time-Series Prediction. IEEE-Transactions on Neural Networks, 4(4):558–569, 1993.
661:
The name "neural gas" is because one can imagine it to be what an SOM would be like if there is no underlying graph, and all points are free to move without the bonds that bind them together.
826:
1138:
Iqbal, Hafsa; Campo, Damian; Baydoun, Mohamad; Marcenaro, Lucio; Martin, David; Regazzoni, Carlo (2019). "Clustering
Optimization for Abnormality Detection in Semi-Autonomous Systems".
203:
592:
552:
878:
612:
572:
357:
330:
165:
879:
MICAI 2000: Advances in artificial intelligence : Mexican
International Conference on Artificial Intelligence, Acapulco, Mexico, April 2000 : proceedings
278:
115:
658:
neighbors in the underlying graph, then they will always tend to move together, no matter whether the two vectors happen to be neighbors in the
Euclidean space.
632:
300:
249:
227:
135:
692:
The winner node and its topological neighbors (connected by an edge) are moving towards the current input by different fractions of their respective errors.
903:
Angelopoulou, Anastassia; Psarrou, Alexandra; Garcia
Rodriguez, Jose; Revett, Kenneth (2005). "Computer Vision for Biomedical Image Applications". In
645:. By adapting not only the closest feature vector but all of them with a step size decreasing with increasing distance order, compared to (online)
649:
a much more robust convergence of the algorithm can be achieved. The neural gas model does not delete a node and also does not create new nodes.
909:
Computer vision for biomedical image applications: first international workshop, CVBIA 2005, Beijing, China, October 21, 2005 : proceedings
940:
Fernando
Canales; Max Chacon (2007). "Progress in Pattern Recognition, Image Analysis and Applications". In Luis Rueda; Domingo Mery (eds.).
873:
1295:
698:
If the winner node and the second-winner are connected by an edge, such an edge is set to 0. Else, an edge is created between them.
1157:
1076:
959:
924:
887:
678:
Fritzke describes the growing neural gas (GNG) as an incremental network model that learns topological relations by using a "
1321:
522:{\displaystyle w_{i_{k}}^{t+1}=w_{i_{k}}^{t}+\varepsilon \cdot e^{-k/\lambda }\cdot (x-w_{i_{k}}^{t}),k=0,\cdots ,N-1}
1231:
1190:
1011:
Marsland, Stephen; Shapiro, Jonathan; Nehmzow, Ulrich (2002). "A self-organising network that grows when required".
701:
If there are edges with an age larger than a threshold, they are removed. Nodes without connections are eliminated.
1306:
1103:
Ridella, Sandro; Rovetta, Stefano; Zunino, Rodolfo (1998). "Plastic algorithm for adaptive vector quantisation".
766:
670:
neural gas. A performance-oriented approach that avoids the risk of overfitting is the
Plastic Neural gas model.
902:
979:
1301:
138:
1214:
Ancona, Fabio; Rovetta, Stefano; Zunino, Rodolfo (1997). "Hardware implementation of the neural gas".
1173:
Ancona, Fabio; Rovetta, Stefano; Zunino, Rodolfo (1996). "A parallel approach to plastic neural gas".
874:"Competitive learning methods for efficient Vector Quantizations in a speech recognition environment"
847:
30:
1025:
170:
86:
577:
537:
1289:
1059:
Prudent, Yann; Ennaji, Abdellatif (2005). "An incremental growing neural gas learns topologies".
686:
It is calculated the errors (distances) between the two closest nodes to the current input data.
1020:
597:
557:
1140:
1st International Workshop on Multimodal Understanding and Learning for Embodied Applications
748:
335:
308:
143:
254:
91:
8:
66:
54:
45:. The neural gas is a simple algorithm for finding optimal data representations based on
34:
1298:
Unsupervised Neural Networks (including Self-organizing map) in Java with source codes.
1237:
1196:
1120:
1082:
646:
617:
285:
234:
212:
120:
70:
58:
1034:
1275:
1227:
1186:
1153:
1072:
1038:
955:
920:
883:
1241:
1200:
1086:
1271:
1219:
1178:
1143:
1124:
1112:
1064:
1030:
945:
912:
679:
638:
74:
62:
50:
38:
950:
944:. Lecture Notes in Computer Science. Vol. 4756. Springer. pp. 684–693.
689:
The error of the winner node (only the closest one) is respectively accumulated.
1068:
752:
46:
42:
1061:
Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005
1315:
1223:
1182:
642:
1148:
911:. Lecture Notes in Computer Science. Vol. 3765. Springer. p. 210.
1042:
916:
1116:
876:. In Osvaldo Cairó; L. Enrique Sucar; Francisco J. Cantú-Ortiz (eds.).
751:
scenario, the ability to grow and shrink is suited to the more general
1262:
Martinetz, T.; Schulten, K. (1994). "Topology representing networks".
904:
20:
1216:
Proceedings of International Conference on Neural Networks (ICNN'97)
1175:
Proceedings of International Conference on Neural Networks (ICNN'96)
858:
845:
708:
The accumulated error of all nodes is decreased by a given factor.
695:
The age of all edges connected to the winner node are incremented.
1292:
Javascript simulator for Neural Gas (and other network models)
634:
so that the algorithm converges after many adaptation steps.
871:
637:
The adaptation step of the neural gas can be interpreted as
1137:
939:
359:
the index of the second closest feature vector, and so on.
1010:
769:
620:
600:
580:
560:
540:
368:
338:
311:
288:
257:
237:
215:
173:
146:
123:
94:
1102:
1213:
1172:
820:
626:
606:
586:
566:
546:
521:
351:
324:
294:
272:
243:
221:
197:
159:
129:
109:
1307:A GNG and GWR Classifier implementation in Matlab
1261:
984:Advances in Neural Information Processing Systems
716:
1313:
980:"A Growing Neural Gas Network Learns Topologies"
747:While growing-only methods only cater for the
69:. As a robustly converging alternative to the
1058:
740:"memory length" of the stream of input data.
846:Thomas Martinetz and Klaus Schulten (1991).
554:can be understood as the learning rate, and
332:be the index of the closest feature vector,
302:and each feature vector. Rank the distances.
821:{\displaystyle i_{0},i_{1},\ldots ,i_{N-1}}
1302:formal description of Neural gas algorithm
848:"A "neural gas" network learns topologies"
1147:
1024:
949:
907:; Tianzi Jiang; Changshui Zhang (eds.).
725:
977:
872:F. Curatelli; O. Mayora-Iberra (2000).
1314:
1296:Java Competitive Learning Applications
1098:
1096:
652:
734:
673:
1054:
1052:
1006:
1004:
973:
971:
1105:Neural Computing & Applications
1093:
1063:. Vol. 2. pp. 1211–1216.
13:
1252:
758:
14:
1333:
1283:
1218:. Vol. 2. pp. 991–994.
1177:. Vol. 1. pp. 126–130.
1049:
1001:
968:
1207:
1166:
1131:
933:
896:
865:
839:
717:Incremental growing neural gas
486:
455:
362:Update each feature vector by:
267:
261:
104:
98:
1:
1035:10.1016/s0893-6080(02)00078-3
832:
282:Compute the distance between
198:{\displaystyle i=1,\cdots ,N}
1276:10.1016/0893-6080(94)90109-0
951:10.1007/978-3-540-76725-1_71
614:are reduced with increasing
587:{\displaystyle \varepsilon }
547:{\displaystyle \varepsilon }
80:
7:
664:
574:as the neighborhood range.
85:Suppose we want to model a
10:
1338:
1322:Artificial neural networks
1069:10.1109/IJCNN.2005.1556026
855:Artificial Neural Networks
37:and introduced in 1991 by
18:
882:. Springer. p. 109.
137:using a finite number of
57:is an issue, for example
31:artificial neural network
16:Artificial neural network
1224:10.1109/ICNN.1997.616161
1183:10.1109/ICNN.1996.548878
607:{\displaystyle \lambda }
567:{\displaystyle \lambda }
87:probability distribution
19:Not to be confused with
1149:10.1145/3347450.3357657
978:Fritzke, Bernd (1995).
822:
628:
608:
588:
568:
548:
523:
353:
326:
296:
274:
245:
223:
199:
161:
131:
111:
823:
726:Growing when required
629:
609:
589:
569:
549:
524:
354:
352:{\displaystyle i_{1}}
327:
325:{\displaystyle i_{0}}
297:
275:
246:
224:
200:
162:
160:{\displaystyle w_{i}}
132:
112:
767:
763:To find the ranking
749:incremental learning
618:
598:
578:
558:
538:
366:
336:
309:
286:
273:{\displaystyle P(x)}
255:
235:
213:
171:
144:
121:
110:{\displaystyle P(x)}
92:
73:it is also used for
917:10.1007/11569541_22
861:. pp. 397–402.
653:Comparison with SOM
485:
421:
396:
231:Sample data vector
209:For each time step
67:pattern recognition
55:vector quantization
35:self-organizing map
1142:. pp. 33–41.
1117:10.1007/BF01413708
818:
735:Plastic neural gas
674:Growing neural gas
647:k-means clustering
624:
604:
584:
564:
544:
534:In the algorithm,
519:
464:
400:
369:
349:
322:
292:
270:
241:
219:
195:
157:
127:
107:
71:k-means clustering
59:speech recognition
33:, inspired by the
1159:978-1-4503-6918-3
1078:978-0-7803-9048-5
961:978-3-540-76724-4
926:978-3-540-29411-5
889:978-3-540-67354-5
627:{\displaystyle t}
295:{\displaystyle x}
244:{\displaystyle x}
222:{\displaystyle t}
130:{\displaystyle x}
1329:
1279:
1246:
1245:
1211:
1205:
1204:
1170:
1164:
1163:
1151:
1135:
1129:
1128:
1100:
1091:
1090:
1056:
1047:
1046:
1028:
1019:(8): 1041–1058.
1008:
999:
998:
996:
995:
975:
966:
965:
953:
937:
931:
930:
900:
894:
893:
869:
863:
862:
852:
843:
827:
825:
824:
819:
817:
816:
792:
791:
779:
778:
639:gradient descent
633:
631:
630:
625:
613:
611:
610:
605:
593:
591:
590:
585:
573:
571:
570:
565:
553:
551:
550:
545:
528:
526:
525:
520:
484:
479:
478:
477:
451:
450:
446:
420:
415:
414:
413:
395:
384:
383:
382:
358:
356:
355:
350:
348:
347:
331:
329:
328:
323:
321:
320:
301:
299:
298:
293:
279:
277:
276:
271:
250:
248:
247:
242:
228:
226:
225:
220:
204:
202:
201:
196:
166:
164:
163:
158:
156:
155:
136:
134:
133:
128:
117:of data vectors
116:
114:
113:
108:
75:cluster analysis
63:image processing
51:data compression
39:Thomas Martinetz
1337:
1336:
1332:
1331:
1330:
1328:
1327:
1326:
1312:
1311:
1286:
1264:Neural Networks
1255:
1253:Further reading
1250:
1249:
1234:
1212:
1208:
1193:
1171:
1167:
1160:
1136:
1132:
1101:
1094:
1079:
1057:
1050:
1013:Neural Networks
1009:
1002:
993:
991:
976:
969:
962:
938:
934:
927:
901:
897:
890:
870:
866:
850:
844:
840:
835:
806:
802:
787:
783:
774:
770:
768:
765:
764:
761:
759:Implementations
737:
728:
719:
676:
667:
655:
619:
616:
615:
599:
596:
595:
579:
576:
575:
559:
556:
555:
539:
536:
535:
480:
473:
469:
468:
442:
435:
431:
416:
409:
405:
404:
385:
378:
374:
373:
367:
364:
363:
343:
339:
337:
334:
333:
316:
312:
310:
307:
306:
287:
284:
283:
256:
253:
252:
236:
233:
232:
214:
211:
210:
172:
169:
168:
151:
147:
145:
142:
141:
139:feature vectors
122:
119:
118:
93:
90:
89:
83:
47:feature vectors
24:
17:
12:
11:
5:
1335:
1325:
1324:
1310:
1309:
1304:
1299:
1293:
1285:
1284:External links
1282:
1281:
1280:
1270:(3): 507–522.
1259:
1254:
1251:
1248:
1247:
1232:
1206:
1191:
1165:
1158:
1130:
1092:
1077:
1048:
1026:10.1.1.14.8763
1000:
967:
960:
932:
925:
895:
888:
864:
837:
836:
834:
831:
815:
812:
809:
805:
801:
798:
795:
790:
786:
782:
777:
773:
760:
757:
753:streaming data
736:
733:
727:
724:
718:
715:
714:
713:
709:
706:
702:
699:
696:
693:
690:
687:
675:
672:
666:
663:
654:
651:
623:
603:
583:
563:
543:
532:
531:
530:
529:
518:
515:
512:
509:
506:
503:
500:
497:
494:
491:
488:
483:
476:
472:
467:
463:
460:
457:
454:
449:
445:
441:
438:
434:
430:
427:
424:
419:
412:
408:
403:
399:
394:
391:
388:
381:
377:
372:
360:
346:
342:
319:
315:
303:
291:
280:
269:
266:
263:
260:
240:
218:
194:
191:
188:
185:
182:
179:
176:
154:
150:
126:
106:
103:
100:
97:
82:
79:
43:Klaus Schulten
15:
9:
6:
4:
3:
2:
1334:
1323:
1320:
1319:
1317:
1308:
1305:
1303:
1300:
1297:
1294:
1291:
1288:
1287:
1277:
1273:
1269:
1265:
1260:
1257:
1256:
1243:
1239:
1235:
1233:0-7803-4122-8
1229:
1225:
1221:
1217:
1210:
1202:
1198:
1194:
1192:0-7803-3210-5
1188:
1184:
1180:
1176:
1169:
1161:
1155:
1150:
1145:
1141:
1134:
1126:
1122:
1118:
1114:
1110:
1106:
1099:
1097:
1088:
1084:
1080:
1074:
1070:
1066:
1062:
1055:
1053:
1044:
1040:
1036:
1032:
1027:
1022:
1018:
1014:
1007:
1005:
989:
985:
981:
974:
972:
963:
957:
952:
947:
943:
936:
928:
922:
918:
914:
910:
906:
899:
891:
885:
881:
880:
875:
868:
860:
856:
849:
842:
838:
830:
813:
810:
807:
803:
799:
796:
793:
788:
784:
780:
775:
771:
756:
754:
750:
745:
741:
732:
723:
710:
707:
703:
700:
697:
694:
691:
688:
685:
684:
683:
681:
671:
662:
659:
650:
648:
644:
643:cost function
640:
635:
621:
601:
581:
561:
541:
516:
513:
510:
507:
504:
501:
498:
495:
492:
489:
481:
474:
470:
465:
461:
458:
452:
447:
443:
439:
436:
432:
428:
425:
422:
417:
410:
406:
401:
397:
392:
389:
386:
379:
375:
370:
361:
344:
340:
317:
313:
304:
289:
281:
264:
258:
238:
230:
229:
216:
208:
207:
206:
192:
189:
186:
183:
180:
177:
174:
152:
148:
140:
124:
101:
95:
88:
78:
76:
72:
68:
64:
60:
56:
52:
48:
44:
40:
36:
32:
28:
22:
1267:
1263:
1215:
1209:
1174:
1168:
1139:
1133:
1108:
1104:
1060:
1016:
1012:
992:. Retrieved
987:
983:
941:
935:
908:
898:
877:
867:
854:
841:
762:
746:
742:
738:
729:
720:
677:
668:
660:
656:
636:
533:
84:
26:
25:
1290:DemoGNG.js
994:2016-04-26
833:References
27:Neural gas
1111:: 37–51.
1021:CiteSeerX
990:: 625–632
905:Yanxi Liu
811:−
797:…
755:problem.
602:λ
582:ε
562:λ
542:ε
514:−
505:⋯
462:−
453:⋅
448:λ
437:−
429:⋅
426:ε
187:⋯
81:Algorithm
21:Nerve gas
1316:Category
1242:62480597
1201:61686854
1087:41517545
1043:12416693
859:Elsevier
665:Variants
167:, where
1125:1184174
1240:
1230:
1199:
1189:
1156:
1123:
1085:
1075:
1041:
1023:
958:
923:
886:
712:nodes.
29:is an
1238:S2CID
1197:S2CID
1121:S2CID
1083:S2CID
851:(PDF)
641:on a
251:from
1228:ISBN
1187:ISBN
1154:ISBN
1073:ISBN
1039:PMID
956:ISBN
921:ISBN
884:ISBN
680:Hebb
594:and
305:Let
41:and
1272:doi
1220:doi
1179:doi
1144:doi
1113:doi
1065:doi
1031:doi
946:doi
913:doi
65:or
53:or
1318::
1266:.
1236:.
1226:.
1195:.
1185:.
1152:.
1119:.
1107:.
1095:^
1081:.
1071:.
1051:^
1037:.
1029:.
1017:15
1015:.
1003:^
986:.
982:.
970:^
954:.
919:.
857:.
853:.
205:.
77:.
61:,
1278:.
1274::
1268:7
1244:.
1222::
1203:.
1181::
1162:.
1146::
1127:.
1115::
1109:7
1089:.
1067::
1045:.
1033::
997:.
988:7
964:.
948::
929:.
915::
892:.
814:1
808:N
804:i
800:,
794:,
789:1
785:i
781:,
776:0
772:i
622:t
517:1
511:N
508:,
502:,
499:0
496:=
493:k
490:,
487:)
482:t
475:k
471:i
466:w
459:x
456:(
444:/
440:k
433:e
423:+
418:t
411:k
407:i
402:w
398:=
393:1
390:+
387:t
380:k
376:i
371:w
345:1
341:i
318:0
314:i
290:x
268:)
265:x
262:(
259:P
239:x
217:t
193:N
190:,
184:,
181:1
178:=
175:i
153:i
149:w
125:x
105:)
102:x
99:(
96:P
23:.
Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.